CI/CD Pipeline

Relevant source files

• .github/workflows/ci.yml
• docker-compose.yml

Purpose and Scope

This document describes the automated testing pipeline implemented through GitHub Actions that validates the Mosquitto MQTT broker service can be successfully deployed using Docker Compose. The pipeline is triggered on code changes and performs basic health checks to ensure the broker container starts and reaches a running state.

Note : This page covers automated testing of the core infrastructure. For manual testing and local development setup, see Local Development Setup. For production deployment considerations, see Production Considerations.

Overview

The CI/CD pipeline is implemented as a GitHub Actions workflow defined in .github/workflows/ci.yml:1-43 The pipeline focuses exclusively on testing the mosquitto service, as the cloudflared service requires a valid CLOUDFLARE_TUNNEL_TOKEN which cannot be securely provided in the public CI environment.

Key Characteristics:

• Scope : Tests only the mosquitto service from docker-compose.yml:4-9
• Trigger Events : Push and pull request events to the main branch
• Test Environment : Ubuntu runner with Docker and Docker Compose
• Validation Method : Container state inspection with retry logic
• Test Duration : Maximum 100 seconds (10 attempts × 10 seconds)

Sources: .github/workflows/ci.yml:1-43 docker-compose.yml:1-18

Workflow Configuration

Trigger Conditions

The workflow is configured to run on two types of GitHub events:

This ensures all changes to the main branch and all pull requests targeting main are automatically validated before merge.

Sources: .github/workflows/ci.yml:3-9

Job Definition

The workflow defines a single job named test-mosquitto:

The job executes on GitHub's ubuntu-latest runner, which provides Docker pre-installed but requires Docker Compose to be explicitly installed.

Sources: .github/workflows/ci.yml:11-13

Workflow Architecture

Complete Pipeline Flow

Sources: .github/workflows/ci.yml:1-43 docker-compose.yml:4-9

Step-by-Step Execution

Sources: .github/workflows/ci.yml:1-43 docker-compose.yml:4-9

Test Steps Detail

Step 1: Repository Checkout

Clones the repository to the runner's workspace using the official GitHub Actions checkout action. This provides access to docker-compose.yml:1-18 and mosquitto.conf:1-7

Sources: .github/workflows/ci.yml:16-17

Step 2: Docker Compose Installation

Installs Docker Compose on the Ubuntu runner. Although Docker Engine is pre-installed on ubuntu-latest, Docker Compose must be explicitly installed through the package manager.

Sources: .github/workflows/ci.yml:19-22

Step 3: Service Startup

Starts only the mosquitto service in detached mode. The -d flag ensures the container runs in the background, allowing the workflow to proceed to health checks. Note that the cloudflared service defined in docker-compose.yml:11-17 is explicitly excluded, as it requires a CLOUDFLARE_TUNNEL_TOKEN which is not available in the CI environment.

Sources: .github/workflows/ci.yml:24-25 docker-compose.yml:4-9

Step 4: Health Check Loop

The health check implements a retry pattern with the following parameters:

Health Check Implementation

Sources: .github/workflows/ci.yml:27-39

Inspection Command

The health check uses Docker's inspect command with format filtering:

This command:

1. Queries the container's state using docker inspect
2. Extracts the Status field from the container state
3. Checks if the status contains the string running
4. Returns exit code 0 if found, non-zero otherwise

Sources: .github/workflows/ci.yml30

Step 5: Service Teardown

Executes regardless of health check success or failure. Stops and removes the mosquitto container, ensuring clean resource management. The workflow does not use conditional execution (e.g., if: always()) because Docker Compose steps default to always running.

Sources: .github/workflows/ci.yml:41-42

What Is Tested

The CI pipeline validates the following aspects:

Sources: .github/workflows/ci.yml:1-43 docker-compose.yml:4-9 mosquitto.conf:1-7

What Is NOT Tested

The following aspects are explicitly excluded from CI testing:

Cloudflared Service

The cloudflared service requires a valid CLOUDFLARE_TUNNEL_TOKEN from docker-compose.yml17 and .env.sample:1-6 This token:

• Authenticates the tunnel client with Cloudflare's network
• Is environment-specific and cannot be shared across deployments
• Would expose the tunnel to public access if leaked in CI logs

Sources: docker-compose.yml:11-17 .github/workflows/ci.yml:24-25

Functional Testing

The pipeline does not perform:

Sources: .github/workflows/ci.yml:1-43 mosquitto.conf:1-7

Success and Failure Scenarios

Success Criteria

The workflow succeeds when:

docker inspect --format='{{.State.Status}}' mosquitto
# Returns: running
# Within 10 attempts (100 seconds)

This indicates the mosquitto container:

1. Started successfully using the configuration from mosquitto.conf:1-7
2. Loaded the volume mount from docker-compose.yml8
3. Initialized the Mosquitto broker process
4. Reached a stable running state

Exit Code : 0 (.github/workflows/ci.yml32)

Sources: .github/workflows/ci.yml:29-32

Failure Scenarios

The workflow fails in the following conditions:

Timeout Failure

Waiting for Mosquitto to be healthy...
(Repeats 10 times)
Mosquitto did not become healthy in time

Exit Code : 1 (.github/workflows/ci.yml39)

Potential Causes :

• Configuration error in mosquitto.conf:1-7
• Missing or invalid volume mount in docker-compose.yml8
• Resource constraints on the GitHub Actions runner
• Network issues preventing image pull from Docker Hub

Sources: .github/workflows/ci.yml:38-39

Docker Compose Failures

Failures during docker-compose up step prevent health checks from running:

Sources: docker-compose.yml:1-18 .github/workflows/ci.yml:24-25

Running Tests Locally

Developers can replicate the CI environment locally using the same commands:

Note : Local testing may require Docker and Docker Compose to be installed. See Prerequisites for installation instructions.

Sources: .github/workflows/ci.yml:24-42

Debugging Failed Workflows

When a workflow fails in GitHub Actions:

graph TB
    Failure["Workflow Failure Detected"]
CheckLogs["Review GitHub Actions logs"]
IdentifyStep{"Which step failed?"}
CheckoutFail["Checkout Step\n[ci.yml:16-17]"]
ComposeFail["Docker Compose Install\n[ci.yml:19-22]"]
StartupFail["Service Startup\n[ci.yml:24-25]"]
HealthFail["Health Check Timeout\n[ci.yml:27-39]"]
CheckRepo["Verify repository permissions"]
CheckRunner["Check runner environment"]
CheckImage["Verify docker-compose.yml syntax\nCheck image availability"]
CheckConfig["Review mosquitto.conf\nCheck container logs"]
Failure --> CheckLogs
 
   CheckLogs --> IdentifyStep
    
 
   IdentifyStep -->|Step 1| CheckoutFail
 
   IdentifyStep -->|Step 2| ComposeFail
 
   IdentifyStep -->|Step 3| StartupFail
 
   IdentifyStep -->|Step 4| HealthFail
    
 
   CheckoutFail --> CheckRepo
 
   ComposeFail --> CheckRunner
 
   StartupFail --> CheckImage
 
   HealthFail --> CheckConfig

Viewing Logs

1. Navigate to the Actions tab in the GitHub repository
2. Select the failed workflow run
3. Click on the test-mosquitto job
4. Expand the failing step to view output

Common Debugging Steps

Sources: .github/workflows/ci.yml:1-43

Retrieving Container Logs

GitHub Actions does not automatically capture container logs. To debug container-level issues, modify the workflow temporarily:

This step would appear after .github/workflows/ci.yml39 and before .github/workflows/ci.yml:41-42

Sources: .github/workflows/ci.yml:27-42

Limitations and Considerations

Scope Limitations

Sources: .github/workflows/ci.yml:1-43 mosquitto.conf:1-7

Runner Environment Differences

The GitHub Actions ubuntu-latest runner may differ from production environments:

• Architecture : Typically x86_64; production might use ARM
• Network : Isolated; production might have firewall rules
• Resources : Limited CPU/memory; production might have different constraints
• Docker Version : May lag behind latest Docker Engine releases

Sources: .github/workflows/ci.yml13

Performance Considerations

The 100-second timeout .github/workflows/ci.yml:29-39 is conservative but may be insufficient if:

• Docker Hub implements rate limiting on image pulls
• The GitHub Actions runner is under heavy load
• Network latency to Docker Hub is high

In production deployments, containers typically start within 5-10 seconds.

Sources: .github/workflows/ci.yml:27-39

Future Enhancements

Potential improvements to the CI pipeline:

Extended Test Coverage

Each enhancement would require additional test steps and potentially separate jobs in .github/workflows/ci.yml:11-42

Sources: .github/workflows/ci.yml:1-43

Integration Testing

For systems deploying the protected features (see Topic Access Control) and Encrypted Retained Messages), additional test jobs could validate:

• ACL enforcement with test credentials
• Encrypted volume mounting and data persistence
• Multi-client pub/sub scenarios

Sources: .github/workflows/ci.yml:1-43

Summary

The CI/CD pipeline provides basic validation that the Mosquitto MQTT broker can be successfully deployed using the repository's Docker Compose configuration. The pipeline:

• Executes on every push and pull request to main
• Tests only the mosquitto service (excludes cloudflared)
• Validates container startup and health within 100 seconds
• Ensures configuration files are syntactically valid
• Provides immediate feedback to developers on infrastructure changes

While limited in scope, this pipeline prevents basic configuration errors from reaching production and provides a foundation for more comprehensive testing in the future.

Sources: .github/workflows/ci.yml:1-43 docker-compose.yml:1-18 mosquitto.conf:1-7