Development

Relevant source files

• .github/workflows/ci.yml
• .gitignore
• docker-compose.yml

Purpose and Scope

This document provides an overview of the development workflow and practices for the Docker MQTT Mosquitto Cloudflare Tunnel system. It covers the essential processes for setting up a local development environment, understanding the automated testing pipeline, and following version control best practices.

This page serves as an entry point for developers who want to contribute to or extend the system. For detailed instructions on:

• Setting up your local development environment, see Local Development Setup
• Understanding the automated testing system, see CI/CD Pipeline
• Managing secrets and version control, see Version Control Practices

For production deployment guidance, see Production Considerations.

Sources: README.md, docker-compose.yml, .gitignore, .github/workflows/ci.yml

Development Workflow Overview

The development workflow follows a standard pattern for Docker-based systems with cloud integration. Developers work with configuration files tracked in version control while managing secrets locally.

Development File Structure

The following table summarizes the files developers interact with and their roles:

Sources: .gitignore, docker-compose.yml, .env.sample (referenced in README)

graph TB
    subgraph "Version Control [GitHub]"
        Repo["Repository"]
ComposeYML["docker-compose.yml"]
MosqConf["mosquitto.conf"]
EnvSample[".env.sample"]
GitIgnore[".gitignore"]
CIWorkflow[".github/workflows/ci.yml"]
end
    
    subgraph "Developer Workstation"
        LocalClone["Local Git Clone"]
EnvActual[".env\n(Not Tracked)"]
DataDir["data/\n(Not Tracked)"]
DockerEngine["Docker Engine"]
end
    
    subgraph "External Services"
        CFDashboard["Cloudflare Zero Trust Dashboard"]
CFToken["CLOUDFLARE_TUNNEL_TOKEN"]
end
    
    subgraph "Running Services [Docker Compose]"
        MosquittoContainer["mosquitto Container\n(eclipse-mosquitto:latest)"]
CloudflaredContainer["cloudflared Container\n(cloudflare/cloudflared:latest)"]
end
    
 
   Repo -->|git clone| LocalClone
 
   LocalClone -->|Contains| ComposeYML
 
   LocalClone -->|Contains| MosqConf
 
   LocalClone -->|Contains| EnvSample
 
   LocalClone -->|Contains| GitIgnore
    
 
   GitIgnore -.->|Excludes| EnvActual
 
   GitIgnore -.->|Excludes| DataDir
    
 
   EnvSample -->|Template for| EnvActual
 
   CFDashboard -->|Provides| CFToken
 
   CFToken -->|Stored in| EnvActual
    
 
   ComposeYML -->|docker-compose up| DockerEngine
 
   DockerEngine -->|Starts| MosquittoContainer
 
   DockerEngine -->|Starts| CloudflaredContainer
    
 
   MosqConf -.->|Volume Mount| MosquittoContainer
 
   EnvActual -.->|Environment Variable| CloudflaredContainer
 
   MosquittoContainer -.->|Creates| DataDir

Development Environment Components

Diagram: Development Environment Component Relationships

This diagram shows how version-controlled files, local secrets, and Docker services interact during development. The .gitignore file .gitignore:1-2 prevents sensitive files from being committed, while docker-compose.yml docker-compose.yml:1-18 orchestrates both services.

Sources: .gitignore, docker-compose.yml, .env.sample (referenced in README)

Continuous Integration Pipeline

The system includes automated testing through GitHub Actions that validates the mosquitto service can start successfully.

graph TB
    subgraph "Trigger Events"
        PushMain["Push to main branch"]
PullRequest["Pull Request to main"]
end
    
    subgraph "GitHub Actions Runner [ubuntu-latest]"
        Checkout["Checkout repository\n(actions/checkout@v2)"]
SetupDocker["Install docker-compose\n(apt-get install)"]
StartService["docker-compose up -d mosquitto"]
subgraph "Health Check Loop"
            Iterate["for i in {1..10}"]
Inspect["docker inspect --format='{{.State.Status}}' mosquitto"]
CheckRunning{"Status == 'running'?"}
Sleep["sleep 10"]
end
        
        Teardown["docker-compose down"]
end
    
    subgraph "Test Result"
        Success["Exit 0: Success"]
Failure["Exit 1: Timeout"]
end
    
 
   PushMain -->|Triggers| Checkout
 
   PullRequest -->|Triggers| Checkout
    
 
   Checkout --> SetupDocker
 
   SetupDocker --> StartService
 
   StartService --> Iterate
    
 
   Iterate --> Inspect
 
   Inspect --> CheckRunning
 
   CheckRunning -->|Yes| Success
 
   CheckRunning -->|No| Sleep
 
   Sleep --> Iterate
 
   Iterate -.->|After 10 attempts| Failure
    
 
   Success --> Teardown
 
   Failure --> Teardown

CI Workflow Structure

Diagram: CI Workflow Execution Flow

The CI workflow defined in .github/workflows/ci.yml:1-43 executes a series of steps to validate the mosquitto service. The health check loop .github/workflows/ci.yml:28-39 polls the container status using docker inspect with a maximum of 10 attempts at 10-second intervals.

Sources: .github/workflows/ci.yml

Service Orchestration

The docker-compose.yml file defines two services that developers work with:

mosquitto Service

The mosquitto service configuration docker-compose.yml:4-9 includes:

• Image: eclipse-mosquitto:latest
• Container Name: mosquitto
• Volume Mount: ./mosquitto.conf:/mosquitto/config/mosquitto.conf
• Restart Policy: unless-stopped

This service does not expose any ports to the host, as traffic routing is handled internally through the Docker network to the cloudflared service.

cloudflared Service

The cloudflared service configuration docker-compose.yml:11-17 includes:

• Image: cloudflare/cloudflared:latest
• Container Name: cloudflared
• Command: tunnel --no-autoupdate run --token ${CLOUDFLARE_TUNNEL_TOKEN}
• Environment: Receives CLOUDFLARE_TUNNEL_TOKEN from .env file
• Restart Policy: unless-stopped

Sources: docker-compose.yml

Version Control Strategy

The repository implements a clear separation between public configuration and private secrets:

Tracked Files

Files committed to version control:

• Service definitions (docker-compose.yml)
• Public configuration (mosquitto.conf)
• Environment templates (.env.sample)
• CI workflows (.github/workflows/ci.yml)

Excluded Files

Files excluded via .gitignore .gitignore:1-2:

• .env - Contains the CLOUDFLARE_TUNNEL_TOKEN secret
• data/* - Runtime data directory used by mosquitto for persistence

This exclusion pattern ensures sensitive credentials never enter version control while maintaining clear documentation through .env.sample.

Sources: .gitignore

Testing Strategy

The automated testing focuses on validating successful service startup rather than functional MQTT testing:

Note that the cloudflared service is not tested in CI because it requires a valid CLOUDFLARE_TUNNEL_TOKEN, which is not available in the test environment. The CI validates only that the mosquitto service can successfully start in isolation.

Sources: .github/workflows/ci.yml

Development Commands

Developers use the following docker-compose commands during development:

These commands interact with the service definitions in docker-compose.yml:3-17

Sources: docker-compose.yml, .github/workflows/ci.yml

Configuration Management

Environment Variables

The system uses environment variable substitution in docker-compose.yml docker-compose.yml:14-17:

The CLOUDFLARE_TUNNEL_TOKEN variable must be defined in the .env file for the cloudflared service to authenticate with Cloudflare's network.

Volume Mounts

The mosquitto configuration is provided via volume mount docker-compose.yml:7-8:

This allows developers to modify mosquitto.conf and restart the service without rebuilding any Docker images.

Sources: docker-compose.yml

Next Steps for Developers

After understanding this overview:

1. Setup Local Environment: Follow the detailed instructions in Local Development Setup to configure your workstation
2. Understand CI Pipeline: Review CI/CD Pipeline for details on how automated tests validate changes
3. Learn Version Control: Read Version Control Practices for guidelines on managing secrets and contributions

For advanced configuration topics such as ACLs and encryption, see Advanced Topics.

Sources: All files in this repository