Troubleshooting

Relevant source files

• .github/workflows/ci.yml
• README.md
• docker-compose.yml
• mosquitto.conf

This page provides diagnostic procedures and solutions for common issues encountered when deploying and operating the Docker MQTT Mosquitto Cloudflare Tunnel system. It covers problems related to container startup, tunnel connectivity, MQTT client connections, and configuration errors.

For production deployment considerations and hardening, see Production Considerations. For security-related configuration, see Security Model.

Diagnostic Overview

The following decision tree helps identify the component experiencing issues:

Sources : System architecture from README.md, docker-compose.yml, mosquitto.conf

Container Verification

Checking Container Status

First, verify that both the mosquitto and cloudflared containers are running:

Expected output should show both containers with status "Up":

If containers are missing or have status "Exited" or "Restarting", proceed to the relevant troubleshooting section below.

Sources : docker-compose.yml:4-9, docker-compose.yml:11-17

graph LR
 
   DockerPS["docker ps"] --> CheckMosquitto{"mosquitto\ncontainer visible?"}
DockerPS --> CheckCF{"cloudflared\ncontainer visible?"}
CheckMosquitto -->|No| MosquittoDown["Mosquitto stopped\nor failed"]
CheckMosquitto -->|Yes| MosquittoStatus["Check Status field"]
CheckCF -->|No| CFDown["Cloudflared stopped\nor failed"]
CheckCF -->|Yes| CFStatus["Check Status field"]
MosquittoStatus --> StatusUp{"Status: Up?"}
StatusUp -->|No| MosquittoRestarting["Container restarting\nCheck logs"]
StatusUp -->|Yes| MosquittoOK["Mosquitto running"]
CFStatus --> CFStatusUp{"Status: Up?"}
CFStatusUp -->|No| CFRestarting["Container restarting\nCheck token"]
CFStatusUp -->|Yes| CFOK["Cloudflared running"]

Docker Compose Startup Failures

Issue: Containers Fail to Start

Symptoms :

• Running docker compose up results in immediate exit
• Error messages about missing files or invalid configuration
• Containers repeatedly restart

Common Causes :

Mosquitto Container Exits Immediately

Diagnostic Steps :

1. Check mosquitto logs:

2. Common error patterns:

Configuration File Syntax Errors :

The mosquitto.conf file must have valid syntax. Each listener directive must be complete:

listener <port>
[allow_anonymous true|false]
[protocol websockets|mqtt]

Verify the configuration at mosquitto.conf:1-6 matches the expected format:

• Line 1: listener 1883 (standard MQTT)
• Line 2: allow_anonymous true
• Line 4: listener 9001 (WebSocket)
• Line 5: protocol websockets

Sources : mosquitto.conf:1-6, docker-compose.yml:4-9

Cloudflared Container Exits Immediately

Diagnostic Steps :

1. Check cloudflared logs:

2. Identify error type:

Token Configuration Issues :

The environment variable must be correctly passed from .env to the container. Verify:

1. .env file exists in project root with format:

CLOUDFLARE_TUNNEL_TOKEN=your_actual_token_here

2. The token is referenced in docker-compose.yml14 as ${CLOUDFLARE_TUNNEL_TOKEN}
3. The environment variable is declared in docker-compose.yml:16-17

Sources : docker-compose.yml:11-17, README.md:47-53

Cloudflared Tunnel Issues

Tunnel Connection Failures

Symptoms :

• cloudflared container runs but clients cannot connect
• Logs show repeated connection attempts
• Cloudflare dashboard shows tunnel as "Inactive"

Diagnostic Commands :

Common Issues :

1. Token Mismatch : The token in .env doesn't match the tunnel configuration

   • Solution: Copy exact token from Cloudflare dashboard as shown in README.md:47-51
   • Recreate .env file with correct format

2. Network Isolation : Docker container cannot reach Cloudflare edge servers

   • Solution: Verify Docker networking, check corporate firewall/proxy settings
   • Test connectivity: docker exec cloudflared ping cloudflare.com

3. Tunnel Deleted in Dashboard : The tunnel was deleted but token still in use

   • Solution: Create new tunnel in Cloudflare Zero Trust, update token in .env

Sources : docker-compose.yml:11-17, README.md:23-54

Public Hostname Routing Issues

Symptoms :

• Tunnel shows "Active" in Cloudflare dashboard
• MQTT clients receive connection refused or timeout errors
• No traffic reaches mosquitto container

Verification Steps :

1. Check public hostname configuration in Cloudflare dashboard:

   • Navigate to: Zero Trust > Networks > Tunnels > [Your Tunnel] > Public Hostname
   • Verify service type is "HTTP" (see README.md:61)
   • Verify URL points to mosquitto:9001 (see README.md:62)

2. Test internal routing:

Common Configuration Errors :

Docker Network Resolution :

The hostname mosquitto in the URL resolves via Docker's internal DNS to the container_name specified in docker-compose.yml6 If the container name is changed, update the Cloudflare public hostname URL accordingly.

Sources : README.md:56-66, docker-compose.yml:4-9

MQTT Client Connection Issues

Connection Refused or Timeout

Client Configuration Requirements :

For successful connection through Cloudflare Tunnel, clients must:

1. Use the public hostname configured in Cloudflare Zero Trust (e.g., mqtt.example.com)
2. Use WebSocket protocol when connecting through the tunnel
3. Use standard port (80 for HTTP, 443 for HTTPS) - do not specify 9001
4. Not require TLS at the MQTT level (Cloudflare provides TLS termination)

Example Client Configurations :

Common Mistakes :

1. Specifying Port 9001 : Clients should not include :9001 in the hostname. Cloudflare listens on standard ports and routes internally.

2. Using Standard MQTT Instead of WebSocket : Port 9001 is configured with protocol websockets in mosquitto.conf5 Standard MQTT clients will fail.

3. Direct Connection Attempts : Clients attempting to connect directly to the Docker host IP will fail. All connections must go through Cloudflare.

Sources : mosquitto.conf:4-5, README.md:59-66

Protocol Mismatch Errors

Symptoms :

• Client connects but immediately disconnects
• Error messages about unexpected packet format
• "Bad request" or "HTTP 400" errors

Root Cause Analysis :

The mosquitto broker has two listeners with different protocols:

Solutions :

1. For External Clients (through Cloudflare) :

   • Must use WebSocket-capable MQTT libraries
   • Connection URL format: ws:// or wss:// (not mqtt://)
   • The Cloudflare tunnel routes to port 9001 which requires WebSocket protocol

2. For Internal/Development Testing :

   • Standard MQTT clients can connect directly to port 1883
   • Requires Docker port exposure (not configured by default)
   • Not recommended for production use

Sources : mosquitto.conf:1-5, README.md:62

Anonymous Access Confusion

Symptoms :

• Clients expecting authentication prompts receive none
• Security concerns about unauthenticated access
• Confusion about access control

Current Configuration :

The allow_anonymous true directive at mosquitto.conf2 permits connections without username/password authentication. This is intentional for the base configuration but may not be suitable for all deployments.

Security Implications :

When Anonymous Access is Appropriate :

• Trusted environment where all clients are controlled
• Network-level security (Cloudflare Tunnel) provides sufficient protection
• Simplicity is prioritized over fine-grained access control

When to Require Authentication :

• Multi-tenant environments
• Untrusted client devices
• Compliance requirements for access logging
• Topic-level access control needed

For authentication and ACL configuration, see the [protected-no-wildcard branch](https://github.com/jzombie/docker-mqtt-mosquitto-cloudflare-tunnel/blob/8a829fda/protected-no-wildcard branch) or consult Topic Access Control (ACL)).

Sources : mosquitto.conf:2, README.md:5-11

Mosquitto Configuration Issues

Configuration File Syntax Errors

The mosquitto.conf file uses a simple key value format. Common syntax errors include:

Valid Configuration Structure :

listener <port>
[listener-specific options]

listener <port>
[listener-specific options]

Reference the working configuration at mosquitto.conf:1-6

Sources : mosquitto.conf:1-6

Listener Configuration Problems

Issue: Ports Not Available

If mosquitto logs show Error: Address already in use, another service is using port 1883 or 9001.

Diagnostic :

Solutions :

1. Stop conflicting service
2. Change mosquitto listener ports in mosquitto.conf
3. If changing ports, update Cloudflare public hostname URL accordingly

Issue: WebSocket Listener Not Working

If WebSocket clients cannot connect but standard MQTT clients can:

1. Verify protocol websockets directive exists at mosquitto.conf5
2. Confirm it's associated with the correct listener (9001)
3. Check that Cloudflare public hostname routes to port 9001, not 1883

Sources : mosquitto.conf:1-6, README.md:62

Health Check and CI/CD Issues

GitHub Actions CI Failures

The CI pipeline at .github/workflows/ci.yml:1-42 tests mosquitto startup. If CI fails, it indicates a fundamental issue with the configuration.

CI Workflow Troubleshooting :

Common CI Failure Causes :

1. Invalid mosquitto.conf :

   • Syntax errors prevent broker startup
   • Solution: Test locally with docker compose up mosquitto

2. Missing mosquitto.conf :

   • File not committed to repository
   • Check .gitignore doesn't exclude it

3. Port Conflicts in CI Runner :

   • Unlikely but possible on shared runners
   • CI only starts mosquitto, not cloudflared, to avoid needing real tunnel token

Local Replication of CI Test :

Sources : .github/workflows/ci.yml:1-42

Health Check Loop Timeout

The CI health check uses a 10-iteration loop with 10-second sleep intervals (.github/workflows/ci.yml:29-39), allowing up to 100 seconds for mosquitto to become healthy.

If Health Check Times Out :

1. Check Image Pull Time : First run pulls eclipse-mosquitto:latest, which may take time

   • Not typically an issue as GitHub runners have good bandwidth

2. Check Container Logs :

Look for configuration errors or startup failures

3. Check System Resources : Insufficient memory/CPU could slow startup

   • Mosquitto is lightweight; this is rarely the issue

4. Verify Docker Engine : Docker daemon issues prevent container creation

Sources : .github/workflows/ci.yml:27-39

Debugging Techniques

Log Analysis

Viewing Real-Time Logs :

Key Log Indicators :

Sources : docker-compose.yml:4-17, mosquitto.conf:1-6

Container Inspection

Detailed Container State :

Common Inspection Checks :

1. Restart Count : High restart count indicates recurring failures

2. Environment Variables : Verify cloudflared receives token

3. Network Connectivity : Check both containers are on same network

Sources : docker-compose.yml:1-18

Testing MQTT Connectivity

Internal Testing (Within Docker Network) :

External Testing (Through Cloudflare Tunnel) :

For WebSocket testing, use a WebSocket-capable MQTT client:

Sources : mosquitto.conf:1-6, README.md:59-66

Network Troubleshooting

Docker Network Investigation :

Common Network Issues :

1. Containers on Different Networks : Should both be on default compose network
2. DNS Resolution Failure : Container name mosquitto doesn't resolve
3. Firewall Within Container : Unlikely with standard images

Sources : docker-compose.yml:1-18

Environment Variable Issues

.env File Problems

Verification Checklist :

Common .env Mistakes :

1. Spaces Around Equals : KEY = value (incorrect) vs KEY=value (correct)
2. Quotes : Not needed unless value contains spaces
3. Comments : Use # at start of line, not inline
4. File Name : Must be exactly .env, not env or .env.txt
5. Encoding : Must be UTF-8, not UTF-16 or other encodings
6. Line Endings : Use Unix line endings (LF), not Windows (CRLF)

Verification Command :

Sources : .env.sample, docker-compose.yml:14-17, README.md:51

Token Format Issues

The CLOUDFLARE_TUNNEL_TOKEN is a long base64-encoded string. Common issues:

1. Truncated Token : Token copied incompletely from dashboard

   • Tokens are typically 200+ characters
   • Verify entire token was copied

2. Corrupted Token : Extra characters or line breaks inserted

   • Token should be single line with no spaces
   • Common when copying from certain terminals

3. Wrong Token Type : Using API key instead of tunnel token

   • Token should start with eyJ (base64-encoded JSON)
   • Obtained from specific tunnel creation flow in README.md:47-51

Sources : README.md:47-51, docker-compose.yml:14-17

Additional Resources

For ongoing issues not covered in this troubleshooting guide:

• Mosquitto Documentation : https://mosquitto.org/documentation/
• Cloudflare Tunnel Documentation : https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/
• Docker Compose Documentation : https://docs.docker.com/compose/

For advanced configuration topics:

• Topic-based access control: See Topic Access Control (ACL))
• Production deployment: See Production Considerations
• Security hardening: See Security Model

Sources : README.md:15-21, README.md:5-11