Encrypted Retained Messages

Relevant source files

• README.md
• mosquitto.conf

Purpose and Scope

This document describes the encrypted retained messages feature available in the protected-no-wildcard branch of the repository. This feature provides transparent encryption of MQTT retained messages using gocryptfs, combined with automatic persistence after each message. This documentation covers the architecture, encryption mechanism, and auto-save functionality for retained messages.

For information about topic-based access control also available in the protected-no-wildcard branch, see Topic Access Control (ACL)). For the base system configuration without these features, see Mosquitto Configuration.

Sources: README.md:5-11

Overview

The protected-no-wildcard branch extends the base system with two key enhancements for retained messages:

1. Encryption of retained messages using gocryptfs
2. Automatic persistence after every message

This feature set is designed for scenarios requiring at-rest encryption of MQTT retained messages while maintaining the standard MQTT protocol interface for clients.

Sources: README.md:5-11

MQTT Retained Messages

What Are Retained Messages

Retained messages in MQTT are messages that the broker stores and delivers to new subscribers immediately upon subscription, even if the original publisher is no longer connected. When a client publishes a message with the retain flag set to true, the broker:

1. Stores the message associated with the topic
2. Delivers the message to any future subscribers to that topic
3. Replaces any previously retained message on the same topic

Storage Requirements

By default, Mosquitto stores retained messages in memory and optionally persists them to disk in the data/ directory. The encrypted retained messages feature adds a transparent encryption layer to this persistence mechanism.

Sources: mosquitto.conf:1-6

Encryption Architecture

gocryptfs Overview

The protected-no-wildcard branch uses gocryptfs, a FUSE filesystem that provides transparent encryption. gocryptfs operates at the filesystem layer, meaning:

• Files written to the mounted directory are automatically encrypted
• Files read from the mounted directory are automatically decrypted
• The encryption is transparent to applications (Mosquitto in this case)

Architecture Diagram

Sources: README.md8

Encryption Flow

Sources: README.md8

Auto-Save Mechanism

Persistence Behavior

The protected-no-wildcard branch implements automatic persistence of retained messages after every message. This differs from standard Mosquitto behavior, which may batch persistence operations or persist on a schedule.

Benefits

Auto-Save Architecture

Sources: README.md9

Integration with Docker Architecture

Volume Mounting

The encrypted retained messages feature requires a gocryptfs mount to be established before Mosquitto starts. This involves:

1. Creating an encrypted filesystem on the host
2. Mounting the encrypted filesystem using gocryptfs within the container
3. Configuring Mosquitto to use the mounted directory for persistence
4. Ensuring the encryption key is securely managed

graph TB
    subgraph "Container Startup Sequence"
        Start["Container Start"]
InitGocryptfs["Initialize gocryptfs"]
MountEncrypted["Mount Encrypted FS"]
StartMosquitto["Start mosquitto"]
Ready["Ready for Connections"]
end
    
    subgraph "Container Shutdown Sequence"
        Shutdown["Shutdown Signal"]
StopMosquitto["Stop mosquitto"]
FlushData["Flush All Data"]
UnmountFS["Unmount gocryptfs"]
Stop["Container Stop"]
end
    
 
   Start --> InitGocryptfs
 
   InitGocryptfs --> MountEncrypted
 
   MountEncrypted --> StartMosquitto
 
   StartMosquitto --> Ready
    
 
   Shutdown --> StopMosquitto
 
   StopMosquitto --> FlushData
 
   FlushData --> UnmountFS
 
   UnmountFS --> Stop

Container Lifecycle

Sources: README.md:5-11

Security Considerations

Encryption Key Management

The gocryptfs encryption requires a master key to encrypt and decrypt the filesystem. Key management considerations include:

• Key Storage: The encryption key must be securely stored and accessible to the container
• Key Rotation: Procedures for rotating encryption keys if compromised
• Access Control: Limiting which processes can access the encryption key

Threat Model

Performance Implications

Transparent encryption adds computational overhead:

• CPU Usage: Encryption/decryption operations consume CPU cycles
• I/O Latency: Additional layer between application and disk
• Auto-Save Impact: Per-message persistence increases disk I/O frequency

Sources: README.md8

Accessing the Implementation

Branch Location

The encrypted retained messages feature is implemented in the protected-no-wildcard branch of the repository. To access:

Comparing with Main Branch

To view the complete set of changes required to implement this feature:

https://github.com/jzombie/docker-mqtt-mosquitto-cloudflare-tunnel/compare/main...protected-no-wildcard

This diff shows:

• gocryptfs integration modifications
• Mosquitto configuration changes for encrypted persistence
• Auto-save implementation details
• Container orchestration updates

Sources: README.md11

Use Cases

Compliance Requirements

Encrypted retained messages are beneficial for:

• Regulatory Compliance: Meeting data-at-rest encryption requirements (GDPR, HIPAA, etc.)
• Multi-Tenant Environments: Protecting tenant data from infrastructure administrators
• Sensitive Data: IoT applications handling personally identifiable information (PII)

Deployment Scenarios

Sources: README.md:5-11

Relationship to Base System

Differences from Main Branch

The base system (main branch) provides:

• Unencrypted retained message storage
• Standard Mosquitto persistence behavior
• Simpler deployment without encryption overhead

The protected-no-wildcard branch adds:

• Transparent encryption layer via gocryptfs
• Guaranteed per-message persistence
• Additional ACL features (see Topic Access Control (ACL)))

Migration Path

To migrate from the main branch to the encrypted variant:

1. Back up existing retained messages from the data/ directory
2. Initialize a gocryptfs encrypted filesystem
3. Deploy the protected-no-wildcard branch configuration
4. Manually republish retained messages (encryption is not retroactive)

Sources: README.md:5-11

Summary

The encrypted retained messages feature provides at-rest encryption for MQTT retained messages through gocryptfs integration and automatic per-message persistence. This feature is available in the protected-no-wildcard branch and is designed for deployments requiring data protection compliance or handling sensitive information. The implementation maintains the standard MQTT protocol interface while adding a transparent encryption layer at the filesystem level.

For access control features that complement this encryption, see Topic Access Control (ACL)). For general deployment procedures, see Deployment.

Sources: README.md:5-11