ZeroMQ Pub/Sub with FlatBuffers Schemas
Typed, versioned messages between C++ and Python processes with zero-copy reads.
- Runs out of the box: yes
- Tests: yes
- CI: yes
- Docs: yes
- Used in production: no
Purpose#
A C++ publisher and a Python subscriber that share one FlatBuffers schema. Topics are prefixed with the schema version so a subscriber can ignore messages it cannot decode, and the build regenerates both language bindings from the same file.
The example also shows how to detect a slow subscriber with the high-water mark and how to record the stream to disk without copying message bodies.
What you will learn#
- Define a versioned FlatBuffers schema and generate C++ and Python bindings in CMake
- Wire PUB/SUB sockets with topic prefixes that encode the schema version
- Detect and report slow subscribers using the high-water mark
- Read fields without copying the message buffer
Media#
Technologies & features#
- Technologies
- C++PythonZeroMQFlatBuffersCMake
- Features
- Publish-Subscribe MessagingSchema-Driven Serialization
- Topics
- Messaging & IPCprimaryData & Persistence
Lessons learned#
- Worked
Encoding the schema version in the topic prefix let old subscribers keep running while a new publisher rolled out.
- Didn’t work
Relying on default socket buffers hid dropped messages. Setting a low high-water mark and logging the drop counter made the loss visible.
- Decision
PUB/SUB was chosen over request-reply because telemetry tolerates drops and publishers must never block on a subscriber.
Try it#
git clone https://git.example.internal/mission-software/zmq-fbs-pubsub.git
cd zmq-fbs-pubsub
cmake -S . -B build && cmake --build build -j2
./build/publisher &
python3 py/subscriber.pyCommands come from the owner’s README. Credentials are never stored here.
