Some of the content described in this article is available in meshIQ Manage versions 11.1.1 and later.
What is Kafka Offset
In Kafka, an offset is a unique identifier for a position in a partition of a Kafka topic, representing the position of a consumer in the stream of messages within that partition. Kafka topics are split into multiple partitions, and each partition is an ordered, immutable sequence of messages. Every message in a partition is assigned a sequential offset, starting at 0, which helps Kafka track the position of messages and allows consumers to easily start reading from any point in the message stream. Since each partition in a Kafka topic has its own offset sequence, it ensures that messages are ordered correctly within each partition.
Navigating Kafka Messages
When browsing Kafka topics, customers can request the last 'n' messages or search for specific message content or keys. The offset allows for efficient browsing, ensuring you can view messages from any point in time. Since Kafka topics may have multiple partitions, each partition's messages are accessed using its own offsets.
Displaying Real Kafka Offsets
Prior to version 11.1.1, the system used a custom Navigator/Manage offset as an internal index. After the update, we started displaying the actual Kafka message offset, which is assigned by Kafka starting from offset 0 by default.
When you click on Kafka messages, the Partition ID, Offset, and Key are displayed.