Some of the content described in this article is available in meshIQ Manage versions 12.1 and greater.
For a quick overview of what's new or changed, visit the meshIQ Highlights page for a version-by-version breakdown.
Kafka Topic Widget Field Descriptions
| Field Name | Description |
|---|---|
| Topic Name | Name of the Kafka topic. |
| Total Partitions | Number of partitions configured for the topic. You can only increase this number after it is set. Increases can’t be rolled back, because Kafka doesn’t allow the partition count to be reduced. |
| Preferred Leader Replicas | Number of replicas that are responsible for handling all read and write requests for the partition. |
| Total Messages | Total number of messages written to the topic since it was created, summed across all partitions (the sum of each partition’s log-end offset). Messages already removed by retention or compaction are still counted. |
| Available Messages | Number of messages currently held in the topic and available to consume, summed across all partitions (log-end offset minus log-start offset). This is lower than Total Messages when retention or compaction has removed older messages. |
| Consumer Groups | Number of consumer groups for the topic. A consumer group is a set of consumers that read the topic’s data in parallel. This lets data be consumed faster and increases the volume that can be processed. |
| Manager Name | Name of the Kafka manager (the cluster connection defined in meshIQ Manage) that the topic belongs to and that collects its data. |
| Last Data Update Timestamp | Time elapsed since the topic’s data was last updated from the cluster, shown as hh:mm:ss. Use it to check how current the values in the row are. |
| Bytes In / Minute | Rate at which data is written to the topic by producers, in bytes per minute, summed across all brokers. Based on the Kafka broker metric BytesInPerSec. |
| Bytes Out / Minute | Rate at which data is read from the topic by consumers, in bytes per minute, summed across all brokers. Based on BytesOutPerSec. Replication traffic between brokers isn’t included. |
| Messages In / Minute | Number of messages produced to the topic per minute, summed across all brokers. Based on MessagesInPerSec. |
| Total Fetch Requests / Minute | Number of fetch requests per minute made against the topic by consumers and follower replicas. Based on TotalFetchRequestsPerSec. |
| Total Produce Requests / Minute | Number of produce requests per minute received for the topic. Based on TotalProduceRequestsPerSec. |
| Failed Produce Requests / Minute | Number of produce requests per minute for the topic that failed, for example because of NotEnoughReplicas, an oversized message, or an authorization error. A value above zero means producers are losing writes and should be investigated. Based on FailedProduceRequestsPerSec. |
Field descriptions display numeric values rounded to a maximum of two decimal places.
Accessing the Topic Properties
- Select the required topic.
- From the Selected menu, click Properties. The Kafka Topic Properties dialog opens.
- The dialog contains the following tabs:
- General
- Configuration
- Custom Attributes
| General Tab | |
|---|---|
| Field Name | Description |
| Topic Name | Name of the Kafka topic. |
| Is Internal | Internal topics are used internally by Kafka Streams. |
| Partition Count | After you set the number of partitions, you can only choose to increase this number. You will not be able to roll back any increases later, since decreases in the number of partitions are not permitted. |
| Replication Factor | Total number of replicas of the data that will exist once the data is replicated on additional brokers. |
| Preferred Leader Replicas | Number of replica that is responsible for handling all read and write requests for the partition |
| Replicas not in sync | Indicates the number of replicas that are not in sync (are missing commits) |
| Consumer Groups | Number of consumer groups for the topic. A consumer group is a group of consumers that reads data at the same time. By allowing data to be consumed at a faster rate, it increases the volume of data that can be consumed. |
| Compression Type |
"Side" on which data is compressed: producer or broker Kafka configuration parameter compression.type. |
| Minimum In-sync Replicas |
Minimum number of brokers that you want to maintain as in-sync, that is, having the latest messages for a given partition. Kafka configuration parameter min.insync.replicas. |
| Cleanup Policy |
delete (default) or compact. The “delete” policy discards old log segments based on their retention time or size limit. The “compact” policy retains the latest value for each key but compacts the log. Kafka configuration parameter cleanup.policy. |
| Bytes per Segment |
Size of each segment of a partition. Kafka configuration parameter segment.bytes. |
| Retention (ms) |
Length of time in millisecond (ms) that messages within the topic are retained. Kafka configuration parameter retention.ms. |
| Max Message Size (bytes) |
Maximum size of a message that Kafka is allowed to send. Kafka configuration parameter max.message.bytes. |
| Unclean Leader Election |
true or false. If this property is set to true, and if all leaders die, Kafka can choose a replica that is missing commits (a non-In-Sync Replica) to become the new leader. Kafka configuration parameter unclean.leader.election.enable. |
| Retention (bytes) |
When the delete Cleanup Policy is in effect, this controls the maximum size a partition can reach before old log segments are discarded. Kafka configuration parameter retention.bytes. |