meshIQ Kafka Console™ is a tool for monitoring and managing Kafka clusters. It consists of Apache ZooKeeper and Apache Kafka.
Cluster list
When you select Kafka from the welcome page, the Cluster page shows the list of registered clusters in a card view and provides basic cluster information and status. If the cluster status is abnormal, the cluster name is displayed in red.
- Security protocol
- Is ACL enabled?
- Authentication mechanism
- Number of brokers
- Number of partitions
- Number of topics
- Messages generated per second
- Message consumption per second
- Zookeeper connection status
Select a cluster
To monitor detailed metrics of a cluster or perform tasks such as topic management, select Manage at the bottom of the card of that cluster.
Change the target cluster
You can quickly change from managing one cluster to managing another. You can return to the welcome screen by clicking the meshIQ Kafka Console icon at the top left of the screen, or switch to another cluster by clicking the cluster shortcut displayed immediately below.
Register a new cluster
You can register a new Kafka cluster you want to manage by pressing Register on the cluster list page, or from the welcome page by selecting Register a new Cluster. Clusters that can be registered are as follows.
- meshIQ Kafka Console distribution Kafka cluster. All settings are ready, including metric collection configuration.
- Apache Kafka distribution. JMX Exporter, Node Exporter must be configured to collect metrics. Otherwise, some notification features will not be available.
- Amazon MSK + Prometheus Open Monitoring. MSK clusters configured with open monitoring do not support all features.
Click Next.
Register bootstrap server information
When using a Kafka client, the bootstrap server address of the Kafka cluster is always required. meshIQ Kafka Console is also a Kafka client, so this information is essential. When registering a Kafka cluster with authentication/authorization settings, security information and superuser privileges are required. Enter one broker address and port for the Kafka cluster, enter security information as needed, and then proceed to the review step.
Set up security protocols
The currently supported security protocols are as follows:
- PLAINTEXT
- SASL_PLAINTEXT
Support for SSL and SASL_SSL protocols is planned.
Set up the authentication mechanism
The following are the supported authentication mechanisms:
- PLAIN
- SCRAM-SHA-256/512
- OAUTHBEARER
Support for the Kerberos mechanism is planned.
Validate the cluster
Select Validate Cluster.
Review cluster information
After entering the bootstrap server information and verifying the cluster, most of the information will be retrieved automatically. The cluster information review screen consists of three main parts:
- Cluster Summary
- Kafka broker information
- ZooKeeper Ensemble Information
Cluster summary
Select the cluster name and cluster type to be used for registration and management in meshIQ Kafka Console.
Setting up Kafka broker information and metrics collection
A list is displayed of all brokers that make up the Kafka cluster. If a broker is stopped, it may not appear in the list. If metrics collection is set up, you can activate the function and modify the port number.
ZooKeeper ensemble information and metrics collection settings
A list is displayed of ZooKeeper servers that are connected to the Kafka cluster. If metrics collection is set up, you can activate the function and modify the port number. If your account is configured to use authentication, you will need to enter authentication information.