Creating a connection to a Kafka cluster running in Confluent Cloud

Confluent Cloud is a popular environment for running Kafka. In order to access it from meshIQ Cloud is very simple. You will need access to the Confluent Cloud web interface,

1. In the Confluent Cloud web interface, open your cluster and navigate to Overview.
2. On the left hand side, click on Clients, select Consumers tab, then click on New client
3. Select Java.  
4. Create Cluster API Key if you have not previously created one.
5. Save the connection info into a file by copying/pasting it (be sure API key is included).
6. In another browser window in meshIQ Manage, create a new remote Kafka definition. If you need assistance to do this, see Adding your first instance to meshIQ Cloud. You can find instructions for creating a remote definition either manually (as described below) or through an import process in the Create Remote Kafka Manager Connection article using the file created above 
   1. Set Node name to REMOTE_KAFKA (or the name you are using).
   2. Set Cluster Name to a meaningful name for this cluster.
   3. Complete or verify the bootstrap server from the Configuration snippet.
7. Select the Config tab
   1. Create or verify each property line for each property from the snippet.   
8. (Optional) if you want to connect to the Schema Registry, Select the Schema Registry Tab
   1. Locate the Schema Information on Cluster tab in the Confluent Cloud web interface
   2. Set a meaningful name for this registry 
   3. Copy the Schema URL 
   4. Set the Username and credentials from the schema
9. Click OK to save the settings 

You can now view the various Kafka objects see Kafka viewlets

This article is applicable to both meshIQ Cloud and Enterprise environments.