See the below sections for information on how to import or export remote Kafka clusters (nsqcmkafka.jar) in bulk using the nsqmigr utility.
Export (get) remote Kafka clusters
Perform the following to bulk export remote Kafka clusters:
- Run a command similar to the following to create an information (.info) containing a list of all remote Kafka clusters. Please note that the -K parameter must be uppercase.
nsqmigr -g -K -f<filename>.infoFor a full list of commands, enter the command: nsqmigr -?
The following command will generate the file, remote_kafka.info, and will write all remote Kafka clusters from the workgroup server database to it. The file will be created in C:\nastel\apwmq\config\groups\remote_kafka.info.
C:\nastel\apwmq\bin>nsqmigr -g -K -fC:\nastel\apwmq\config\groups\remote_kafka.info
- The file is now created. The default export directory is [APWMQ_HOME]/config/groups, but can be overwritten by using the -O parameter.
All remote Kafka clusters and their associated fields will be written to the information file created. The fields are as follows:
The following is an example of an information (.info) file.
Import (put) remote Kafka clusters
Perform the following to bulk import remote Kafka clusters:
- You will need to create an information (.info) file to define each remote Kafka cluster being imported. To create this file, either:
- manually create this file to define each remote Kafka cluster being imported. Add the fields listed in the Exported fields section above for each remote Kafka cluster. See Example file above for a sample; or
- follow the steps in the Export (get) remote Kafka clusters section above to export the remote Kafka clusters. Update the file accordingly.
- Place the file in:
- Run a command similar to the following to import the file. Please note that the -K parameter must be uppercase.
nsqmigr -p -K -f<filename>.infoExample:
The following command will import the remote Kafka clusters from the remote_kafka.info file located in C:\nastel\apwmq\config\groups.
C:\nastel\apwmq\bin>nsqmigr -p -K -fC:\nastel\apwmq\config\groups\remote_kafka.info
- The remote Kafka clusters are now imported in to the WGS database. The info file is automatically backed up in [APWMQ_HOME]/config/groups/workgroup/history and is not deleted.