How do I manually update the Kafka component?

The following instructions apply to XRay versions 1.5 and 1.6 and to the meshIQ Platform versions 11 and later. The meshIQ Platform version 11.1 uses Kafka 2.13-3.5.1.

Customers who would like to manually update the Apache Kafka component may follow the instructions below.

1. Stop the Kafka service by using the command ./stop.sh kafkasrv in $APIN_HOME/sbin.
2. To update the Kafka component, download the latest version of Kafka 3.x from https://kafka.apache.org/downloads and place the file in $APIN_HOME/kafka.
3. Unlink the current version of Kafka by using the command unlink current.
4. To update the files, please follow the instructions below for Upgrading Kafka to 3.4.0.
5. Using the command ln -s kafka 3.x current, link the current Kafka version.
6. After all steps have been completed, restart the Kafka service by using the command ./start.sh kafkasrv to apply the changes.
7. Recreate Kafka topics using the command ./config.sh kaf in $APIN_HOME/sbin.

Upgrading Kafka to 3.4.0

To configure Kafka 3.4.0, edit server.properties in config subfolder within the Kafka distribution:

•  Add the following properties under Server Basics:

message.max.bytes=10485760
replica.fetch.max.bytes=10485760

• Change the following properties:
• under Socket Server Settings:

  num.network.threads from 3 to 8
  num.io.threads from 8 to 16

• By default, log.dirs is set to the folder shown below. But you can change it if you do not want to use the default folder.

log.dirs=/tmp/kafka-logs

• under Log Basics:
  num.partitions from 1 to 16
• under Log Flush Policy, uncomment the following line:

log.segment.bytes=1073741824

• zookeeper.connect from default value to localhost:2181/xraykafka