Skip to main content
Top
Review meshIQ Products (G2.com)

How do I manually update the Kafka component?

To upgrade any Kafka component manually from one version to another version follow the below instructions.

  1. Download the latest version of Kafka component Kafka2.13- xxx.tar.gz from https://kafka.apache.org/downloads and place the file in $APIN_HOME/kafka.
  2. Stop the Kafka service by using the command ./stop.sh kafkasrv in $APIN_HOME/sbin.
  3. Navigate back to Kafka directory present in $APIN_HOME/kafka.

  4. Untar the downloaded Kafka package using the command.
     

    tar -zxvf kafka2.13-xxx.tar.gz
  5. Kafka2.13-xxx folder is obtained which will be the current version.

  6. Unlink the old version of Kafka by using the command.
     

    Unlink Current

  7. Link the current version of kafka by using the command. 

    ln -s kafka2.13-xxx current

  8. Open the untared folder and edit the server.properties file present in the $APIN_HOME/kafka/kafka2.13-xxx/config/
     

    cd/$APIN_HOME/kafka/kafka2.13-xxx/config/
view server.properties

  9. Add the following properties under Server Basics:

    message.max.bytes=10485760
replica.fetch.max.bytes=10485760

  10. Under zookeeper add the following line

    zookeeper.connect=localhost:2181/xraykafka

  11. Change the following properties: 

      • Under Socket Server Settings:

        Change num.network.threads from 3 to 8

        num.network.threads=8
        Change num.io.threads from 8 to 16
        num.io.threads=16
      • By default, log.dirs is set to the folder shown below. But you can change it if you do not want to use the default folder.
        log.dirs=/tmp/kafka-logs
      • Under Log Basics:
        Change num.partitions from 1 to 16
        num.partitions=16
      • Under Log Flush Policy, uncomment the following line:
        log.segment.bytes=107374182
  12. Save the server.properties file after the above changes.
  13. After all steps have been completed, restart the Kafka service from /opt/meshiq/sbin by using the command ./start.sh kafkasrv to apply the changes.
  14. Recreate Kafka topics using the command ./config.sh kaf in $APIN_HOME/sbin.