How to configure XRay with Solr, Kafka, and ZooKeeper clusters – meshIQ

This article applies to XRay versions 1.5 and later.

Beginning with version 1.6, Java 11 is the minimum version required to run XRay.

Create a ZooKeeper cluster

To create a ZooKeeper cluster:

Download the latest version of ZooKeeper and install it in all the VMs.
Create the folder /opt/nastel.
Under the nastel directory, create folders for java, logs, sbin, and zookeeper.
Update Java to the latest version (JDK 11.0.19).
Install ZooKeeper under the zookeeper folder. Link to the current version using the following command:
```
tar -zxvf zookeeper.XXX.tar.gz
```
```
ln -s zookeeper.XXX current
```
Then under /opt/nastel/zookeeper/current/conf, copy over the sample config file:
```
cp zoo.cfg
```
Open the configuration file using the command:
```
vi zoo.cfg
```

Add the following:

# The number of milliseconds of each tick
tickTime=2000

# The number of ticks that the initial
# synchronization phase can take
initLimit=10

# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5

# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/var/lib/zookeeper

# the port at which the clients will connect
clientPort=2181

server.1=172.16.31.28:2888:3888
server.2=172.16.31.29:2888:3888
server.3=172.16.31.62:2888:3888

# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60

#

# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.

#

# https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance

#

# The number of snapshots to retain in dataDir
autopurge.snapRetainCount=7

# Purge task interval in hours
# Set to "0" to disable auto purge feature
autopurge.purgeInterval=7


## Metrics Providers

#

# https://prometheus.io Metrics Exporter

#metricsProvider.className=org.apache.zookeeper.metrics.prometheus.PrometheusMetricsProvider

#metricsProvider.httpHost=0.0.0.0

#metricsProvider.httpPort=7000

#metricsProvider.exportJvmInfo=true


admin.serverPort=8181

admin.enableServer=false


4lw.commands.whitelist=mntr,conf,ruok

Create a zookeeper directory in the lib folder in each of the servers using the command:

    sudo touch /var/lib/zookeeper/myid

View the myid file:
```
vi myid 
```
Enter the appropriate value (1, 2, or 3) in the myid files of each of the VMs.
Start the services.

Solr cluster

To create a Solr cluster, do the following:

Downloaded the separate solr package (containing java, logs, solr, and misc folders) from the meshIQ ftp site.
Create the folder /opt/nastel.
Place the downloaded tar file under the nastel folder.
Untar the file using the following command:
```
tar -zxvf solr.xxx.tar.gz
```
Go to /opt/nastel/sbin using the command:
```
vi apin_env.sh
```
Change the ZooKeeper host and provide the zkroot, following the example below:
```
export ZK_HOST=172.16.31.28:2181/solrcluster2
```
To connect the external ZooKeeper to the Solr instance, uncomment the following line in the /opt/nastel/solr/current/bin/solr.in.sh file:
```
ZK_HOST="173.16.31.28:2181,173.16.31.29:2181,173.16.31.62:2181/solrcluster2"
```
Go to opt/nastel/sbin and run the following command:
```
./config.sh solr
```
The services will be started automatically.

Kafka Cluster

To configure Kafka in the cluster, do the following:

Download the latest Kafka version.
In all the servers, create the /opt/nastel folder.
Within the nastel folder, create java, logs, sbin, and kafka folders.
Install the Autopilot m6 folder in one of the Kafka servers.
Go to /opt/nastel/kafka/current/config.
```
vi server.properties
```
Add the ZK host in the server.properties file.
Export the Java path:

 export JAVA_HOME=/opt/nastel/java/current
 export JRE_HOME=/opt/nastel/java/current
 export PATH=$JAVA_HOME/bin:$PATH

Go to /opt/nastel/sbin and create Kafka topics using the command:
```
./config.sh kaf
```
In order to check whether the topics were created in the other servers also, run the following command:
```
./kafka-topics.sh  --bootstrap-server 172.16.31.31:9092 --list
```