meshIQ Platform Deployment Pack - Version 12.1

Follow below steps to get started with the meshIQ setup.

Before running this procedure, please verify the prerequisites below.

Prerequisites

Linux System:

• Operating System:  Linux 64 bit OS
• CPU:  Minimum 16 Cores virtual CPUs
• Memory:  Minimum 32GB RAM 
• File System:  Minimum 100 GB free space, SSD preferred
• Recommended Install folder: /opt
• Required: Dedicated Server Instance 

Some steps below require root access.

1. USERID: 

Create the Linux user ID (UID) and group which will be used to start all meshIQ services.

The Recommended user and group are meshiq:meshiq, this can be changed to any preferred user product owner.

The UID and group must own all files and directories under deployment path.

2. SWAP:

It is highly recommended to turn Swap off within the operating system on all virtual machine servers running meshIQ Track. 

To disable swap, perform the following:

a) Identify configured swap devices and files with the following command:

 cat /proc/swaps or swapon --show

or            

b) Turn off all swap devices and files with the following command:

 sudo swapoff -a

c) Comment out any swap matching references found in /etc/fstab.

3. LIMITS:

Modify /etc/security/limits.conf and add below for your user:

Replace meshiq with the user that will start Track processes.

          meshiq     soft    nofile  65536
          meshiq     hard    nofile  65536
          meshiq     soft    nproc   65536
          meshiq     hard    nproc   65536

4.    SOFTWARE REQUIREMENTS: 

Ensure 'curl' is installed

Ensure native 'PostgreSQL' is NOT running, Installation using its own PostgreSQL database implementation.
 

This meshIQ Deployment pack is pre-configured to use PostgreSQL database as default and requires to run meshIQ.

Ensure 'ansible' installed on the system that will run ansible commands: 

command: pip install ansible

Control Node (where Ansible is installed):

• SSH port (22) open between control node and managed nodes
• DNS resolution or /etc/hosts entries for hostnames in inventory
• sshpass -for password-based SSH auth
• paramiko - optional SSH fallback, alternative to sshpass if sshpass cannot be installed or used. 

pip install paramiko

Managed Nodes (targets). 

• Ensure 'python' version >= 3.10+ is installed
• Install path for meshIQ deployment must exists
• Install user must own and have rights to deploy to the install path.

5. NETWORK PORTS:

The following list contains all processes network ports that are involved in meshIQ deployment.

DOMAIN SERVER (ATPNAMES) ...................... 2323, 3000 (TCP)
    DOMAIN_SERVER REST API .................... 8018 (TCP)
    PERMITS REST API .......................... 8020 (TCP)

CEP SERVER (ATPNODE) .......................... 3005 (TCP)
    STREAMING ................................. 6580 (TCP)

WEB SERVER (apache-tomcat) .................... 8080 (TCP)

ZOOKEEPER ..................................... 2181 (TCP)

ACTIVEMQ SERVER ............................... 61616 (TCP)
    ACTIVEMQ WEB CONSOLE ...................... 8161 (TCP)

KAFKA SERVER .................................. 9092 (TCP)

POSTGRESQL (postgres) ......................... 5432 (TCP)

PINOT BROKER .................................. 8099 (TCP)
PINOT CONTROLLER .............................. 9000 (TCP)
PINOT SERVER .................................. 8098 (TCP)

WORKGROUP SERVER .............................. 4010 (TCP & UDP)
    REST API .................................. 8019 (TCP)

    MQ AGENT (nsqmq) .......................... 5010 (TCP & UDP)
    MQ AGENT JAVA (nsqcmmq.jar) ............... 5015 (TCP & UDP)

    MQ CONNECTION MANAGER (nsqcmmq.jar) ....... 5025 (TCP)
        IBM LISTENERS ......................... 1414 (TCP)

    IIB/ACE CONNECTION MANAGER (nsqcmace.jar) . 5577 (TCP)
        IIB/ACE NODE .......................... 4414 (TCP)

    EMS CONNECTION MANAGER (nsqcmems.jar) ..... 5556 (TCP)
        TIBCO EMS URL ......................... 7222 (TCP)

    KAFKA CONNECTION MANAGER (nsqcmkafka.jar) . 5566 (TCP)
        KAFKA SERVER .......................... 9092 (TCP)

    SOLACE CONNECTION MANAGER (nsqcmsolace.jar) 5588 (TCP)
        SOLACE URL PORT ....................... 8980 (TCP)

    RABBITMQ CONNECTION MANAGER (nsqcmrabbitmq.jar)
                                               5599 (TCP)
        RABBITMQ URL PORT ..................... 15672 (TCP)

    ACTIVEMQ CONNECTION MANAGER (nsqcmactivemq.jar)
                                               5592 (TCP)
        ACTIVEMQ URL PORT ..................... 8161 (TCP)

The machine on which the install will be done respects all prerequisites for installation described above.

For more information and to get started, please refer to the URL link:
https://customers.meshiq.com/hc/en-us/sections/41902118859795-Management

https://customers.meshiq.com/hc/en-us/sections/41902120341011-Tracking

Setup Steps:

FIREWALL: Make sure the below ports are open between servers for successful meshIQ deployment.
   2323, 3000, 3005, 2181, 61616, 9092, 9000, 4010, 8020, 8080, 5432, 8099, 8018, 6580
   See above under "5. NETWORK PORTS:" for port reference.

ANSIBLE DEPLOMENENT ONLY, (WHERE ALL SERVICES ARE RUN ON THE SAME OR MULTIPLE SERVERS.):

1.  Untar the "meshiq_deployment_xx.x_Vx.x.tar.gz" package to any path of your choice ...

SHELL COMMAND: 

tar -zxvf meshiq_deployment_xx.x_Vx.x.tar.gz

2. REVIEW, VERIFY AND SPECIFY SETTINGS FOR REMOTE SERVERS:

Change to "meshiq-ansible-deployment" folder and modify file "inventory.ini"

SHELL COMMAND: cd meshiq-ansible-deployment

SHELL COMMAND: vi inventory.ini

You will find below content is from inventeory.ini file:

deployment contains "meshiq" as a root directory no need to specify it in "deploy_root" variable.

 [all:vars]

deploy_root=/opt

ansible_user=meshiq     

EX: [domain:vars] is for domain service and [web:vars] is for tomcat service variables, and so on.

Remote install path must exists and remote install user must have full authority for that path.

 Next change "localhost" or IP with server location for appropriate services.

Same IP for the server will indicate that these services are running on that same server.

Configure Target Servers

[domain]
localhost

[cep_wgs]
localhost

[cep_track]
localhost

[cep_observe]
localhost

[web]
localhost

[base]
localhost

[pinot]
localhost

[gateway]
localhost

If you intend to run all services on the same server then put the same IP for all services above.

3. TO START THE DEPLOYMENT, AND FOR INITIAL SETUP:

 Run below command from "meshiq-ansible-deployment" directory and follow on screen instructions.

SHELL COMMAND: 

./ansible-playbook -i inventory.ini meshiq-playbook.yml

By default ansible uses transport "sshpass" to connect to remote servers, if you prefer to use "paramiko" as an alternative then use below command: both sshpass or paramiko need to be installed, refer to above

SHELL COMMAND: 

ANSIBLE_TRANSPORT=paramiko ansible-playbook -i inventory.ini meshiq-playbook.yml

Ansible deployment will configure and start appropriate services automatically based the the settings above.

LICENSE:

For this deployment we are using a default trial license.

Ensure to request a new 12.1 license file and apply it to all remote servers.

YOU HAVE COMPLETED THE MESHIQ ANSIBLE DEPLOMENENT.

To access the meshIQ UI from a web browser, use the following address: 

http://<serverip>:8080/manage
http://<serverip>:8080/track
http://<serverip>:8080/observe
http://<serverip>:8080/control-center

Use the following default log in credentials:

If "tomcat" and "Work Group Server(WGS)" running on separate servers, once connected in web "manage" UI you will need to change WGS IP from 127.0.0.1 to WGS server location IP.

STREAMING DATA TO TRACK:

In order to stream data to Track, a token id is needed.

DEFAULT TOKEN ID: DefaultToken

DEFAULT STREAM URL: http://<serverip>:6580

ANSIBLE USEFUL COMMANDS:

Selective service deployment: 
./ansible-playbook -i inventory.ini meshiq-playbook.yml --limit web
./ansible-playbook -i inventory.ini meshiq-playbook.yml --limit 'all:!domain:!cep_wgs:!cep_track:!cep_observe:!web:!track'