The meshIQ platform supports an integration using a data source for Grafana. That data source will issue queries for various data sources within the meshIQ platform. This includes analytic and tracking data, expert metrics, and administrative data.

In support of this initiative, a REST endpoint is now available as an M6 expert, which will provide the data. The query language used, jKQL, is the same as can be used within the XRay interface when used with or without Grafana. 

Installing the meshIQ data source plugin for Grafana

The instructions below describe four options for installing the meshIQ data source plugin: in a container or using a package. After install, you will be ready for setup.

The default port for Grafana, port 3000, is also the port used by the domain server. So if you install Grafana locally on the same machine as the domain server, you need to change the port for either Grafana or the Domain server. The port for the Domain server can be changed in [AP_HOME]/naming/node.properties:
property server.user.url.port = 3000

Option 1: Grafana installation (RedHat Linux)

This example shows installation using the RPM package for RedHat Linux.

1. Download Grafana using the following link:
   https://grafana.com/docs/grafana/latest/setup-grafana/installation/ 
2. Edit the /etc/grafana/grafana.ini file and uncomment the required ports. The following example shows uncommented lines.
   
3. Start Grafana using the systemctl command:

sudo systemctl daemon-reload
sudo systemctl start grafana-server
sudo systemctl status grafana-server
sudo systemctl enable grafana-server.service

4. Check the status of Grafana. If the server is running, it will show as active.
5. Access the Grafana user interface using the URL: http://localhost:3000.

 

Option 2: Installing Grafana on Linux using the Enterprise Edition (tar.gz file)

1. Download Grafana using the following link:
   https://grafana.com/grafana/download 
2. Edit the ini file as follows:

   [plugins]
   allow_loading_unsigned_plugins = meshiq-datasource
   # Redirect to correct domain if host header does not match domain 
   # Prevents DNS rebinding attacks

   For example:
   
3. Start Grafana from the bin directory. Navigate to the bin directory and run the following:

[nastel@xray11-m1 etc]$ cd /opt/nastel/grafana-10.2.0/
[nastel@xray11-m1 bin]$ ./grafana-server --config /etc/grafana/grafana.ini

Example:

 

Option 3. Running the meshIQ data source plugin in a container

You can run Grafana with the meshIQ data source plugin in a Docker container. Follow these steps:

1. Pull the Grafana image:

docker pull meshiq/meshiq-grafana

2. Run the Grafana container:

docker run --name meshiq-grafana -d -p 3000:3000 meshiq/meshiq-grafana

 

Option 4. Using a packaged meshIQ data source plugin

1. Install Grafana (if you haven’t already). Follow the instructions provided in the official Grafana documentation for your specific operating system.
2. Obtain the packaged meshIQ datasource plugin for Grafana by logging in to the meshIQ software repository and going to the AP6/Deployment_Pack/meshiq_platform/DSX directory.
3. Unpack and place the plugin in the Grafana plugins directory (e.g., /var/lib/grafana/plugins).
4. Edit the grafana.ini configuration file (usually located at /etc/grafana/grafana.ini).
5. Add plugin ID to the allow_loading_unsigned_plugins configuration option:

allow_loading_unsigned_plugins=meshiq-datasource

6. Save the configuration file.
7. Restart Grafana using the appropriate method:
   • If you are using Grafana from meshIQ, navigate to /opt/meshiq/grafana-xx/bin and run the following:

./grafana-server --config etc/grafana/grafana.ini

1. • If Grafana was locally installed using yum install (or apt-get), run the following:

sudo systemctl start grafana-server

8. Log into the Grafana UI.
9. To set the admin password in Grafana, run the following command from the bin directory:

./grafana-cli admin reset-admin-password <new password>

Remember to adjust the paths and configurations according to your specific setup.

Now you’re ready to use the meshIQ data source plugin in Grafana!

After completing the steps for one of the four options above, you can proceed to set up the meshIQ-Grafana integration using the instructions below.

Set up the meshIQ-Grafana integration

Jump ahead Step 1: Create a meshIQ data source Step 2: Create a dashboard Tips How to create a dashboard Add a visualization Examples Save the dashboard Step 3: Set up alerts Step 4: Create contact points Step 5: Add a notification policy You can pause notifications temporarily or at scheduled times using Silences and Mute timings.

 

Step 1: Create a meshIQ data source

In Grafana, you set up data sources for each application you want to connect to. The data source setup allows you to obtain data from that application. 

To set up a data source for meshIQ, do the following:

1. Open the menu  on the left side of the Grafana window and click the Connections button . 
2. The Add new connection page opens. The potential data sources you can create are listed.
   
3. Search for meshIQ and select it.
4. Enter a Name for the Data source.
5. Turn on the Default toggle to make this the default Data source.
   
6. Fill in the required fields:

• • Service URL: Data for the meshIQ-Grafana integration is provided by a REST endpoint that is available as an expert. The Service URL is the address where the expert is running.
    DSX or meshIQ Platform Thin Edition:
    http://cep-dsx:8580
    meshIQ Platform Simple, Typical or Big Edition:
    http://ds-api:8080
    RedHat Linux (RPM package) or Enterprise Edition (tar.gz) for Track:
    http://<ip>/ds-api/jkql 
  • Access Token:
    DSX (Grafana integration): Enter
    1298eae0-b27c-4175-9081-aa665c49e653
    meshIQ Platform (Thin, Simple, Typical, or Big): Enter
    grafana-default
    RedHat Linux (RPM package):
    grafana-default

7. Click Save & test to confirm that the expert service is working.
8. (Optional) The Autocomplete service is recommended to help you complete jKQL queries by offering suggestions as you type. To turn on Autocomplete, click the Enable Auto-complete toggle. Add the Autocomplete Service URL:
   DSX or meshIQ Platform Thin Edition:
   http://cep-dsx:7580
   meshIQ Platform Simple, Typical or Big Edition:
   http://cep-service:7580
   
9. Click Save & test to confirm that the Autocomplete service is working. A confirmation message is displayed:
   
10. Once the connection is successfully tested, you can expand Advanced Settings and  view the Max data points limit. The Max data points limit is the number of rows that the server can return on the back end. This limit comes from the CEP server configuration. The default value is 10000. Changes to it would need to be made in the DSX configuration.
    

Step 2: Create a dashboard

Jump ahead Tips How to create a dashboard Add a visualization Examples Data is missing a number field  Autocomplete "contains"  match Fields View Properties with FIELDS ALL  Adjust the parameters  Save the dashboard

Tips

• When writing a query, use the up and down arrow keys to select a field. Press Tab on your keyboard to make a selection.
• If you haven't started typing yet, or if you have closed Autocomplete, you can turn autosuggestions back on. Either starting typing again, or press Ctrl + spacebar.
• To run query, press Ctrl+ Enter. (Do not press only Enter.)
• Turn on Table view at any time to view your results as a table .

 

How to create a dashboard

Now that the Data Source has been created, you can create a new dashboard. 

1. Open the menu on the left side of the Grafana window.
   
2. Click the Dashboards option. The Dashboards page opens.
   
3. Select New > New Dashboard.
   

The Start your new dashboard… page opens. You are prompted to add a visualization. 

Add a visualization

Add a new visualization:

1. Click + Add visualization. The Select data source page opens.
2. Select the meshIQ data source. In the lower left panel, the meshIQ data source is selected.
   
3. Note the Query options that are in effect for the query. Among these Query options is Max data points, a Grafana setting. The Max data points value represents Grafana's determination of how much data it can display, based on the "width of the panel." The Max data points limit under Advanced Settings is prefilled based on the configuration settings for the data source. When you run a jKQL query, the number of data points returned is the lesser of the two values (Max data points limit or Max data points).
   
4. Begin typing a query (such as a GET query). Autocomplete offers suggestions. See the following sections:

• • • Autocomplete "contains"  match
    • Fields
    • View Properties with Fields ALL 

4. Use the examples below to create a query.
5. Run the query by pressing Control + Enter on your keyboard.

Examples

Data is missing a number field 

get itemtypes 

If your data is missing a number field, you are prompted to view it as a table or open visualization suggestions.

Click Switch to table. (You can also turn on Table view at any time to view your results as a table .)

The data is displayed as a table: 

Autocomplete "contains"  match

get ems

As you type a query, Autocomplete will offer suggestions. These suggestions include all partial matches, including "contains" matches.

Fields

get WgsEmsQueue FIELDS 

Add 'FIELDS' to your query and Autocomplete provides a list of fields in the table you specify. 

Use the up and down arrow keyboard keys to select a field. To add the selected field to your query, press Tab on your keyboard.

View Properties with FIELDS ALL 

By default, properties are not included. If you want to see all the properties, run a query with "FIELDS ALL".

Get WgsEmsQueue FIELDS ALL

Adjust the parameters 

You can make changes to the Advanced Settings to adjust results after they are returned.

In the example below, the Max Data Points Limit is 5.

You can change the Max Data Points Limit to 2, return to the jKQL prompt, and press Enter on your keyboard. The table results are updated.

Save the dashboard

1. Click Save.
    
2. The Save dashboard area opens. Enter a Dashboard name and select a Folder.
   
3. Click Save. After the dashboard is saved, its panel is displayed:
   

To resize the panel, place your mouse in the lower right corner until it changes to an nw-resize cursor: . Click and drag up and left to make the panel smaller, or down and right to make it larger.

Use the vertical ellipses at the top right of the panel to open a menu with panel options such as edit, share, explore, inspect, or remove.

Step 3: Set up alerts

See Alert Rules.

Step 4: Create contact points

Contact points consist of an integration method for sending notifications and a list of notification recipients. Integration options include Discord, Email, Slack, and others. In this example we will use email.

For Email integrations, Grafana comes with a default contact point called grafana-default-email. When this email address is set up, it is used as the default when no other contact point has been specified.

1. To create an Email contact point:
   Select the Grafana menu , expand the Alerting section and select Contact points.
   
2. Enter a Name and select Email from the Integration list. To add multiple email addresses, separate them with semicolons.
   
3. To send one email to all recipients instead of a separate email to each one, select Single email. In this case, all recipients will be able to see the other recipients.
   
   For more information about template variables, refer to the Grafana documentation at https://grafana.com/docs/grafana/latest/dashboards/variables/#templates.
4. By default, contact points receive a resolved (OK) message for a resolved alert that they have previously received notifications for. If you do not want to send the resolved message, select Disable resolved message.
   

Step 5: Add a notification policy

A notification policy manages notifications by tying alert rules to contact points according to matching labels. For example, if a contact point has the value 'Devs' for the label 'teams' and an alert rule also has the value 'Devs' for the label 'teams' then the alert rule will use the notification policy.

1. Select the Grafana menu , expand the Alerting section and select Notification policies.
   
2. Click New nested policy.
3. Use the Matching labels section to narrow down the alert rules that you want to be subject to this notification policy. Alert rules with labels that match this policy will become subject to it.
   
4. Select the Contact point that you want to receive notifications according to this policy. See Create Contact points.
5. Click Save policy.
   When the alert rule conditions are met for the specified length of time, an email notification like the one below is sent to matching contact points.