The functionality described in this article is available in the meshIQ Platform versions 11.1.1 and later. See meshIQ platform Highlights v11.1 for an overview of feature changes.
meshIQ supports the OpenTelemetry Standard.
The meshIQ OpenTelemetry Exporter sends metrics directly to OpenTelemetry. Once the data is there, it can be picked up by another tool such as Prometheus.
- Before configuring meshIQ to send data to OpenTelemetry, ensure that an OpenTelemetry Collector is running and properly configured. You can refer the https://opentelemetry.io/docs/collector/quick-start/ for instructions on setting up the collector. Once the collector is up and running, you can proceed with configuring meshIQ to export data to it.
Required Attributes
otel_exporter_otlp_endpoint
This is the OTLP/gRPC endpoint where meshIQ sends the OpenTelemetry data. Only (Google RPC) gRPC is supported. (HTTP is not supported.)
- gRPC default value: http://localhost:4317
OpenTelemetry Configuration Steps
OpenTelemetry can be configured in meshIQ in the following way:
-
Make sure your OpenTelemetry Collector is running and reachable.
-
Navigate to
/opt/meshiq/platform/localhost/node.properties
. -
Alternatively, open the Enterprise Manager, right-click on the
cep-service
, selectnode.properties
, and add the following property: - Restart the
cep-service
. - Once meshIQ is configured, the data will be sent to the OpenTelemetry Collector, which can then forward the metrics to other tools like Prometheus or Grafana.
- In Grafana UI, go to the meshIQ plugin dashboard run the following queries Example Query and Results.
- You should now be able to see the Track metrics data in the browser using the following URL
http://<my-endpoint-api>:8889/metrics
Setting up Permissions
To export data to OpenTelemetry, a user must have permissions to run Invoke queries for getting data. The access token used by Grafana for authorization must have Execute permission selected in token setup under Token Options.
Example Query and Results
Our sample Query returns policy names, sensor names, sensor values, and child count:
Get Sensor fields PolicyName as 'policy_name', SensorName as 'sensor_name', SensorValue as 'sensor_value', ChildSensorCount as 'child_count' where PolicyManagerName='DOMAIN_SERVER_Facts' and SensorName contains '%'
To export data to OpenTelemetry, add the Invoke command Invoke OtlpExporter to the query:
Get Sensor fields PolicyName as 'policy_name', SensorName as 'sensor_name', SensorValue as 'sensor_value', ChildSensorCount as 'child_count' where PolicyManagerName='DOMAIN_SERVER_Facts' and SensorName contains '%' | Invoke OtlpExporter
Results are sent to OpenTelemetry.
How results are formatted
In data from query results, values can be numeric or non-numeric.
- All numeric columns, such as sensor value and child count in the examples above, are assumed to be numeric metrics.
- All nonnumerical values, such as policy name and sensor name in the examples above, are converted to strings and sent as labels.
When results are formatted, they are grouped by numeric column. In the example below, results are in two groupds: one for child_count and one for sensor_value. Each group consists of four lines of metrics, each with two labels, provide the individual numeric values.
# HELP child_count
# TYPE child_count gauge
child_count{job=”otlp_exporter”, policy_name=”SYS_node_health.bsv”,SENSOR_NAME=”Fact Storage Utilization %”} 0
child_count{job=”otlp_exporter”, policy_name=”SYS_node_health.bsv”,SENSOR_NAME=”Fact Publish Utilization %”} 0
child_count{job=”otlp_exporter”, policy_name=”SYS_node_health.bsv”,SENSOR_NAME=”Memory Utilization %”} 0
child_count{job=”otlp_exporter”, policy_name=”SYS_node_health.bsv”,SENSOR_NAME=”Rule Engine CPU Idle %”} 0
# HELP sensor_value
# TYPE sensor_value gauge
sensor_value{job=”otlp_exporter”, policy_name=”SYS_node_health.bsv”,SENSOR_NAME=”Fact Storage Utilization %”} 84.8
sensor_value{job=”otlp_exporter”, policy_name=”SYS_node_health.bsv”,SENSOR_NAME=”Fact Publish Utilization %”} 0
sensor_value{job=”otlp_exporter”, policy_name=”SYS_node_health.bsv”,SENSOR_NAME=”Memory Utilization %”} 27.15
sensor_value{job=”otlp_exporter”, policy_name=”SYS_node_health.bsv”,SENSOR_NAME=”Rule Engine CPU Idle %”} 99.99