Prometheus Remote Write HTTP Endpoint

TL;DR

Build TNT4J-Streams Servlet

There are two ways to build the TNT4J-Streams WAR package:

• When building the whole tnt4j-streams project, Maven will produce tnt4j-streams-<VERSION>.war, where all reactor-enabled modules and dependencies are built into that WAR package.

• When individually building the tnt4j-streams-servlet module, Maven will produce tnt4j-streams-servlet-<VERSION>.war, where all module dependencies are built into that WAR package.

In either case, the WAR package contains classes and resources (like web.xml) provided by the tnt4j-streams-servlet module.

Web-App Deployment Descriptor (web.xml within WAR package)

Defines the servlet initialization parameter set and servlet mapping.

Init params

• streams.configs.dir — TNT4J-Streams configuration files location path. It shall contain these files: tnt4j.properties, log4j2.xml, and tnt-data-source.xml. Optional if setting individual configuration files with init params tnt4j.config, log4j2.config, and streams.config.
  Default value: ${catalina.base}/conf/Catalina/localhost/tnt4j-streams

• tnt4j.config — TNT4J configuration file path.
  Default value: tnt4j.properties

• log4j2.config — log4j v2 configuration file path.
  Default value: log4j2.xml

• streams.config — TNT4J-Streams data source/parsers configuration file path.
  Default value: tnt-data-source.xml

Servlet mapping

• Default URL pattern: /

log4j2

TNT4J

Stream (tnt-data-source.xml)

Parsers (parsers.xml)

Unwraps Prometheus-sent metrics data packages into a collection of time-series maps, then allows metrics filtering (see Metrics filtering), and performs time-series map field-value parsing into TNT4J activities and snapshots.

This file defines these parsers:

• PrometheusParser — bootstrap parser recognizing the received metrics data package format (binary, text, etc.).

• MetricsParserStr — parses Prometheus Remote Write request data if it was sent as JSON, picking a collection of time-series entries.

• TimeSeriesParser — aggregates all time-series entries provided by MetricsParserStr under a single ACTIVITY.

• SnappyCompressedMetricsParser — converts Prometheus Snappy-compressed binary data to a collection of time-series maps and produces an ACTIVITY entity containing snapshots for every time-series entry produced by TimeSeriesEntryParser.

• TimeSeriesEntryParser — builds a time-series–entry–map–parsed SNAPSHOT entity.

• MetricsMetaDataParser — time-series map fields group metadata parser.

• ExemplarsParser — time-series map fields group exemplars parser.

• SamplesParser — time-series map fields group samples parser.

• LabelsParser — time-series map fields group labels parser.

And these pre-parsers:

• SnappyBinPreParser — performs Snappy-compressed binary data decompression.

• PrometheusReqMapPreParser — parses binary data into a Prometheus Remote.WriteRequest message and extracts it into a time-series entries map.

Metrics Filtering

You may want to stream only a subset of the provided time series. To filter time series by label, implement filtering logic in the SnappyCompressedMetricsParser parser field (embedded activity) MetricsDataProto transformation. The default transformation does not perform any filtering:

Collection<Map<String, ?>> tsColl = new ArrayList<>()

for (Map.Entry<String, Collection<Map<String, ?>>> tsCollE : $fieldValue.entrySet()) {
    tsColl.addAll(tsCollE.getValue())
}

return tsColl

import org.apache.commons.lang3.StringUtils

Collection<Map<String, ?>> tsColl = new ArrayList<> ()

for (Map.Entry<String, Collection<Map<String, ?>>> tsCollE : $fieldValue.entrySet ()) {
    if (StringUtils.startsWithAny (tsCollE.getKey (), "kafka_consumer_")) {  //NON-NLS
        tsColl.addAll (tsCollE.getValue ())
    }
}

return tsColl

Streamed Metrics Data

For Prometheus Remote Write–provided metrics packages, TNT4J-Streams produces an ACTIVITY entity containing:

• Name has value PrometheusTimeSeries

• StartTime/EndTime having current stream runtime timestamp

Time-series entries are placed as child SNAPSHOTs of that ACTIVITY entity. Each snapshot contains the following fields:

Labels group produces

• Instance — value from group field instance

• Job — value from group field job

• Type — value from group field type

• Name — value from group field name

Samples group produces

• StartTime/EndTime — value from group field timestamp

• Value — value from group field value

Metadata group produces

• MetricType — value from group field type

• Help — value from group field help

• MetricFamilyName — value from group field family

• Unit — value from group field unit

Additional mappings

• Category — value mapped from the time-series entry name to the JMX domain. Mapping is done in the TimeSeriesEntryParser parser field domain transformation.

• Name — value from labels group field __name__

The rest of the metrics line fields are mapped into snapshot properties using their original names (1:1). Usually, these may be new or very rare fields.