This article focuses on the Core module of TNT4J-Streams—its purpose (data collection and transformation), required prerequisites, and how to build, package, run samples, test, and troubleshoot the core functionality.
Modules
- (M) marked modules are mandatory
- (O) marked modules are optional
- (U) marked modules are utility modules (e.g., performs compiled assemblies packaging)
Modules list:
-
Core(M) - major module implementing data streaming (collection and transformation) features.Maven dependency:
<dependency> <groupId>com.jkoolcloud.tnt4j.streams</groupId> <artifactId>tnt4j-streams-core</artifactId> <version>12.0.0</version> </dependency> -
Elastic-Beats(O) - Elastic Beats–provided data streaming moduleMaven dependency:
<dependency> <groupId>com.jkoolcloud.tnt4j.streams</groupId> <artifactId>tnt4j-streams-elastic-beats</artifactId> <version>12.0.0</version> </dependency> -
Flume-Plugin(O) - Apache Flume–provided data streaming moduleMaven dependency:
<dependency> <groupId>com.jkoolcloud.tnt4j.streams</groupId> <artifactId>tnt4j-streams-flume-plugin</artifactId> <version>12.0.0</version> </dependency> -
Fs(O) - JSR-203 compliant FileSystem provided files data streaming module.Maven dependency:
<dependency> <groupId>com.jkoolcloud.tnt4j.streams</groupId> <artifactId>tnt4j-streams-fs</artifactId> <version>12.0.0</version> </dependency> -
Hdfs(O) - HDFS (Apache Hadoop) provided data streaming module.Maven dependency:
<dependency> <groupId>com.jkoolcloud.tnt4j.streams</groupId> <artifactId>tnt4j-streams-hdfs</artifactId> <version>12.0.0</version> </dependency> -
JMS(O) - JMS (Java Message Service) provided data streaming module.Maven dependency:
<dependency> <groupId>com.jkoolcloud.tnt4j.streams</groupId> <artifactId>tnt4j-streams-jms</artifactId> <version>12.0.0</version> </dependency> -
Kafka(O) - Apache Kafka provided data streaming module.Maven dependency:
<dependency> <groupId>com.jkoolcloud.tnt4j.streams</groupId> <artifactId>tnt4j-streams-kafka</artifactId> <version>12.0.0</version> </dependency> -
Mqtt(O) - MQTT provided data streaming module.Maven dependency:
<dependency> <groupId>com.jkoolcloud.tnt4j.streams</groupId> <artifactId>tnt4j-streams-mqtt</artifactId> <version>12.0.0</version> </dependency> -
MsOffice(O) - MS Office Excel-provided data streaming module.Maven dependency:
<dependency> <groupId>com.jkoolcloud.tnt4j.streams</groupId> <artifactId>tnt4j-streams-msoffice</artifactId> <version>12.0.0</version> </dependency> -
WMQ(O) - IBM MQ-provided data streaming module.Maven dependency:
<dependency> <groupId>com.jkoolcloud.tnt4j.streams</groupId> <artifactId>tnt4j-streams-wmq</artifactId> <version>12.0.0</version> </dependency> -
WS(O) - web-service (or OS command)-provided data streaming module.Maven dependency:
<dependency> <groupId>com.jkoolcloud.tnt4j.streams</groupId> <artifactId>tnt4j-streams-ws</artifactId> <version>12.0.0</version> </dependency> -
Chronicle(O) - Chronicle Queue-provided data streaming module.Maven dependency:
<dependency> <groupId>com.jkoolcloud.tnt4j.streams</groupId> <artifactId>tnt4j-streams-chronicle</artifactId> <version>12.0.0</version> </dependency> -
Servlet(M) - packaging streams as servlet module.Maven dependency:
<dependency> <groupId>com.jkoolcloud.tnt4j.streams</groupId> <artifactId>tnt4j-streams-servlet</artifactId> <version>12.0.0</version> </dependency> -
Compress(O) - Compressed data pre-parsers module.Maven dependency:
<dependency> <groupId>com.jkoolcloud.tnt4j.streams</groupId> <artifactId>tnt4j-streams-compress</artifactId> <version>12.0.0</version> </dependency> -
Protobuf(O) - Protobuf messages pre-parsers module.Maven dependency:
<dependency> <groupId>com.jkoolcloud.tnt4j.streams</groupId> <artifactId>tnt4j-streams-protobuf</artifactId> <version>12.0.0</version> </dependency> -
Prometheus(O) - Prometheus Remote-Write protobuf messages pre-parsers module.Maven dependency:
<dependency> <groupId>com.jkoolcloud.tnt4j.streams</groupId> <artifactId>tnt4j-streams-prometheus</artifactId> <version>12.0.0</version> </dependency> -
Samples(O) - integration into custom API sample module.Maven dependency:
<dependency> <groupId>com.jkoolcloud.tnt4j.streams</groupId> <artifactId>tnt4j-streams-samples</artifactId> <version>12.0.0</version> </dependency> -
Distribution(OU) - distributable package build module.
All optional modules (extensions) depend on the core module and cannot be built or run without it.
The Samples module provides no additional features to the TNT4J streaming framework; it contains only streams API usage samples. The Distribution module performs Maven post-build release assembly delivery to the build/ directory.
Requirements
- JDK 17+
- Apache Maven 3
- TNT4J
- JESL
All other required dependencies are defined in project modules’ pom.xml files.
If Maven is run in online mode, it will download these dependencies automatically.
Manually Installed Dependencies
Some required or optional dependencies may not be available in the public Maven repository.
In such cases, manually download those dependencies into the module’s lib directory and install them into your local Maven Repository by running the Maven script lib/pom.xml with the install goal.
Building
To build the project, run the Maven goal:
clean packageTo build the project and install it to the local repository, run:
clean install-
To create distributable release assemblies, use one of the following profiles:
For a binary distribution (including the
testpackage), run:mvn -P pack-assemblyFor a distribution that includes binary (with
testpackage), source, and Javadoc, run:mvn -P pack-sources-and-javadocs,pack-assembly
To generate the Maven-required source and Javadoc packages, use the profile:
pack-sources-and-javadocsTo create a Maven Central-compliant release with source, Javadoc, and all signed packages, use the profile:
final-releaseBy default, Maven builds all modules defined in the
tnt4j-streams/pom.xmlfile.If you do not want to build certain optional modules (e.g.,
WMQ), comment them out in the POM file.
Alternatively, you can specify a preferred set of modules to build using the-plor--projectsargument (comma-separated list of modules), along with the-amor--also-makeargument. For example:
mvn -pl tnt4j-streams-core,tnt4j-streams-samples,tnt4j-streams--distribution -am clean install -P pack-assembly
or
mvn --projects tnt4j-streams-core,tnt4j-streams-samples,tnt4j-streams--distribution --also-make clean install -P pack-assembly
The modules list must be comma-separated without spaces.
The above commands build only tnt4j-streams-core, tnt4j-streams-samples, and tnt4j-streams--distribution. Release assemblies are built to the build/ directory.
Troubleshooting:
Sometimes Maven fails to resolve dependencies correctly. If configuration looks fine but Maven still complains about missing artifacts, try deleting the local Maven repository (e.g., on Windows:
C:\Users\[username]\.m2\repository).-
If you use Java 9 as your compiler and encounter
java.lang.NoClassDefFoundError: javax/xml/bind/JAXBException, add thejavacparameter:--add-modules java.xml.bind
to include JAXB classes on the classpath.
Running samples
Testing of TNT4J-Streams
Requirements
Testing using Maven
Maven test execution is disabled by default.
Enable tests with:
-DskipTests=false
or use the run-tests profile.
Running Manually from IDE
Run the JUnit test suite for the corresponding module:
core →
AllStreamsCoreTestselastic-beats →
AllElasticBeatsTestsflume-plugin →
AllFlumeTestsfs →
AllFsStreamTestshdfs →
AllHdfsStreamTestsjms →
AllJMSStreamTestskafka →
AllKafkaStreamTestsmqtt →
AllMqttStreamTestswmq →
AllWmqStreamTestsws →
AllWsStreamTestsmsoffice →
AllMsOfficeStreamTestschronicle →
AllChronicleStreamTestsservlet →
AllServletStreamTestscompress →
AllCompressStreamTestsprotobuf →
AllProtobufStreamTestsprometheus →
AllPrometheusStreamTests