Skip to main content
Top
Review meshIQ Products (G2.com)

Stream-JMX: Auto Dumps, Sampling Control, and Conditional Actions

This article explains how Stream-JMX manages application state, sampling, and actions through TNT4J features.
It covers automatic dumps, custom sampler behavior, conditional actions, and Java Flight Recorder integration.

Auto-Generating Application State Dump

Stream-JMX utilizes the TNT4J state dump capability to generate application state dumps

1. Dump on VM shutdown:

java -Dtnt4j.dump.on.vm.shutdown=true -Dtnt4j.dump.provider.default=true -Dtnt4j.dump.folder=./ ...

2. Dump on uncaught thread exceptions:

java -Dtnt4j.dump.on.exception=true -Dtnt4j.dump.provider.default=true -Dtnt4j.dump.folder=./ ...

-Dtnt4j.dump.folder=./ specifies the destination folder where dump (.dump) files will be created (default is the current working directory).

By default, Stream-JMX will generate dumps with the following info:

  • Java Properties Dump -- PropertiesDumpProvider
  • Java Runtime Dump -- MXBeanDumpProvider
  • Thread Stack Dump -- ThreadDumpProvider
  • Thread Deadlock Dump -- ThreadDumpProvider
  • Logging Statistics Dump -- LoggerDumpProvider

You may create your own dump providers and handlers.

Overriding Default SamplerFactory

SamplerFactory instances are used to generate Sampler implementation for a specific runtime environment. Stream-JMX supplies sampler and sampler factories for standard JVMs, J2EE, JBoss, IBM WebSphere Application Server, IBM WebSphere Liberty server. You may want to override default SamplerFactory with your own or an alternative by specifying:

java -Dtnt4j.stream.jmx.sampler.factory=com.jkoolcloud.tnt4j.stream.jmx.impl.PlatformSamplerFactory ...

SamplerFactory is used to generate instances of the underlying sampler implementations (objects that provide sampling of underlying mbeans).

// return default or user defined SamplerFactory implementation
SamplerFactory factory = DefaultSamplerFactory.getInstance();
...

Managing Sample Behavior

Stream-JMX provides a way to intercept sampling events such as pre, during, and post sample runs to control sample behavior. See the SampleListener interface for more details. Applications can register more than one listener per Sampler. Each listener is called in registration order.

In addition to intercepting sample events, applications may want to control how one or more attributes are sampled and whether each sample is reported/logged. See example below:

// return default or user defined SamplerFactory implementation
SamplerFactory factory = DefaultSamplerFactory.getInstance();
// create an instance of the sampler that will sample mbeans
Sampler sampler = factory.newInstance();
Map<String, Object> samplerCfg = new HashMap<>(3);
samplerCfg.put(SampleHandler.CFG_INCLUDE_FILTER, Sampler.JMX_FILTER_ALL);
samplerCfg.put(Sampler.CFG_SAMPLING_PERIOD, 30000);
sampler.setSchedule(samplerCfg).addListener(new MySampleListener())).run();

Below is a sample of what MySampleListener may look like:

class MySampleListener implements SampleListener {
    @Override
    public void getStats(SampleContext context, Map<String, Object> stats) {
        // add your own stats to the map
    }

    @Override
    public void register(SampleContext context, ObjectName oName) {
        System.out.println("Register mbean: " + oName + ", mbean.server=" + context.getMBeanServer());
    }

    @Override
    public void unregister(SampleContext context, ObjectName oName) {
        System.out.println("Unregister mbean: " + oName + ", mbean.server=" + context.getMBeanServer());
    }

    @Override
    public void pre(SampleContext context, Activity activity) {
        // called once per sample, beginning of each sample
        // set activity to NOOP to disable further sampling
        // no other attribute will be sampled during current sample
        if (some-condition) {
            activity.setType(OpType.NOOP);
        }
    }

    @Override
    public void pre(SampleContext context, AttributeSample sample) {
        // called once before attribute is sampled
        // set exclude to true to skip sampling this attribute
        sample.excludeNext(sample.getAttributeInfo().isReadable());
    }

    @Override
    public void post(SampleContext context, AttributeSample sample) {
        // called once after attribute is sampled
        Object value = sample.get();
    }

    @Override
    public void post(SampleContext context, Activity activity) {
        // called once per sample, end of each sample
        // set activity to NOOP to disable sampling reporting
        if (some-condition) {
            activity.setType(OpType.NOOP);
        }
    }

    @Override
    public void error(SampleContext context, Throwable ex) {
        // called once for every exception that occurs not associated with a sample
        ex.printStackTrace();
    }

    @Override
    public void error(SampleContext context, AttributeSample sample) {
        // called once for every exception that occurs during each sample
        Throwable ex = sample.getError();
        ex.printStackTrace();
    }
}

Conditions and Actions

Stream-JMX allows you to associate conditions with user-defined actions based on values of MBean attributes on each sampling interval. For example, what if you wanted to set up an action when a specific MBean attribute exceeds a certain threshold?

Stream-JMX AttributeCondition and AttributeAction interfaces allow you to call your action at runtime every time a condition evaluates to true. See example below:

// return default or user defined SamplerFactory implementation
SamplerFactory factory = DefaultSamplerFactory.getInstance();
// create an instance of the sampler that will sample mbeans
Sampler sampler = factory.newInstance();
// create a condition when ThreadCount > 100
AttributeCondition myCondition = new SimpleCondition("java.lang:type=Threading", "ThreadCount", 100, ">");
// schedule collection (ping) for given MBean filter and 30000 ms sampling period
Map<String, Object> samplerCfg = new HashMap<>(3);
samplerCfg.put(SampleHandler.CFG_INCLUDE_FILTER, Sampler.JMX_FILTER_ALL);
samplerCfg.put(Sampler.CFG_SAMPLING_PERIOD, 30000);
sampler.setSchedule(samplerCfg).register(myCondition, new MyAttributeAction()).run();

Below is a sample of what MyAttributeAction may look like:

public class MyAttributeAction implements AttributeAction {
    @Override
    public Object action(SampleContext context, AttributeCondition cond, AttributeSample sample) {
        Activity activity = sample.getActivity();
        // obtain a collection of all sampled metrics
        Collection<Snapshot> metrics = activity.getSnapshots();
        System.out.println("MyAction called with value=" + sample.get()
            + ", age.usec=" + sample.ageUsec()
            + ", count=" + metrics.size());
        return null;
    }
}

Java Flight Recorder (JFR) Recordings for tnt4j-stream-jmx

There are shell scripts dedicated to running tnt4j-stream-jmx recordings.

These scripts can be found in tnt4j-stream-jmx delivery package directory bin/jfr:

  • record-duration.sh/record-duration.cmd - runs a tnt4j-stream-jmx  JFR recording for the configured duration, until the max recording file size is reached, or until the JVM exits.
  • record-rolling.sh/record-rolling.cmd - runs a tnt4j-stream-jmx JFR recording until the configured maximum age is reached, the max recording file size is exceeded, the user stops recording, or the JVM exits.
  • record-spikes.sh/record-spikes.cmd - runs a tnt4j-stream-jmx JFR recording when configured OS resource usage thresholds are reached. Monitored OS resources are:
    • cpu use - gets triggered when > 80% is reached
    • memory use - gets triggered when > 80% is reached
    • disk io - gets triggered when > 80% is reached
    • network throughput (Linux only) - gets triggered when reception or transition volume gets > 100Mb/s
    • file descriptor use - gets triggered when > 80% is reached