JMX and Spring – Part 3

This article is the last one of this series. Take a look at Part 1 and Part 2.

In this last article of the series I’ll show how to use the native JMX support within the JDK to implement a notification mechanism which alerts a listener when the HEAP memory is above a certain threshold.

As discussed in my previous article this approach is ideal because is push instead of pull, is not intrusive and places minimal computing demand on your application.

These are the key components to the solution illustrated in this article:

  • MemoryWarningService: This component acts as a listener and registers itself with the Memory MBean to receive notifications. It is configurable with a threshold in the form of a percentage between 0 and 1 (where 1 is 100%)
  • MemoryThreadDumper: This component is invoked when the MemoryWarningService is notified that the HEAP usage is above the threshold and its responsibility is to write a thread dump to a file
  • MemoryWarningServiceConfigurator: This component is an MBean and exposes a method to change the threshold of the MemoryWarningService.

The solution provides also a MemoryHeapFiller class used to fill up the HEAP while testing the application and a MemTest class to bootstrap the Spring environment.

While the application is running (play with the MemoryHeapFiller settings) You can fire the JConsole at URL: service:jmx:rmi://localhost/jndi/rmi://localhost:8888/jemosJmxConnector connecting as jemosAdmin / secure and change the threshold to various values.

The code is not meant for production: it is not robust, there are numerous comments missing, and the filename where to write the thread dump is hard-coded; it represents, however, a good starting point.

The code is attached below. You will need Maven to build it.

Download Jemos-jmx-experiments-0.0.1-SNAPSHOT-project

I tried a scenario with initial threshold to be 0.5, I changed it to 0.3 and then to 0.8. The results are shown below:

2011-08-15 21:53:21 ClassPathXmlApplicationContext [INFO] Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@a4a63d8: startup date [Mon Aug 15 21:53:21 BST 2011]; root of context hierarchy
2011-08-15 21:53:21 XmlBeanDefinitionReader [INFO] Loading XML bean definitions from class path resource [jemos-jmx-appCtx.xml]
2011-08-15 21:53:21 PropertyPlaceholderConfigurer [INFO] Loading properties file from class path resource [jemos-jmx.properties]
2011-08-15 21:53:21 PropertyPlaceholderConfigurer [INFO] Loading properties file from URL [file:/C:/Users/mtedone/.secure/jmxconnector-credentials.properties]
2011-08-15 21:53:21 ThreadPoolTaskScheduler [INFO] Initializing ExecutorService  'myScheduler'
2011-08-15 21:53:21 ClassPathXmlApplicationContext [INFO] Bean 'myScheduler' of type [class org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2011-08-15 21:53:21 DefaultListableBeanFactory [INFO] Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@30296f76: defining beans [propertyConfigurer,loggerConfigurator,memoryWarningServiceConfigurator,memoryHeapFiller,memoryThreadDumper,org.springframework.context.annotation.internalConfigurationAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor,mbeanExporter,jemosJmxServer,rmiRegistry,clientConnector,memoryMxBean,memoryWarningService,org.springframework.scheduling.annotation.internalAsyncAnnotationProcessor,org.springframework.scheduling.annotation.internalScheduledAnnotationProcessor,myScheduler]; root of factory hierarchy
2011-08-15 21:53:21 AnnotationMBeanExporter [INFO] Registering beans for JMX exposure on startup
2011-08-15 21:53:21 RmiRegistryFactoryBean [INFO] Looking for RMI registry at port '8888'
2011-08-15 21:53:23 RmiRegistryFactoryBean [INFO] Could not detect RMI registry - creating new one
2011-08-15 21:53:23 ConnectorServerFactoryBean [INFO] JMX connector server started: javax.management.remote.rmi.RMIConnectorServer@4355d3a3
2011-08-15 21:53:23 AnnotationMBeanExporter [INFO] Bean with name 'jemosJmxServer' has been autodetected for JMX exposure
2011-08-15 21:53:23 AnnotationMBeanExporter [INFO] Bean with name 'loggerConfigurator' has been autodetected for JMX exposure
2011-08-15 21:53:23 AnnotationMBeanExporter [INFO] Bean with name 'memoryWarningServiceConfigurator' has been autodetected for JMX exposure
2011-08-15 21:53:23 AnnotationMBeanExporter [INFO] Located managed bean 'loggerConfigurator': registering with JMX server as MBean [jemos.mbeans:type=config,name=LoggingConfiguration]
2011-08-15 21:53:23 AnnotationMBeanExporter [INFO] Located MBean 'jemosJmxServer': registering with JMX server as MBean [jemos.mbeans:name=jemosJmxServer,type=RMIConnectorServer]
2011-08-15 21:53:23 AnnotationMBeanExporter [INFO] Located managed bean 'memoryWarningServiceConfigurator': registering with JMX server as MBean [jemos.mbeans:type=config,name=MemoryWarningServiceConfiguration]
2011-08-15 21:53:23 MemoryWarningService [INFO] Percentage is: 0.5
2011-08-15 21:53:23 MemoryWarningService [INFO] Listener added to JMX bean
Adding data...
Adding data...
Adding data...
Adding data...
Adding data...
Adding data...
Adding data...
2011-08-15 21:53:37 MemoryWarningService [INFO] Percentage is: 0.3
2011-08-15 21:53:37 MemoryWarningServiceConfigurator [INFO] Memory threshold set to 0.3
Adding data...
2011-08-15 21:53:38 MemoryWarningService [WARN] Memory usage low!!!
2011-08-15 21:53:38 MemoryWarningService [WARN] percentageUsed = 0.3815679398794023
2011-08-15 21:53:38 MemoryThreadDumper [WARN] Stacks dumped to: C:/tmp/stacks.dump
Adding data...
Adding data...
Adding data...
2011-08-15 21:53:45 MemoryWarningService [INFO] Percentage is: 0.8
2011-08-15 21:53:45 MemoryWarningServiceConfigurator [INFO] Memory threshold set to 0.8
Adding data...
Adding data...
Adding data...
Adding data...
Adding data...
Adding data...
Adding data...
2011-08-15 21:54:01 MemoryWarningService [WARN] Memory usage low!!!
2011-08-15 21:54:01 MemoryWarningService [WARN] percentageUsed = 0.8383333266727508
2011-08-15 21:54:02 MemoryThreadDumper [WARN] Stacks dumped to: C:/tmp/stacks.dump
Adding data...
Adding data...
Adding data...
Exception in thread "JMX server connection timeout 24" java.lang.OutOfMemoryError: Java heap space

The Memory Warning Service

package uk.co.jemos.experiments.jmx;

import java.lang.management.ManagementFactory;
import java.lang.management.MemoryNotificationInfo;
import java.lang.management.MemoryPoolMXBean;
import java.lang.management.MemoryType;

import javax.annotation.PostConstruct;
import javax.management.Notification;
import javax.management.NotificationEmitter;
import javax.management.NotificationListener;

import org.springframework.beans.factory.annotation.Autowired;

/**
 * A component which sends notifications when the HEAP memory is above a certain
 * threshold.
 * 
 * @author mtedone
 * 
 */
public class MemoryWarningService implements NotificationListener {

    /** This bean's name */
    public static final String MBEAN_NAME = "jemos.mbeans:type=monitoring,name=MemoryWarningService";

    /** The application logger */
    private static final org.apache.log4j.Logger LOG = org.apache.log4j.Logger
            .getLogger(MemoryWarningService.class);

    @Autowired
    private NotificationEmitter memoryMxBean;

    @Autowired
    private MemoryThreadDumper threadDumper;

    /** A pool of Memory MX Beans specialised in HEAP management */
    private static final MemoryPoolMXBean tenuredGenPool = findTenuredGenPool();

    /**
     * {@inheritDoc}
     */
    @Override
    public void handleNotification(Notification notification, Object handback) {

        if (notification.getType().equals(
                MemoryNotificationInfo.MEMORY_THRESHOLD_EXCEEDED)) {
            long maxMemory = tenuredGenPool.getUsage().getMax();
            long usedMemory = tenuredGenPool.getUsage().getUsed();
            LOG.warn("Memory usage low!!!");
            double percentageUsed = (double) usedMemory / maxMemory;
            LOG.warn("percentageUsed = " + percentageUsed);
            threadDumper.dumpStacks();

        } else {
            LOG.info("Other notification received..."
                    + notification.getMessage());
        }

    }

    /**
     * It sets the threshold percentage.
     * 
     * @param percentage
     */
    public void setPercentageUsageThreshold(double percentage) {
        if (percentage <= 0.0 || percentage > 1.0) {
            throw new IllegalArgumentException("Percentage not in range");
        } else {
            LOG.info("Percentage is: " + percentage);
        }
        long maxMemory = tenuredGenPool.getUsage().getMax();
        long warningThreshold = (long) (maxMemory * percentage);
        tenuredGenPool.setUsageThreshold(warningThreshold);
    }

    @PostConstruct
    public void completeSetup() {
        memoryMxBean.addNotificationListener(this, null, null);
        LOG.info("Listener added to JMX bean");
    }

    /**
     * Tenured Space Pool can be determined by it being of type HEAP and by it
     * being possible to set the usage threshold.
     */
    private static MemoryPoolMXBean findTenuredGenPool() {
        for (MemoryPoolMXBean pool : ManagementFactory.getMemoryPoolMXBeans()) {
            // I don't know whether this approach is better, or whether
            // we should rather check for the pool name "Tenured Gen"?
            if (pool.getType() == MemoryType.HEAP
                    && pool.isUsageThresholdSupported()) {
                return pool;
            }
        }
        throw new AssertionError("Could not find tenured space");
    }

}

The Memory Thread Dumper

package uk.co.jemos.experiments.jmx;

import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.Writer;
import java.lang.management.ManagementFactory;
import java.lang.management.ThreadInfo;
import java.lang.management.ThreadMXBean;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;

import org.apache.commons.io.IOUtils;
import org.springframework.stereotype.Component;

/**
 * This component dumps the thread stack to the file system.
 * 
 * @author mtedone
 * 
 */
@Component
public class MemoryThreadDumper {

    /** The application logger */
    private static final org.apache.log4j.Logger LOG = org.apache.log4j.Logger
            .getLogger(MemoryThreadDumper.class);

    /**
     * It dumps the Thread stacks
     * 
     * @throws IOException
     */
    public void dumpStacks() {

        // hard-coded: This needs to be changed to a property or something
        String stackFileName = "C:/tmp/stacks.dump";

        ThreadMXBean mxBean = ManagementFactory.getThreadMXBean();
        ThreadInfo[] threadInfos = mxBean.getThreadInfo(
                mxBean.getAllThreadIds(), 0);
        Map<Long, ThreadInfo> threadInfoMap = new HashMap<Long, ThreadInfo>();
        for (ThreadInfo threadInfo : threadInfos) {
            threadInfoMap.put(threadInfo.getThreadId(), threadInfo);
        }

        File dumpFile = new File(stackFileName);
        BufferedWriter writer = null;
        try {
            writer = new BufferedWriter(new FileWriter(dumpFile));
            this.dumpTraces(mxBean, threadInfoMap, writer);

            LOG.warn("Stacks dumped to: " + stackFileName);

        } catch (IOException e) {
            throw new IllegalStateException(
                    "An exception occurred while writing the thread dump");
        } finally {
            IOUtils.closeQuietly(writer);
        }

    }

    private void dumpTraces(ThreadMXBean mxBean,
            Map<Long, ThreadInfo> threadInfoMap, Writer writer)
            throws IOException {
        Map<Thread, StackTraceElement[]> stacks = Thread.getAllStackTraces();
        writer.write("Dump of "
                + stacks.size()
                + " thread at "
                + new SimpleDateFormat("yyyy/MM/dd HH:mm:ss z")
                        .format(new Date(System.currentTimeMillis())) + "\n\n");
        for (Map.Entry<Thread, StackTraceElement[]> entry : stacks.entrySet()) {
            Thread thread = entry.getKey();
            writer.write("\"" + thread.getName() + "\" prio="
                    + thread.getPriority() + " tid=" + thread.getId() + " "
                    + thread.getState() + " "
                    + (thread.isDaemon() ? "deamon" : "worker") + "\n");
            ThreadInfo threadInfo = threadInfoMap.get(thread.getId());
            if (threadInfo != null) {
                writer.write("    native=" + threadInfo.isInNative()
                        + ", suspended=" + threadInfo.isSuspended()
                        + ", block=" + threadInfo.getBlockedCount() + ", wait="
                        + threadInfo.getWaitedCount() + "\n");
                writer.write("    lock=" + threadInfo.getLockName()
                        + " owned by " + threadInfo.getLockOwnerName() + " ("
                        + threadInfo.getLockOwnerId() + "), cpu="
                        + mxBean.getThreadCpuTime(threadInfo.getThreadId())
                        / 1000000L + ", user="
                        + mxBean.getThreadUserTime(threadInfo.getThreadId())
                        / 1000000L + "\n");
            }
            for (StackTraceElement element : entry.getValue()) {
                writer.write("        ");
                writer.write(element.toString());
                writer.write("\n");
            }
            writer.write("\n");
        }
    }

}

The Memory Service Configuration MBean

package uk.co.jemos.experiments.jmx.mbeans;

import org.springframework.beans.BeansException;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;
import org.springframework.jmx.export.annotation.ManagedOperation;
import org.springframework.jmx.export.annotation.ManagedOperationParameter;
import org.springframework.jmx.export.annotation.ManagedOperationParameters;
import org.springframework.jmx.export.annotation.ManagedResource;
import org.springframework.stereotype.Component;

import uk.co.jemos.experiments.jmx.MemoryWarningService;

@Component
@ManagedResource(objectName = MemoryWarningServiceConfigurator.MBEAN_NAME, //
description = "Allows clients to set the memory threshold")
public class MemoryWarningServiceConfigurator implements
        ApplicationContextAware {

    

    /** The application logger */
    private static final org.apache.log4j.Logger LOG = org.apache.log4j.Logger
            .getLogger(MemoryWarningServiceConfigurator.class);

    public static final String MBEAN_NAME = "jemos.mbeans:type=config,name=MemoryWarningServiceConfiguration";

    private ApplicationContext ctx;

    @ManagedOperation(description = "Sets the memory threshold for the memory warning system")
    @ManagedOperationParameters({ @ManagedOperationParameter(description = "The memory threshold", name = "memoryThreshold"), })
    public void setMemoryThreshold(double memoryThreshold) {

        MemoryWarningService memoryWarningService = (MemoryWarningService) ctx
                .getBean("memoryWarningService");
        memoryWarningService.setPercentageUsageThreshold(memoryThreshold);

        LOG.info("Memory threshold set to " + memoryThreshold);
    }

    @Override
    public void setApplicationContext(ApplicationContext applicationContext)
            throws BeansException {
        ctx = applicationContext;

    }


}

The Spring configuration

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
    xmlns:util="http://www.springframework.org/schema/util"
    xmlns:task="http://www.springframework.org/schema/task"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
        http://www.springframework.org/schema/task http://www.springframework.org/schema/task/spring-task-3.0.xsd
        http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-3.0.xsd">


    <bean id="propertyConfigurer" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"  >
        <property name="locations">
            <list>
                <value>classpath:jemos-jmx.properties</value>
                <value>file:///${user.home}/.secure/jmxconnector-credentials.properties
                </value>
            </list>
        </property>
    </bean>

    <context:component-scan base-package="uk.co.jemos.experiments.jmx" />

    <context:mbean-export default-domain="jemos.mbeans" />

    <bean id="jemosJmxServer" class="org.springframework.jmx.support.ConnectorServerFactoryBean"       
        depends-on="rmiRegistry">
        <property name="objectName" value="connector:name=rmi" />
        <property name="serviceUrl"
            value="service:jmx:rmi://localhost/jndi/rmi://localhost:${jemos.jmx.rmi.port}/jemosJmxConnector" />
        <property name="environment">
            <!-- the following is only valid when the sun jmx implementation is used -->
            <map>
                <entry key="jmx.remote.x.password.file" value="${user.home}/.secure/jmxremote.password" />
                <entry key="jmx.remote.x.access.file" value="${user.home}/.secure/jmxremote.access" />
            </map>
        </property>
    </bean>

    <bean id="rmiRegistry" class="org.springframework.remoting.rmi.RmiRegistryFactoryBean">
        <property name="port" value="${jemos.jmx.rmi.port}" />
    </bean>

    <bean id="clientConnector" class="org.springframework.jmx.support.MBeanServerConnectionFactoryBean"       
        depends-on="jemosJmxServer">
        <property name="serviceUrl"
            value="service:jmx:rmi://localhost/jndi/rmi://localhost:${jemos.jmx.rmi.port}/jemosJmxConnector" />
        <property name="environment">
            <map>
                <entry key="jmx.remote.credentials">
                    <bean
                        factory-method="commaDelimitedListToStringArray">
                        <constructor-arg value="${jmx.username},${jmx.password}" />
                    </bean>
                </entry>
            </map>
        </property>
    </bean>


    <bean id="memoryMxBean" class="java.lang.management.ManagementFactory"
        factory-method="getMemoryMXBean" />
        
    <bean id="memoryWarningService" class="uk.co.jemos.experiments.jmx.MemoryWarningService">
      <property name="percentageUsageThreshold" value="0.5" />
    </bean>
        

    <task:annotation-driven scheduler="myScheduler" />   

    <task:scheduler id="myScheduler" pool-size="10" />    
        


</beans>

 

Reference: JMX and Spring – Part 3 from our JCG partner Marco Tedone at the Marco Tedone’s blog blog.

Do you want to know how to develop your skillset to become a Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

JPA Mini Book

Learn how to leverage the power of JPA in order to create robust and flexible Java applications. With this Mini Book, you will get introduced to JPA and smoothly transition to more advanced concepts.

JVM Troubleshooting Guide

The Java virtual machine is really the foundation of any Java EE platform. Learn how to master it with this advanced guide!

Given email address is already subscribed, thank you!
Oops. Something went wrong. Please try again later.
Please provide a valid email address.
Thank you, your sign-up request was successful! Please check your e-mail inbox.
Please complete the CAPTCHA.
Please fill in the required fields.

Leave a Reply


6 × = forty two



Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.
Do you want to know how to develop your skillset and become a ...
Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

Get ready to Rock!
You can download the complementary eBooks using the links below:
Close