Heroku and Java – From Newbie to Beginner, Part 2

The problem

So after a few days I could get back to my little Recaps project. I started with checking logs and found something like this:
 
 
 
 
 
 
 

2012-03-04T01:52:51+00:00 heroku[web.1]: Idling
2012-03-04T01:52:53+00:00 heroku[web.1]: Stopping process with SIGTERM
2012-03-04T01:53:03+00:00 heroku[web.1]: Error R12 (Exit timeout) -> Process failed to exit within 10 seconds of SIGTERM
2012-03-04T01:53:03+00:00 heroku[web.1]: Stopping process with SIGKILL

I don’t know what you think but whenever I see ‘Error’ in my logs I get concerned. So I decided to remove that nasty line. What transpired was not pleasant.

The solution

This seems like a simple problem. I started up with finding what this SIGTERM is all about. I knew it was Linux signal, I just wanted to know what Heroku is actually doing. So basically sometimes Heroku just sends SIGTERM to your process so that it’s allowed to gracefully shut down. This is very simple.

As I mentioned in my last post, I decided to use Jetty instead of Grizzly. At first I decided to use jetty-runner to run my web application and it worked fine, resources were scanned as Jersey servlet started up. Deploying to Heroku was also easy and with modified Procfile the application started up.

Nevertheless the application did not react correctly to SIGTERM, so without delving deeper in jetty-runner configuration, I decided to just use the embedded Jetty server. It’s very simple and running foreman start made the application actually start. So without further consideration I just deployed the changed application to Heroku. To check whether the error appears again, after first startup I just did heroku restart and connected to logs in another terminal. But the exit timeout error message was still there. My mistake there – I did not test the application whether it will exit properly when using foreman. So again, foreman start and then ctrl+c just to see what happens (later I tried kill -s TERM procand got similar output):

pbu@pbudesk ~/recaps $ foreman start
21:57:27 web.1     | started with pid 9603
21:57:27 web.1     | 0    [main] INFO  org.eclipse.jetty.server.Server  - jetty-8.1.1.v20120215
21:57:27 web.1     | 110  [main] INFO  org.eclipse.jetty.webapp.StandardDescriptorProcessor  - NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
21:57:27 web.1     | 132  [main] INFO  org.eclipse.jetty.server.handler.ContextHandler  - started o.e.j.w.WebAppContext{/,file:/home/pbu/Devel/IdeaProjects/recaps/webmodule/src/main/webapp/},webmodule/src/main/webapp
21:57:27 web.1     | 133  [main] INFO  org.eclipse.jetty.server.handler.ContextHandler  - started o.e.j.w.WebAppContext{/,file:/home/pbu/Devel/IdeaProjects/recaps/webmodule/src/main/webapp/},webmodule/src/main/webapp
21:57:27 web.1     | 183  [main] INFO  org.eclipse.jetty.server.AbstractConnector  - Started SelectChannelConnector@0.0.0.0:5000
^CSIGINT received
21:57:57 system    | sending SIGTERM to all processes
21:57:57 system    | sending SIGTERM to pid 9603
21:57:57 web.1     | process terminated
pbu@pbudesk ~/recaps $

OK, so when foreman receives SIGINT it sends SIGTERM to all processes, cool – probably Heroku dynos behave the same. Still, it wasn’t a graceful shutdown, but Jetty has a graceful shutdown section that mentions two nice properties: gracefulShutdown and stopAtShutdown. The modified class looks like this:

public class Serve {
    public static void main(String[] args) throws Exception {
        int port = Integer.valueOf(System.getenv("PORT"));

        Server jetty = new Server(port);

        WebAppContext context = new WebAppContext();
        context.setContextPath("/");
        String webapp = "webmodule/src/main/webapp";
        context.setWar(webapp);
        context.setResourceBase(webapp);

        jetty.setHandler(context);

        jetty.setGracefulShutdown(1000);
        jetty.setStopAtShutdown(true);

        jetty.start();
        jetty.join();
    }
}

Running foreman again and using ctrl+c proves this to be working! Great!

pbu@pbudesk ~/recaps $ foreman start
22:11:47 web.1     | started with pid 9863
22:11:47 web.1     | 0    [main] INFO  org.eclipse.jetty.server.Server  - jetty-8.1.1.v20120215
22:11:47 web.1     | 110  [main] INFO  org.eclipse.jetty.webapp.StandardDescriptorProcessor  - NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
22:11:47 web.1     | 131  [main] INFO  org.eclipse.jetty.server.handler.ContextHandler  - started o.e.j.w.WebAppContext{/,file:/home/pbu/Devel/IdeaProjects/recaps/webmodule/src/main/webapp/},webmodule/src/main/webapp
22:11:47 web.1     | 132  [main] INFO  org.eclipse.jetty.server.handler.ContextHandler  - started o.e.j.w.WebAppContext{/,file:/home/pbu/Devel/IdeaProjects/recaps/webmodule/src/main/webapp/},webmodule/src/main/webapp
22:11:48 web.1     | 183  [main] INFO  org.eclipse.jetty.server.AbstractConnector  - Started SelectChannelConnector@0.0.0.0:5000
^C22:11:49 web.1     | 1969 [Thread-1] INFO  org.eclipse.jetty.server.Server  - Graceful shutdown SelectChannelConnector@0.0.0.0:5000
22:11:49 web.1     | 1970 [Thread-1] INFO  org.eclipse.jetty.server.Server  - Graceful shutdown o.e.j.w.WebAppContext{/,file:/home/pbu/Devel/IdeaProjects/recaps/webmodule/src/main/webapp/},webmodule/src/main/webapp
SIGINT received
22:11:49 system    | sending SIGTERM to all processes
22:11:49 system    | sending SIGTERM to pid 9863
22:11:50 web.1     | 2982 [Thread-1] INFO  org.eclipse.jetty.server.handler.ContextHandler  - stopped o.e.j.w.WebAppContext{/,file:/home/pbu/Devel/IdeaProjects/recaps/webmodule/src/main/webapp/},webmodule/src/main/webapp
22:11:50 web.1     | process terminated
pbu@pbudesk ~/recaps $
So off to deploy it to the cloud! Again, deploy, heroku restart and watch logs... but it doesn't work.

Different way

After initial failure I’ve tried a different approach. I found out that you can register shutdown hooks – very easy thing. To do that, just register a new thread with Runtime.getRuntime().addShutdownHook(Thread) method:

public class Serve {
    public static void main(String[] args) throws Exception {
        Runtime.getRuntime().addShutdownHook(new Thread() {
            @Override
            public void run() {
                System.out.println("Shutting down by shutdown hook");
            }
        });
        int port = Integer.valueOf(System.getenv("PORT"));

        Server jetty = new Server(port);

        WebAppContext context = new WebAppContext();
        context.setContextPath("/");
        String webapp = "webmodule/src/main/webapp";
        context.setWar(webapp);
        context.setResourceBase(webapp);

        jetty.setHandler(context);

        jetty.setGracefulShutdown(1000);
        jetty.setStopAtShutdown(true);

        jetty.start();
        jetty.join();
    }
}

Final test with foreman proves it to be working too, however once again it doesn’t work on Heroku.

At this point, I have no idea how to get rid of that timeout. It is not very important, I just wanted to check whether I can somehow react to it, but to no avail. For now I guess I’ll just contact Heroku, maybe they’ll help. Another option could be trying embedded Tomcat, but maybe at a later time. For now, I have other things to do, like checking out Jelastic.
 

Related Whitepaper:

The Forrester Wave: Enterprise Public Cloud Platforms

One size does not fit all developer needs.

Public cloud platforms are key to unlocking the flexibility, productivity, and economic advantages of cloud computing. Download this Forrester Research, Inc. report to learn: 1. Which public cloud platforms are best suited to each type of development pro 2. Forrester’s evaluation of the 14 most significant public cloud platforms 3. Why Salesforce was cited as a leader in all four categories of AD&D leaders

Get it Now!  

Leave a Reply


eight − 7 =



Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.

Sign up for our Newsletter

15,153 insiders are already enjoying weekly updates and complimentary whitepapers! Join them now to gain exclusive access to the latest news in the Java world, as well as insights about Android, Scala, Groovy and other related technologies.

As an extra bonus, by joining you will get our brand new e-books, published by Java Code Geeks and their JCG partners for your reading pleasure! Enter your info and stay on top of things,

  • Fresh trends
  • Cases and examples
  • Research and insights
  • Two complimentary e-books
Get tutored by the Geeks! JCG Academy is a fact... Join Now
Hello. Add your message here.