Enterprise Java

Async Servlet Feature of Servlet 3

Before we jump into understanding what Async Servlet is, let’s try to understand why do we need it. Let’s say we have a Servlet that takes a lot of time to process, something like below.
 
 
 
 
 
 
 
 

LongRunningServlet.java

package com.journaldev.servlet;

import java.io.IOException;
import java.io.PrintWriter;

import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

@WebServlet("/LongRunningServlet")
public class LongRunningServlet extends HttpServlet {
	private static final long serialVersionUID = 1L;

	protected void doGet(HttpServletRequest request,
			HttpServletResponse response) throws ServletException, IOException {
		long startTime = System.currentTimeMillis();
		System.out.println("LongRunningServlet Start::Name="
				+ Thread.currentThread().getName() + "::ID="
				+ Thread.currentThread().getId());

		String time = request.getParameter("time");
		int secs = Integer.valueOf(time);
		// max 10 seconds
		if (secs > 10000)
			secs = 10000;

		longProcessing(secs);

		PrintWriter out = response.getWriter();
		long endTime = System.currentTimeMillis();
		out.write("Processing done for " + secs + " milliseconds!!");
		System.out.println("LongRunningServlet Start::Name="
				+ Thread.currentThread().getName() + "::ID="
				+ Thread.currentThread().getId() + "::Time Taken="
				+ (endTime - startTime) + " ms.");
	}

	private void longProcessing(int secs) {
		// wait for given time before finishing
		try {
			Thread.sleep(secs);
		} catch (InterruptedException e) {
			e.printStackTrace();
		}
	}

}

If we hit above servlet through browser with URL as http://localhost:8080/AsyncServletExample/LongRunningServlet?time=8000, we get response as “Processing done for 8000 milliseconds!!” after 8 seconds. Now if you will look into server logs, you will get following log:

LongRunningServlet Start::Name=http-bio-8080-exec-34::ID=103
LongRunningServlet Start::Name=http-bio-8080-exec-34::ID=103::Time Taken=8002 ms.

So our Servlet Thread was running for ~8+ seconds, although most of the processing has nothing to do with the servlet request or response.

This can leads to Thread Starvation – Since our servlet thread is blocked until all the processing is done, if server gets a lot of requests to process, it will hit the maximum servlet thread limit and further requests will get Connection Refused errors.

Prior to Servlet 3.0, there were container specific solution for these long running threads where we can spawn a separate worker thread to do the heavy task and then return the response to client. The servlet thread returns to the servlet pool after starting the worker thread. Tomcat’s Comet, WebLogic’s FutureResponseServlet and WebSphere’s Asynchronous Request Dispatcher are some of the example of implementation of asynchronous processing.

The problem with container specific solution is that we can’t move to other servlet container without changing our application code, that’s why Async Servlet support was added in Servlet 3.0 to provide standard way for asynchronous processing in servlets.

Asynchronous Servlet Implementation

Let’s see steps to implement async servlet and then we will provide async supported servlet for above example.

  1. First of all the servlet where we want to provide async support should have @WebServlet annotation with asyncSupported value as true.
  2. Since the actual work is to be delegated to another thread, we should have a thread pool implementation. We can create thread pool using Executors framework and use servlet context listener to initiate the thread pool.
  3. We need to get instance of AsyncContext through ServletRequest.startAsync() method. AsyncContext provides methods to get the ServletRequest and ServletResponse object references. It also provides method to forward the request to another resource using dispatch() method.
  4. We should have a Runnable implementation where we will do the heavy processing and then use AsyncContext object to either dispatch the request to another resource or write response using ServletResponse object. Once the processing is finished, we should call AsyncContext.complete() method to let container know that async processing is finished.
  5. We can add AsyncListener implementation to the AsyncContext object to implement callback methods – we can use this to provide error response to client incase of error or timeout while async thread processing. We can also do some cleanup activity here.

Once we will complete our project for Async servlet Example, it will look like below image.

Async-Servlet-Example

Initializing Worker Thread Pool in Servlet Context Listener

AppContextListener.java

package com.journaldev.servlet.async;

import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import javax.servlet.annotation.WebListener;

@WebListener
public class AppContextListener implements ServletContextListener {

	public void contextInitialized(ServletContextEvent servletContextEvent) {

		// create the thread pool
		ThreadPoolExecutor executor = new ThreadPoolExecutor(100, 200, 50000L,
				TimeUnit.MILLISECONDS, new ArrayBlockingQueue<Runnable>(100));
		servletContextEvent.getServletContext().setAttribute("executor",
				executor);

	}

	public void contextDestroyed(ServletContextEvent servletContextEvent) {
		ThreadPoolExecutor executor = (ThreadPoolExecutor) servletContextEvent
				.getServletContext().getAttribute("executor");
		executor.shutdown();
	}

}

The implementation is pretty straight forward, if you are not familiar with Executors framework, please read Thread Pool Executor.

For more details about listeners, please read Servlet Listener Tutorial.

Worker Thread Implementation

AsyncRequestProcessor.java

package com.journaldev.servlet.async;

import java.io.IOException;
import java.io.PrintWriter;

import javax.servlet.AsyncContext;

public class AsyncRequestProcessor implements Runnable {

	private AsyncContext asyncContext;
	private int secs;

	public AsyncRequestProcessor() {
	}

	public AsyncRequestProcessor(AsyncContext asyncCtx, int secs) {
		this.asyncContext = asyncCtx;
		this.secs = secs;
	}

	@Override
	public void run() {
		System.out.println("Async Supported? "
				+ asyncContext.getRequest().isAsyncSupported());
		longProcessing(secs);
		try {
			PrintWriter out = asyncContext.getResponse().getWriter();
			out.write("Processing done for " + secs + " milliseconds!!");
		} catch (IOException e) {
			e.printStackTrace();
		}
		//complete the processing
		asyncContext.complete();
	}

	private void longProcessing(int secs) {
		// wait for given time before finishing
		try {
			Thread.sleep(secs);
		} catch (InterruptedException e) {
			e.printStackTrace();
		}
	}
}

Notice the use of AsyncContext and it’s usage in getting request and response objects and then completing the async processing with complete() method call.

AsyncListener Implementation

AppAsyncListener.java

package com.journaldev.servlet.async;

import java.io.IOException;
import java.io.PrintWriter;

import javax.servlet.AsyncEvent;
import javax.servlet.AsyncListener;
import javax.servlet.ServletResponse;
import javax.servlet.annotation.WebListener;

@WebListener
public class AppAsyncListener implements AsyncListener {

	@Override
	public void onComplete(AsyncEvent asyncEvent) throws IOException {
		System.out.println("AppAsyncListener onComplete");
		// we can do resource cleanup activity here
	}

	@Override
	public void onError(AsyncEvent asyncEvent) throws IOException {
		System.out.println("AppAsyncListener onError");
		//we can return error response to client
	}

	@Override
	public void onStartAsync(AsyncEvent asyncEvent) throws IOException {
		System.out.println("AppAsyncListener onStartAsync");
		//we can log the event here
	}

	@Override
	public void onTimeout(AsyncEvent asyncEvent) throws IOException {
		System.out.println("AppAsyncListener onTimeout");
		//we can send appropriate response to client
		ServletResponse response = asyncEvent.getAsyncContext().getResponse();
		PrintWriter out = response.getWriter();
		out.write("TimeOut Error in Processing");
	}

}

Notice the implementation of onTimeout() method where we are sending timeout response to client.

Here is the implementation of our async servlet, notice the use of AsyncContext and ThreadPoolExecutor for processing.

AsyncLongRunningServlet.java

package com.journaldev.servlet.async;

import java.io.IOException;
import java.util.concurrent.ThreadPoolExecutor;

import javax.servlet.AsyncContext;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

@WebServlet(urlPatterns = "/AsyncLongRunningServlet", asyncSupported = true)
public class AsyncLongRunningServlet extends HttpServlet {
	private static final long serialVersionUID = 1L;

	protected void doGet(HttpServletRequest request,
			HttpServletResponse response) throws ServletException, IOException {
		long startTime = System.currentTimeMillis();
		System.out.println("AsyncLongRunningServlet Start::Name="
				+ Thread.currentThread().getName() + "::ID="
				+ Thread.currentThread().getId());

		request.setAttribute("org.apache.catalina.ASYNC_SUPPORTED", true);

		String time = request.getParameter("time");
		int secs = Integer.valueOf(time);
		// max 10 seconds
		if (secs > 10000)
			secs = 10000;

		AsyncContext asyncCtx = request.startAsync();
		asyncCtx.addListener(new AppAsyncListener());
		asyncCtx.setTimeout(9000);

		ThreadPoolExecutor executor = (ThreadPoolExecutor) request
				.getServletContext().getAttribute("executor");

		executor.execute(new AsyncRequestProcessor(asyncCtx, secs));
		long endTime = System.currentTimeMillis();
		System.out.println("AsyncLongRunningServlet End::Name="
				+ Thread.currentThread().getName() + "::ID="
				+ Thread.currentThread().getId() + "::Time Taken="
				+ (endTime - startTime) + " ms.");
	}

}

Run Async Servlet

Now when we will run above servlet with URL as http://localhost:8080/AsyncServletExample/AsyncLongRunningServlet?time=8000 we get the same response and logs as:

AsyncLongRunningServlet Start::Name=http-bio-8080-exec-50::ID=124
AsyncLongRunningServlet End::Name=http-bio-8080-exec-50::ID=124::Time Taken=1 ms.
Async Supported? true
AppAsyncListener onComplete

If we run with time as 9999, timeout occurs and we get response at client side as “TimeOut Error in Processing” and in logs:

AsyncLongRunningServlet Start::Name=http-bio-8080-exec-44::ID=117
AsyncLongRunningServlet End::Name=http-bio-8080-exec-44::ID=117::Time Taken=1 ms.
Async Supported? true
AppAsyncListener onTimeout
AppAsyncListener onError
AppAsyncListener onComplete
Exception in thread "pool-5-thread-6" java.lang.IllegalStateException: The request associated with the AsyncContext has already completed processing.
	at org.apache.catalina.core.AsyncContextImpl.check(AsyncContextImpl.java:439)
	at org.apache.catalina.core.AsyncContextImpl.getResponse(AsyncContextImpl.java:197)
	at com.journaldev.servlet.async.AsyncRequestProcessor.run(AsyncRequestProcessor.java:27)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
	at java.lang.Thread.run(Thread.java:680)

Notice that servlet thread finished execution quickly and all the major processing work is happening in other thread.

Thats all for Async Servlet, I hope you liked it.

 

Reference: Async Servlet Feature of Servlet 3 from our JCG partner Pankaj Kumar at the Developer Recipes blog.
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

10 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
satish kumar
satish kumar
10 years ago

great job

Vijay
Vijay
10 years ago

Nice article.
Some questions
– Is the long running processing handed off to another thread just to free up the servlet thread or unblock the client as well from the call? Or does the client still block until a reply is sent?
– Is it ok to create your own threads inside a servlet container? Somehow it has been discouraged to do so I thought. In fact thats the reason why JEE7 has some facility to do so in controlled manner (Managed thread pool or something).

Thanks.

Evan
Evan
9 years ago

Hi, I think this asyncSupport feature is not useful. It just shift the work to another thread. We can also achieve this by adding more request threads.

Rehman
Rehman
7 years ago
Reply to  Evan

usually servlet can handle only n HTTP Requests, and (n+1)th request get on hold till one request gets completed let say n=2 (just for explanation, actually n going to be much high value), which mean if have then requests R0, R1, R2 … R09 made in parallel then out of these ten requests only two will be served and remaining going to wait lets say R0, R1 are heavy request which taking long time to process and remaining are very short which takes very short time. if asynchSupport not been used then if webserver scheduled R0, R1 first then all… Read more »

Murali
Murali
9 years ago

Nice article.
So here heavy code is moved to worker threads and main servlet listening thread will return to pool. In this way high availability of thread you are achieving. Is Async feature is only to handle more number of requests ? My question is , will client will be blocked until servlet sends response or client will do its pending job and will be updated only when servlet send response.

Xgeoff
Xgeoff
9 years ago

The method for implementing async functions is so awful. This is why node.js is so popular. I know you didn’t write the spec, and you’re just illustrating how it works, but this is awful. Look at how much code you need to write for a simple async servlet.awful, just awful.

Valery-Sh
Valery-Sh
8 years ago
Reply to  Xgeoff

Try do the same long task in node.js (it’s blocking task). There are also Groovy and Scala ( and other langs for JVM ) where the code may look like in JavaScript and even better.

pranav
pranav
8 years ago

Thanks, very good explanations.

Sheroo Pratap
Sheroo Pratap
8 years ago

In AsyncListener were getting AsyncEvent from which we can get ServletsRequest and ServletsResponse Object, how i can get HttpServletRequest and HttpServletResponse object on same trigger

Sheroo Pratap
Sheroo Pratap
8 years ago

on the same trigger i need to send response to the client.

Back to top button