The problem or Why are Stuck Threads evil?
WebLogic Server automatically detects when a thread in an execute queue becomes “stuck.” Because a stuck thread cannot complete its current work or accept new work, the server logs a message each time it diagnoses a stuck thread. If all threads in an execute queue become stuck, the server changes its health state to either “warning” or “critical” depending on the execute queue:
- If all threads in the default queue become stuck, the server changes its health state to “critical.” (You can set up the Node Manager application to automatically shut down and restart servers in the critical health state. For more information, see “Node Manager Capabilities” in Configuring and Managing WebLogic Server.)
- If all threads in weblogic.admin.HTTP, weblogic.admin.RMI, or a user-defined execute queue become stuck, the server changes its health state to “warning.”
What you can do to avoid your application completely fail?
- Shut down the application if there are stuck threads in the application. The application is shutdown by bringing it into admin mode. All Work Managers belonging to the application are shut down, and behave as described above.
- Mark the server instance as failed and shut it down it down if there are stuck threads in the server. In a cluster, clustered clients that are connected or attempting to connect will fail over to another cluster member.
How to identify the problem?
Tools to help you with analyzing the Thread Dumps can be:
How to workaround the problem?
Below are some posts on how to create a Work Manager
Test: How to create a Stuck Thread?
How to create a Stuck Thread in order to test your weblogic settings? Put a breakpoint in a backing bean or model method that is called with you request. If you wait in the breakpoint for Stuck Max Thread Time, you notice a Stuck Thread trace will be shown in servers log:
<16 =?? 2011 12:28:22 ?? EET><Error><WebLogicServer><BEA-000337><[STUCK] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "134" seconds working on the request "weblogic.servlet.internal.ServletRequestImpl@6e6f4718[ GET /---/---/----/---/days.xhtml HTTP/1.1 Connection: keep-alive User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.120 Safari/535.2 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Encoding: gzip,deflate,sdch Accept-Language: en-GB,en-US;q=0.8,en;q=0.6 Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3 Cookie: JSESSIONID=DYG5TDTZSnKLTFw5CMMdLCD9sPsZS4Jqlmxj9wdGNyt1BnPcfNrR!-1520792836 ]", which is more than the configured time (StuckThreadMaxTime) of "60" seconds. Stack trace: --------------------------------------------(--------------------.java:83) javax.faces.component.UIComponentBase.encodeBegin(UIComponentBase.java:823) com.sun.faces.renderkit.html_basic.HtmlBasicRenderer.encodeRecursive(HtmlBasicRenderer.java:285) com.sun.faces.renderkit.html_basic.GridRenderer.renderRow(GridRenderer.java:185) com.sun.faces.renderkit.html_basic.GridRenderer.encodeChildren(GridRenderer.java:129) javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:848) org.primefaces.renderkit.CoreRenderer.renderChild(CoreRenderer.java:55) org.primefaces.renderkit.CoreRenderer.renderChildren(CoreRenderer.java:43) org.primefaces.component.fieldset.FieldsetRenderer.encodeContent(FieldsetRenderer.java:95) org.primefaces.component.fieldset.FieldsetRenderer.encodeMarkup(FieldsetRenderer.java:76) org.primefaces.component.fieldset.FieldsetRenderer.encodeEnd(FieldsetRenderer.java:53) javax.faces.component.UIComponentBase.encodeEnd(UIComponentBase.java:878) javax.faces.component.UIComponent.encodeAll(UIComponent.java:1620) javax.faces.render.Renderer.encodeChildren(Renderer.java:168) javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:848) org.primefaces.renderkit.CoreRenderer.renderChild(CoreRenderer.java:55) org.primefaces.renderkit.CoreRenderer.renderChildren(CoreRenderer.java:43) org.primefaces.component.panel.PanelRenderer.encodeContent(PanelRenderer.java:229) org.primefaces.component.panel.PanelRenderer.encodeMarkup(PanelRenderer.java:152)
- Excellent post by Frank Munz: WebLogic Stuck Threads: Creating, Understanding and Dealing with them. Updated for Weblogic 12c. Includes sample app for creating Stuck Thread too.
- Maxence Button excellent post: http://m-button.blogspot.com/2008/07/using-wlst-to-perform-regular.html
Java Platform, Enterprise Edition is a widely used platform for enterprise server programming in the Java programming language.
This book covers exciting recipes on securing, tuning and extending enterprise applications using a Java EE 6 implementation.The book starts with the essential changes in Java EE 6. Then they will dive into the implementation of some of the new features of the JPA 2.0 specification, and look at implementing auditing for relational data stores.They will then look into how they can enable security for their software system using Java EE built-in features as well as using the well-known Spring Security framework. They will then look at recipes on testing various Java EE technologies including JPA, EJB, JSF, and Web services.Next they will explore various ways to extend a Java EE environment with the use of additional dynamic languages as well as frameworks.At the end of the book, they will cover managing enterprise application deployment and configuration, and recipes that will help you debug problems and enhance the performance of your applications.