Featured FREE Whitepapers

What's New Here?


Introduction to BTrace for Java applications

The aim of this article is to learn how to dynamically trace/observe running Java applications (JDK 6+) using BTrace without changing the code and configuration params of the applications. What is BTrace? BTrace is an open source project that was started in 2007 and was originally owned by two people – A.Sundararajan and K. Balasubramanian. It gained fame thanks to Java One 2008 conference. BTrace helps us to pinpoint complicated code problems in the application. The problems that can be solved using BTrace include code bugs, unpredictable flows, concurrency problems and performance problems that happen under specific and usually hard to reproduce circumstances. BTrace dynamically (without restarting application) instruments (changes) bytecode of an application in the way that programmer defines. The goal of the instrumentation is to see what happens in a specific area of the code. If used beyond this scope it may harm applications’ flow and therefore is forbidden by a validator. For example let’s try to solve following problem – an important file gets deleted occasionally once a day. We want to find the code that does this. Therefore we would like to change ‘delete’ method of java.io.File and print a stack trace of the calling thread if the file name fits. With BTrace we can do this by writing a short and straightforward Java code. The following schema displays principles of BTrace work.In Target JVM there is a dynamically inserted BTrace agent (using attach API). BTrace client (either BTrace Command Line or Visual VM with BTrace plugin) sends commands to the agent and gets responses. BTrace agent instruments classes that are being loaded into Target JVM and reloads classes that have already been loaded. Writing a Tracing script Writing BTrace scripts is pretty simple and straightforward and has concepts similar to Aspect Oriented Programming concepts. import com.sun.btrace.annotations.*; import com.sun.btrace.BTraceUtils;@BTrace public class HelloWorld {@OnMethod(clazz="java.io.File",method="") public static void onNewFileCreated(String fileName) { BTraceUtils.println("New file is being created"); BTraceUtils.println(fileName); }Each BTrace script consists of Probes (pointcuts in Aspects slang) and Actions (advices). A probe defines when the instrumentation should be executed. Action defines the instrumentation. Probe can consist of the following:Method entry/exit Line number Field updated/accessed Method call/return (within specified method(s)) Exception throw (before) Synchronization entry/exit Timer Constructor entryAbilities and limitations of BTrace AbilitiesDynamically connect to any java6 + application and run any(*) codeLimitationsMust run with the same user as the traced application is running with – due to security concerns Supports Hotspot JVM only Is compiled separately from the target application – is not familiar with application classes May crush target application – I ran BTrace on our longevity setup which was heavily loaded with many Tracing scripts. Target application didn’t crush even once.Advanced BTrace BTrace community consists de facto of a single developer that works regularly on the project. Therefore don’t expect too much improvement and many new features in the next releases. (This is a good opportunity to contribute to the project as each developer will enhance the development significantly). Documentation of the project should be significantly improved – I found myself guessing many times while integrating this framework into the existing frameworks at my job. BTrace forum is a good way to get answers about the framework. In order to understand how the magic of BTrace works, knowledge in three fields is required – ‘Java Agents Development’, ‘Bytecode manipulation in java’ and ‘Java attach api’. I may write about some of these fields in the future. Related linked http://www.parleys.com/#st=5&id=1618&sl=1 http://kenai.com/projects/btrace Reference: Introduction to BTrace from our JCG partner Art Gourevitch at the The Art of Java blog....

JavaFX 2 GameTutorial Part 4

Introduction This is part four of a six part series related to a JavaFX 2 Game Tutorial. If you’ve missed Part 1, Part 2, or Part 3, I encourage you to go through them before beginning this tutorial. To recap, in Part 3 I gave you a little history of the many classic arcade style games and the different input devices that were used. I then showed you how to create a simple game similar to the famous arcade ‘Asteroids’. The controls (movement of the ship) were, however, more similar to those of the PC game ‘Star Craft’. In Part 3, you should have a good understanding of how to receive input from your keyboard and mouse.Figure 1 JavaFX 2 Game Tutorial Part 4This tutorial is about tweaking Part 2’s game engine and updating the existing ‘Asteroids’ style game from Part 3 to handle collision detection. In this tutorial, I will briefly talk about sprites and how to handle collision detection. The spaceship will now have the ability to generate a force field to protect itself from enemies and asteroids. This is reminiscent of the classic arcade ‘Asteroids Deluxe’. If you want to run the demo, scroll down and click on the WebStart button below. Please read the requirements before launching the game. What is a Sprite? According to Wikipedia, “a sprite is a two-dimensional image or animation that is integrated into a larger scene.” From the Java gaming world view, a sprite is an object containing image frames and additional data based on the context of an actor to be animated onto the scene area. In the days of Walt Disney, when cartoons were drawn with a pencil and paper, the artist produced many drawings that became animations. This example points to the creation of the flip book. I’m sure you’ve created flip books as a kid. I know I did. I used to doodle and make cool animations with all the corners of my notebooks. In our Asteroid type game, I created a sprite object which contains all of the images (ImageView) of the ship pre-rotated just like a flip book. To animate the ship turning, I made the current frame visible and the rest of the frames not visible. Similar to a flip book, it will appear to be rotated about its center (pivot) point. A sprite can also contain other information, such as velocity or health points.  Collision Detection When actors or sprites animate across the scene, the game engine will check each sprite against other sprites to determine if they have collided into one another. This process should be very efficient especially when you have numerous sprites moving about the screen. There are tradeoffs when it comes to being efficient. Because each cycle in the game loop will check for collision, being more accurate usually degrades your performance. Many games will use the bounding region of the image to determine if two sprites have collided into one another. Some games use rectangles as bounding regions. Shown below in figure 2 are two sprites colliding:Figure 2 Bounding box as a rectangular collision region.I’m sure you’ll know by now that most actors (images) in games don’t appear rectangular when pixels surrounding the actor are transparent. However, the actor or image is indeed rectangular even if the pixels are transparent or not.Figure 3 depicts an actor imageThose games which use rectangular bounding regions usually make the bounding box inscribed in the sprite’s image. Shown below in figure 4 two rectangular bounding regions (orange and green) are inscribed in the spaceship image.Figure 4 Two rectangles used as collision bounding boxes.I’m sure you will notice that the nose tip of the ship and wings are not covered by either bounding box. This means that when an asteroid overlaps the unbounded region of the sprite the collision will not occur. Some games use this strategy; you will notice that the sprites’ rectangular bounding regions are small and placed in key areas of the sprite image. Greater precision will be found with better algorithms for polygons and other non-rectangular shapes. In this blog entry, I basically use circles as bounding regions and not rectangles. I could have made each sprite contain an array of collision shapes, but instead, I chose to have only one collision region for each sprite. Each collision region will be a circle shape on the scene graph. For the spaceship, I inscribed the circle based on the center point of the ship with the radius extended to the cock pit of the ship. Shown below in figure 5 the bounded circular collision area of the ship is depicted as a red circle.Figure 5 The ship’s collision region.I chose a circle as the bounding region because of the relative ease to determine the collision of two objects based on the distance formula (Pythagorean theorem) which only requires each sprite’s bounding region’s center point and their radii. After calculating the distance based on the two center points, you will compare the result to see if it is less than or equal to the sum of the two radii. If the result does indeed come out to be less than or equal to the sum of the two radii then a collision has occurred. Figure 6 depicts how the distance formula relates to the two center points of circular bounding regions.Figure 6 distance formula between two center points.The following code creates the main game loop from the GameWorld class: @Override public void handle(javafx.event.ActionEvent event) {// update actors updateSprites();// check for collision checkCollisions();// removed dead things cleanupSprites();} The code below creates the checkCollision() method from the GameWorld class: protected void checkCollisions() { // check other sprite's collisions spriteManager.resetCollisionsToCheck(); // check each sprite against other sprite objects. for (Sprite spriteA : spriteManager.getCollisionsToCheck()) { for (Sprite spriteB : spriteManager.getAllSprites()) { if (handleCollision(spriteA, spriteB)) { // The break helps optimize the collisions // The break statement means one object only hits another // object as opposed to one hitting many objects. // To be more accurate comment out the break statement. break; } } } }The derived Game World (TheExpanse) class’ implementation of its handleCollision() method: /** * How to handle the collision of two sprite objects. * * @param spriteA Sprite from the first list. * @param spriteB Sprite from the second list. * @return boolean returns a true if the two sprites have collided otherwise false. */ @Override protected boolean handleCollision(Sprite spriteA, Sprite spriteB) { if (spriteA != spriteB) { if (spriteA.collide(spriteB)) {if (spriteA != myShip) { spriteA.handleDeath(this); } if (spriteB != myShip) { spriteB.handleDeath(this); } } }return false; } The Sprite Class’ default implementation of its collide() method using the distance formula: public boolean collide(Sprite other) {if (collisionBounds == null || other.collisionBounds == null) { return false; }// determine it's size Circle otherSphere = other.collisionBounds; Circle thisSphere = collisionBounds; Point2D otherCenter = otherSphere.localToScene(otherSphere.getCenterX(), otherSphere.getCenterY()); Point2D thisCenter = thisSphere.localToScene(thisSphere.getCenterX(), thisSphere.getCenterY()); double dx = otherCenter.getX() - thisCenter.getX(); double dy = otherCenter.getY() - thisCenter.getY(); double distance = Math.sqrt(dx * dx + dy * dy); double minDist = otherSphere.getRadius() + thisSphere.getRadius();return (distance < minDist); } The Sprite Class’ default implementation of its handleDeath() method: public void handleDeath(GameWorld gameWorld) { gameWorld.getSpriteManager().addSpritesToBeRemoved(this); } The Atom (an asteroid or missile) class will override the handleDeath() method: public void handleDeath(GameWorld gameWorld) { implode(gameWorld); super.handleDeath(gameWorld); } JavaFX 2 Sprite and Collision Demo This simple demo game will be a mix between StarCraft and Asteroids. When using the mouse to navigate the ship, you will notice that the controls will resemble StarCraft’s Battle Cruiser. The objective is to fire your weapon at the spheres before they hit your spaceship or other spheres which implode upon impact. Because this is a simple tutorial or even a game in its early stages of development, the game doesn’t keep track of the score. I encourage you to go to GitHub to download the code and enhance the game. For the sake of brevity, I will not be showing all of the code changes, but I trust you will visit GitHub here: https://github.com/carldea/JFXGen for all the demos and source code. Requirements:Java 7 or later JavaFX 2.1 or later Windows XP or later (Should be available soon for Linux/MacOS)A simple Asteroid type game called ‘The Expanse’. Instructions:Right mouse click (on Windows) to fly ship. Left mouse click (left click on Windows mouse) to fire weapon. Key press ’2? to change to large missiles. (blue circular projectiles) Other key press defaults to smaller missiles. (red circular projectiles) Space bar key press will toggle a force field to protect the ship from enemies and asteroids.Click on the Launch button below to start the demo: Continue to Part 5 of this tutorial. Related articles Definition of a sprite: http://en.wikipedia.org/wiki/Sprite_%28computer_graphics%29 Walt Disney: http://en.wikipedia.org/wiki/Walt_Disney How to make a flip book: http://www.bitrebels.com/design/how-to-create-a-flip-book/ JavaFX’s ImageView : http://docs.oracle.com/javafx/2/api/javafx/scene/image/ImageView.html Collision detection: http://zetcode.com/tutorials/javagamestutorial/collision/ AABBs Collision detection in Java: http://www.youtube.com/watch?v=JIxV-LXqa1g Pythagorean theorem: http://en.wikipedia.org/wiki/Pythagorean_theorem Distance formula: http://en.wikipedia.org/wiki/Distance Serious game of Asteroids Deluxe (Youtube): http://www.youtube.com/watch?v=6DG-GJENHgg Reference: JavaFX 2 GameTutorial Part 4 from our JCG partner Carl Dea at the Carl’s FX Blog blog....

Where do Security Requirements come from?

One of the problems in building a secure application is that it’s not always clear what the security requirements are and where they are supposed to come from. Are security requirements supposed to come from the customer? Are they specified in the regulatory and compliance environment? Or are they implicit in the type of application that you are building – an online web store, real-time industrial control, a payroll system, a military weapons command system, an enterprise mobile app – and in the platform and technologies that you are using to build the system? The answer is: all of the above. In a recent post on how to manage security requirements in Agile projects, Rohit Sethi offers a useful way of looking at security requirements for any kind of software development work, Agile or not. He breaks security requirements down into 2 types: Security Stories A need to do, or not do, something in the system, which can be expressed in a story or a change request like other features. There’s no one way to get these requirements. Some requirements will come directly from the customer or operations or infosec or compliance or a partner, stated explicitly – if not always completely. You need to look harder for other security requirements, take time to understand the compliance and regulatory and legal requirements for the business and wade through the legalese to understand what your obligations are – both in the design of the system, and in how you design and build and operate it. Other requirements come from the basic patterns of the system that you are building. Any team developing an online web store or an enterprise mobile app should have a good understanding of the basic security problems that they may need to take care of – different kinds of systems have well-understood general requirements for authentication, authorization, auditing and logging, data privacy and confidentiality. Security stories are tracked and managed and prioritized like the rest your development work – the trick is to make sure that they are taken seriously by the team and by the customer, but at least they are visible to everyone on the project. Technical Security Constraints But there are other requirements that aren’t clear to the business and other stake holders – they’re hidden from outside of the development team, and sometimes hidden from the people on the development team as well. These are the things that developers have to do, or not do, to design and build software in a secure way given the architecture and the platform and technologies that they are using. Unlike requirements which are part of the problem space, these constraints are part of the solution space, part of the job of developing software properly. Some technical constraints are obvious to good, experienced developers who know how to write good, safe software: defensive programming, logging and auditing, not hard-coding passwords or storing secrets in the clear, using prepared statements for relational database access, and so on. Other technical constraints are much more obscure and specialised – developers may not understand or even be aware of these constraints unless they have training in secure software development and a good technical knowledge of the platform and language(s) and frameworks that they are working with, how to use them properly, what their security weak points are. Understanding Security requirements is part of your job as a Developer In Beautiful Security: Forcing Firms to Focus, Jim Routh explains that software security, like software quality, is generally implied in requirements for an application. In the same way that we expect a burger to be served hot and fresh, not stale and at room temperature, nobody should have to explicitly state that the software has to work, that you can’t deliver junk that is full of bugs and isn’t reliable, or code that isn’t secure. “… clearly, the omission of security from explicit requirements is no reason to believe that customers don’t care about it, or don’t understand its importance.” It’s up to us as developers to understand the business problems that we’re trying to solve and what our customers and partners need from a system – including understanding and confirming their security requirements. It’s up to us as developers to understand what kind of system we are building, and what the security and reliability and safety requirements for that kind of system should be. It’s up to us as developers to understand the platform and tools that we’re using, and how to use them properly. It’s up to us to find and understand security requirements and security constraints for a system. We can’t depend on other people to tell us these things, to make all of the steps explicit. This is our job, it’s what we get paid for. Reference: Where do Security Requirements come from? from our JCG partner Jim Bird at the Building Real Software blog....

java.lang.NoClassDefFoundError: How to resolve – Part 2

This article is part 2 of our NoClassDefFoundError troubleshooting series. Take a look at part 1. It will focus and describe the simplest type of NoClassDefFoundError problem. This article is ideal for Java beginners and I highly recommend that you compile and run the sample Java program yourself. The following writing format will be used going forward and will provide you with: - Description of the problem case and type of NoClassDefFoundError - Sample Java program “simulating” the problem case - ClassLoader chain view - Recommendations and resolution strategies NoClassDefFoundError problem case 1 – missing JAR file The first problem case we will cover is related to a Java program packaging and / or classpath problem. A typical Java program can include one or many JAR files created at compile time. NoClassDefFoundError can often be observed when you forget to add JAR file(s) containing Java classes referenced by your Java or Java EE application. This type of problem is normally not hard to resolve once you analyze the Java Exception and missing Java class name. Sample Java program The following simple Java program is split as per below: - The main Java program NoClassDefFoundErrorSimulator - The caller Java class CallerClassA - The referencing Java class ReferencingClassA - A util class for ClassLoader and logging related facilities JavaEETrainingUtil This program is simple attempting to create a new instance and execute a method of the Java class CallerClassA which is referencing the class ReferencingClassA. It will demonstrate how a simple classpath problem can trigger NoClassDefFoundError. The program is also displaying detail on the current class loader chain at class loading time in order to help you keep track of this process. This will be especially useful for future and more complex problem cases when dealing with larger class loader chains. package org.ph.javaee.training1;import org.ph.javaee.training.util.JavaEETrainingUtil;/** * NoClassDefFoundErrorTraining1 * @author Pierre-Hugues Charbonneau * */ public class NoClassDefFoundErrorSimulator { /** * @param args */ public static void main(String[] args) { System.out.println("java.lang.NoClassDefFoundError Simulator - Training 1"); System.out.println("Author: Pierre-Hugues Charbonneau"); System.out.println("http://javaeesupportpatterns.blogspot.com"); // Print current Classloader context System.out.println("\nCurrent ClassLoader chain: "+JavaEETrainingUtil.getCurrentClassloaderDetail()); // 1. Create a new instance of CallerClassA CallerClassA caller = new CallerClassA(); // 2. Execute method of the caller caller.doSomething(); System.out.println("done!"); } }package org.ph.javaee.training1;import org.ph.javaee.training.util.JavaEETrainingUtil;/** * CallerClassA * @author Pierre-Hugues Charbonneau * */ public class CallerClassA { private final static String CLAZZ = CallerClassA.class.getName(); static { System.out.println("Classloading of "+CLAZZ+" in progress..."+JavaEETrainingUtil.getCurrentClassloaderDetail()); } public CallerClassA() { System.out.println("Creating a new instance of "+CallerClassA.class.getName()+"..."); } public void doSomething() { // Create a new instance of ReferencingClassA ReferencingClassA referencingClass = new ReferencingClassA(); } }package org.ph.javaee.training1;import org.ph.javaee.training.util.JavaEETrainingUtil;/** * ReferencingClassA * @author Pierre-Hugues Charbonneau * */ public class ReferencingClassA {private final static String CLAZZ = ReferencingClassA.class.getName(); static { System.out.println("Classloading of "+CLAZZ+" in progress..."+JavaEETrainingUtil.getCurrentClassloaderDetail()); } public ReferencingClassA() { System.out.println("Creating a new instance of "+ReferencingClassA.class.getName()+"..."); } public void doSomething() { //nothing to do... } }package org.ph.javaee.training.util;import java.util.Stack; import java.lang.ClassLoader;/** * JavaEETrainingUtil * @author Pierre-Hugues Charbonneau * */ public class JavaEETrainingUtil { /** * getCurrentClassloaderDetail * @return */ public static String getCurrentClassloaderDetail() { StringBuffer classLoaderDetail = new StringBuffer(); Stack<ClassLoader> classLoaderStack = new Stack<ClassLoader>(); ClassLoader currentClassLoader = Thread.currentThread().getContextClassLoader(); classLoaderDetail.append("\n-----------------------------------------------------------------\n"); // Build a Stack of the current ClassLoader chain while (currentClassLoader != null) { classLoaderStack.push(currentClassLoader); currentClassLoader = currentClassLoader.getParent(); } // Print ClassLoader parent chain while(classLoaderStack.size() > 0) { ClassLoader classLoader = classLoaderStack.pop(); // Print current classLoaderDetail.append(classLoader); if (classLoaderStack.size() > 0) { classLoaderDetail.append("\n--- delegation ---\n"); } else { classLoaderDetail.append(" **Current ClassLoader**"); } } classLoaderDetail.append("\n-----------------------------------------------------------------\n"); return classLoaderDetail.toString(); } }Problem reproduction In order to replicate the problem, we will simply “voluntary” omit one of the JAR files from the classpath that contains the referencing Java class ReferencingClassA. The Java program is packaged as per below: - MainProgram.jar (contains NoClassDefFoundErrorSimulator.class and JavaEETrainingUtil.class ) -CallerClassA.jar (contains CallerClassA.class ) - ReferencingClassA.jar (contains ReferencingClassA.class ) Now, let’s run the program as is: Recommendations and resolution strategies ## Baseline (normal execution) ..\bin>java -classpath CallerClassA.jar;ReferencingClassA.jar;MainProgram.jar org.ph.javaee.training1.NoClassDefFoundErrorSimulatorjava.lang.NoClassDefFoundError Simulator - Training 1 Author: Pierre-Hugues Charbonneauhttp://javaeesupportpatterns.blogspot.comCurrent ClassLoader chain: ----------------------------------------------------------------- sun.misc.Launcher$ExtClassLoader@17c1e333 --- delegation --- sun.misc.Launcher$AppClassLoader@214c4ac9 **Current ClassLoader** -----------------------------------------------------------------Classloading of org.ph.javaee.training1.CallerClassA in progress... ----------------------------------------------------------------- sun.misc.Launcher$ExtClassLoader@17c1e333 --- delegation --- sun.misc.Launcher$AppClassLoader@214c4ac9 **Current ClassLoader** -----------------------------------------------------------------Creating a new instance of org.ph.javaee.training1.CallerClassA... Classloading of org.ph.javaee.training1.ReferencingClassA in progress... ----------------------------------------------------------------- sun.misc.Launcher$ExtClassLoader@17c1e333 --- delegation --- sun.misc.Launcher$AppClassLoader@214c4ac9 **Current ClassLoader** -----------------------------------------------------------------Creating a new instance of org.ph.javaee.training1.ReferencingClassA... done!What happened? The removal of the ReferencingClassA.jar, containing ReferencingClassA, did prevent the current class loader to locate this referencing Java class at runtime leading to ClassNotFoundException and NoClassDefFoundError. This is the typical Exception that you will get if you omit JAR file(s) from your Java start-up classpath or within an EAR / WAR for Java EE related applications. ClassLoader view Now let’s review the ClassLoader chain so you can properly understand this problem case. As you saw from the Java program output logging, the following Java ClassLoaders were found: Classloading of org.ph.javaee.training1.CallerClassA in progress... ----------------------------------------------------------------- sun.misc.Launcher$ExtClassLoader@17c1e333 --- delegation --- sun.misc.Launcher$AppClassLoader@214c4ac9 **Current ClassLoader** -----------------------------------------------------------------** Please note that the Java bootstrap class loader is responsible to load the core JDK classes and is written in native code ** ## sun.misc.Launcher$AppClassLoader This is the system class loader responsible to load our application code found from the Java classpath specified at start-up. ##sun.misc.Launcher$ExtClassLoader This is the extension class loader responsible to load code in the extensions directories (<java_home>/lib/ext, or any other directory specified by the java.ext.dirs system property). As you can see from the Java program logging output, the extension class loader is the actual super parent of the system class loader. Our sample Java program was loaded at the system class loader level. Please note that this class loader chain is very simple for this problem case since we did not create child class loaders at this point. This will be covered in future articles. Recommendations and resolution strategies Now find below my recommendations and resolution strategies for NoClassDefFoundError problem case 1: - Review the java.lang.NoClassDefFoundError error and identify the missing Java class  -Verify and locate the missing Java class from your compile / build environment  -Determine if the missing Java class is from your application code, third part API or even the Java EE container itself. Verify where the missing JAR file(s) is / are expected to be found - Once found, verify your runtime environment Java classpath for any typo or missing JAR file(s) - If the problem is triggered from a Java EE application, perform the same above steps but verify the packaging of your EAR / WAR file for missing JAR and other library file dependencies such as MANIFEST. Please feel free to post any question or comment. The part 3 will be available shortly. Reference: java.lang.NoClassDefFoundError: How to resolve – Part 2 from our JCG partner Pierre-Hugues Charbonneau at the Java EE Support Patterns & Java Tutorial blog....

The importance of measuring your IT operations

How could one tell if an IT organisation is working well? A simple measure could be to check the delivered business functionalities versus the requested ones; if IT delivered all that was requested by the business one could assume that the IT organisation works well; viceversa if IT didn’t deliver the majority/totality of what was requested it seems clear that it’s not working well. However the number of actual versus projected deliverables is a rather superficial way to measure the effectiveness of an IT department; what measures the effectiveness of an IT organisation is the answer to the following question: “Is the IT department under consideration performing at its best?”. Suddendly the number of deliverables loses significance; how does one measure whether an IT organisation is performing at its best? I believe the answer lies in the accurate measurement of IT Operations (IT OPS). In fact only thorough data collection allows an IT organisation to answer two simple questions:How is IT performing? (Where are we at now?) How would we like IT to perform? (Where do we want to go?)Regardless of the IT methodology used, the answer to the two questions above requires meticolous measurement of IT OPS.The challenge now is to identify the metrics that need to be measured in order to answer the two questions above. In this first article I’m just going to suggest some of them, while in subsequent articles I’ll attempt to describe each one of them in more detail and provide a framework on how one could go about measuring them.I think that the following are fundamental metrics in IT OPS:Feature Average Lead Time (FALT). This is the average number of days it takes for a request, from the moment was originated, to hit production. Development Cost for a Deployed Feature (DECODEF). This is the development cost of getting a feature deployed into production. It comes down to bare people cost. Keep The Lights On (KTLO) cost for an information system. This is the infrastructure cost for a particular service. Number and Cost of Production Bugs after a release (NCPB). This measure is meant to provide both a figure around quality and a figure around quantity. Cost of mainteining Legacy Systems (COLS). This is the people cost in interacting with Legacy systems for issue investigation, small enhancements, production bugs Cost of Evergreening (COE). This is the cost of keeping the various technologies to a version which is supported by third party vendors. Business Value of each Deliverable to production (BUVD). This is perhaphs the most difficult figure to get since it’s not an exact science (how would one quantify the business value of Evergreening?)Why is measuring IT OPS so important? Well, my answer is that while deliverables stats tell you whether your IT is delivering, IT OPS lay the foundation for a Continuous Improvement culture and it’s only through continuous improvement that an IT organisation can reach operational excellence.In subsequent articles I’ll try to delve in detail on each IT OPS measurement; I will try to provide a framework that can be used as is or customised to obtain sensisble figures around the above measurements.Reference: The importance of measuring your IT operations from our JCG partner Marco Tedone at the Marco Tedone’s blog blog....

ElasticMQ message replication with JGroups

ElasticMQ is a messaging server, with a Scala, Java, and an Amazon SQS-compatible interface. It supports guaranteed messaging by replicating the messages across a cluster of servers, and message persistence with journalling.Message replication is one of the core features of ElasticMQ. However if you look at the code, it’s only a handful of classes, the longest having 76 lines (remember that this is Scala, though ;) ). That is because ElasticMQ uses JGroups as the underlying communication library. JGroups is quite old, especially for a Java library – the first release was in 1999 (!). But it’s quite far from being outdated and obsolete – it has a nice API, works without problems, has a good community; and as any Java library cooperates well with Scala.JGroups has a lot of useful features:reliable multicast cluster management failure detection node discovery many years of performance improvementswhich are extensively used for implementing replication in ElasticMQ. Below is a summary of how it’s done.How does an ElasticMQ cluster work?In a single ElasticMQ cluster one node is always the master. You can only execute operations against this node. The results of each operation are then replicated to the other members. There are a couple of options related to blocking; the replication can be fully asynchronous, or it can wait until at least one or all nodes ack the operation. To make sure that in case of cluster partitions the same messages aren’t received from different partitions, only the partition which has at least half+1 of the nodes operations is active.The central concept in ElasticMQ is the message storage. A storage executes commands (such as a send message command, delete message command etc.). The replication layer is just a wrapper around any other storage. Note though, that we can only replicate the resulting storage mutations (so after the command is executed), not the original command itself. For example, if the command is “receive a message”, the results of executing it may be different on each machine. Hence we replicate only the change of the message visibility (in ElasticMQ, similarly to Amazon SQS, if a message is received it is blocked from a subsequent receive for a specified period of time), if receiving a message succeedes. You can see this basic logic in JGroupsReplicatedStorage.Initializing the clusterBefore we get to the replication itself though, the first thing to do is to initialize the cluster; that is done in ReplicatedStorageConfigurator. As a parameter there, we need a JGroups configuration file, which is a stack of protocols. You don’t really need to know what each of these protocols do and what all those configuration parameters mean. The two most useful are udp.xml and tcp.xml. The first one should be used if you have multicast in your network, the second – if all communication should go through TCP (e.g. on EC2). In the latter case you will also need to provide the list of initial IPs. The list doesn’t have to be exhaustive, just a list of seeds.Having a protocol stack, ElasticMQ creates a JChannel, and connects it, which simply means connecting to the cluster. And that’s in fact all you need to do to create a cluster with JGroups – pretty simple, right? As you can see at the end of ReplicatedStorageConfigurator, the first thing after connecting is a call to channel.getState(null, 0). This will go to the current master node (more on master election later), fetch the state (current queues and messages) and apply it on the current node (see the very simple JGroupsStateTransferMessageListener – handles both sending and receiving). There are two important things to note here. First is that this transfer doesn’t block the whole cluster from normal operation. Second is that if an operation is executed during the state transfer, it will also be replicated. So it may happen that one command is executed twice on the new node. That doesn’t matter though, as each replicated command is idempotent, so may be applied many times. In other scenarios some application-side mechanism would have to be implemented to prevent such situations.Replicating dataFinally we get to the core: replicating the commands. On the sender side, this is handled by JGroupsReplicationMessageSender. Again, not a very complicated class. It uses the MessageDispatcher “building block” from JGroups which, apart from multicasing messages across the cluster, enables you to wait until a specified number of nodes receive it. On the receiving side, we have JGroupsRequestHandler. Again, pretty simple. When a message is received it is simply sent to the storage.Cluster managementYou may have also noticed the SetMaster special message. This is needed for the user to be able to read the current master’s node address. Master election (deciding which node is the master) is handled entirely with JGroups. There isn’t a specific algorithm in JGroups to elect a master, however we can use the fact that each node has the same cluster view, which is represented by the JGroups View class. All we need to do is simply get the first (or last, or 3rd, etc. – anything as long as it’s the same on all nodes) element from that list, and set it as the master.Cluster view is handled by the last “core” replication class, JGroupsMembershipListener. Two things happen there. The viewAccepted callback is called whenever a new node joins or leaves the cluster; each node with the same (well, equal :) ) instance of the View class. The master broadcasts its address (which is the ElasticMQ server address, not the internal JGroups cluster-communication address) in a separate thread. It’s a very easy mistake to perform a blocking operation in one of the JGroups callback methods. You should never do that, as the whole stack can then lock. We also need the FLUSH protocol (which is always added during the cluster setup); this protocol makes sure that no new messages are sent when before the new view has been installed by all nodes, and hence we make sure that a new node always receives the master information.The membership listener also handles cluster merges. Again, JGroups gives us a view of the partitions that were merged and the new combined view. In ElasticMQ, all partitions except the primary parition (which is the largest one) request a state transfer, just as after connecting to the cluster. That way the data is kept in a consistent state.Summing upIt’s also worth noting that ElasticMQ’s replication is fully tested using ScalaTest. Each test creates a cluster of in-memory storages, creating new nodes or simulating node crashes. See the JGroupsReplicatedStorageTest class.Having the mechanisms from JGroups it is pretty straightforward to implement cluster communication. As always, though, you need to remember about some traps when it comes to concurrency (e.g. that there may be cluster activity when a new node joins; that partitions and merges can happen anytime; that there’s no ordering between normal messages and cluster view changes; that messages may be sent during state transfer; etc.). However both the JGroups tutorial and manual is pretty comprehensive and given the additional help from the forums (thanks Bela!), you should be good to go.You can try how the replication works in practice by downloading a standalone distribution of ElasticMQ or running it embedded.Reference: Implementing message replication in ElasticMQ with JGroups from our JCG partner Adam Warski at the Blog of Adam Warski blog....

Dynamic forms, JSF world was long waiting for

The new PrimeFaces Extensions release 0.5.0 brought a new DynaForm component. Normally, we can build a form quite straightforward by h:panelGrid oder p:panelGrid if the count of rows / columns, positions of elements, etc. are known. That’s true for static forms. But it’s not possible to use h:panelGrid oder p:panelGrid if a form is described dynamically, at runtime. E.g. if the entire form’s definition is placed in a database or a XML file. DynaForm makes possible to build a dynamic form with labels, inputs, selects and any other elements by model. There aren’t limitations. Explore all its features in the showcase. Let’s show how to build a simple dynamic form. Main steps: Create model instance: DynaFormModel model = new DynaFormModel(); Add row to regular grid: DynaFormRow row = model.createRegularRow(); Add label: DynaFormLabel label = row.addLabel(value, colspan, rowspan); Add editable control: DynaFormControl control = row.addControl(data, type, colspan, rowspan); Set relationship between label and control (optional): label.setForControl(control); Repeat four last steps as many times as needed. What is could look like from UI point of view? A screenshot after a failed form validation:The main tag is pe:dynaForm. Child tag pe:dynaFormControl matches created in Java controls by ” type” attribute. This is usually a “one to many” relation. XHTML / Java (controller bean and model) code to the dynamic form above is listed below. <h:panelGroup id="dynaFormGroup"> <p:messages id="messages" showSummary="true"/><pe:dynaForm id="dynaForm" value="#{dynaFormController.model}" var="data"> <pe:dynaFormControl type="input" for="txt"> <p:inputText id="txt" value="#{data.value}" required="#{data.required}"/> </pe:dynaFormControl> <pe:dynaFormControl type="calendar" for="cal" styleClass="calendar"> <p:calendar id="cal" value="#{data.value}" required="#{data.required}" showOn="button"/> </pe:dynaFormControl> <pe:dynaFormControl type="select" for="sel" styleClass="select"> <p:selectOneMenu id="sel" value="#{data.value}" required="#{data.required}"> <f:selectItems value="#{dynaFormController.languages}"/> </p:selectOneMenu> </pe:dynaFormControl> <pe:dynaFormControl type="textarea" for="tarea"> <p:inputTextarea id="tarea" value="#{data.value}" required="#{data.required}" autoResize="false"/> </pe:dynaFormControl> <pe:dynaFormControl type="rating" for="rat"> <p:rating id="rat" value="#{data.value}" required="#{data.required}"/> </pe:dynaFormControl><f:facet name="buttonBar"> <p:commandButton value="Submit" action="#{dynaFormController.submitForm}" process="dynaForm" update="_mainForm_dynaFormGroup"/> <p:commandButton type="reset" value="Reset" style="margin-left: 5px;"/> </f:facet> </pe:dynaForm> </h:panelGroup>@ManagedBean @ViewScoped public class DynaFormController implements Serializable {private DynaFormModel model;private static List<SelectItem> LANGUAGES = new ArrayList<SelectItem>();public DynaFormController() { model = new DynaFormModel();// add rows, labels and editable controls // set relationship between label and editable controls to support outputLabel with "for" attribute// 1. row DynaFormRow row = model.createRegularRow();DynaFormLabel label11 = row.addLabel("Author", 1, 1); DynaFormControl control12 = row.addControl(new BookProperty("Author", true), "input", 1, 1); label11.setForControl(control12);DynaFormLabel label13 = row.addLabel("ISBN", 1, 1); DynaFormControl control14 = row.addControl(new BookProperty("ISBN", true), "input", 1, 1); label13.setForControl(control14);// 2. row row = model.createRegularRow();DynaFormLabel label21 = row.addLabel("Title", 1, 1); DynaFormControl control22 = row.addControl(new BookProperty("Title", false), "input", 3, 1); label21.setForControl(control22);// 3. row row = model.createRegularRow();DynaFormLabel label31 = row.addLabel("Publisher", 1, 1); DynaFormControl control32 = row.addControl(new BookProperty("Publisher", false), "input", 1, 1); label31.setForControl(control32);DynaFormLabel label33 = row.addLabel("Published on", 1, 1); DynaFormControl control34 = row.addControl(new BookProperty("Published on", false), "calendar", 1, 1); label33.setForControl(control34);// 4. row row = model.createRegularRow();DynaFormLabel label41 = row.addLabel("Language", 1, 1); DynaFormControl control42 = row.addControl(new BookProperty("Language", false), "select", 1, 1); label41.setForControl(control42);DynaFormLabel label43 = row.addLabel("Description", 1, 2); DynaFormControl control44 = row.addControl(new BookProperty("Description", false), "textarea", 1, 2); label43.setForControl(control44);// 5. row row = model.createRegularRow();DynaFormLabel label51 = row.addLabel("Rating", 1, 1); DynaFormControl control52 = row.addControl(new BookProperty("Rating", 3, true), "rating", 1, 1); label51.setForControl(control52); }public DynaFormModel getModel() { return model; }public String submitForm() { ... // do something }public List<SelectItem> getLanguages() { if (LANGUAGES.isEmpty()) { LANGUAGES.add(new SelectItem("en", "English")); LANGUAGES.add(new SelectItem("de", "German")); LANGUAGES.add(new SelectItem("ru", "Russian")); LANGUAGES.add(new SelectItem("tr", "Turkish")); }return LANGUAGES; } }public class BookProperty implements Serializable {private String name; private Object value; private boolean required;public BookProperty(String name, boolean required) { this.name = name; this.required = required; }public BookProperty(String name, Object value, boolean required) { this.name = name; this.value = value; this.required = required; }// getter // setter }You see that one of important features is a buil-in support for labels. DynaForm renders labels automatically – no need to write p:outputLabel. Another feature is autoSubmit flag. It allows pass form parameters in URL, build a form on page load and submit it automatically. More highlights: expandable extended view area (grid), open / close state saving, widget’s client-side API, various facets. Next screenshots demonstrate how to build dynamic forms with lables above fields and various elements like PrimeFaces separator. Two forms in this example are switched by clicking on the “Switch model” link. Note, XHTML code with pe:dynaForm stays the same, only Java model gets changed.Explore the corresponding code in the use case Yet another forms. Reference: Dynamic forms, JSF world was long waiting for from our JCG partner Oleg Varaksin at the Thoughts on software development blog....

Product-Burndown-Charts and Sprint-Burndown-Charts in SCRUM Projects

Product-Backlog-Charts and Sprint-Backlog-Charts are used in almost all Agile approaches. In the following article the terminology of SCRUM is used, e.g. User Stories, Product Owner, Product Backlog, Sprint and Sprint Planning Meeting.The described Backlog-Charts are also useful in other agile methodologies and even in non-agile settings. Actually, they are a general technique to estimate efforts in situations where other estimation methods are difficult to apply and/or when historical data are not yet available.Product-Burndown-ChartA Product-Burndown-Chart depicts story points of all user stories in the so called product backlog, where the product backlog is a simple ranked list of all functional and non-functional requirements described as user stories. The chart displays story points for each completed sprint, so it depicts the completion of requirements over time. Backlog and Product-Burndown-Chart is usually updated at the end of each sprint. New user stories can be added and/or removed before each sprint planning meeting. Figure 1: Sample Product-Burndown-Chart A simplified Product-Burndown-Chart is shown in Figure 1. Here the team had a constant velocity of 50 story points per sprint. In the second sprint some user stories (with 100 story points) had been added and in the fifth sprint some user stories (100 story points) have been removed. The sprint with the number zero is just used to show the initial backlog value before the first sprint (Table 1 contains the raw data of the graph).Table 1: Data of Product Burndown-Chart-SampleSprint Remaining New Removed Done0 600 0 0 -501 550 0 0 -502 500 100 0 -503 550 0 0 -504 500 0 0 -505 450 0 -100 -506 300 0 0 -507 250 0 0 -508 200 0 0 -50Sometimes it makes sense, to use instead of story points a simpler measure for complexity of tasks. This can be an enumeration like {large, medium, small} which can be mapped in the backlog to numbers. This can be useful for maintenance and/or operations teams. The important point is that a relative complexity measure is available and used for estimation.The purpose of the Product-Burndown-Chart is to show the team and stakeholders how many sprints will be needed to implement all remaining user stories or how many user stories can be completed in the remaining time.After some User Stories have been completed, the needed working hours for the related tasks can be summed up and divided by the story points. That gives the velocity of the team, which can be used to improve the planning quality.Sprint-Burndown-ChartA Sprint-Burndown-Chart shows all remaining work of the so called the sprint backlog. This sprint backlog contains all work for the current sprint as committed by the team. Usually it is displayed as remaining work in ideal hours for each work day. The Sprint-Burndown-Chart is updated on a daily basis by the team – often before the stand-up meeting of the next work day.   Figure 2: Sample Sprint-Burndown-ChartLike in the Product Burndown there is a working day zero to show the initially planned work before the first work day is finished. The purpose of the Sprint-Burndown-Chart is to show the team if all planned tasks can be finished till the end of the sprint.Reference: How to use Product-Burndown-Charts and Sprint-Burndown-Charts not only in SCRUM Projects? from our JCG partner Markus Sprunck at the Software Engineering Candies blog....

Just in Time Compiler (JIT) in Hotspot

The Just In Time Compiler (JIT) concept and more generally adaptive optimization is well known concept in many languages besides Java (.Net, Lua, JRuby). In order to explain what is JIT Compiler I want to start with a definition of compiler concept. According to wikipedia compiler is ‘ a computer program that transforms the source language into another computer language (the target language)’. We are all familiar with static java compiler (javac) that compiles human readable .java files to a byte code that can be interpreted by JVM – .class files. Then what does JIT compile? The answer will given a moment later after explanation of what is ‘Just in Time’.According to most researches, 80% of execution time is spent in executing 20% of code. That would be great if there was a way to determine those 20% of code and to optimize them. That’s exactly what JIT does – during runtime it gathers statistics, finds the ‘hot’ code compiles it from JVM interpreted bytecode (that is stored in .class files) to a native code that is executed directly by Operating System and heavily optimizes it. Smallest compilation unit is single method. Compilation and statistics gathering is done in parallel to program execution by special threads. During statistics gathering the compiler makes hypotheses about code function and as the time passes tries to prove or to disprove them. If the hypothesis is dis-proven the code is deoptimized and recompiled again. The name ‘Hotspot’ of Sun (Oracle) JVM is chosen because of the ability of this Virtual Machine to find ‘hot’ spots in code. What optimizations does JIT? Let’s look closely at more optimizations done by JIT.Inline methods – instead of calling method on an instance of the object it copies the method to caller code. The hot methods should be located as close to the caller as possible to prevent any overhead. Eliminate locks if monitor is not reachable from other threads Replace interface with direct method calls for method implemented only once to eliminate calling of virtual functions overhead Join adjacent synchronized blocks on the same object Eliminate dead code Drop memory write for non-volatile variables Remove prechecking NullPointerException and IndexOutOfBoundsException Et ceteraWhen the Java VM invokes a Java method, it uses an invoker method as specified in the method block of the loaded class object. The Java VM has several invoker methods, for example, a different invoker is used if the method is synchronized or if it is a native method. The JIT compiler uses its own invoker. Sun production releases check the method access bit for value ACC_MACHINE_COMPILED to notify the interpreter that the code for this method has already been compiled and stored in the loaded class. JIT compiler compiles the method block into native code for this method and stores that in the code block for that method. Once the code has been compiled the ACC_MACHINE_COMPILED bit, which is used on the Sun platform, is set. How do we know what JIT is doing in our program and how can it be controlled? First of all to disable JIT Djava.compiler=NONE parameter can be used. There are 2 types of JIT compilers in Hotspot – one is used for client program and one for server (-server option in VM parameters). Program, running on server enjoys usually from more resources than program running on client and to server program top throughput is usually more important. Hence JIT in server is more resource consuming and gathering statistics takes more time to make the statistics more accurate. For client program gathering statics for a method lasts 1500 method calls, for server 15000. These default values can be changed by – XX:CompileThreshold=XXX VM parameter. In order to find out whether default value is good for you try enabling ‘XX:+PrintCompilation’ and ‘-XX:-CITime’ parameters that print JIT statistics and time CPU spent by JIT. Benchmarks Most of the benchmarks show that JITed code runs 10 to 20 times faster than interpreted code. There are many benchmarks done. Below given result graphs of two of them:Its worth to mention that programs that run in JIT mode, but are still in ‘learning mode’ run much slower than non JITed programs. Drawbacks of JIT JIT Increases level of unpredictability and complexity in Java program. It adds another layer that developers don’t really understand. Example of possible bugs – ‘happens before relations’ in concurrency. JIT can easily reorder code if the change is safe for a program running in single thread. To solve this problem developers make hints to JIT using ‘synchronized’ word or explicit locking. Increases non heap memory footprint – JITed code is stored in ‘Code Cache’ generation. Advanced JIT JIT and garbage collection.For GC to occur program must reach safe points. For this purpose JIT injects yieldpoints at regular intervals in native code. In addition to scanning of stack to find root references, registers must be scanned as they may hold objects created by JITReference: Just in Time Compiler (JIT) in Hotspot from our JCG partner Art Gourevitch at the The Art of Java blog....

Jetty and Eclipse Integration in 3 Steps

This tutorial will walk you through out how to integrate Jetty and Eclipse and how to run a web application on Jetty server inside Eclipse. Steps:Install Jetty Eclipse plugin Create web application Run web application1 – Installing Jetty Eclipse PluginWhen you add a server to the Servers view, you will not see an option for Jetty as you will find for Tomcat, JBoss, Apache, etc. First you need to install a plugin. Go to Eclipse -> Install new Software menu. Click on add and type Jetty for Name and http://run-jetty-run.googlecode.com/svn/trunk/updatesite for Location. Select the Jetty plugin to install. Click on Next and follow the installation:2 – Creating a Web Application When you restart Eclipse, got o Project Explorer view or the New menu and click on New -> Dynamic Web Project:Configure the Project, create a name for it and click on Next:Click on Next:Configure the Web Module:And the project is create. Create also a index.html file. The project structure should look like this: 3 – Running the Web Application Select the application you want to run on Jetty. Click on the Run button -> Run Configurations.Configure your app on Jetty as shown in the picture bellow and click on Run:Wait for the server to start. You should get something like the following on your log:Open a browser and test the application!Done! Happy Coding! Reference: Jetty and Eclipse Integration in 3 Steps from our JCG partner Loiane Groner at the Loiane Groner’s blog blog....
Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.

Sign up for our Newsletter

20,709 insiders are already enjoying weekly updates and complimentary whitepapers! Join them now to gain exclusive access to the latest news in the Java world, as well as insights about Android, Scala, Groovy and other related technologies.

As an extra bonus, by joining you will get our brand new e-books, published by Java Code Geeks and their JCG partners for your reading pleasure! Enter your info and stay on top of things,

  • Fresh trends
  • Cases and examples
  • Research and insights
  • Two complimentary e-books