Featured FREE Whitepapers

What's New Here?


Giving up on Eclipse Juno

In my last blog I posted about my Eclipse 4.2 Juno setup; as much as a reference for me in case I needed to do a reinstall as anything else. What I didn’t talk about then was the issues I’ve been having generally with Juno. I had thought it was my own installation being screwy, but things haven’t improved much since. The main issues I’ve been hitting are:continued modal(!) exception dialogs on certain code completions general sluggishness with the editors HTML editors in particular that are sluggishI could possible live with the last two issues, but the first issue is driving me potty: I can get that modal dialog something like 4 times a minute when I’m into my groove. Not much fun… I’ve realized that I’ve started developing muscle memory to deal with it! As for the performance issues, it would seem that I am not alone, and that things aren’t likely to improve quickly. So, for me, it’s back to Eclipse 3.x. I’ve decided to go with Eclipse 3.7.2; there is an Eclipse 3.8 that was released at the same time as 4.2, but the Eclipse foundation didn’t create the usual distros, and my guess is that it’s not gonna receive much attention if it has bugs etc. Eclipse 3.7 was working just fine for me, so that’s what I’ll use for now. And, let’s see how things are in a year from now. If they don’t fix it, it could be the end of 10 years of using this IDE. PS: I know how this is meant to work: as a user of open source, it is one’s responsibility to contribute back to help improve the product. But my open source time is spent on Apache Isis, and Eclipse is just too big to be able to provide any meaningful help. Don’t forget to share! Reference: Giving up on Eclipse Juno from our JCG partner Dan Haywood at the Dan Haywood blog blog....

Spring Shell Project Released

Spring Source released Spring Shell yesterday. Spring Shell is an interactive shell that can be easily extended with commands using a Spring based programming model. It has been extracted from the Spring Roo project by removing the OSGi dependencies and turned into a standalone project. This allows for an easier adoption by those who wish to use only interactive shell features. At the same time, this decoupling facilitates focusing into improving shell functionality and making it an easily reusable component across various projects. Some of its features include:POJO based programming model Spring classpath scanning features Inherits the Roo Shell Features Tab completion Contextual awareness Command Hiding Help assistance Scripting and script recording Command prompt customizationCheck out the Spring Shell page and the relevant documentation. Happy coding!...

Play 2.0 framework and XA transactions

XA transactions are useful and out of the box, Play 2.0 today does not have support for them. Here I show how to add that support: First off, some examples when XA is useful: – JPA uses two physical connections if you use entities from two different persistence.xml – those two connections might need to be committed in one transaction, so XA is your only option – Committing a change in a database and at the same time committing a message to JMS. E.g. you want to guarantee that an email is sent after you successfully commit an order to the database, asynchronously. There are other ways, but JMS provides a transactional way to do this with little overhead in having to think about failure. – Writing to a physically different database because of any of several political reasons (legacy system, different department responsible for different database server / different budgets). – See http://docs.codehaus.org/display/BTM/FAQ#FAQ-WhywouldIneedatransactionmanager So the way I see it, XA is something Play needs to ‘support’. Adding support is very easy. I have created a play plugin which is based on Bitronix. Resources are configured in the Bitronix JNDI tree (why on earth does Play use a config file rather than JNDI?! anyway…) You start the transaction like this, ‘withXaTransaction': def someControllerMethod = Action { withXaTransaction { ctx => TicketRepository.addValidation(user.get, bookingRef, ctx) ValidationRepository.addValidation(bookingRef, user.get, ctx) } val tickets = TicketRepository.getByEventUid(eventUid) Ok(views.html.ticketsInEvent(eventUid, getTickets(eventUid), user, eventValidationForm)) }The ctx object is an XAContext (my own class) which lets you look up resources like a datasource, or set rollback in case of a failure. So the validation repo does this, using ScalaQuery (I used ‘withSession’ rather than ‘withTransaction!’): def addValidation(bookingRef: String, validator: User, ctx: XAContext) = { val ds = ctx.lookupDS("jdbc/maxant/scalabook_admin") Database.forDataSource(ds) withSession { implicit db: Session => Validations.insert(Validation(bookingRef, validator.email, new java.sql.Timestamp(now))) } }And the ticket repo does the following with JMS: def addValidation(user: User, bookingRef: String, ctx: XAContext) = {val xml = {bookingRef} {user.email}val qcf = ctx.lookupCF("jms/maxant/scalabook/ticketvalidations") val qc = qcf.createConnection("ticketValidation","password") val qs = qc.createSession(false, Session.AUTO_ACKNOWLEDGE) val q = qs.createQueue("ticketValidationQueue") //val q = ctx.lookup(QUEUE).asInstanceOf[Queue] val sender = qs.createProducer(q) val m = qs.createTextMessage(xml.toString) sender.send(m) sender.close qs.close qc.close }I’ve tested it with writing to MySQL and sending a JMS message to JBoss (HornetQ) and it seems to work well (except getting hornetQ to play with Bitronix was a bitch – see here: https://community.jboss.org/thread/206180?tstart=0). The scala code for the XA support is: package ch.maxant.scalabook.play20.plugins.xasupportimport play.api.mvc.RequestHeader import play.api.mvc.Results import play.api.mvc.Request import play.api.mvc.AnyContent import play.api.mvc.Result import play.api.mvc.Action import play.api.mvc.Security import play.api._ import play.api.mvc._ import play.api.data._ import play.api.data.Forms._ import ch.maxant.scalabook.persistence.UserRepository import bitronix.tm.TransactionManagerServices import java.util.Hashtable import javax.naming.Context._ import javax.naming.InitialContext import javax.sql.DataSource import bitronix.tm.BitronixTransaction import java.io.File import org.scalaquery.session.Database import org.scalaquery.SQueryException import scala.collection.mutable.ListBuffer import java.sql.Connection import java.sql.SQLException import org.scalaquery.session.Session import bitronix.tm.BitronixTransactionManager import javax.jms.ConnectionFactoryclass XAContext {private val env = new Hashtable[String, String]() env.put(INITIAL_CONTEXT_FACTORY, "bitronix.tm.jndi.BitronixInitialContextFactory") private val namingCtx = new InitialContext(env);var rollbackOnly = false def lookup(name: String) = { namingCtx.lookup(name) } def lookupDS(name: String) = { lookup(name).asInstanceOf[DataSource] } def lookupCF(name: String) = { lookup(name).asInstanceOf[ConnectionFactory] } }trait XASupport { self: Controller =>private lazy val tm = play.api.Play.current.plugin[XASupportPlugin] match { case Some(plugin) => plugin.tm case None => throw new Exception("There is no XASupport plugin registered. Make sure it is enabled. See play documentation. (Hint: add it to play.plugins)") }/** * Use this flow control to make resources used inside `f` commit with the XA protocol. * Conditions: get resources like drivers or connection factories out of the context passed to f. * Connections are opened and closed as normal, for example by the withSession flow control offered * by ScalaQuery / SLICK. */ def withXaTransaction[T](f: XAContext => T): T = { tm.begin//get a ref to the transaction, in case when we want to commit we are no longer on the same thread and TLS has lost the TX. //we have no idea what happens inside f! they might spawn new threads or send work to akka asyncly val t = tm.getCurrentTransaction Logger("XASupport").info("Started XA transaction " + t.getGtrid()) val ctx = new XAContext() var completed = false try{ val result = f(ctx) completed = true if(!ctx.rollbackOnly){ Logger("XASupport").info("committing " + t.getGtrid() + "...") t.commit Logger("XASupport").info("committed " + t.getGtrid()) } result }finally{ if(!completed || ctx.rollbackOnly){ //in case of exception, or in case of set rollbackOnly = true Logger("XASupport").warn("rolling back (completed=" + completed + "/ctx.rollbackOnly=" + ctx.rollbackOnly) t.rollback } } } }class XASupportPlugin(app: play.Application) extends Plugin {protected[plugins] var tm: BitronixTransactionManager = null override def onStart { //TODO how about getting config out of jar! val file = new File(".", "app/bitronix-default-config.properties").getAbsolutePath Logger("XASupport").info("Using Bitronix config at " + file) val prop = System.getProperty("bitronix.tm.configuration", file) //default System.setProperty("bitronix.tm.configuration", prop) //override with default, if not set//start the TM tm = TransactionManagerServices.getTransactionManager Logger("XASupport").info("Started TM with resource config " + TransactionManagerServices.getConfiguration.getResourceConfigurationFilename) }override def onStop { //on graceful shutdown, we want to shutdown the TM too Logger("XASupport").info("Shutting down TM") tm.shutdown Logger("XASupport").info("TM shut down") }}Use the code as you like, I’m giving it away for free :-) Just don’t complain if it don’t work ;-) It would be nice to see this plugin extended and turned into something a little more production ready. Even nicer would be for Play to support a transaction manager natively, including fetching resources out of JNDI. Happy coding and don’t forget to share! Reference: Play 2.0 framework and XA transactions from our JCG partner Ant Kutschera at the The Kitchen in the Zoo blog....

Android ListView example with Image and Text

In this tutorial I am going to show you how to create an Android ListView with images and text. You will learn how to load an image from a resource and how to set text to TextView . Here is the screenshot of the finished ListView.Android List View example on Samsung Galaxy Y s5360ItemDetails class will help us to set and get item data : package com.jsupport.listviewimages;public class ItemDetails {public String getName() { return name; } public void setName(String name) { this.name = name; } public String getItemDescription() { return itemDescription; } public void setItemDescription(String itemDescription) { this.itemDescription = itemDescription; } public String getPrice() { return price; } public void setPrice(String price) { this.price = price; } public int getImageNumber() { return imageNumber; } public void setImageNumber(int imageNumber) { this.imageNumber = imageNumber; }private String name ; private String itemDescription; private String price; private int imageNumber;}ItemListBaseAdapter Which is extended from the BaseAdapter and sets item details and the image package com.jsupport.listviewimages;import java.util.ArrayList;import android.content.Context; import android.view.LayoutInflater; import android.view.View; import android.view.ViewGroup; import android.widget.BaseAdapter; import android.widget.ImageView; import android.widget.TextView;public class ItemListBaseAdapter extends BaseAdapter { private static ArrayList<ItemDetails> itemDetailsrrayList; private Integer[] imgid = { R.drawable.p1, R.drawable.bb2, R.drawable.p2, R.drawable.bb5, R.drawable.bb6, R.drawable.d1 }; private LayoutInflater l_Inflater;public ItemListBaseAdapter(Context context, ArrayList<ItemDetails> results) { itemDetailsrrayList = results; l_Inflater = LayoutInflater.from(context); }public int getCount() { return itemDetailsrrayList.size(); }public Object getItem(int position) { return itemDetailsrrayList.get(position); }public long getItemId(int position) { return position; }public View getView(int position, View convertView, ViewGroup parent) { ViewHolder holder; if (convertView == null) { convertView = l_Inflater.inflate(R.layout.item_details_view, null); holder = new ViewHolder(); holder.txt_itemName = (TextView) convertView.findViewById(R.id.name); holder.txt_itemDescription = (TextView) convertView.findViewById(R.id.itemDescription); holder.txt_itemPrice = (TextView) convertView.findViewById(R.id.price); holder.itemImage = (ImageView) convertView.findViewById(R.id.photo);convertView.setTag(holder); } else { holder = (ViewHolder) convertView.getTag(); } holder.txt_itemName.setText(itemDetailsrrayList.get(position).getName()); holder.txt_itemDescription.setText(itemDetailsrrayList.get(position).getItemDescription()); holder.txt_itemPrice.setText(itemDetailsrrayList.get(position).getPrice()); holder.itemImage.setImageResource(imgid[itemDetailsrrayList.get(position).getImageNumber() - 1]);return convertView; }static class ViewHolder { TextView txt_itemName; TextView txt_itemDescription; TextView txt_itemPrice; ImageView itemImage; } }ListViewImagesActivity package com.jsupport.listviewimages;import java.util.ArrayList;import android.app.Activity; import android.os.Bundle; import android.view.View; import android.widget.AdapterView; import android.widget.ListView; import android.widget.Toast; import android.widget.AdapterView.OnItemClickListener;public class ListViewImagesActivity extends Activity { /** Called when the activity is first created. */ @Override public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.main);ArrayList<ItemDetails> image_details = GetSearchResults();final ListView lv1 = (ListView) findViewById(R.id.listV_main); lv1.setAdapter(new ItemListBaseAdapter(this, image_details));lv1.setOnItemClickListener(new OnItemClickListener() { @Override public void onItemClick(AdapterView<?> a, View v, int position, long id) { Object o = lv1.getItemAtPosition(position); ItemDetails obj_itemDetails = (ItemDetails)o; Toast.makeText(ListViewImagesActivity.this, 'You have chosen : ' + ' ' + obj_itemDetails.getName(), Toast.LENGTH_LONG).show(); } }); }private ArrayList<ItemDetails> GetSearchResults(){ ArrayList<ItemDetails> results = new ArrayList<ItemDetails>();ItemDetails item_details = new ItemDetails(); item_details.setName('Pizza'); item_details.setItemDescription('Spicy Chiken Pizza'); item_details.setPrice('RS 310.00'); item_details.setImageNumber(1); results.add(item_details);item_details = new ItemDetails(); item_details.setName('Burger'); item_details.setItemDescription('Beef Burger'); item_details.setPrice('RS 350.00'); item_details.setImageNumber(2); results.add(item_details);item_details = new ItemDetails(); item_details.setName('Pizza'); item_details.setItemDescription('Chiken Pizza'); item_details.setPrice('RS 250.00'); item_details.setImageNumber(3); results.add(item_details);item_details = new ItemDetails(); item_details.setName('Burger'); item_details.setItemDescription('Chicken Burger'); item_details.setPrice('RS 350.00'); item_details.setImageNumber(4); results.add(item_details);item_details = new ItemDetails(); item_details.setName('Burger'); item_details.setItemDescription('Fish Burger'); item_details.setPrice('RS 310.00'); item_details.setImageNumber(5); results.add(item_details);item_details = new ItemDetails(); item_details.setName('Mango'); item_details.setItemDescription('Mango Juice'); item_details.setPrice('RS 250.00'); item_details.setImageNumber(6); results.add(item_details);return results; } } Download the complete project: Android ListView Happy coding and don’t forget to share! Reference: Android ListView example with Image and Text from our JCG partner Chathura Wijesinghe at the Java Sri Lankan Support blog....

Hunting a Random Bug – A True Story

A few weeks ago, I finished a bug hunt for the RapidFTR open source project, which took me three evenings. I thought it might be worth sharing the story of the hunt. This article will describe what I did. I will give an overview of my journey to actually find the root cause of what was going on. My goal with this article is to highlight examples of techniques you can use, to actually track down strange and random bug. In the end, I found a pretty clear explanation what and why the problem happened. And it is not a coincidence, that there is a ‘random’ in the title of this article.Start – What was the problem The problem was, that when following the guide for using a development virtual machine, sometimes starting up the application took a long time and then failed. I had the issue right away, after starting the application twice. Other people had the same problem. I was told, nobody really knew, why it was behaving this way – but restarting the VM would probably fix it. It didn’t for me. But I enjoy digging into problems – so I wanted to know, what was going wrong.Rake File to Solr The error occurred when running rake app:run, sometimes. From running rake with --trace --verbose, the debug option, I was able to see, that the application was waiting for sunspot to start the Apache Solr search service. So the question was, where does the process take all the time? The good thing to start with was, that rake and sunspot are written in Ruby. So I read the code and added a few output statements to see, which statement took so long. It turned out, it was the Java process starting Solr.Digging into the Java Apache Solr sends some output to sdtout – but sunspot hides it from you. But as any output is very helpful when debugging, I wanted to see the output. I used ps -aux to find the exact command-line arguments sunspot was issuing and started the process manually from the command-line. Luckily, it still took some time – so I knew the problem was not in the Ruby Code. With the port mapper nmap I found out, that it took between 2 seconds to over 2 minutes to start the service. But the rake task only waited for 10 seconds. Now having the the console output, I could see, that Solr internally uses Jetty – and it was Jetty in version 6.1.3, which took all the time – in the case, it took the time. So I downloaded the tag for version 6.1.3 from source control and looked into the code. I found a system property called DEBUG which increases the output. I enabled it and saw that the last debug output, before it took so long, was something about starting the session service (I’ve lost pointer to the code – unfortunately). This didn’t really help me. I unsuccessfully tried to make the Jetty log work with log4j to eventually see more – but I failed and had to give up for the evening (the 2nd one).Debug the Java Code So if your not getting anything out of the log, you can still use a debugger. The code was however failing on the VM – an Ubuntu box, but not on my local machine. But fortunately Java has a remote debugger. I started the Java process with the remote debugging arguments and attached a listener in IntelliJ with the Jetty code from SVN. I tried to get the execution stopped at a breakpoint near the last log output – but this method was called very often and I didn’t get it to hit the breakpoint at the right moment. However, when something takes very long, it can be helpful just to stop the running process and look at the stack trace. So I stopped the execution and searched for the last method frame from Jetty, which was calling into the JDK – and there I found the following comment: //This operation may block on some systems with low entropy. See this page //for workaround suggestions: //http://docs.codehaus.org/display/JETTY/Connectors+slow+to+startup Log.debug('Init SecureRandom.'); _random=new SecureRandom();The Solution Finally – I did go to the Jetty wiki page referenced. I found out, that Java’s java.util.SecureRandom uses real entropy from the operating system – which can block when there is none available. The solution from the wiki page, just to use the normal java.util.Random, would have been difficult, because the configuration file lives deep inside sunspot and Solr. But the line ‘NB Some workaround reports use /dev/./urandom instead of /dev/urandom’ gave me a hint, were to look otherwise. I googled for the solution and found on Stack Overflow, that you can replace /dev/urandom random with /dev/./urandom in your Java process. This can be done by using the command-line argument -Djava.security.egd=file:/dev/./urandom or by replacing it in the file /usr/lib/jvm/java-6-openjdk/jre/lib/security/java.security (on an Ubuntu box). So what was the matter? Java’s secure random will not accept /dev/urandom as source for random numbers. /dev/urandom is unblocking, which means it will return random numbers which can be guessed – when there is no real entropy available. So Java defaults to /dev/random which will block and wait, until entropy is present. This is the reason, why blocking is very unlikely to occur on a production or development machine – however it did happen, when using a virtual machine, which does not have a lot of network traffic and IO. It happens especially often, when you restart an application often – as you tend to do, when something is not working. The fix is a little funny. Java rejects /dev/urandom based on the string. It does not reject /dev/./urandom which of course points to the same potentially unsafe random number generator.Last Words The fix introduces a potential security risk – but on a development virtual machine this is not a problem. As you can see, finding a bug or problem in a system involves to use techniques, which bring you nearer to the root cause. However, there is no golden path. Sometimes one technique will not get you any more information. So you will have to try a different technique. Sometimes your gut feeling might help you as well. Having arrived here, what are your thoughts on the story? Reference: Hunting a Random Bug – A True Story from our JCG partner Johannes Thones at the Johannes Thones blog blog....

Maven Build Number Plugin – Sample Usage

Suppose we need to add a build number to some artifact (jar, war, etc.). Here I’d like to demonstrate the usage of buildnumber-maven-plugin. This post is based on:http://mojo.codehaus.org/buildnumber-maven-plugin/usage.html http://www.site.lalitbhatt.com/maven-build-number-plugin http://blog.peterlynch.ca/2009/11/buildnumber-maven-plugin-helpful.html http://apollo.ucalgary.ca/tlcprojectswiki/index.php/Public/Project_Versioning_-_Best_Practices#Build_VersioningWe have some project and need to include into jar manifest file sequential build number which isn’t based on VCS (SVN, Git, Mercurial, etc.) revision number. Let’s create appropriate pom.xml file and implement small demo to verify the result. Generate maven project $ mvn archetype:generate -DgroupId=org.halyph -DartifactId=buildNoTest\ -DarchetypeArtifactId=maven-archetype-quickstart \ -DinteractiveMode=false Create the pom.xml <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemalocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelversion>4.0.0</modelversion> <groupid>org.halyph</groupid> <artifactid>buildNoTest</artifactid> <packaging>jar</packaging> <version>1.0-SNAPSHOT</version> <name>buildNoTest</name> <url>http://maven.apache.org</url> <dependencies> <dependency> <groupid>junit</groupid> <artifactid>junit</artifactid> <version>3.8.1</version> <scope>test</scope> </dependency> </dependencies> <properties> <project.build.sourceencoding>UTF-8</project.build.sourceencoding> </properties> <!-- If you have access to scm then you can place actual url's. Otherwise with <revisionOnScmFailure /> you can give some fake URLs as follows. --> <scm> <connection>scm:svn:http://none</connection> <developerconnection>scm:svn:https://none</developerconnection> <url>scm:svn:https://none</url> </scm> <build> <resources> <resource> <directory>src/main/resources</directory> </resource> <resource> <directory>src/main/filtered-resources</directory> <filtering>true</filtering> </resource> </resources> <plugins> <plugin> <groupid>org.codehaus.mojo</groupid> <artifactid>buildnumber-maven-plugin</artifactid> <version>1.1</version> <executions> <execution> <phase>generate-resources</phase> <goals> <goal>create</goal> </goals> </execution> </executions> <configuration> <!-- doCheck and doUpdate actually talk to repository if it's true, Check would check that there are no local changes. Update would update it --> <docheck>false</docheck> <doupdate>false</doupdate> <!-- This ensures that even if we are not connected to scm than also take the version from local .svn file --> <revisiononscmfailure> <!-- Generate sequence build number based on: build number and timestamp --> <format>Build: #{0} ({1,date})</format> <items> <item>buildNumber\d*</item> <item>timestamp</item> </items> </revisiononscmfailure></configuration> </plugin> <plugin> <groupid>org.apache.maven.plugins</groupid> <artifactid>maven-jar-plugin</artifactid> <version>2.1</version> <configuration> <archive> <!-- will put the entries into META-INF/MANIFEST.MF file --> <manifestentries> <implementation-version>${project.version}</implementation-version> <implementation-build>${buildNumber}</implementation-build> </manifestentries> </archive> </configuration> </plugin> </plugins> </build> </project>Create demo application to verify the results package org.halyph;import java.io.IOException; import java.util.ResourceBundle; import java.util.jar.Attributes; import java.util.jar.Manifest;public class App { public static void main( String[] args ) throws IOException { System.out.println('Verify Resource bundle' ); // Check filtered resources based on generated build number ResourceBundle bundle = ResourceBundle.getBundle( 'build' ); String msg = bundle.getString( 'build.message' ); System.out.println(msg);System.out.println('\nVerify Generated MANIFEST.MF Properties' );// Check Manifest file based on generated build number Manifest mf = new Manifest(); mf.read(Thread.currentThread().getContextClassLoader().getResourceAsStream('META-INF/MANIFEST.MF'));Attributes atts = mf.getMainAttributes();System.out.println('Implementation-Versio: ' + atts.getValue('Implementation-Version')); System.out.println('Implementation-Build: ' + atts.getValue('Implementation-Build')); } }Build application several time and Run $ mvn install $ java -cp target\buildNoTest-1.0-SNAPSHOT.jar org.halyph.App Verify Resource bundle Build: #3 (Jun 27, 2012)Verify Generated MANIFEST.MF Properties Implementation-Versio: 1.0-SNAPSHOT Implementation-Build: Build: #3 (Jun 27, 2012)SummaryWe should inform buildnumber-maven-plugin that we won’t use version control revision as build number via adding fake <scm> section into pom.xml and <revisionOnScmFailure /> into buildnumber-maven-plugin <configuration> Implemented custom build number format, see buildnumber-maven-plugin <configuration>/<format> and <configuration>/<items>. Added build number into jar manifest, see maven-jar-plugin pom.xml section Tested if generated build number can be properly added in filtered resourcescreated src\main\filtered-resources\build.properties file build.message=${buildNumber}added resource filtering, see section <resource> flag <filtering>true</filtering>Demo application verifying the filtered resources and build number in jar manifest fileYou can git clone this project https://github.com/halyph/blog-sandbox/tree/master/Maven/blogpost_062712 Reference: Maven Build Number Plugin – Sample Usage from our JCG partner Orest Ivasiv at the Knowledge Is Everything blog....

JavaOne 2012: A Walk Through of Groovy’s AST Transformations

I made the very short walk from Hilton Plaza A/B back to Hilton Golden Gate 3/4/5 to see the presentation ‘Walk through Groovy’s AST Transformations.’ Groovy‘s AST Transformations are something I’ve dabbled with directly a few times, but have more often benefited from others’ work with them. I had started reading the Packt Publishing book Groovy for Domain-Specific Languages, but wanted to attend this presentation to reinvigorate my interest and kick-start my increased use of this powerful tool.Andres Almiray (Canoo) presented this presentation on Groovy AST Transformations. It did not surprise me that most of the audience had Groovy experience given that use of Groovy ASTs is likely more appealing to those with some familiarity with Groovy already. Almiray defined AST Transformations as ‘essentially byte code generation’ that ‘enables compile-time metaprogramming.’ He showed that Groovy has two types of AST Transformations: global and local. The focus of today’s presentation is on global AST transformations. The AST Transformations framework was added to Groovy years ago, but things were made much easier in Groovy 1.7. Almiray covered the Delegate Transformation (@Delegate annotation) in Groovy allows the compiled code to have all of the public methods of the field that was explicitly delegated to. Almiray explained that @Delegate works with interfaces as well as classes. Almiray also explained that any new method defined will take precedent over any delgate method of the same signature. Similarly, the first delegate encountered takes precedent over same method signature of other delegates. Almiray then covered @Singleton and the Singleton Transformation. Almiray stated that the singleton implemented with this transformation meets the definition of a safe singleton described in Josh Bloch’s Effective Java. @Immutable (the Immutable Transformation) was covered next. Just as the @Singleton transformation automatically implemented all necessary rules for singletons, the @Immutable transformation implements the rules for immutable. Almiray noted that there are different exceptions for attempts to set a property on an immutable Groovy class via property set versus method set. The next Groovy AST transformation to be covered was @Category (the Category Transformation). This was the first covered transformation that requires usage within Groovy code (not within Java code) to be fully used. The Mixin Transformation (@Mixin) was also covered. Almiray moved onto coverage of @Grab (Grab Transformation), one which I have posted about before. @Grab is useful for downloading dependencies at runtime. I like it for the same reason that Almiray mentioned: ‘it’s perfect for self-contained scripts.’ Almiray introduced the @Synchronized (Synchronized Transformation) as a Groovier way to specify synchronized blocks. Almiray covered the @Lazy (Lazy Transformation), which is used to only initialize values when actually needed (when first used). Almiray pointed out Groovy’s ability to access classes’ private fields and cautioned that this should only be used for unit testing and only when absolutely necessary in production. Almiray demonstrated use of @Newify (Newify Transformation) before showing a code sample using @Bindable (Bindable Transformation), which he stated was added to Groovy to make use of Swing easier. The transformation makes a class observable and removes the need to write all the code for explicitly doing this. The @Vetoable transformation similarly makes it easier to veto a property change. As I described in my post Easy Groovy Logger Injection and Log Guarding, @Log (Log Transformation) can be very useful (as are @Commons, @Log4j, and @Slf4j). Almiray covered some of my favorite and most often-used Groovy transformations: @ToString (see my post), @EqualsAndHashCode (see my post), @TupleConstructor (see my post), or the combination of them all (@Canonical – see my post). Following coverage of @Canonical and its constituent transformations, Almiray moved onto covering @IndexedProperty (related post). He then listed several others without code samples: @AutoClone, @AutoExternalize, @ConditionalInterrupt, @TimedInterrupt, @ThreadInterrupt, @PackageScope (‘gain back package-level access specificity in Groovy’), @WithReadLock, @WithWriteLock, and @Field (‘mostly used inside scripts’). I was happy to see Almiray mention the addition of @TypeChecked to support ‘static Groovy!’ He referenced a later presentation on new features of Groovy 2.0 to get more details. Almiray mentioned new transformations specific to Grails (@Entity) and to Griffon (@EventPublisher, @PropertyListener, @Treading, and more). He referenced @Scalify and @Bytecode as well. Although I was already familiar with a large percentage of the Groovy AST transformations covered in this presentation, it was still worthwhile to attend and learn of or be reminded of other useful transformations that are available. Don’t forget to share! Reference: JavaOne 2012: A Walk Through of Groovy’s AST Transformations from our JCG partner Dustin Marx at the Inspired by Actual Events blog....

Opinion: Performance Testing

Performance tuning an application is time consuming, and expensive. Useful tests often need dedicated hardware to run on. It’s specialised and time consuming to prepare the ground work and write the various fixtures needed to run, and whose only perceived benefit is preventing a production issue that you don’t even know will happen yet. Stereotypical Scenarios and Outcomes Here’s some stereotypes I’ve encountered:A feature release contains a new feature that wasn’t (or even could not have been) tested adequately that has a major performance issue from the outset. A change to existing code, perhaps to finesse or refine an existing feature, perhaps a change that’s not been requested by the customer, introduces a catastrophic performance issue. A sleeper: a change in the system that only occurs after some time (e.g. running out of values in a 4-byte integer serial column, meaning that inserts into that table require the database engine to scan for unused keys). Change of use: it was designed to be used in one way, and then the customer starts using it in another unanticipated way. Age: the database gets large and queries start to slow down, I’ve heard this referred to is somewhat wooly terms as ‘degradation’.A common resolution to these stereotypes is a herculean effort: late night and weekends with people on the phone asking ‘When will my site be fixed? Who’s responsible for this?’. I’ve heard this called ‘hero culture’. It’s a mentality that can perversely reward those who might have been expected to prevent the problem, as they are the ones best capable of fixing it. After resolution, a period of self-reflection. People are asking what can be done to show willingness to tackle the problem; perhaps in a one-off performance tuning exercise by a specialist, which resolves current issues. But, if the analysis is done by a seconded specialist who’s not part of the team, it’s an exercise whose lessons are not disseminated, and which is not repeatable. Those who do not learn from the past are doomed to repeat it*. This might be a fait-accompli: if performance testing is more expensive than the cost of fixing periodic production issues, then this is the most logical, most cost-effective approach.Many systems, perhaps due to cost or lack of time, have not been developed in a way that is amenable to automated testing. After all, when a system is first written, you might not know if it’s going to be a commercial success, so why spend money making system maintainable if it might never need to be maintained? The inability to test the performance of changes can mean that improvements to the system are prevented; the cost of introducing a bug cannot be mitigated. People start to fear change and the product stagnates. A younger, faster, competitor learns from your success. They quickly write a more modern, cheaper version of your software and starts taking your business.Automated Performance Testing How do you implement this? How do you take a system that might even be hostile to testing, and change it so that releases become more bug free and robust?Foundations You should make sure your code base is reliable before you consider performance testing. Most of this is common sense.Change and test incrementally. Bugs first, features second; new features will only introduce new bugs, so make sure you fix any bugs first. Use a mature developed build system. Be able to write and execute unit tests on your code.Once these are in place, consider using code quality tools, such as or Cobertura, to get metrics and enforce them, failing builds that don’t meet some minimum criteria.Integration Test Integration test forms the first step toward full performance testing. There’s many frameworks, depending on how users or clients interface. If it is a web-app, then you might use Selenium, a web service, a ReST or SOAP client. Generally, a popular framework is a better choice, as it encourages adoption by the rest of your team. Ask yourself – would I rather learn something well-documented, interesting and personally valuable, or wrestle with someone else’s hand-rolled vanity project? Regardless, to run integration tests, you’ll need to be able to:Build your app. Deploy to a test environment. Execute the tests. Report the results.Ideally, you should be able to do this at the touch of a button, otherwise you’ll be the only person who does it, and you’ll lose a lot of the value of your work. As you do this you’ll find that:You better understand the architecture of your app. You know how to create a suitable environment for it. You understand the deployment process. You can deploy it automatically.These are key to automating performance testing.Performance Testing Unit testing, and to some degree integration testing, have binary outcomes: they pass and everyone’s happy, they fail, and there’s a bug to be fixed. To a similar degree, the tools are well supported and everyone knows how to use them. Performance testing is a bit more of an art. Ultimately a performance test produces some measures: a series of numbers, but are those numbers good or bad? Do you want to guess? A single metric, standing on it’s own, can be un-enlightening, but you can look at its relative change to previous measurement. You need to (in order):Expose metrics (noting that you may want to introduce new ones and deprecate old ones). Sample the metrics. Run the same test from the same baseline (e.g. by starting with a freshly provisioned server, loading it with data, and warming it up). Report on the results within a tool.Again, with a single button press. If you deploy your app to one host, where do you run the tests from? What demand might they make of the office network? Do you need multiple hosts and their own LAN? You’ll need to expose your metrics first, and there are a few commercial and open source tools for Java, such as JInspired or Metrics, or, indeed you can roll your own. One feature you might want is exposing the metrics over JMX, which allow sampling. OpenNMS is a network management application that can remotely periodically sample JMX beans, and it is relatively straight forward to get graphs of those metrics. There are, of course, alternatives. Now, if you automatically deploy then performance test on each commit, then you could have the details displayed on your agile wall, so the team can see when performance changes and any hot spots appear. Best of all, once it’s all in place, you don’t need to do much to keep it up to date. Reference: Opinion: Performance Testing from our JCG partner Alex Collins at the Alex Collins ‘s blog blog....

Redis pub/sub using Spring

Continuing to discover the powerful set of Redis features, the one worth mentioning about is out of the box support of pub/sub messaging. Pub/Sub messaging is essential part of many software architectures. Some software systems demand from messaging solution to provide high-performance, scalability, queues persistence and durability, fail-over support, transactions, and many more nice-to-have features, which in Java world mostly always leads to using one of JMS implementation providers. In my previous projects I have actively used Apache ActiveMQ (now moving towards Apache ActiveMQ Apollo). Though it’s a great implementation, sometimes I just needed simple queuing support and Apache ActiveMQ just looked overcomplicated for that. Alternatives? Please welcome Redis pub/sub! If you are already using Redis as key/value store, few additional lines of configuration will bring pub/sub messaging to your application in no time. Spring Data Redis project abstracts very well Redis pub/sub API and provides the model so familiar to everyone who uses Spring capabilities to integrate with JMS. As always, let’s start with the POM configuration file. It’s pretty small and simple, includes necessary Spring dependencies, Spring Data Redis and Jedis, great Java client for Redis. <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemalocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"><modelversion>4.0.0</modelversion> <groupid>com.example.spring</groupid> <artifactid>redis</artifactid> <version>0.0.1-SNAPSHOT</version> <packaging>jar</packaging><properties> <project.build.sourceencoding>UTF-8</project.build.sourceencoding> <spring.version>3.1.1.RELEASE</spring.version> </properties><dependencies> <dependency> <groupid>org.springframework.data</groupid> <artifactid>spring-data-redis</artifactid> <version>1.0.1.RELEASE</version> </dependency><dependency> <groupid>cglib</groupid> <artifactid>cglib-nodep</artifactid> <version>2.2</version> </dependency><dependency> <groupid>log4j</groupid> <artifactid>log4j</artifactid> <version>1.2.16</version> </dependency><dependency> <groupid>redis.clients</groupid> <artifactid>jedis</artifactid> <version>2.0.0</version> <type>jar</type> </dependency><dependency> <groupid>org.springframework</groupid> <artifactid>spring-core</artifactid> <version>${spring.version}</version> </dependency><dependency> <groupid>org.springframework</groupid> <artifactid>spring-context</artifactid> <version>${spring.version}</version> </dependency> </dependencies><build> <plugins> <plugin> <groupid>org.apache.maven.plugins</groupid> <artifactid>maven-compiler-plugin</artifactid> <version>2.3.2</version> <configuration> <source>1.6 <target>1.6</target> </configuration> </plugin> </plugins> </build> </project>Moving on to configuring Spring context, let’s understand what we need to have in order for a publisher to publish some messages and for a consumer to consume them. Knowing the respective Spring abstractions for JMS will help a lot with that.we need connection factory -> JedisConnectionFactory we need a template for publisher to publish messages -> RedisTemplate we need a message listener for consumer to consume messages -> RedisMessageListenerContainerUsing Spring Java configuration, let’s describe our context: package com.example.redis.config;import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.data.redis.connection.jedis.JedisConnectionFactory; import org.springframework.data.redis.core.RedisTemplate; import org.springframework.data.redis.listener.ChannelTopic; import org.springframework.data.redis.listener.RedisMessageListenerContainer; import org.springframework.data.redis.listener.adapter.MessageListenerAdapter; import org.springframework.data.redis.serializer.GenericToStringSerializer; import org.springframework.data.redis.serializer.StringRedisSerializer; import org.springframework.scheduling.annotation.EnableScheduling;import com.example.redis.IRedisPublisher; import com.example.redis.impl.RedisMessageListener; import com.example.redis.impl.RedisPublisherImpl;@Configuration @EnableScheduling public class AppConfig { @Bean JedisConnectionFactory jedisConnectionFactory() { return new JedisConnectionFactory(); }@Bean RedisTemplate< String, Object > redisTemplate() { final RedisTemplate< String, Object > template = new RedisTemplate< String, Object >(); template.setConnectionFactory( jedisConnectionFactory() ); template.setKeySerializer( new StringRedisSerializer() ); template.setHashValueSerializer( new GenericToStringSerializer< Object >( Object.class ) ); template.setValueSerializer( new GenericToStringSerializer< Object >( Object.class ) ); return template; }@Bean MessageListenerAdapter messageListener() { return new MessageListenerAdapter( new RedisMessageListener() ); }@Bean RedisMessageListenerContainer redisContainer() { final RedisMessageListenerContainer container = new RedisMessageListenerContainer();container.setConnectionFactory( jedisConnectionFactory() ); container.addMessageListener( messageListener(), topic() );return container; } @Bean IRedisPublisher redisPublisher() { return new RedisPublisherImpl( redisTemplate(), topic() ); }@Bean ChannelTopic topic() { return new ChannelTopic( 'pubsub:queue' ); } } Very easy and straightforward. The presence of @EnableScheduling annotation is not necessary and is required only for our publisher implementation: the publisher will publish a string message every 100 ms. package com.example.redis.impl;import java.util.concurrent.atomic.AtomicLong;import org.springframework.data.redis.core.RedisTemplate; import org.springframework.data.redis.listener.ChannelTopic; import org.springframework.scheduling.annotation.Scheduled;import com.example.redis.IRedisPublisher;public class RedisPublisherImpl implements IRedisPublisher { private final RedisTemplate< String, Object > template; private final ChannelTopic topic; private final AtomicLong counter = new AtomicLong( 0 );public RedisPublisherImpl( final RedisTemplate< String, Object > template, final ChannelTopic topic ) { this.template = template; this.topic = topic; }@Scheduled( fixedDelay = 100 ) public void publish() { template.convertAndSend( topic.getTopic(), 'Message ' + counter.incrementAndGet() + ', ' + Thread.currentThread().getName() ); } } And finally our message listener implementation (which just prints message on a console). package com.example.redis.impl;import org.springframework.data.redis.connection.Message; import org.springframework.data.redis.connection.MessageListener;public class RedisMessageListener implements MessageListener { @Override public void onMessage( final Message message, final byte[] pattern ) { System.out.println( 'Message received: ' + message.toString() ); } } Awesome, just two small classes, one configuration to wire things together and we have full pub/sub messaging support in our application! Let’s run the application as standalone … package com.example.redis;import org.springframework.context.ApplicationContext; import org.springframework.context.annotation.AnnotationConfigApplicationContext;import com.example.redis.config.AppConfig;public class RedisPubSubStarter { public static void main(String[] args) { new AnnotationConfigApplicationContext( AppConfig.class ); } }… and see following output in a console: ... Message received: Message 1, pool-1-thread-1 Message received: Message 2, pool-1-thread-1 Message received: Message 3, pool-1-thread-1 Message received: Message 4, pool-1-thread-1 Message received: Message 5, pool-1-thread-1 Message received: Message 6, pool-1-thread-1 Message received: Message 7, pool-1-thread-1 Message received: Message 8, pool-1-thread-1 Message received: Message 9, pool-1-thread-1 Message received: Message 10, pool-1-thread-1 Message received: Message 11, pool-1-thread-1 Message received: Message 12, pool-1-thread-1 Message received: Message 13, pool-1-thread-1 Message received: Message 14, pool-1-thread-1 Message received: Message 15, pool-1-thread-1 Message received: Message 16, pool-1-thread-1 ...Great! There is much more which you could do with Redis pub/sub, excellent documentation is available for you on Redis official web site. Reference: Redis pub/sub using Spring from our JCG partner Andrey Redko at the Andriy Redko {devmind} blog....

RateLimiter – discovering Google Guava

RateLimiter class was recently added to Guava libraries (since 13.0) and it is already among my favourite tools. Have a look what the JavaDoc says: Let’s start from a simple example. Say we have a long running process that needs to broadcast its progress to supplied listener: def longRunning(listener: Listener) { var processed = 0 for(item <- items) { //..do work... processed += 1 listener.progressChanged(100.0 * processed / items.size) } }trait Listener { def progressChanged(percentProgress: Double) }Please forgive me the imperative style of this Scala code, but that’s not the point. The problem I want to highlight becomes obvious once we start our application with some concrete listener: class ConsoleListener extends Listener { def progressChanged(percentProgress: Double) { println('Progress: ' + percentProgress) } }longRunning(new ConsoleListener)Imagine that longRunning() method processes millions of items but each iteration takes just a split of a second. The amount of logging messages is just insane, not to mention console output is probably taking much more time than processing itself. You’ve probably faced such a problem several times and have a simple workaround: if(processed % 100 == 0) { listener.progressChanged(100.0 * processed / items.size) }There, I Fixed It! We only print progress every 100th iteration. However this approach has several drawbacks:code is polluted with unrelated logic there is no guarantee that every 100th iteration is slow enough… … or maybe it’s still too slow?What we really want to achieve is to limit the frequency of progress updates (say: two times per second). OK, going deeper into the rabbit hole: def longRunning(listener: Listener) { var processed = 0 var lastUpdateTimestamp = 0L for(item <- items) { //..do work... processed += 1 if(System.currentTimeMillis() - lastUpdateTimestamp > 500) { listener.progressChanged(100.0 * processed / items.size) lastUpdateTimestamp = System.currentTimeMillis() } } }Do you also have a feeling that we are going in the wrong direction? Ladies and gentlemen, I give you RateLimiter: var processed = 0 val limiter = RateLimiter.create(2) for (item <- items) { //..do work... processed += 1 if (limiter.tryAcquire()) { listener.progressChanged(100.0 * processed / items.size) } }Getting better? If the API is not clear: we are first creating a RateLimiter with 2 permits per second. This means we can acquire up to two permits during one second and if we try to do it more often tryAcquire() will return false (or thread will block if acquire() is used instead 1). So the code above guarantees that the listener won’t be called more that two times per second. As a bonus, if you want to completely get rid of unrelated throttling code from the business logic, decorator pattern to the rescue. First let’s create a listener that wraps another (concrete) listener and delegates to it only at a given rate: class RateLimitedListener(target: Listener) extends Listener {val limiter = RateLimiter.create(2)def progressChanged(percentProgress: Double) { if (limiter.tryAcquire()) { target.progressChanged(percentProgress) } } }What’s best about the decorator pattern is that both the code using the listener and the concrete implementation are not aware of the decorator. Also the client code became much simpler (essentially we came back to original): def longRunning(listener: Listener) { var processed = 0 for (item <- items) { //..do work... processed += 1 listener.progressChanged(100.0 * processed / items.size) } }longRunning(new RateLimitedListener(new ConsoleListener))But we’ve only scratched the surface of where RateLimiter can be used! Say we want to avoid aforementioned denial of service attack or slow down automated clients of our API. It’s very simple with RateLimiter and servlet filter: @WebFilter(urlPatterns=Array('/*')) class RateLimiterFilter extends Filter {val limiter = RateLimiter.create(100)def init(filterConfig: FilterConfig) {}def doFilter(request: ServletRequest, response: ServletResponse, chain: FilterChain) { if(limiter.tryAcquire()) { chain.doFilter(request, response) } else { response.asInstanceOf[HttpServletResponse].sendError(SC_TOO_MANY_REQUESTS) } }def destroy() {} }Another self-descriptive sample. This time we limit our API to handle not more than 100 requests per second (of course RateLimiter is thread safe). All HTTP requests that come through our filter are subject to rate limiting. If we cannot handle incoming request, we send HTTP 429 – Too Many Requests error code (not yet available in servlet spec). Alternatively you may wish to block the client for a while instead of eagerly rejecting it. That’s fairly straightforward as well: def doFilter(request: ServletRequest, response: ServletResponse, chain: FilterChain) { limiter.acquire() chain.doFilter(request, response) }limiter.acquire() will block as long as it’s needed to keep desired 100 requests per second limit. Yet another alternative is to use tryAcquire() with timeout (blocking up to given amount of time). Blocking approach is better if you want to avoid sending errors to the client. However under high load it’s easy to imagine almost all HTTP threads blocked waiting for RateLimiter, eventually causing servlet container to reject connections. So dropping of clients can be only partially avoided. This filter is a good starting point to build more sophisticated solutions. Map of rate limiters by IP or user name are good examples. What we haven’t covered yet is acquiring more than one permit at a time. It turns out RateLimiter can also be used e.g. to limit network bandwidth or the amount of data being sent/received. Imagine you create a search servlet and you want to impose that no more than 1000 results are returned per second. In each request user decides how many results she wants to receive per response: it can be 500 requests each containing 2 results or 1 huge request asking for 1000 results at once. But never more than 1000 results within a second on average. Users are free to use their quota as they wish: @WebFilter(urlPatterns = Array ('/search')) class SearchServlet extends HttpServlet {val limiter = RateLimiter.create(1000)override def doGet(req: HttpServletRequest, resp: HttpServletResponse) { val resultsCount = req.getParameter('results').toInt limiter.acquire(resultsCount) //process and return results... } }By default we acquire() one permit per invocation. Non-blocking servlet would call limiter.tryAcquire(resultsCount) and check the results, you know that by now. If you are interested in rate limiting of network traffic, don’t forget to check out my Tenfold increase in server throughput with Servlet 3.0 asynchronous processing. RateLimiter, due to a blocking nature, is not very well suited to write scalable upload/download servers with throttling. The last example I would like to share with you is throttling client code to avoid overloading the server we are talking to. Imagine a batch import/export process that calls some server thousands of times exchanging data. If we don’t throttle the client and there is no rate limiting on the server side, server might get overloaded and crash. RateLimiter is once again very helpful: val limiter = RateLimiter.create(20)def longRunning() { for (item <- items) { limiter.acquire() server.sync(item) } }This sample is very similar to the first one. Difference being that this time we block instead of discard missing permits. Thanks to blocking, external call to server.sync(item) won’t overload the 3rd-party server, calling it at most 20 times per second. Of course if you have several threads interacting with the server, they can all share the same RateLimiter. To wrap-up:RateLimiter allows you to perform certain actions not more often than with a given frequency It’s a small and lightweight class (no threads involved!) You can create thousands of rate limiters (per client?) or share one among several threads We haven’t covered warm-up functionality – if RateLimiter was completely idle for a long time, it will gradually increase allowed frequency over configured time up to configured maximum value instead of allowing maximum frequency from the very beginningI have a feeling that we’ll go back to this class soon. I hope you’ll find it useful in your next project! 1 – I am using Guava 14.0-SNAPSHOT. If 14.0 stable is not available by the time you are reading this, you must use more verbose tryAcquire(1, 0, TimeUnit.MICROSECONDS) instead of tryAcquire() and acquire(1) instead of acquire(). Happy coding and don’t forget to share! Reference: RateLimiter – discovering Google Guava from our JCG partner Tomasz Nurkiewicz at the Java and neighbourhood blog....
Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy | Contact
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.
Do you want to know how to develop your skillset and become a ...
Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

Get ready to Rock!
You can download the complementary eBooks using the links below: