Featured FREE Whitepapers

What's New Here?

career-logo

Mobile Development Job Trends – 2012-08

Today we have the last installment of the job trends posts, mobile development job trends. Everyone is talking about mobile development because it really is the future of interaction. With smart phones and tablets, app development is becoming hugely important even for the enterprise. The terms included in this list were iPhone, Android, WP7 or “Windows Phone“, BlackBerry, Symbian, WebOS and PhoneGap. There is some noise in the data, but not enough to significantly affect the trends. I am watching to see if Apache Cordova starts gaining adoption, but it barely registers on the trend graphs right now.First, let’s look at the basic job trends from Indeed:Based on this graph, you would think that Android demand has surpassed iPhone demand. Generally, this is not quite true because of the introduction of the iPad and iOS. The iPad does not change the demand too much, but adding iOS to the graph really changes the outlook:Now you can see that iOS demand is around 75% larger than Android, with iOS growth outpacing Android in 2012. As you can see, Blackberry is slowly declining as should be expected. WebOS and Symbian never really gained much attention, and the little that they did is obviously declining as well. Windows Phone is showing some growth, just not as quick as Android or iOS. You cannot see it very well in this graph, but PhoneGap is showing some solid growth during this year.Now we look at the short-term trends from SimplyHired:The SimplyHired trends are very difficult to get a good handle on. I tried adding the iPad and iOS to the query, which made the demand decrease. Obviously that makes no sense, so I reverted to only looking at the iPhone demand. Oddly, iPhone demand had a huge dip around May, but rebounded nicely. All other mobile development looks to be declining. Granted, the general trend for Android still looks positive, but I am not sure what is changing the trend during the summer. Blackberry has a slow decline throughout the year as expected. The others barely register in this graph and do not seem to be showing any real growth. I am not sure why SimplyHired has trends that look so much different than Indeed.Lastly, we look at the relative scaling from Indeed, which shows trends based on job growth:This is another graph where things are really strange depending on the terms you add. If I include iPad and iOS, there is very stable growth, but much lower than what is shown above. The growth for iPhone does not seem to match the general trend lines either, but we do know that iOS development is growing rapidly. So, let’s look at a graph that does not include the iPhone line.That looks a lot more readable. You can see that WebOS had a spike in the beginning of 2011, but that has dropped off significantly and been stable for this year. Windows phone shows a very nice growth line, as does Android. PhoneGap is probably the biggest surprise, and something that needs to be watched. While it does not have the gross demand of the other technologies, this type of growth shows it is gaining some serious attention. With the breadth of devices that people need to support, I expect more device agnostic frameworks to become popular. The other technologies, Blackberry and Symbian, barely register on this graph.The next year will be interesting in mobile development as we see more devices introduced and we see the future of some of the older technologies. If you are looking at mobile development, your main focus should be on iPhone development and Android development. If you like the idea of cross-platform development, PhoneGap development is gaining adoption and a solid option to review.Reference: Mobile Development Job Trends – August 2012 from our JCG partner Rob Diana at the Regular Geek blog....
javafx-logo

JavaFX Tutorial – Basics

JavaFX seem to be gaining ground on the RIA space. With the right tools and development support, it wil definitely have its toll on the next best technology “thing!”. I’m not writing any JavaFX review here since there is a lot of Technology critiques out there that probably reviewed it extensively, but instead, I will write a simple tutorial on how can you develop JavaFX application in your MacOSX lion. First some pre-requisites:JavaFX Runtime Environment Java Runtime Environment JavaFX SDK JavaFX Scene Builder JavaFX IDE – I had chosen NetBeans 7 as it already has support for JavaFX.This can all be downloaded on the Oracle Website. You may google it. Requirement: Create a simple application that accepts (you guess it) person details (simple registration), a custom web browser and some analytics. Technology: JavaFX and JPA Step 1: Create the Database and Tables Simple Database and Table, download the SQL file: hereStep 2: Create the User Interface and specify the controller Using the JavaFX Scene Builder, create the user interface.Step 3: Development Code the App! NetBeans has its support for JPA – so I used it to interact with the database.Download the source: here Its basically a very simple application, but I think this sample will give you a brief introduction or a head start on how to actually develop an application using the platform. Not a bad alternative if you want to create Desktop Applications, which of course, .net offers much better solution. Though the main take away here is that JavaFX redefines Java Desktop Application Development – flexible enough to support the best of both worlds (Desktop and Web) and if thats not enough, it also supports mobile phones. Reference: JavaFX Tutorial – Basics from our JCG partner Alvin Reyes at the Alvin “Jay” Reyes Blog blog....
java-logo

Resource Bundle Tricks and Best Practices

Today is resource bundle day. This is the most well known mechanism for internationalization (i18n) in Java in general. Working with it should be a breeze. But there are many little questions that come up while getting your hands dirty with it. If you are feeling the same, this post is for you. Basics The java.util.ResourceBundle defines a standardized way for accessing translations in java. They contain locale-specific resources. Resource bundles belong to families whose members share a common base name, but whose names also have additional components that identify their locales. Each resource bundle in a family contains the same items, but the items have been translated for the locale represented by that resource bundle. Those are key/value pairs. The keys uniquely identify a locale-specific object in the bundle. The most basic example uses the following familiy: Messages.properties Messages_de.properties Messages_en.properties If you need to query a bundle in your application you simple call the ResourceBundle bundle = ResourceBundle.getBundle("Messages");method and query the returned bundle: bundle.getString("welcome.message");If you are wondering which Locale is going to be used here, you are right. The String constructor implicitly uses Locale.getDefault() to resolve the language. That might not be what you want. So you should ResourceBundle bundle = ResourceBundle.getBundle("Messages", locale); You cannot set the locale after you have retrieved the bundle. Every ResourceBundle has one defined locale. Naming stuff   Some thoughts about naming. Name the bundle properties after their contents. You can go a more general way by simply naming them “Messages” and “Errors” etc. but it also is possible to have a bundle per subsystem or component. Whatever fit’s your needs. Maintaining the contents isn’t easy with lots of entries. So any kind of contextual split makes developers happy. The bundle properties files are equivalent to classes; Name them accordingly. And further on you should find a common system for naming your keys. Depending on the split you have chosen for the property files you might also introduce some kind of subsystem or component namespace with your keys. Page prefixes are also possible. Think about this wisely and play around with it. You are aiming to have least possible dublicates in your keys. Encapsulating   As you have seen, you use the string representation of the bundles a lot. The fact that those are actually file-names (or better class-names) you would be better of with a simple enum which encapsulates everything for you: public enum ResourceBundles { MESSAGES("Messages"), ERRORS("Errors"); private String bundleName;ResourceBundles(String bundleName) { this.bundleName = bundleName; }public String getBundleName() { return bundleName; }@Override public String toString() { return bundleName; } }Having this you simply can write ResourceBundle bundle = ResourceBundle.getBundle(MESSAGES.getBundleName());Java Server Faces and ResourceBundles   To use resource bundles in your jsf based application you simple have to define them in your faces-config.xml and use the shortcuts in your xhtml files. <resource-bundle> <base-name>Messages</base-name> <var>msgs</var><h:outputLabel value="#{msgs['welcome.general']}" />JSF takes care of the rest. What about parameter substitution? Think about a key-value pair like the following: welcome.name=Hi {0}! How are you?You can pass the parameter via the f:param tag: <h:outputFormat value="#{msgs['welcome.name']}"> <f:param value="Markus" /> </h:outputFormat>To change the language you have to set a specific locale for your current FacesContext instance. It’s best to do this via a value change listener: public void countryLocaleCodeChanged(ValueChangeEvent e) { String newLocaleValue = e.getNewValue().toString(); //loop country map to compare the locale code for (Map.Entry<String, Object> entry : countries.entrySet()) { if (entry.getValue().toString().equals(newLocaleValue)) { FacesContext.getCurrentInstance() .getViewRoot().setLocale((Locale) entry.getValue()); } } }Resource Bundles in EJBs   JSF obviously is very easily integrated. What about using those bundles in EJBs? It is basically the same. You have the same mechanisms in place to get hand on the bundle and use it. There is one thing that you should keep in mind. You probably don’t want to always use the default locale. So you have to find a way to pass the locale down from the UI. If you are thinking about @Injecting the MessageBundle via a @Produces annotation you have to think more than one time. Especially if you are working with @Stateless EJBs. Those instances get pooled and you have to pass the Locale to any business method that needs to know about the current Locale. You typically would do this with a parameter object or some kind of user session profile. Don’t add the Locale as method signature all over. Resource Bundles from the DB   In most of the cases I see you need to pull the keys from a DB. Given the inner workings of the ResourceBundle (one “class” per locale) you end up having to implement the logic in your own ResourceBundle implementation. Most of the examples you find on the web do this by overriding the handleGetObject(String key) method. I don’t like this approach, especially since we have a far better way using the ResourceBundle.Control mechanism. Now you can override the newBundle() method and return your own ResourceBundle implementation. All you have to do is to set your own Control as a parent with your DatabaseResourceBundle: public DatabaseResourceBundle() { setParent(ResourceBundle.getBundle(BUNDLE_NAME, FacesContext.getCurrentInstance().getViewRoot().getLocale(), new DBControl())); }The DBControl returns MyResourceBundle which is a ListResourceBundle: protected class DBControl extends Control {@Override public ResourceBundle newBundle(String baseName, Locale locale, String format, ClassLoader loader, boolean reload) throws IllegalAccessException, InstantiationException, IOException { return new MyResources(locale); }/** * A simple ListResourceBundle */ protected class MyResources extends ListResourceBundle {private Locale locale;/** * ResourceBundle constructor with locale * * @param locale */ public MyResources(Locale locale) { this.locale = locale; }@Override protected Object[][] getContents() { TypedQuery<ResourceEntity> query = _entityManager.createNamedQuery("ResourceEntity.findForLocale", ResourceEntity.class); query.setParameter("locale", locale);List<ResourceEntity> resources = query.getResultList(); Object[][] all = new Object[resources.size()][2]; int i = 0; for (Iterator<ResourceEntity> it = resources.iterator(); it.hasNext();) { ResourceEntity resource = it.next(); all[i] = new Object[]{resource.getKey(), resource.getValue()}; values.put(resource.getKey(), resource.getValue()); i++; } return all; } } }As you can see, this is backed by an entitymanager and a simple ResourceEntity which has all the fields and NamedQueries necessary for building up the different bundles. @Id @GeneratedValue(strategy = GenerationType.AUTO) private Long id; @Column(name = "i18n_key") private String key; @Column(name = "i18n_value") private String value; @Column(name = "i18n_locale") private Locale locale;By putting the bundles in a private Map<String, String> values = new HashMap<String, String>(); you also have a good way of caching the results after the bundles have been build up for the first time. This still isn’t the best solution as ResourceBundles have a way of caching. But I might dig into this in more detail later. Until now, this bundle is cached forever (or at least until the next redeployment). Rewrite as Language Switch   On last thing to mention is that you also could have some fancy add-ons here. If you already have the JSF language switch magic in place it is simple to add ocpsoft’s rewrite to your application. This is a simple way to encode the language in the URLs like this http://yourhost.com/Bundle-Provider-Tricks/en/index.html All you have to do is to add rewrite to the game by adding two simple dependencies: <dependency> <groupId>org.ocpsoft.rewrite</groupId> <artifactId>rewrite-servlet</artifactId> <version>1.1.0.Final</version> </dependency> <dependency> <groupId>org.ocpsoft.rewrite</groupId> <artifactId>rewrite-integration-faces</artifactId> <version>1.1.0.Final</version> </dependency>Rewrite needs you to add your own ConfigurationProvider which is the central place to hold your rewriting rules. Implement the following: public class BundleTricksProvider extends HttpConfigurationProvider {@Override public Configuration getConfiguration(ServletContext context) { return ConfigurationBuilder.begin() // Locale Switch .addRule(Join.path("/{locale}/{page}.html").to("/{page}.xhtml") .where("page").matches(".*") .where("locale").bindsTo(PhaseBinding.to(El.property("#{languageSwitch.localeCode}")).after(PhaseId.RESTORE_VIEW))); }@Override public int priority() { return 10; } }Next is to add a file named “org.ocpsoft.rewrite.config.ConfigurationProvider” to your META-INF/services folder and put the fully qualified name of your ConfigurationProvider implementation there. One last thing to tweak is the logic in the LanguageSwitch bean. Rewrite isn’t able to trigger a ValueChangeEvent (as far as I know :)) so you have to add some magic to change the Locale while the setter is called. That’s it .. very easy! Reference: Resource Bundle Tricks and Best Practices from our JCG partner Markus Eisele at the Enterprise Software Development with Java blog....
java-logo

BTrace: hidden gem in Java developer toolbox

This post is about BTrace which I am considering as a hidden gem for Java developer. BTrace is a safe, dynamic tracing tool for the Java platform. BTrace can be used to dynamically trace a running Java program (similar to DTrace for OpenSolaris applications and OS). Shortly, the tool allows to inject tracing points without restarting or reconfiguring your Java application while it’s running. Moreover, though there are several ways to do that, the one I would like to discuss today is using JVisualVM tool from standard JDK bundle. What is very cool, BTrace itself uses Java language to define injection trace points. The approach looks very familiar if you ever did aspect-oriented programming (AOP). So let’s get started with a problem: we have an application which uses one of the NoSQL databases (f.e., let it be MongoDB) and suddenly starts to experience significant performance slowdown. Developers suspect that application runs too many queries or updates but cannot say it with confidence. Here BTrace can help. First thing first, let’s run JVisualVM and install BTrace plugin:JVisualVM should be restarted in order for plugin to appear. Now, while our application is up and running, let’s right click on it in JVisualVM applications tree:The following very intuitive BTrace editor (with simple toolbar) should appear:This is a place where tracing instrumentation could be defined and dynamically injected into the running application. BTrace has a very rich model in order to define what exactly should be traced: methods, constructors, method returns, errors, …. Also it supports aggregations out of the box so it quite easy to collect a bunch of metrics while application is running. For our problem, we would like to see which methods related to MongoDB are being executed. As my application uses Spring Data MongoDB, I am interested in which methods of any implementation of org.springframework.data.mongodb.core.MongoOperations interface are being called by application and how long every call takes. So I have defined a very simple BTrace script: import com.sun.btrace.*; import com.sun.btrace.annotations.*; import static com.sun.btrace.BTraceUtils.*;@BTrace public class TracingScript { @TLS private static String method;@OnMethod( clazz = '+org.springframework.data.mongodb.core.MongoOperations', method = '/.*/' ) public static void onMongo( @ProbeClassName String className, @ProbeMethodName String probeMethod, AnyType[] args ) { method = strcat( strcat( className, '::' ), probeMethod ); } @OnMethod( clazz = '+org.springframework.data.mongodb.core.MongoOperations', method = '/.*/', location = @Location( Kind.RETURN ) ) public static void onMongoReturn( @Duration long duration ) { println( strcat( strcat( strcat( strcat( 'Method ', method ), ' executed in ' ), str( duration / 1000 ) ), 'ms' ) ); } } Let me explain briefly what I am doing here. Basically, I would like to know when any method of any implementation of org.springframework.data.mongodb.core.MongoOperations is called (onMongo marks that) and duration of the call (onMongoReturn marks that in turn). Thread-local variable method holds full qualified method name (with a class), while thanks to useful BTrace predefined annotation, duration parameter holds the method execution time (in nanoseconds). Though it’s pure Java, BTrace allows only small subset of Java classes to be used. It’s not a problem as com.sun.btrace.BTraceUtils class provides a lot of useful methods (f.e., strcat) to fill the gaps. Running this script produces following output: ** Compiling the BTrace script ... *** Compiled ** Instrumenting 1 classes ... Method org.springframework.data.mongodb.core.MongoTemplate::maybeEmitEvent executed in 25ms Method org.springframework.data.mongodb.core.MongoTemplate::maybeEmitEvent executed in 3ms Method org.springframework.data.mongodb.core.MongoTemplate::getDb executed in 22ms Method org.springframework.data.mongodb.core.MongoTemplate::prepareCollection executed in 2ms Method org.springframework.data.mongodb.core.MongoTemplate::prepareCollection executed in 19ms Method org.springframework.data.mongodb.core.MongoTemplate::access$100 executed in 2ms Method org.springframework.data.mongodb.core.MongoTemplate::access$100 executed in 1ms Method org.springframework.data.mongodb.core.MongoTemplate::maybeEmitEvent executed in 3ms Method org.springframework.data.mongodb.core.MongoTemplate::maybeEmitEvent executed in 2ms Method org.springframework.data.mongodb.core.MongoTemplate::getDb executed in 2ms Method org.springframework.data.mongodb.core.MongoTemplate::prepareCollection executed in 1ms Method org.springframework.data.mongodb.core.MongoTemplate::prepareCollection executed in 6ms Method org.springframework.data.mongodb.core.MongoTemplate::access$100 executed in 1ms Method org.springframework.data.mongodb.core.MongoTemplate::access$100 executed in 0ms Method org.springframework.data.mongodb.core.MongoTemplate::maybeEmitEvent executed in 2ms Method org.springframework.data.mongodb.core.MongoTemplate::maybeEmitEvent executed in 1ms Method org.springframework.data.mongodb.core.MongoTemplate::getDb executed in 2ms Method org.springframework.data.mongodb.core.MongoTemplate::prepareCollection executed in 1ms Method org.springframework.data.mongodb.core.MongoTemplate::prepareCollection executed in 6ms Method org.springframework.data.mongodb.core.MongoTemplate::access$100 executed in 1ms Method org.springframework.data.mongodb.core.MongoTemplate::access$100 executed in 0ms Method org.springframework.data.mongodb.core.MongoTemplate::maybeEmitEvent executed in 2ms Method org.springframework.data.mongodb.core.MongoTemplate::maybeEmitEvent executed in 1ms ... As you can see, output contains bunch of inner classes which could easily be eliminated by providing more precise method name templates (or maybe even tracing MongoDB driver instead). I have just started to discover BTrace but I definitely see a great value for me as a developer from using this awesome tool. Reference: BTrace: hidden gem in Java developer toolbox from our JCG partner Andrey Redko at the Andriy Redko {devmind} blog....
agile-logo

Contracting in Agile – You try it

One of the key principles in Agile development is “Customer collaboration over contract negotiation” Unfortunately, that means that if you’re trying to follow Agile methods, you’re left without useful guidelines to follow when it comes to contracting and coming up with contracts that fit the way that Agile teams work. Time-and-materials of course is a no-brainer, regardless of how the team works – do the work, track the time and other costs, and charge the customer as you do it. But it’s especially challenging for people who have to work within contract structures such as fixed price / fixed scope, which is the way that many government contracts are awarded and the way that a number of large businesses still contract development work. The advice for Agile teams usually runs something like: it’s up to you to convince the purchaser to change the rules and accept a fuzzier, more balanced way of contracting, with more give-and-take. Something that fits the basic assumptions of Agile development: that costs (mostly the people on the team) and schedule can be fixed, but the scope needs to be flexible and worked out as the project goes on. But in many business situations the people paying for the work aren’t interested in changing how they think and plan – it’s their money and they want what they want when they want it. They are calling the shots. If you don’t comply with the terms of the bidding process, you don’t get the opportunity to work with the customer at all. And the people paying you (your management) also need to know how much it is going to cost and when it is going to be done and what the risks are so they know if they can afford to take the project on. This puts the developers in a difficult (maybe impossible) situation.Money for Nothing and Change for Free Jeff Sutherland, one of the creators of Scrum, proposes a contract structure called “Money for Nothing and your Change for Free”. The development team delivers software incrementally – if they are following Scrum properly, they should start with the work that is most important to the customer first, and deliver what the customer needs the most as early as possible. The customer can terminate the contract at any point (because they’ve already got what they really need), and pay some percentage of the remainder of the contract to compensate the developer for losing the revenue that they planned to get for completing the entire project. So obviously, the payment schedule for the contract can’t be weighted towards the end of the project (no large payments on “final acceptance” since it may never happen). That’s the “money for nothing” part. “Change for free” means that the customer can’t add scope to the project, but can make changes as long as they substitute work still to be done in the backlog with work that is the same size or smaller. So new work can come up, the customer can change their mind, but the overall size of the project remains the same, which means that the team should still be able to deliver the project by the scheduled end date. To do this you have to define, understand and size all of the work that needs to be done upfront – which doesn’t fit well with the iterative, incremental way that Agile teams work. And it ignores the fact that changes still carry a price: the developers have to throw away the time that they spent upfront understanding what needed to be done enough to estimate it and the work that they went in to planning it, and they have to do more work to review and understand the change, estimate it and replan. Change is cheap in Agile development, but it’s not free. If the customer needs to make a handful of changes, the cost isn’t great. But it can become a real drag to delivery and add significant cost if a customer does this dozens or hundreds of times over a project.Fixed Price and Fixed Everything Contracts Fixed Price contracts, and especially what Alistair Cockburn calls Fixed-Everything contracts (fixed-price, fixed-scope and fixed-time too) are a nasty fact of business. Cockburn says that these contracts are usually created out of lack of trust – the people paying for the system to be developed don’t trust the people building the software to do what they need, and try to push the risk onto the development team. Even if people started out trusting each other, these contracts often create an environment where trust breaks down – the customer doesn’t trust the developers, the developers hide things from the customer, and the people who are paying the developers don’t trust anybody. But it’s still a common way to contract work because for many customers it is easier for them to plan around and it makes sense for organizations that think of software development projects as engineering projects and that want to treat software engineering projects the same way as they do building a road or a bridge. This is what we told you we want, this is when we need it, that’s how much you said it was going to cost (including your risk and profit margin), we agree that’s what we’re willing to pay, now go build it and we’ll pay you when you get it done. Cockburn does talk about a case where a team was successful in changing a fixed-everything contract into a time-and-materials contract over time, by working closely with the customer and proving that they could give the customer what they needed. After each delivery, the team would meet with the customer and discuss whether to continue with the contract as written or work on something that customer really needed instead, renegotiating the contract as they went on. I’ve seen this happen, but it’s rare, unless both companies do a lot of work together and the stakes of failure on a project are low. Ken Schwaber admits that fixed price contracting can’t be done with Scrum projects (read the book). Again, the solution is to convince the customer to accept and pay for work in an incremental, iterative way. Martin Fowler says that you can’t deliver a fixed price, fixed time and fixed scope contract without detailed, stable and accurate requirements – which he believes can’t be done. His solution is to fix the price and time, and then work with the customer to deliver what you can by the agreed end date, and hope that this will be enough. The most useful reference I’ve found on contracting in Agile projects is the freely-available Agile Contracts Primer from Practices for Scaling Lean and Agile Development, by Arbogast, Larman And Vodde. Their advice: avoid fixed-priced, fixed-scope (FPFS) contracts, because they are a lose-lose for both customer and supplier. The customer is less likely to get what they need because the supplier will at some point panic over delivery and be forced to cut quality; and if the supplier is able to deliver, the customer has to pay more than they should because of the risk premium that the supplier has to add. And working this way leads to a lack of transparency and to game playing on both sides. But, if you have to do it:Obviously it’s going to require up-front planning and design work to understand and estimate everything that has to get done – which means you have to bend Agile methods a lot. You don’t have to allow changes – you can just work incrementally from the backlog that is defined upfront. Or you can restrict the customer to only changing their mind on priority of work to be done (which gives them transparency and some control), or allow them to substitute a new requirement for an existing requirement of the same size (Sutherland’s “Change for Free”).To succeed in this kind of contract you have to:Invest a lot to do detailed, upfront requirements analysis, some design work, thorough acceptance test definition and estimation – by experienced people who are going to do the work Don’t allow changes in requirements or scope – just replacement / substitution Increase the margin of the contract price Make sure that you understand the problem you are working on – the domain and technology Deliver important things early and hope that the customer will be flexible with you towards the end if you still can’t deliver everything.PMI-ACP on Agile Contracting? For all of the projects that have been delivered using Agile methods, contracting seems to be still a work in progress. There are lots of good ideas and suggestions, but no solid answers. I’ve gone through the study guide materials for the PMI-ACP certification to see what PMI has to say about contracting in Agile projects. There is the same stuff about Sutherland’s “Money for Nothing and your Change for Free” and a few other options. It’s clear that the PMI didn’t take contracting in Agile projects on as a serious problem. This means that they missed another opportunity to help large organizations and people working with large organizations (the kind of people who are going to care about the PMI-ACP certification) to understand how to work with Agile methods in real-life situations. Reference: Contracting in Agile – You try it from our JCG partner Jim Bird at the Building Real Software blog....
java-logo

Java 7: HashMap vs ConcurrentHashMap

As you may have seen from my past performance related articles and HashMap case studies, Java thread safety problems can bring down your Java EE application and the Java EE container fairly easily. One of most common problems I have observed when troubleshooting Java EE performance problems is infinite looping triggered from the non-thread safe HashMap get() and put() operations. This problem is known since several years but recent production problems have forced me to revisit this issue one more time. This article will revisit this classic thread safety problem and demonstrate, using a simple Java program, the risk associated with a wrong usage of the plain old java.util.HashMap data structure involved in a concurrent threads context. This proof of concept exercise will attempt to achieve the following 3 goals:Revisit and compare the Java program performance level between the non-thread safe and thread safe Map data structure implementations (HashMap, Hashtable, synchronized HashMap, ConcurrentHashMap) Replicate and demonstrate the HashMap infinite looping problem using a simple Java program that everybody can compile, run and understand Review the usage of the above Map data structures in a real-life and modern Java EE container implementation such as JBoss AS7For more detail on the ConcurrentHashMap implementation strategy, I highly recommend the great article from Brian Goetz on this subject. Tools and server specifications As a starting point, find below the different tools and software’s used for the exercise:Sun/Oracle JDK & JRE 1.7 64-bit Eclipse Java EE IDE Windows Process Explorer (CPU per Java Thread correlation) JVM Thread Dump (stuck thread analysis and CPU per Thread correlation)The following local computer was used for the problem replication process and performance measurements:Intel(R) Core(TM) i5-2520M CPU @ 2.50Ghz (2 CPU cores, 4 logical cores) 8 GB RAM Windows 7 64-bit* Results and performance of the Java program may vary depending of your workstation or server specifications. Java program In order to help us achieve the above goals, a simple Java program was created as per below:The main Java program is HashMapInfiniteLoopSimulator.java A worker Thread class WorkerThread.java was also createdThe program is performing the following:Initialize different static Map data structures with initial size of 2 Assign the chosen Map to the worker threads (you can chose between 4 Map implementations) Create a certain number of worker threads (as per the header configuration). 3 worker threads were created for this proof of concept NB_THREADS = 3; Each of these worker threads has the same task: lookup and insert a new element in the assigned Map data structure using a random Integer element between 1 – 1 000 000. Each worker thread perform this task for a total of 500K iterations The overall program performs 50 iterations in order to allow enough ramp up time for the HotSpot JVM The concurrent threads context is achieved using the JDK ExecutorServiceAs you can see, the Java program task is fairly simple but complex enough to generate the following critical criteria’s:Generate concurrency against a shared / static Map data structure Use a mix of get() and put() operations in order to attempt to trigger internal locks and / or internal corruption (for the non-thread safe implementation) Use a small Map initial size of 2, forcing the internal HashMap to trigger an internal rehash/resizeFinally, the following parameters can be modified at your convenience: ## Number of worker threads private static final int NB_THREADS = 3;## Number of Java program iterations private static final int NB_TEST_ITERATIONS = 50;## Map data structure assignment. You can choose between 4 structures // Plain old HashMap (since JDK 1.2) threadSafeMap1 = new Hashtable<String, Integer>(2);// Plain old Hashtable (since JDK 1.0) threadSafeMap1 = new Hashtable<String, Integer>(2); // Fully synchronized HashMap threadSafeMap2 = new HashMap<String, Integer>(2); threadSafeMap2 = Collections.synchronizedMap(threadSafeMap2);// ConcurrentHashMap (since JDK 1.5) threadSafeMap3 = new ConcurrentHashMap<String, Integer>(2);/*** Assign map at your convenience ****/ assignedMapForTest = threadSafeMap3;Now find below the source code of our sample program. #### HashMapInfiniteLoopSimulator.java package org.ph.javaee.training4;import java.util.Collections; import java.util.Map; import java.util.HashMap; import java.util.Hashtable;import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors;/** * HashMapInfiniteLoopSimulator * @author Pierre-Hugues Charbonneau * */ public class HashMapInfiniteLoopSimulator {private static final int NB_THREADS = 3; private static final int NB_TEST_ITERATIONS = 50; private static Map<String, Integer> assignedMapForTest = null; private static Map<String, Integer> nonThreadSafeMap = null; private static Map<String, Integer> threadSafeMap1 = null; private static Map<String, Integer> threadSafeMap2 = null; private static Map<String, Integer> threadSafeMap3 = null; /** * Main program * @param args */ public static void main(String[] args) { System.out.println("Infinite Looping HashMap Simulator"); System.out.println("Author: Pierre-Hugues Charbonneau"); System.out.println("http://javaeesupportpatterns.blogspot.com"); for (int i=0; i<NB_TEST_ITERATIONS; i++) { // Plain old HashMap (since JDK 1.2) nonThreadSafeMap = new HashMap<String, Integer>(2); // Plain old Hashtable (since JDK 1.0) threadSafeMap1 = new Hashtable<String, Integer>(2); // Fully synchronized HashMap threadSafeMap2 = new HashMap<String, Integer>(2); threadSafeMap2 = Collections.synchronizedMap(threadSafeMap2); // ConcurrentHashMap (since JDK 1.5) threadSafeMap3 = new ConcurrentHashMap<String, Integer>(2); // ConcurrentHashMap /*** Assign map at your convenience ****/ assignedMapForTest = threadSafeMap3; long timeBefore = System.currentTimeMillis(); long timeAfter = 0; Float totalProcessingTime = null; ExecutorService executor = Executors.newFixedThreadPool(NB_THREADS); for (int j = 0; j < NB_THREADS; j++) { /** Assign the Map at your convenience **/ Runnable worker = new WorkerThread(assignedMapForTest); executor.execute(worker); } // This will make the executor accept no new threads // and finish all existing threads in the queue executor.shutdown(); // Wait until all threads are finish while (!executor.isTerminated()) { } timeAfter = System.currentTimeMillis(); totalProcessingTime = new Float( (float) (timeAfter - timeBefore) / (float) 1000); System.out.println("All threads completed in "+totalProcessingTime+" seconds"); } }}#### WorkerThread.java package org.ph.javaee.training4;import java.util.Map;/** * WorkerThread * * @author Pierre-Hugues Charbonneau * */ public class WorkerThread implements Runnable { private Map<String, Integer> map = null;public WorkerThread(Map<String, Integer> assignedMap) { this.map = assignedMap; }@Override public void run() { for (int i=0; i<500000; i++) { // Return 2 integers between 1-1000000 inclusive Integer newInteger1 = (int) Math.ceil(Math.random() * 1000000); Integer newInteger2 = (int) Math.ceil(Math.random() * 1000000); // 1. Attempt to retrieve a random Integer element Integer retrievedInteger = map.get(String.valueOf(newInteger1)); // 2. Attempt to insert a random Integer element map.put(String.valueOf(newInteger2), newInteger2); } }}Performance comparison between thread safe Map implementations The first goal is to compare the performance level of our program when using different thread safe Map implementations:Plain old Hashtable (since JDK 1.0) Fully synchronized HashMap (via Collections.synchronizedMap()) ConcurrentHashMap (since JDK 1.5)Find below the graphical results of the execution of the Java program for each iteration along with a sample of the program console output.# Output when using ConcurrentHashMap Infinite Looping HashMap Simulator Author: Pierre-Hugues Charbonneauhttp://javaeesupportpatterns.blogspot.comAll threads completed in 0.984 seconds All threads completed in 0.908 seconds All threads completed in 0.706 seconds All threads completed in 1.068 seconds All threads completed in 0.621 seconds All threads completed in 0.594 seconds All threads completed in 0.569 seconds All threads completed in 0.599 seconds ………………As you can see, the ConcurrentHashMap is the clear winner here, taking in average only half a second (after an initial ramp-up) for all 3 worker threads to concurrently read and insert data within a 500K looping statement against the assigned shared Map. Please note that no problem was found with the program execution e.g. no hang situation. The performance boost is definitely due to the improved ConcurrentHashMap performance such as the non-blocking get() operation. The 2 other Map implementations performance level was fairly similar with a small advantage for the synchronized HashMap. HashMap infinite looping problem replication The next objective is to replicate the HashMap infinite looping problem observed so often from Java EE production environments. In order to do that, you simply need to assign the non-thread safe HashMap implementation as per code snippet below: /*** Assign map at your convenience ****/ assignedMapForTest = nonThreadSafeMap;Running the program as is using the non-thread safe HashMap should lead to:No output other than the program header Significant CPU increase observed from the system At some point the Java program will hang and you will be forced to kill the Java processWhat happened? In order to understand this situation and confirm the problem, we will perform a CPU per Thread analysis from the Windows OS using Process Explorer and JVM Thread Dump. 1 – Run the program again then quickly capture the thread per CPU data from Process Explorer as per below. Under explore.exe you will need to right click over the javaw.exe and select properties. The threads tab will be displayed. We can see overall 4 threads using almost all the CPU of our system.2 – Now you have to quickly capture a JVM Thread Dump using the JDK 1.7 jstack utility. For our example, we can see our 3 worker threads which seems busy/stuck performing get() and put() operations. ..\jdk1.7.0\bin>jstack 272 2012-08-29 14:07:26 Full thread dump Java HotSpot(TM) 64-Bit Server VM (21.0-b17 mixed mode):"pool-1-thread-3" prio=6 tid=0x0000000006a3c000 nid=0x18a0 runnable [0x0000000007ebe000] java.lang.Thread.State: RUNNABLE at java.util.HashMap.put(Unknown Source) at org.ph.javaee.training4.WorkerThread.run(WorkerThread.java:32) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)"pool-1-thread-2" prio=6 tid=0x0000000006a3b800 nid=0x6d4 runnable [0x000000000805f000] java.lang.Thread.State: RUNNABLE at java.util.HashMap.get(Unknown Source) at org.ph.javaee.training4.WorkerThread.run(WorkerThread.java:29) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)"pool-1-thread-1" prio=6 tid=0x0000000006a3a800 nid=0x2bc runnable [0x0000000007d9e000] java.lang.Thread.State: RUNNABLE at java.util.HashMap.put(Unknown Source) at org.ph.javaee.training4.WorkerThread.run(WorkerThread.java:32) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) ..............It is now time to convert the Process Explorer thread ID DECIMAL format to HEXA format as per below. The HEXA value allows us to map and identify each thread as per below: ## TID: 1748 (nid=0X6D4)Thread name: pool-1-thread-2 CPU @25.71% Task: Worker thread executing a HashMap.get() operationat java.util.HashMap.get(Unknown Source) at org.ph.javaee.training4.WorkerThread.run(WorkerThread.java:29) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)## TID: 700 (nid=0X2BC)Thread name: pool-1-thread-1 CPU @23.55% Task: Worker thread executing a HashMap.put() operationat java.util.HashMap.put(Unknown Source) at org.ph.javaee.training4.WorkerThread.run(WorkerThread.java:32) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)## TID: 6304 (nid=0X18A0)Thread name: pool-1-thread-3 CPU @12.02% Task: Worker thread executing a HashMap.put() operationat java.util.HashMap.put(Unknown Source) at org.ph.javaee.training4.WorkerThread.run(WorkerThread.java:32) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)## TID: 5944 (nid=0X1738)Thread name: pool-1-thread-1 CPU @20.88% Task: Main Java program execution"main" prio=6 tid=0x0000000001e2b000 nid=0x1738 runnable [0x00000000029df000] java.lang.Thread.State: RUNNABLE at org.ph.javaee.training4.HashMapInfiniteLoopSimulator.main(HashMapInfiniteLoopSimulator.java:75) As you can see, the above correlation and analysis is quite revealing. Our main Java program is in a hang state because our 3 worker threads are using lot of CPU and not going anywhere. They may appear ‘stuck’ performing HashMap get() & put() but in fact they are all involved in an infinite loop condition. This is exactly what we wanted to replicate. HashMap infinite looping deep dive Now let’s push the analysis one step further to better understand this looping condition. For this purpose, we added tracing code within the JDK 1.7 HashMap Java class itself in order to understand what is happening. Similar logging was added for the put() operation and also a trace indicating that the internal & automatic rehash/resize got triggered. The tracing added in get() and put() operations allows us to determine if the for() loop is dealing with circular dependency which would explain the infinite looping condition. #### HashMap.java get() operation public V get(Object key) { if (key == null) return getForNullKey(); int hash = hash(key.hashCode()); /*** P-H add-on- iteration counter ***/ int iterations = 1; for (Entry<K,V> e = table[indexFor(hash, table.length)]; e != null; e = e.next) { /*** Circular dependency check ***/ Entry<K,V> currentEntry = e; Entry<K,V> nextEntry = e.next; Entry<K,V> nextNextEntry = e.next != null?e.next.next:null; K currentKey = currentEntry.key; K nextNextKey = nextNextEntry != null?(nextNextEntry.key != null?nextNextEntry.key:null):null; System.out.println("HashMap.get() #Iterations : "+iterations++); if (currentKey != null && nextNextKey != null ) { if (currentKey == nextNextKey || currentKey.equals(nextNextKey)) System.out.println(" ** Circular Dependency detected! ["+currentEntry+"]["+nextEntry+"]"+"]["+nextNextEntry+"]"); } /***** END ***/ Object k; if (e.hash == hash && ((k = e.key) == key || key.equals(k))) return e.value; } return null; }HashMap.get() #Iterations : 1 HashMap.put() #Iterations : 1 HashMap.put() #Iterations : 1 HashMap.put() #Iterations : 1 HashMap.put() #Iterations : 1 HashMap.resize() in progress... HashMap.put() #Iterations : 1 HashMap.put() #Iterations : 2 HashMap.resize() in progress... HashMap.resize() in progress... HashMap.put() #Iterations : 1 HashMap.put() #Iterations : 2 HashMap.put() #Iterations : 1 HashMap.get() #Iterations : 1 HashMap.get() #Iterations : 1 HashMap.put() #Iterations : 1 HashMap.get() #Iterations : 1 HashMap.get() #Iterations : 1 HashMap.put() #Iterations : 1 HashMap.get() #Iterations : 1 HashMap.put() #Iterations : 1 ** Circular Dependency detected! [362565=362565][333326=333326]][362565=362565] HashMap.put() #Iterations : 2 ** Circular Dependency detected! [333326=333326][362565=362565]][333326=333326] HashMap.put() #Iterations : 1 HashMap.put() #Iterations : 1 HashMap.get() #Iterations : 1 HashMap.put() #Iterations : 1 ............................. HashMap.put() #Iterations : 56823Again, the added logging was quite revealing. We can see that following a few internal HashMap.resize() the internal structure became affected, creating circular dependency conditions and triggering this infinite looping condition (#iterations increasing and increasing…) with no exit condition. It is also showing that the resize() / rehash operation is the most at risk of internal corruption, especially when using the default HashMap size of 16. This means that the initial size of the HashMap appears to be a big factor in the risk & problem replication. Finally, it is interesting to note that we were able to successfully run the test case with the non-thread safe HashMap by assigning an initial size setting at 1000000, preventing any resize at all. Find below the merged graph results:The HashMap was our top performer but only when preventing an internal resize. Again, this is definitely not a solution to the thread safe risk but just a way to demonstrate that the resize operation is the most at risk given the entire manipulation of the HashMap performed at that time. The ConcurrentHashMap, by far, is our overall winner by providing both fast performance and thread safety against that test case. JBoss AS7 Map data structures usage We will now conclude this article by looking at the different Map implementations within a modern Java EE container implementation such as JBoss AS 7.1.2. You can obtain the latest source code from the github master branch. Find below the report:Total JBoss AS7.1.2 Java files (August 28, 2012 snapshot): 7302 Total Java classes using java.util.Hashtable: 72 Total Java classes using java.util.HashMap: 512 Total Java classes using synchronized HashMap: 18 Total Java classes using ConcurrentHashMap: 46Hashtable references were found mainly within the test suite components and from naming and JNDI related implementations. This low usage is not a surprise here. References to the java.util.HashMap were found from 512 Java classes. Again not a surprise given how common this implementation is since the last several years. However, it is important to mention that a good ratio was found either from local variables (not shared across threads), synchronized HashMap or manual synchronization safeguard so “technically” thread safe and not exposed to the above infinite looping condition (pending/hidden bugs is still a reality given the complexity with Java concurrency programming…this case study involving Oracle Service Bus 11g is a perfect example). A low usage of synchronized HashMap was found with only 18 Java classes from packages such as JMS, EJB3, RMI and clustering. Finally, find below a breakdown of the ConcurrentHashMap usage which was our main interest here. As you will see below, this Map implementation is used by critical JBoss components layers such as the Web container, EJB3 implementation etc. ## JBoss Single Sign On Used to manage internal SSO ID’s involving concurrent Thread access Total: 1 ## JBoss Java EE & Web Container Not surprising here since lot of internal Map data structures are used to manage the http sessions objects, deployment registry, clustering & replication, statistics etc. with heavy concurrent Thread access. Total: 11 ## JBoss JNDI & Security Layer Used by highly concurrent structures such as internal JNDI security management. Total: 4 ## JBoss domain & managed server management, rollout plans… Total: 7 ## JBoss EJB3 Used by data structures such as File Timer persistence store, application Exception, Entity Bean cache, serialization, passivation… Total: 8 ## JBoss kernel, Thread Pools & protocol management Used by high concurrent Threads Map data structures involved in handling and dispatching/processing incoming requests such as HTTP. Total: 3 ## JBoss connectors such as JDBC/XA DataSources… Total: 2 ## Weld (reference implementation of JSR-299: Contexts and Dependency Injection for the JavaTM EE platform) Used in the context of ClassLoader and concurrent static Map data structures involving concurrent Threads access. Total: 3 ## JBoss Test Suite Used in some integration testing test cases such as an internal Data Store, ClassLoader testing etc. Total: 3 Final words I hope this article has helped you revisit this classic problem and understand one of the common problems and risks associated with a wrong usage of the non-thread safe HashMap implementation. My main recommendation to you is to be careful when using an HashMap in a concurrent threads context. Unless you are a Java concurrency expert, I recommend that you use ConcurrentHashMap instead which offers a very good balance between performance and thread safety. As usual, extra due diligence is always recommended such as performing cycles of load & performance testing. This will allow you to detect thread safety and / or performance problems before you promote the solution to your client production environment. Reference: Java 7: HashMap vs ConcurrentHashMap from our JCG partner Pierre-Hugues Charbonneau at the Java EE Support Patterns & Java Tutorial blog....
software-development-2-logo

Product Management Slowing You Down?

Does product management slow down your company?What Causes Your Business to Be Slow?Paul Young put out the call for the third annual You Might Be A Product Manager… list. If you are spending your holiday wondering if Jason Calacanis is right, and product management is actually preventing your company from being successful, you might be a product manager.Facebook’s success — and mistakes — are based on its developer-driven culture, not because Zuckerberg is some evil mastermind. The Zuckerberg Doctrine: Developers design products with significantly improved speed and functionality compared to product managers and designers, outweighing potential mistakes and drawbacks. Jason Calacanis, Launch newsletter 002Sound like link bait? Maybe, but you’re probably a product manager :). Jason’s put his money where his mouth is – he changed the way things are done at Mahalo (his human-curated search / answers company). In under 30 days, we completely overhauled our product-development process, removing everything between the developer and iterating on the product. We eliminated positions and process. We made it clear the developers were to make the decisions even if those decisions resulted in a developer being 50 percent slower because they were busy *thinking* about the product (as opposed to just transcribing features from the product manager wireframes). Jason Calacanis, Launch newsletter 002As you read through the details of the analysis in Mr. Calacanis’ newsletter, you’ll see that his position is not ultimately as generalized as the above quotes appear to be – Mr. Calacanis (who I’ve seen, and to whom I’ve listened for several years, but never met) is talking specifically about startups. However, he uses AOL, Yahoo, MySpace and Google as his “bad” examples – not exactly startups.I think this logical flaw may be leading Mr. Calacanis to demonize the wrong bad actors.  ProductCamp Austin 6Implicit Product DecisionsWhen I was still writing software and leading teams that were writing software, I would occasionally point out that all software is designed – even if someone sits down and just starts typing code. There is, at the minimum, implicit design happening in the mind of the programmer. It might not be “good” design, but there is always design. There is always a method to the madness, just not always a considered method. The same is true of product management.The creation of every feature and capability, in every product, is preceded by the notion that having this capability is a good idea. That’s what product managers do – decide which capabilities a product should have. Eliminating product managers does not eliminate product management.If “developer led” companies are still doing product management, how then, are they moving faster? Mr. Calacanis addresses valid points – his “bad examples” do seem to move pretty slowly, and his “good examples” do seem to move faster. So what’s really different? John and Roger and I will be framing the discussion at ProductCamp around how to move faster, and not throw the baby out with the bathwater.I might describe what Mahalo has apparently done as follows:Mahalo, with product managers involved in product decisions, was not moving as fast as Mr. Calacanis desired. So they reorganized so that product managers were no longer involved in the process – in hopes of having a faster process. Mr. Calacanis indicated that in a trade-off between “better” and “faster,” he would prefer “faster.”AgilityThe core of the agile development philosophy is – “fail fast. learn. improve.”All of the other stuff in the implementations of agile comes back to this – consider the main ideas in the agile manifesto*People over process: empowerment to fail and learn and improve. Value working software: learning is experiential, and you can’t fail or improve without shipping. Collaboration: You have to understand someone else’s problem before you can solve it. Too many products emerge from insular and isolated “exploration.” Encourage, don’t inhibit change: If you punish failure you prevent learning. If you prevent that new knowledge from being applied, you make learning irrelevant.*Agreed – those aren’t the words used in the manifesto. Same ideas, though.When you talk about business agility, while the mechanics might be different, the goals are the same. Fail fast. Learn. Improve.Mr. Calacanis, in my interpretation, is saying that this is exactly what he wants the Mahalo team to do. I applaud that goal.Does he have to “kill all the product managers” in order to infuse his company with (business) agility?Is product management the antithesis of agility? By definition, product management is still happening – just without product managers. Maybe the product management process has room for improvement…  Market DrivenAs a good product manager, you are market driven. What does that mean to agility?Fail Fast. Maybe you’re making decisions that delay launches until you know “the right product.” That would hurt agility. Lean. Are you listening to your customers, and learning from them? Great! Improve. Does what you’ve learned lead to trying something different, with a hypothesis that it will be better this time?OK, so being market driven will help your company learn and improve. That’s the up-side. It may also enable (but not cause) some corporate dysfunction. If your organization punishes failure, or is afraid of mis-steps, and you’re market driven, you have already heard this conversation – don’t (schedule | design | release) until we get [feedback X] or [insight Y].A tight coupling with your market is a powerful tool. It can be used for good (learning) or evil (avoid failure).Whew. Good to know that being market driven is not the source of the problem. An organization that is afraid of failure is the problem.Mahalo’s experiment may work – Mr. Calacanis clearly intends to encourage the “fail fast” element. So, when his developers are doing product management, they will have an opportunity to succeed by being market driven.  BureaucracyIn fairness, Mr. Calacanis is really only prescribing the “no product managers” approach for startups. Startups are not particularly bureaucratic. Perhaps Mahalo was becoming bureaucratic. It is easy to see the big companies mentioned in the newsletter as being rife with bureaucracy. Even if you could fail fast, learn, and improve in a world of t-crossing and sign-offs, your definition of “fast” would not match your competitors.Piloting your company would be like flying a Cessna twin-prop airplane in a world of super-sonic Gulfstream jets.How are product managers introducing overhead into your product creation process? What parts of product management are “not worth the delays?”There are very real “slow things down” activities in the product creation process.Some activities can be removed. Some activities can be improved. Most activities can be done in parallel with the product creation process, eliminating delays.What Have You Seen?There are a lot of stories from the trenches out there – what are you seeing? Have you tackled this already? How did you make it better.I hope that our session focuses on helping attendees, in a very real way, make their product creation process more effective, and make their businesses more agile.Reference: Product Management Slowing You Down? from our JCG partner Scott Sehlhorst at the Business Analysis | Product Management | Software Requirements blog....
agile-logo

A Prototype is Worth a Thousand Lines of Code

A picture is worth a thousand words. A prototype is worth a thousand lines of code. Two key elements of product management – and of agile development are elicitation and feedback. Low fidelity artifacts can significantly improve both. Polished, codified prototypes can create problems that prevent you from getting the benefits of communication. Prototyping Anti-Pattern David Bernstein has three good quick-read articles about prototyping in the agile zone. The first one depicts the primary anti-pattern of prototyping – mis-set expectations. As I was walking him through the different screens I could see he was starting to get uneasy. As I talked I could see him becoming paler and tenser, as if he saw a ghost. After a while I asked him if he was ok. He finally said, “You mean to tell me that I just wrote you a check for over $100,000 for less than a week of work?” Prototyping Caveat The second article is quick reminder of one of the goals of a prototype. The purpose of a prototype is to elicit feedback from others or verify a design approach will work. When doing this we generally only concern ourselves with the “happy path” through our code. A Prototype is Not a Product David’s third article explores a little more about the mis-setting of expectations, and provides a good parallel from the film industry. Prototypes are rough sketches without the robustness of a final product and it is often confusing to users when we show them a half-baked version of their project for feedback, even though they know it is not yet finished. Agile Prototyping on a Napkin David’s advice is (good and) primarily about using low-fidelity prototypes to avoid disrupting the elicitation and feedback process. Folks in the user experience community have known this for a long time, and adapted their processes accordingly. Back in 2006, I wrote about prototype fidelity as part of exploring ways to use this insight in requirements gathering. Prototyping, in particular, is a great way to elicit implicit requirements – those unspoken (“you should have asked!”) requirements that your customer or stakeholder assumed you knew, or didn’t think to tell you.Multiple Levels of Interaction David focused on the initial “will this approach work?” elements of getting feedback on a design – and by inference, some low-fidelity user acceptance testing that the proposed approach will meet your customer’s requirements. Jan Miksovsky wrote an excellent article (also in 2006) that provides excellent guidance on how much fidelity to build into your prototype, depending on where you are in the design process. This is important – the type of feedback you need at different stages of your design process varies. Jan proposes that the first prototypes be rough sketches, just as David points out. Jan goes on to show how and when to add additional fidelity to your prototypes (read Jan’s article to see his great visual examples). Like I said, the user experience community has known about this for a long time.Prototypes Are For Two-Way Communication The key element, and the reason for creating prototypes, is to get two-way communication. A prototype is not just a status update about your design. A prototype is the start of a conversation. Prototyping is not only useful for design conversations, it is critical to understanding your requirements. Yes, a prototype will you get feedback about the design-execution of your proposed solution. More importantly, a prototype can help you make sure you’re solving the right market problems.Prototypes Are More Than Interface Mock-Ups The first thing we think about, and everything you’ve read so far in this article, is about getting feedback on the user interface of the product. Prototypes are also useful for gathering business rules*, understanding system complexity, and defining, clarifying, and validating requirements. *In that article, I show prototypes as “not being useful” for gathering business rules. At that time, I was thinking in terms of interface mock-ups only, not other prototypes. Use interface mock-ups when you need feedback about user interaction. Use other artifacts when you need feedback about other aspects of your product. You can use prototypes as an active-listening technique when gathering market data too. There are lots of ways to draw stuff to help with communication. I’ve regularly used flow charts to prototype processes. Within those flow-charts, which are initially prototypes of how the product will behave, are decision diamonds. Those decision diamonds prototype the enforcement of business rules within the to-be-created product. In the spirit of DRY (don’t repeat yourself), a quick polishing of your process diagram can serve as a persistent artifact of the requirements. Just like a user interface mock-up. In more complicated scenarios, I’ve also used UML statecharts and data flow diagrams to prototype behavior. In the section on up-front planning in last week’s article on risk management and release cadence, I talked about the mental leap that people need to make to envision a product that doesn’t yet exist. Prototypes are like big springboards that help people leap that chasm of imagination. Important Safety Tip Don’t use the wrong prototype (mock-up, flow chart, etc) for the wrong conversation or with the wrong stakeholder. Showing when and where business rules are being enforced (in the process being designed) is great for getting feedback about your interpretation and potential embodiment of those rules. It will be a disaster if you try and have this conversation with someone who is not the owner of those policies – like a representative user. Sometimes, policies are “owned” by non-technical people who struggle to read diagrams. You may have to walk them through how the diagram works. They’re smart – they’ll get it. Just remember – use the right prototype to emphasize the right ideas, and elicit the right feedback. Reference: A Prototype is Worth a Thousand Lines of Code from our JCG partner Scott Sehlhorst at the Business Analysis | Product Management | Software Requirements blog....
javafx-logo

JavaFX 2 GameTutorial Part 5

Introduction This is part five of a six part series related to a JavaFX 2 Game Tutorial. I know it’s been a long time since I blogged about gaming, but hopefully you’re still with me. If you would like a recap, please read Part 1, Part 2, Part 3, and Part 4 to find out where we left off. If you are up to date, then let’s get started! In this blog entry we will be incorporating sounds into our game.Figure 1: JavaFX Sound FXThere are many elements which make games incredibly fun such as animated effects, collisions, AI, and input. However, one of the most important ingredients to game play is sound. When games incorporate sound effects and music, the gamer will become highly immersed (ugh… like not realizing you are about to see the sun rise). Before we get into the details, let me give you some background history on sound used in PC games for the home computer. If you want to skip the history and get down to business, jump to the section called the ‘Sound Manager Service.’ The Sound Manager Service is responsible for maintaining sound assets used during the game. If you are really impatient and don’t care about the implementation details, jump down to ‘JavaFX Sound Demo.’ Important note: Remember to read the requirements before launching the demo.History If you want to understand today, you have to search yesterday. ~Pearl Buck Back in the day, when I was growing up, I learned that the Apple ][ computer was capable of playing sounds. The Apple ][ had one speaker that was only able to produce simple tones (8 bit mono sound). When I first generated tones (Mary had a little lamb), I was totally amazed. If you are interested in machine code using Applesoft Basic's peek and poke commands to compose music, visit 8 bit Sound and Fury. Even though 8 bits seemed very simple (because there were so few values), it was not. When creating sound effects for games, one of the most difficult things to manage was the timing or duration of the tones in conjunction with the sprites flying across the screen in a (near) simultaneous fashion. In the 90s during the reign of the Intel x86 architecture (the PC), the most popular sound card was called the Sound Blaster 16 made by Creative Technologies. In its prime, this sound card was quite amazing when playing games because it was a separate card having a chip set with the ability to play midi sounds and music in stereo (two channels). The sound card was bundled with a CD rom player allowing one to pop in a music CD. Another cool feature of the Sound Blaster was its 15-pin MIDI/Joystick multiport enabling game input devices to be connected. Today (the future), sound cards are able to support surround sound (3D audio effects), various sound formats, record, various music formats, midi, and mixing. Multitasking enables modern computers to play sounds/music on parallel tracks (simultaneously). Next, we will be creating a sound manager service that will be added to the game engine framework library (JFXGen).Sound Manager Service The GameWorld class contains services such as the sprite manager and (more recently) a reference to an instance of a SoundManager (singleton). The sound manager service is responsible for managing all of the game's sound effects. This service allows the developer to load sound clips (AudioClip) using the loadSoundEffects() method. After loading sound effects each audio clip can be retrieved using a unique id (String) mapped to the sound. The last method is the shutdown() method. When the application is exited, the stop method will invoke the GameWorld's shutdown() method which, in turn, calls the SoundManager object's shutdown to clean up any resources. The SoundManager instance has a thread pool that gets gracefully shutdown. Note: For brevity, I designed the SoundManager class to play simple audio clips, though not music, during the game. If you want to add music, please refer to the JavaDoc on the Media and MediaPlayer APIs. Shown below is the SoundManager class diagram:Figure 2: Class Diagram of the Sound ManagerThe following is source code for the SoundManager class: package carlfx.gameengine;import javafx.scene.media.AudioClip;import java.net.URL; import java.util.HashMap; import java.util.Map; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors;/** * Responsible for loading sound media to be played using an id or key. * Contains all sounds for use later. *</pre> <pre> * User: cdea */ public class SoundManager { ExecutorService soundPool = Executors.newFixedThreadPool(2); Map<String, AudioClip> soundEffectsMap = new HashMap<>();/** * Constructor to create a simple thread pool. * * @param numberOfThreads - number of threads to use media players in the map. */ public SoundManager(int numberOfThreads) { soundPool = Executors.newFixedThreadPool(numberOfThreads); }/** * Load a sound into a map to later be played based on the id. * * @param id - The identifier for a sound. * @param url - The url location of the media or audio resource. Usually in src/main/resources directory. */ public void loadSoundEffects(String id, URL url) { AudioClip sound = new AudioClip(url.toExternalForm()); soundEffectsMap.put(id, sound); }/** * Lookup a name resource to play sound based on the id. * * @param id identifier for a sound to be played. */ public void playSound(final String id) { Runnable soundPlay = new Runnable() { @Override public void run() { soundEffectsMap.get(id).play(); } }; soundPool.execute(soundPlay); }/** * Stop all threads and media players. */ public void shutdown() { soundPool.shutdown(); }}How do I play sound effects in JavaFX? In JavaFX 2, you can play small sound files efficiently with less overhead by using the AudioClip API. This API allows a sound to be played repeatably. An example would be a gamer firing the weapon (left mouse press) which makes a laser sound “pew pew!” Speaking of lasers in the demo game, I used a free sound file from the website FreeSound.org having the Creative Commons license. Since the file was a wav file format, it was larger than it needed to be. So, I decided to convert the file to an mp3 sound format. I felt it was important to reduce the size of the file (smaller footprint) for faster loading. When converting the file to an mp3 sound format, I used Sony’s Sound Forge software. Shown below is a code snippet to play small sound files: AudioClip sound = new AudioClip('laser.mp3'); sound.play();How do I play music in JavaFX? Although the SoundManager(my implementation) doesn’t play music, it is easy to add the capability. The following code snippet shows how to load an MP3 file to be played using the Media and MediaPlayer API:       Media media = new Media('hymetojoy.mp3'); MediaPlayer player = MediaPlayerBuilder.create() .media(media) .onReady( new Runnable() { @Override public void run() { player.play(); }) .build();JavaFX Sound Demo Requirements:Java 7 or later JavaFX 2.1 or later Windows XP or later (Should be available soon for Linux/MacOS)A simple Asteroid type game called ‘The Expanse’. Instructions:Right mouse click (on Windows) to fly ship. Left mouse click (left click on Windows mouse) to fire weapon. Key press ’2? to change to large missiles. (blue circular projectiles) Other key press defaults to smaller missiles. (red circular projectiles) Space bar key press will toggle a force field to protect the ship from enemies and asteroids.Click on the Launch button below to start the demo:Useful LinksApple ][ specs: http://apple2history.org/history/ah03/ 8 bit on the Apple ][: http://eightbitsoundandfury.ld8.org/programming.html Sound Blaster : http://en.wikipedia.org/wiki/Sound_Blaster JFXGen: https://github.com/carldea/JFXGen JavaFX’s AudioClip API: http://docs.oracle.com/javafx/2/api/javafx/scene/media/AudioClip.html Sony Sound Forge: http://www.sonycreativesoftware.com/soundforgesoftware Freesound.org: http://www.freesound.org Laser sound from Freesound.org: http://www.freesound.org/people/THE_bizniss/sounds/39459/ Creative commons license: http://creativecommons.org/licenses/sampling+/1.0/ Media API: http://docs.oracle.com/javafx/2/api/javafx/scene/media/Media.html MediaPlayer API: http://docs.oracle.com/javafx/2/api/javafx/scene/media/MediaPlayer.html Reference: JavaFX 2 GameTutorial Part 5 from our JCG partner Carl Dea at the Carl’s FX Blog blog....
android-logo

Android – Load images from web and caching

Have you gone through my previous article for Loading Remote Images in android?, actually it was the solution for loading images from web but not caching into the local storage so that it has to load image every times, instead here in this article i am going to show you code/demo for loading images from web as well as to cache images in local also, so that whenever image has to be loaded it will check and load image from local if cached already.Solution: MainActivity.java package com.technotalkative.loadwebimage;import com.technotalkative.loadwebimage.R; import com.technotalkative.loadwebimage.imageutils.ImageLoader;import android.app.Activity; import android.os.Bundle; import android.view.View; import android.widget.ImageView;public class MainActivity extends Activity {private ImageView imgView; private ImageLoader imgLoader; private String strURL = "http://technotalkative.com/android.jpg"; @Override public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); imgView = (ImageView) findViewById(R.id.imageView1); imgLoader = new ImageLoader(this); } public void btnLoadImageClick(View v){ imgLoader.DisplayImage(strURL, imgView); } }ImageLoader.java Using DisplayImage(Url, ImageView) method of ImageLoader class, you can load and cache image. You just need to provide the web url of image and the imageview in which you want to display loaded image. package com.technotalkative.loadwebimage.imageutils;import java.io.File; import java.io.FileInputStream; import java.io.FileNotFoundException; import java.io.FileOutputStream; import java.io.InputStream; import java.io.OutputStream; import java.net.HttpURLConnection; import java.net.URL; import java.util.Collections; import java.util.Map; import java.util.WeakHashMap; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors;import android.app.Activity; import android.content.Context; import android.graphics.Bitmap; import android.graphics.BitmapFactory; import android.widget.ImageView;import com.technotalkative.loadwebimage.R;public class ImageLoader { MemoryCache memoryCache=new MemoryCache(); FileCache fileCache; private Map<ImageView, String> imageViews=Collections.synchronizedMap(new WeakHashMap<ImageView, String>()); ExecutorService executorService; public ImageLoader(Context context){ fileCache=new FileCache(context); executorService=Executors.newFixedThreadPool(5); } final int stub_id=R.drawable.ic_launcher; public void DisplayImage(String url, ImageView imageView) { imageViews.put(imageView, url); Bitmap bitmap=memoryCache.get(url); if(bitmap!=null) imageView.setImageBitmap(bitmap); else { queuePhoto(url, imageView); imageView.setImageResource(stub_id); } } private void queuePhoto(String url, ImageView imageView) { PhotoToLoad p=new PhotoToLoad(url, imageView); executorService.submit(new PhotosLoader(p)); } private Bitmap getBitmap(String url) { File f=fileCache.getFile(url); //from SD cache Bitmap b = decodeFile(f); if(b!=null) return b; //from web try { Bitmap bitmap=null; URL imageUrl = new URL(url); HttpURLConnection conn = (HttpURLConnection)imageUrl.openConnection(); conn.setConnectTimeout(30000); conn.setReadTimeout(30000); conn.setInstanceFollowRedirects(true); InputStream is=conn.getInputStream(); OutputStream os = new FileOutputStream(f); Utils.CopyStream(is, os); os.close(); bitmap = decodeFile(f); return bitmap; } catch (Throwable ex){ ex.printStackTrace(); if(ex instanceof OutOfMemoryError) memoryCache.clear(); return null; } }//decodes image and scales it to reduce memory consumption private Bitmap decodeFile(File f){ try { //decode image size BitmapFactory.Options o = new BitmapFactory.Options(); o.inJustDecodeBounds = true; BitmapFactory.decodeStream(new FileInputStream(f),null,o); //Find the correct scale value. It should be the power of 2. final int REQUIRED_SIZE=70; int width_tmp=o.outWidth, height_tmp=o.outHeight; int scale=1; while(true){ if(width_tmp/2<REQUIRED_SIZE || height_tmp/2<REQUIRED_SIZE) break; width_tmp/=2; height_tmp/=2; scale*=2; } //decode with inSampleSize BitmapFactory.Options o2 = new BitmapFactory.Options(); o2.inSampleSize=scale; return BitmapFactory.decodeStream(new FileInputStream(f), null, o2); } catch (FileNotFoundException e) {} return null; } //Task for the queue private class PhotoToLoad { public String url; public ImageView imageView; public PhotoToLoad(String u, ImageView i){ url=u; imageView=i; } } class PhotosLoader implements Runnable { PhotoToLoad photoToLoad; PhotosLoader(PhotoToLoad photoToLoad){ this.photoToLoad=photoToLoad; } @Override public void run() { if(imageViewReused(photoToLoad)) return; Bitmap bmp=getBitmap(photoToLoad.url); memoryCache.put(photoToLoad.url, bmp); if(imageViewReused(photoToLoad)) return; BitmapDisplayer bd=new BitmapDisplayer(bmp, photoToLoad); Activity a=(Activity)photoToLoad.imageView.getContext(); a.runOnUiThread(bd); } } boolean imageViewReused(PhotoToLoad photoToLoad){ String tag=imageViews.get(photoToLoad.imageView); if(tag==null || !tag.equals(photoToLoad.url)) return true; return false; } //Used to display bitmap in the UI thread class BitmapDisplayer implements Runnable { Bitmap bitmap; PhotoToLoad photoToLoad; public BitmapDisplayer(Bitmap b, PhotoToLoad p){bitmap=b;photoToLoad=p;} public void run() { if(imageViewReused(photoToLoad)) return; if(bitmap!=null) photoToLoad.imageView.setImageBitmap(bitmap); else photoToLoad.imageView.setImageResource(stub_id); } }public void clearCache() { memoryCache.clear(); fileCache.clear(); }}These are the required classes for loading image asynchronously and caching into the local memory storage. 1) MemoryCache 2) FileCache 3) Utils MemoryCache.java package com.technotalkative.loadwebimage.imageutils;import java.util.Collections; import java.util.Iterator; import java.util.LinkedHashMap; import java.util.Map; import java.util.Map.Entry; import android.graphics.Bitmap; import android.util.Log;public class MemoryCache {private static final String TAG = "MemoryCache"; private Map<String, Bitmap> cache=Collections.synchronizedMap( new LinkedHashMap<String, Bitmap>(10,1.5f,true));//Last argument true for LRU ordering private long size=0;//current allocated size private long limit=1000000;//max memory in bytespublic MemoryCache(){ //use 25% of available heap size setLimit(Runtime.getRuntime().maxMemory()/4); } public void setLimit(long new_limit){ limit=new_limit; Log.i(TAG, "MemoryCache will use up to "+limit/1024./1024.+"MB"); }public Bitmap get(String id){ try{ if(!cache.containsKey(id)) return null; //NullPointerException sometimes happen here http://code.google.com/p/osmdroid/issues/detail?id=78 return cache.get(id); }catch(NullPointerException ex){ ex.printStackTrace(); return null; } }public void put(String id, Bitmap bitmap){ try{ if(cache.containsKey(id)) size-=getSizeInBytes(cache.get(id)); cache.put(id, bitmap); size+=getSizeInBytes(bitmap); checkSize(); }catch(Throwable th){ th.printStackTrace(); } } private void checkSize() { Log.i(TAG, "cache size="+size+" length="+cache.size()); if(size>limit){ Iterator<Entry<String, Bitmap>> iter=cache.entrySet().iterator();//least recently accessed item will be the first one iterated while(iter.hasNext()){ Entry<String, Bitmap> entry=iter.next(); size-=getSizeInBytes(entry.getValue()); iter.remove(); if(size<=limit) break; } Log.i(TAG, "Clean cache. New size "+cache.size()); } }public void clear() { try{ //NullPointerException sometimes happen here http://code.google.com/p/osmdroid/issues/detail?id=78 cache.clear(); size=0; }catch(NullPointerException ex){ ex.printStackTrace(); } }long getSizeInBytes(Bitmap bitmap) { if(bitmap==null) return 0; return bitmap.getRowBytes() * bitmap.getHeight(); } }FileCache.java Using FileCache, we will create TTImages_cache folder for caching images into it. Also to load image if synced already. We can use clear() method of FileCache class to clear synced images. package com.technotalkative.loadwebimage.imageutils;import java.io.File; import android.content.Context;public class FileCache { private File cacheDir; public FileCache(Context context){ //Find the dir to save cached images if (android.os.Environment.getExternalStorageState().equals(android.os.Environment.MEDIA_MOUNTED)) cacheDir=new File(android.os.Environment.getExternalStorageDirectory(),"TTImages_cache"); else cacheDir=context.getCacheDir(); if(!cacheDir.exists()) cacheDir.mkdirs(); } public File getFile(String url){ //I identify images by hashcode. Not a perfect solution, good for the demo. String filename=String.valueOf(url.hashCode()); //Another possible solution (thanks to grantland) //String filename = URLEncoder.encode(url); File f = new File(cacheDir, filename); return f; } public void clear(){ File[] files=cacheDir.listFiles(); if(files==null) return; for(File f:files) f.delete(); }}Utils.java package com.technotalkative.loadwebimage.imageutils;import java.io.InputStream; import java.io.OutputStream;public class Utils { public static void CopyStream(InputStream is, OutputStream os) { final int buffer_size=1024; try { byte[] bytes=new byte[buffer_size]; for(;;) { int count=is.read(bytes, 0, buffer_size); if(count==-1) break; os.write(bytes, 0, count); } } catch(Exception ex){} } }activity_main.xml <RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools" android:layout_width="match_parent" android:layout_height="match_parent" ><Button android:id="@+id/btnLoadImage" android:layout_width="wrap_content" android:layout_height="wrap_content" android:layout_centerHorizontal="true" android:onClick="btnLoadImageClick" android:text="Load Image" /><ImageView android:id="@+id/imageView1" android:layout_width="match_parent" android:layout_height="200dp" android:layout_centerInParent="true" android:padding="10dp" android:scaleType="fitXY" android:src="@drawable/ic_launcher" /></RelativeLayout>Note: Don’t forget to include INTERNET and WRITE_EXTERNAL_STORAGE permission in AndroidManifest.xml <uses-permission android:name="android.permission.INTERNET" /> <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />Download this example from here: Android – Load images from web and syncing Reference: Android – Load images from web and caching from our JCG partner Paresh N. Mayani at the TechnoTalkative blog....
Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy | Contact
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.
Do you want to know how to develop your skillset and become a ...
Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

Get ready to Rock!
You can download the complementary eBooks using the links below:
Close