Featured FREE Whitepapers

What's New Here?


Create WildFly OpenShift application using Command Line Tools

A new instance of WildFly can be easily provisioned on OpenShift by using the quick start. Just a single click, and you are ready to go! Generally power users of OpenShift use Command Line Tools. However you could not create WildFly cartridge using the CLI tools. But bug# 1134134 is now resolved. And so now rhc cartridge-list shows:         workspaces> rhc cartridge-list jbossas-7 JBoss Application Server 7 web jboss-dv-6.0.0 (!) JBoss Data Virtualization 6 web jbosseap-6 (*) JBoss Enterprise Application Platform 6 web jenkins-1 Jenkins Server web nodejs-0.10 Node.js 0.10 web perl-5.10 Perl 5.10 web php-5.3 PHP 5.3 web php-5.4 PHP 5.4 web zend-6.1 PHP 5.4 with Zend Server 6.1 web python-2.6 Python 2.6 web python-2.7 Python 2.7 web python-3.3 Python 3.3 web ruby-1.8 Ruby 1.8 web ruby-1.9 Ruby 1.9 web ruby-2.0 Ruby 2.0 web jbossews-1.0 Tomcat 6 (JBoss EWS 1.0) web jbossews-2.0 Tomcat 7 (JBoss EWS 2.0) web jboss-vertx-2.1 (!) Vert.x 2.1 web jboss-wildfly-8 (!) WildFly Application Server 8.1.0.Final web diy-0.1 Do-It-Yourself 0.1 web 10gen-mms-agent-0.1 10gen Mongo Monitoring Service Agent addon cron-1.4 Cron 1.4 addon jenkins-client-1 Jenkins Client addon mongodb-2.4 MongoDB 2.4 addon mysql-5.1 MySQL 5.1 addon mysql-5.5 MySQL 5.5 addon phpmyadmin-4 phpMyAdmin 4.0 addon postgresql-8.4 PostgreSQL 8.4 addon postgresql-9.2 PostgreSQL 9.2 addon rockmongo-1.1 RockMongo 1.1 addon switchyard-0 SwitchYard 0.8.0 addon haproxy-1.4 Web Load Balancer addonNote: Web cartridges can only be added to new applications.(*) denotes a cartridge with additional usage costs.(!) denotes a cartridge that will not receive automatic security updates. The newly added cartridge of WildFly 8 is shown in bold. And so now a new WildFly instance can be easily provisioned using the CLI as: workspaces> rhc app-create mywildfly jboss-wildfly-8 Application Options ------------------- Domain: milestogo Cartridges: jboss-wildfly-8 Gear Size: default Scaling: noCreating application 'mywildfly' ... Artifacts deployed: ./ROOT.war doneWildFly 8 administrator added. Please make note of these credentials:Username: adminLWEMEdW Password: 9u6jpWuQU6hg run 'rhc port-forward mywildfly' to access the web admin area on port 9990.Waiting for your DNS name to be available ... doneCloning into 'mywildfly'... Warning: Permanently added the RSA host key for IP address '' to the list of known hosts.Your application 'mywildfly' is now available.URL: http://mywildfly-milestogo.rhcloud.com/ SSH to: 545ac1d0e0b8cd8ad800018d@mywildfly-milestogo.rhcloud.com Git remote: ssh://545ac1d0e0b8cd8ad800018d@mywildfly-milestogo.rhcloud.com/~/git/mywildfly.git/ Cloned to: /Users/arungupta/workspaces/mywildflyRun 'rhc show-app mywildfly' for more details about your app. And then the application’s main page is accessible as shown:And the application can be deleted as: workspaces> rhc app-delete mywildfly This is a non-reversible action! Your application code and data will be permanently deleted if you continue!Are you sure you want to delete the application 'mywildfly'? (yes|no): yesDeleting application 'mywildfly' ... deleted Simple, isn’t it ? See several other OpenShift getting-started related blog entries here.Reference: Create WildFly OpenShift application using Command Line Tools from our JCG partner Arun Gupta at the Miles to go 2.0 … blog....

Spring boot based websocket application and capturing http session id

I was involved in a project recently where we needed to capture the http session id for a websocket request – the reason was to determine the number of websocket sessions utilizing the same underlying http session. The way to do this is based on a sample utilizing the new spring-session module and is described here. The trick to capturing the http session id is in understanding that before a websocket connection is established between the browser and the server, there is a handshake phase negotiated over http and the session id is passed to the server during this handshake phase. Spring Websocket support provides a nice way to register a HandShakeInterceptor, which can be used to capture the http session id and set this in the sub-protocol(typically STOMP) headers. First, this is the way to capture the session id and set it to a header: public class HttpSessionIdHandshakeInterceptor implements HandshakeInterceptor {@Override public boolean beforeHandshake(ServerHttpRequest request, ServerHttpResponse response, WebSocketHandler wsHandler, Map<String, Object> attributes) throws Exception { if (request instanceof ServletServerHttpRequest) { ServletServerHttpRequest servletRequest = (ServletServerHttpRequest) request; HttpSession session = servletRequest.getServletRequest().getSession(false); if (session != null) { attributes.put("HTTPSESSIONID", session.getId()); } } return true; }public void afterHandshake(ServerHttpRequest request, ServerHttpResponse response, WebSocketHandler wsHandler, Exception ex) { } } And to register this HandshakeInterceptor with Spring Websocket support: @Configuration @EnableWebSocketMessageBroker public class WebSocketDefaultConfig extends AbstractWebSocketMessageBrokerConfigurer {@Override public void configureMessageBroker(MessageBrokerRegistry config) { config.enableSimpleBroker("/topic/", "/queue/"); config.setApplicationDestinationPrefixes("/app"); }@Override public void registerStompEndpoints(StompEndpointRegistry registry) { registry.addEndpoint("/chat").withSockJS().setInterceptors(httpSessionIdHandshakeInterceptor()); }@Bean public HttpSessionIdHandshakeInterceptor httpSessionIdHandshakeInterceptor() { return new HttpSessionIdHandshakeInterceptor(); }} Now that the session id is a part of the STOMP headers, this can be grabbed as a STOMP header, the following is a sample where it is being grabbed when subscriptions are registered to the server: @Component public class StompSubscribeEventListener implements ApplicationListener<SessionSubscribeEvent> {private static final Logger logger = LoggerFactory.getLogger(StompSubscribeEventListener.class);@Override public void onApplicationEvent(SessionSubscribeEvent sessionSubscribeEvent) { StompHeaderAccessor headerAccessor = StompHeaderAccessor.wrap(sessionSubscribeEvent.getMessage()); logger.info(headerAccessor.getSessionAttributes().get("HTTPSESSIONID").toString()); } } or it can be grabbed from a controller method handling websocket messages as a MessageHeaders parameter: @MessageMapping("/chats/{chatRoomId}") public void handleChat(@Payload ChatMessage message, @DestinationVariable("chatRoomId") String chatRoomId, MessageHeaders messageHeaders, Principal user) { logger.info(messageHeaders.toString()); this.simpMessagingTemplate.convertAndSend("/topic/chats." + chatRoomId, "[" + getTimestamp() + "]:" + user.getName() + ":" + message.getMessage()); }Here is a complete working sample which implements this pattern.Reference: Spring boot based websocket application and capturing http session id from our JCG partner Biju Kunjummen at the all and sundry blog....

Using Groovy for Bash (shell) Operations

Recently I needed to create a groovy script that deletes some directories in a Linux machine. Here’s why:                  We have a server for doing scheduled jobs. Jobs such as ETL from one DB to another, File to DB etc. The server activates clients, which are located in the machines we want to have action on them. Most (almost all) of the jobs are written in groovy scripts. Part of our CI process is deploying a WAR into a dedicated server. Then, we have a script that among other things uses soft-link to direct ‘webapps’ to the newly created directory. This deployment happens once an hour, which fills up the dedicated server quickly.So I needed to create a script that checks all directories in the correct location and deletes old ones. I decided to keep the latest 4 directories. It’s currently a magic number in the script. If I want / need I can make it as an input parameter. But I decided to start simple. I decided to do it very simple:List all directories with prefix webapp_ in a known location Sort them by time, descending, and run delete on all starting index 4.def numberOfDirectoriesToKeep = 4 def webappsDir = new File('/usr/local/tomcat/tomcat_aps') def webDirectories = webappsDir.listFiles().grep(~/.*webapps_.*/) def numberOfWeappsDirectories = webDirectories.size();if (numberOfWeappsDirectories >= numberOfDirectoriesToKeep) { webDirectories.sort{it.lastModified() }.reverse()[numberOfDirectoriesToKeep..numberOfWeappsDirectories-1].each { logger.info("Deleteing ${it}"); // here we'll delete the file. First try was doing a Java/groovy command of deleting directories } } else { logger.info("Too few web directories") } It didn’t work. Files were not deleted. It happened that the agent runs as a different user than the one that runs tomcat. The agent did not have permissions to remove the directories. My solution was to run a shell command with sudo. I found references at:http://www.joergm.com/2010/09/executing-shell-commands-in-groovy/and:http://groovy.codehaus.org/Executing+External+Processes+From+GroovyTo make a long story short, here’s the full script: import org.slf4j.Logger import com.my.ProcessingJobResultdef Logger logger = jobLogger //ProcessingJobResult is proprietary def ProcessingJobResult result = jobResulttry { logger.info("Deleting old webapps from CI - START") def numberOfDirectoriesToKeep = 4 // Can be externalized to input parameter def webappsDir = new File('/usr/local/tomcat/tomcat_aps') def webDirectories = webappsDir.listFiles().grep(~/.*webapps_.*/) def numberOfWeappsDirectories = webDirectories.size();if (numberOfWeappsDirectories >= numberOfDirectoriesToKeep) { webDirectories.sort{it.lastModified() }.reverse()[numberOfDirectoriesToKeep..numberOfWeappsDirectories-1].each { logger.info("Deleteing ${it}"); def deleteCommand = "sudo -u tomcat rm -rf " + it.toString(); deleteCommand.execute(); } } else { logger.info("Too few web directories") } result.status = Boolean.TRUE result.resultDescription = "Deleting old webapps from CI ended" logger.info("Deleting old webapps from CI - DONE") } catch (Exception e) { logger.error(e.message, e) result.status = Boolean.FALSE result.resultError = e.message }return result BTW, there’s a minor bug of indexes, which I decided not to fix (now), as we always have more directories.Reference: Using Groovy for Bash (shell) Operations from our JCG partner Eyal Golan at the Learning and Improving as a Craftsman Developer blog....

Clustering reactmq with akka-cluster

In the last two posts on reactmq, I described how to write a reactive, persistent message queue. The queue has the following characteristics:                  there is a single broker storing the messages, with potentially multiple clients, either sending or receiving messages provides at-least-once-delivery; receiving a message blocks the message; unless it is deleted within some time it will be available for delivery again sending & receiving messages is reactive thanks to akka-streams: back-pressure is applied if any of the components cannot keep up with the load messages are persistent, using akka-persistence: all send/receive/delete events are persisted in a journal, and when the system is restarted the state of the queue actor is re-createdThe event store can be replicated by using a replicated journal implementation, e.g. Cassandra. However, there is one piece missing: clustering the broker itself. Luckily, akka-cluster is there to help! Let’s see how using it we can make sure that there’s always a broker running in our cluster.Cluster setup First of all we need a working cluster. We will need two new dependencies in our build.sbt file, akka-cluster and akka-contrib, as we will be using some contributed extensions. Secondly, we need to provide configuration for the ActorSystem with clustering enabled. It’s good to make it clear what’s the purpose of a config file, hence there’s the cluster-broker-template.conf file: akka { actor.provider = "akka.cluster.ClusterActorRefProvider"   remote.netty.tcp.port = 0 // should be overriden remote.netty.tcp.hostname = ""   cluster { seed-nodes = [ "akka.tcp://broker@", "akka.tcp://broker@", "akka.tcp://broker@" ] auto-down-unreachable-after = 10s roles = [ "broker" ] role.broker.min-nr-of-members = 2 }   extensions = [ "akka.contrib.pattern.ClusterReceptionistExtension" ] } Going through the settings:we need to use the ClusterActorRefProvider to communicate with actors on other cluster nodes as we will be using the same config file for multiple nodes (for local testing), the port will be overridden in code. Apart from the port, we also need to specify the hostname to which cluster communication will bind. the node needs an initial list of seed nodes, with which it will try to communicate on start-up to form a cluster we’re using auto-downing, which causes nodes to be declared down after 10 seconds. This can cause partitions, however we also specify that there need to be at least 2 nodes alive for the cluster fragment to be operative (we will have 3 nodes in total, so with 2 we are safe against partitions) finally, we are declaring that any cluster nodes started using the config file will have the broker role and use the cluster receptionist extension – but more on that later.Actually starting a clustered actor system is now very simple, see the BrokerManager class: class BrokerManager(clusterPort: Int) { // … val conf = ConfigFactory .parseString(s"akka.remote.netty.tcp.port=$clusterPort") .withFallback(ConfigFactory.load("cluster-broker-template"))   val system = ActorSystem(s"broker", conf) // ... } To specify the port we create configuration from a string with only the port part specified, and to use the other settings, we fall back to the configuration from the template file. Starting cluster nodes is just a matter of starting three simple applications: object ClusteredBroker1 extends App { new BrokerManager(9171).run() } Cluster singleton Having the cluster ready, we now have to start the broker on one of the nodes. At any time, there should be only one broker running – otherwise the queue would get corrupted. For that the Cluster Singleton contrib extension is perfect. To use the extension we need to create an actor, which will be managed by the singleton extension and started on only one cluster node. Hence we create the BrokerManagerActor and we can now start the singleton: def run() { // … system.actorOf(ClusterSingletonManager.props( singletonProps = Props(classOf[BrokerManagerActor], clusterPort), singletonName = "broker", terminationMessage = PoisonPill, role = Some("broker")), name = "broker-manager") }   class BrokerManagerActor(clusterPort: Int) extends Actor { val sendServerAddress = new InetSocketAddress( "localhost", clusterPort + 10) val receiveServerAddress = new InetSocketAddress( "localhost", clusterPort + 20)   override def preStart() = { super.preStart() new Broker(sendServerAddress, receiveServerAddress)(context.system) .run() }   override def receive = { case _ => } } In the ClusterSingletonManager properties, we specify the actor to run, the message that can be used to terminate the actor, and the cluster roles on which the actor can run (our clusters’ nodes have only one role, broker). The BrokerManagerActor takes a port (should be unique to each node if we want to run a couple on localhost), and creates basing on that an address on which the socket for new queue-message-sending-clients will listen, and another for the queue-message-receive-client listening socket. Cluster client & receptionist: cluster side We now have a single broker running in the cluster, but how can clients know what’s the address of the singleton? Well, we can just ask the BrokerManagerActor actor for that! This can be done by a simple message exchange: case object GetBrokerAddresses case class BrokerAddresses(sendServerAddress: InetSocketAddress, receiveServerAddress: InetSocketAddress)   class BrokerManagerActor(clusterPort: Int) extends Actor { // as above, plus: override def receive = { case GetBrokerAddresses => sender() ! BrokerAddresses( sendServerAddress, receiveServerAddress) } } One problem remains, though. The clients that want to use our message queue need not be members of the cluster. Here the Cluster Client contrib extension can help. On the cluster node side, the client extension provides a receptionist, with which actors, which want to be visible outside can register. And that’s what our BrokerManagerActor does on startup: class BrokerManagerActor(clusterPort: Int) extends Actor { // … override def preStart() = { // ... ClusterReceptionistExtension(context.system) .registerService(self) } } Cluster client & receptionist: client side As for the clients itself, they also need some configuration to communicate with the cluster: akka { actor.provider = "akka.remote.RemoteActorRefProvider"   remote.netty.tcp.port = 0 remote.netty.tcp.hostname = "" }   cluster.client.initial-contact-points = [ "akka.tcp://broker@", "akka.tcp://broker@", "akka.tcp://broker@" ] Again going through the settings:to communicate with remote actors (which live in the cluster), we need to use the RemoteActorRefProvider (the ClusterARP is a richer version of the RemoteARP) similarly to the seed nodes, we need to provide seed contact points, so that the client has some way of initiating communication with the clusterActually initiating a cluster client is quite straightforward, we need to create an actor system and create an actor which will communicate with the cluster: val conf = ConfigFactory.load("cluster-client") implicit val system = ActorSystem(name, conf)   val initialContacts = conf .getStringList("cluster.client.initial-contact-points") .asScala.map { case AddressFromURIString(addr) => system.actorSelection( RootActorPath(addr) / "user" / "receptionist") }.toSet   val clusterClient = system.actorOf( ClusterClient.props(initialContacts), "cluster-client") To start a client of our message queue (we have two types of clients: one sends messages to the queue, the other received messages from it), we need to find out what’s the address of the broker. To do that, we ask (using Akka’s ask pattern) the broker registered with the receptionist about its address: clusterClient ? ClusterClient.Send( "/user/broker-manager/broker", GetBrokerAddresses, localAffinity = false) .mapTo[BrokerAddresses] .flatMap { ba => logger.info(s"Connecting a $name using broker address $ba.") runClient(ba, system) } Finally, when the client stream completes (e.g. because a broker is down), we try to restart it after 1 second. Probably some exponential back-off mechanism would be useful here. The runnable application for running the queue-message senders and receivers uses the same code as the single-node one, the difference being that the broker address is obtained from the cluster: object ClusterReceiver extends App with ClusterClientSupport { start("receiver", (ba, system) => new Receiver(ba.receiveServerAddress)(system).run()) } Running If you try to run ClusterReceiver (any number), ClusterSender (any number), ClusteredBroker1, ClusteredBroker2 and ClusteredBroker3, you will see that that the messages flow from senders, through a single running broker, to the receivers. You can kill a broker node, and after a couple of seconds another one will be started on another cluster node, and the senders/receivers will re-connect. I’d say that’s quite nice for the pretty small amount of code we’ve written! Summing up Our message queue now is:reactive, using akka-streams persistent, using akka-persistence clustered, using akka-clusterAnd the best side is, in the code we don’t have to deal with any details of back-pressure handling, storing messages to disk or communicating and reaching consensus in the cluster. We now have a truly reactive application. Thanks to Endre from the Akka team for helping out with adding error handling to the streams. The whole code is available on GitHub. Enjoy!Reference: Clustering reactmq with akka-cluster from our JCG partner Adam Warski at the Blog of Adam Warski blog....

Efficient Creation of Eclipse Modules with Maven Archetype

Maven Archetype is a project templating toolkit that provides developers with the means to generate parameterized versions of built-in or custom scaffolding artefacts. Recently I applied it to our Xiliary P2 repository to achieve automation of Eclipse module stubs creation. As it worked out well enough, I thought it might be worth while to share my experiences in this post.         Maven Archetype Maven Archetype allows programmers to setup scaffolding quickly and consistent with respect to conventions and best practices prescribed by a project or organization. It comes with set of predefined templates to ease generation of commonly used structures. For a list of default archetypes provided by Maven please refer to Introduction to Archetypes, section Provided Archetypes. Generation of a web-app project stub for example can be based on the archetype maven-archetype-webapp: mvn archetype:generate \ -DgroupId=com.codeaffine \ -DartifactId=com.codeaffine.webapp \ -Dversion=1.0-SNAPSHOT \ -DarchetypeGroupId=org.apache.maven.archetypes \ -DarchetypeArtifactId=maven-archetype-webapp \ -DarchetypeVersion=1.0 \ -DinteractiveMode=false The parameters groupId, artifactId and version are used to create a project root folder containing an appropriate configured project object model definition (pom.xml). Whereas the archetypeXXX arguments specify which template to employ. Based on the web-app archetype Maven provides a pom that sets the build lifecycle packaging attribute to war and produces the following directory and file structure: com.codeaffine.webapp |-- pom.xml `-- src `-- main |-- resources `-- webapp |-- WEB-INF | `-- web.xml `-- index.jsp If you happen to work with the Maven Integration for Eclipse you can select the New Project wizard for Maven projects to generate an eclipse project derived from a particular archetype:The selection shown by the image creates the same structure as in the command line example above. Additionally it provides Eclipse project specific files and settings and imports the generated project automatically into the workspace of the IDE. Custom Archetype Creation While the predefined templates are good for a quick-start, they are obviously not sufficient to employ project or organization specific conventions. The Eclipse IDE for example allows to configure all kind of settings in files located within the generated scaffolding structure. Hence it would be helpful to include such presets in a custom archetype. Luckily Maven Archetype facilitates the creation of custom template definitions as explained in the online documentation Guide to Creating Archetypes. However instead of building up the archetype from scratch, I found it more efficient to use the create-from-project option as described by Dirk Reinemann. I was able to do this because I already had a couple of Eclipse plug-ins, fragments and features I could use as prototypes. I found a tycho-eclipse-plugin-archetype definition at GitHub providing a template for generating eclipse modules with test fragments and repository site in one step, which seems to provide a good quick start for eclipse plug-in development. Create from Project To create a Maven Archetype from a given project copy it to a working directory and remove all files that should not go into the archetype package. This project torso now provides all the files and the directory structure that make up the scaffolding. Ensure that the root folder of the torso also contains a simple pom.xml as explained in step one of the Guide to Creating Archetypes. After that navigate to the folder where the pom resides and execute the following command: mvn archetype:create-from-project This generates the archetype development structure stored in the subfolder target/generated-sources/archetype. It contains a pom for the new archetype that is derived from the one which was placed in the root folder of the project torso. Furthermore there is a sub path src/main/resources/archetype-resources that contains a copy of the scaffolding structure and resources. The file src/main/resources/META-INF/maven/archetype-metadata.xml is the ArchetypeDescriptor, which lists all the files that will be contained in newly created template and categorizes them, so they can be processed correctly by the generation mechanism. Now it is possible to package the archetype and give it a first try to see if it works as expected so far. To do so navigate to the folder where the archetype’s pom resides and run: mvn install This makes the archetype available in your local repository. Using it for the first time is as easy as in the web-app example above and should look somewhat like the following snippet: mvn archetype:generate \ -DarchetypeArtifactId=foo.artefactId \ -DarchetypeGroupId=foo.groupId \ -DarchetypeVersion=foo.version If done properly Maven should now have created a project stub that basically looks the same as the one composed in the the project torso. Adjustments Unfortunately there is still more work to do. Eclipse plug-ins, fragments and features provide their own meta descriptors containing identifiers, version numbers, names and the like. And of course we expect those values to be reasonably prepopulated by the template processor. Maven Archetypes handles this with properties which can be declared in the ArchetypeDescriptor (see above). <requiredProperties> <requiredProperty key="identifier"></requiredProperty> </requiredProperties> Now you can refer to this property in arbitrary resources of the archetype using the following syntax: [...] Bundle-SymbolicName: ${identifier} [...] Initialization of the property can be done by setting it as system parameter of the command line for example: mvn archetype:generate \ -DarchetypeArtifactId=foo.artefactId \ -DarchetypeGroupId=foo.groupId \ -DarchetypeVersion=foo.version \ -Didentifier=foo.identifier \ Another problem for plug-ins and fragments is e.g. the empty or non existing source folder referred to by the .project definition file. Maven ignores empty directories during template processing. But the following snippet shows how to configure the descriptor to create such a folders nevertheless: <fileSets> <fileSet filtered="true" encoding="UTF-8"> <directory>src</directory> <includes> <include>**/*.java</include> </includes> </fileSet> [...] For more details on descriptor configuration please refer to the online documentation. Assembling the Pieces Given this knowledge I was able to create Maven Archetype artefacts for plug-in, test fragment and feature definition stubs that match the Xiliary development presets. This means each stub comes with the specific settings for codeformatting, execution environment, compile error/warning preferences and the like out of the box. For flexibility reasons I decided to go with three individual artefacts instead of one and wired them together using a little script. This is because most of the time I need to create all three stubs in one step. Although this renders the Eclipse New Project wizard unusable, it is not a big deal as the only benefit would be the automatic workspace import of the stubs.The only manual tasks left are the registration of the new modules in the parent pom of the repository’s build definition and the addition af a new feature entry in the P2 related catagory.xml. Conclusion This post gave an short introduction to Maven Archetype and showed how it can be used to automate Eclipse module creation. With the custom archetypes described above in place, it now takes about a minute to add a new feature definition with plug-in and test fragment to the workspace and build definition. And being development and build ready within a minute isn’t that bad compared to the previous manual create, configure, copy and paste litany… So you want to have look at the archetype sources by yourself, the definitions are located in the com.codeaffine.xiliary.archetype project of the Xiliary repository at GitHub.Reference: Efficient Creation of Eclipse Modules with Maven Archetype from our JCG partner Frank Appel at the Code Affine blog....

When null checking miserably fails

Disclaimer Before going on I have to state that the techniques described in this article serve no practical purpose when we program Java. It is like a crossword or puzzle. It main train your brain in logical thinking, may develop your Java language knowledge or even your thinking skills. It is like a trick a magician performs. At the end you realize that nothing is what it looks like. Never do such tricks in real life programming that you may need to apply to solve this mind twister.       The Problem I recently read an article that described the debugging case when: if(trouble != null && !trouble.isEmpty()) { System.out.println(“fine here: ” + trouble); }else{ System.out.println(“not so fine here: ” + trouble); } was printing out: fine here: null The actual bug was that the string contained “null”, a.k.a. the characters ‘n’, ‘u’, ‘l’ and ‘l’. May happen in real life especially when you concatenate strings without checking the nullity of a variable. Then I started to think about other similar strange code and debug situation. Could I make it so that the variable is not only “null” string with these characters but really null? Seems to be crazy? Have a look at the code: package com.javax0.blog.nullisnotnull;public class NullIsNotNull {public static void troubled(){ String trouble = new String("hallo"); Object z = trouble != null && !trouble.toString().isEmpty() ? trouble.toString() : ""; if (z == null) { System.out.println("z is really " + z + "?"); } } } Will it ever print out the: z is really null? question. The fact is that you can create a Java class containing a public static void main() so that starting that class as a Java application the sentence will be printed when main() invokes the method troubled(). In other words: I really invoke the method troubled() and the solution is not that main() prints the sentence. In this case the variable z is not only printed as “null” but it really is null. Hints The solution should not involvereflection byte code manipulation calling JNI special class loaders java agent annotation processorThese are too heavy tools. You do not need such armory for the purpose. Hint #1 If I change the code so that the variable z is String it does not even compile:  If it confused you even more, then sorry. Read on! Hint #2 In the Java language String is an identifier and not a keyword. The Java Language Specification section 3.9 may give more information on the significance of this. Hint #3 The method toString() in class Object has a return type java.lang.String. You may want to read my article about the difference between the name, simple name and canonical name of a class. It may shed some light and increase the hit count of the article. Hint #4 To use a class declared in the same package you need not import that package. Solution The solution is to create a class named String in the same package. In that case the compiler will use this class instead of java.lang.String. The ternary operator in the code is simple magician trick. Something to diverge your attention from the important point. The major point is that String is not java.lang.String in the code above. If you still can not find out how to create the trick class, click on the collapsed source code block to see it in all glory: package com.javax0.blog.nullisnotnull;class String { private java.lang.String jString; private boolean first = true;public String(java.lang.String s) { jString = s; }public boolean isEmpty() { return jString.isEmpty(); }@Override public java.lang.String toString() { if( first ){ first = false; return jString; } return null; }public static void main(java.lang.String[] args) { NullIsNotNull.troubled(); } }Reference: When null checking miserably fails from our JCG partner Peter Verhas at the Java Deep blog....

Don’t Waste Time Writing Perfect Code

A system can last for 5 or 10 or even 20 or more years. But the life of specific lines of code, even of designs, is often much shorter: months or days or even minutes when you’re iterating through different approaches to a solution. Some code matters more than other code Researching how code changes over time, Michael Feathers has identified a power curve in code bases. Every system has code, often a lot of it, that is written once and is never changed. But a small amount of code, including the code that is most important and useful, is changed over and over again, refactored or rewritten from scratch several times. As you get more experience with a system, or with a problem domain or an architectural approach, it should get easier to know and to predict what code will change all the time, and what code will never change: what code matters, and what code doesn’t. Should we try to write Perfect Code? We know that we should write clean code, code that is consistent, obvious and as simple as possible. Some people take this to extremes, and push themselves to write code that is as beautiful and elegant and as close to perfect as they can get, obsessively refactoring and agonizing over each detail. But if code is only going to be written once and never changed, or at the other extreme if it is changing all the time, isn’t writing perfect code as wasteful and unnecessary (and impossible to achieve) as trying to write perfect requirements or trying to come up with a perfect design upfront? You Can’t Write Perfect Software. Did that hurt? It shouldn’t. Accept it as an axiom of life. Embrace it. Celebrate it. Because perfect software doesn’t exist. No one in the brief history of computing has ever written a piece of perfect software. It’s unlikely that you’ll be the first. And unless you accept this as a fact, you’ll end up wasting time and energy chasing an impossible dream.” Andrew Hunt, The Pragmatic Programmer: from Journeyman to Master Code that is written once doesn’t need to be beautiful and elegant. It has to be correct. It has to be understandable – because code that is never changed may still be read many times over the life of the system. It doesn’t have to be clean and tight – just clean enough. Copy and paste and other short cuts in this code can be allowed, at least up to a point. This is code that never needs to be polished. This is code that doesn’t need to be refactored (until and unless you need to change it), even if other code around it is changing. This is code that isn’t worth spending extra time on. What about the code that you are changing all of the time? Agonizing over style and coming up with the most elegant solution is a waste of time, because this code will probably be changed again, maybe even rewritten, in a few days or weeks. And so is obsessively refactoring code each time that you make a change, or refactoring code that you aren’t changing because it could be better. Code can always be better. But that’s not important. What matters is: Does the code do what it is supposed to do – is it correct and usable and efficient? Can it handle errors and bad data without crashing – or at least fail safely? Is it easy to debug? Is it easy and safe to change? These aren’t subjective aspects of beauty. These are practical measures that make the difference between success and failure. Pragmatic Coding and Refactoring The core idea of Lean Development is: don’t waste time on things that aren’t important. This should inform how we write code, and how we refactor it, how we review it, how we test it. Only refactor what you need to, in order to get the job done – what Martin Fowler calls opportunistic refactoring (comprehension, cleanup, Boy Scout rule stuff) and preparatory refactoring. Enough to make a change easier and safer, and no more. If you’re not changing the code, it doesn’t really matter what it looks like. In code reviews, focus only on what is important. Is the code correct? Is it defensive? Is it secure? Can you follow it? Is it safe to change? Forget about style (unless style gets in the way of understandability). Let your IDE take care of formatting. No arguments over whether the code could be “more OO”. It doesn’t matter if it properly follows this or that pattern as long as it makes sense. It doesn’t matter if you like it or not. Whether you could have done it in a nicer way isn’t important – unless you’re teaching someone who is new to the platform and the language, and you’re expected to do some mentoring as part of code review. Write tests that matter. Tests that cover the main paths and the important exception cases. Tests that give you the most information and the most confidence with the least amount of work. Big fat tests, or small focused tests – it doesn’t matter, and it doesn’t matter if you write the tests before you write the code or after, as long as they do the job. It’s not (Just) About the Code The architectural and engineering metaphors have never been valid for software. We aren’t designing and building bridges or skyscrapers that will stay essentially the same for years or generations. We’re building something much more plastic and abstract, more ephemeral. Code is written to be changed – that is why it’s called “software”. “After five years of use and modification, the source for a successful software program is often completely unrecognizable from its original form, while a successful building after five years is virtually untouched.” Kevin Tate, Sustainable Software DevelopmentWe need to look at code as a temporary artefact of our work: …we’re led to fetishize code, sometimes in the face of more important things. Often we suffer under the illusion that the valuable thing produced in shipping a product is the code, when it might actually be an understanding of the problem domain, progress on design conundrums, or even customer feedback. Dan Grover, Code and Creative Destruction Iterative development teaches us to experiment and examine the results of our work – did we solve the problem, if we didn’t, what did we learn, how can we improve? The software that we are building is never done. Even if the design and the code are right, they may only be right for a while, until circumstances demand that they be changed again or replaced with something else that fits better. We need to write good code: code that is understandable, correct, safe and secure. We need to refactor and review it, and write good useful tests, all the while knowing that some of this code, or maybe all of it, could be thrown out soon, or that it may never be looked at again, or that it may not get used at all. We need to recognize that some of our work will necessarily be wasted, and optimize for this. Do what needs to be done, and no more. Don’t waste time trying to write perfect code.Reference: Don’t Waste Time Writing Perfect Code from our JCG partner Jim Bird at the Building Real Software blog....

On Java Generics and Erasure

“Generics are erased during compilation” is common knowledge (well, type parameters and arguments are actually the ones erased). That happens due to “type erasure”. But it’s wrong that everything specified inside the <..> symbols is erased, as many developers are assuming. See the code below:               public class ClassTest { public static void main(String[] args) throws Exception { ParameterizedType type = (ParameterizedType) Bar.class.getGenericSuperclass(); System.out.println(type.getActualTypeArguments()[0]); ParameterizedType fieldType = (ParameterizedType) Foo.class.getField("children").getGenericType(); System.out.println(fieldType.getActualTypeArguments()[0]); ParameterizedType paramType = (ParameterizedType) Foo.class.getMethod("foo", List.class) .getGenericParameterTypes()[0]; System.out.println(paramType.getActualTypeArguments()[0]); System.out.println(Foo.class.getTypeParameters()[0] .getBounds()[0]); } class Foo<E extends CharSequence> { public List<Bar> children = new ArrayList<Bar>(); public List<StringBuilder> foo(List<String> foo) {return null; } public void bar(List<? extends String> param) {} } class Bar extends Foo<String> {} } Do you know what that prints?class java.lang.String class ClassTest$Bar class java.lang.String class java.lang.StringBuilder interface java.lang.CharSequenceYou see that every single type argument is preserved and is accessible via reflection at runtime. But then what is “type erasure”? Something must be erased? Yes. In fact, all of them are, except the structural ones – everything above is related to the structure of the classes, rather than the program flow. In other words, the metadata about the type arguments of a class and its field and methods is preserved to be accessed via reflection. The rest, however, is erased. For example, the following code: List<String> list = new ArrayList<>(); Iterator<String> it = list.iterator(); while (it.hasNext()) { String s = it.next(); } will actually be transformed to this (the bytecode of the two snippets is identical): List list = new ArrayList(); Iterator it = list.iterator(); while (it.hasNext()) { String s = (String) it.next(); } So, all type arguments you have defined in the bodies of your methods will be removed and casts will be added where needed. Also, if a method is defined to accept List<T>, this T will be transformed to Object (or to its bound, if such is declared. And that’s why you can’t do new T() (by the way, one open question about this erasure). So far we covered the first two points of the type erasure definition. The third one is about bridge methods. And I’ve illustrated it with this stackoverflow question (and answer). Two “morals” of all this. First, java generics are complicated. But you can use them without understanding all the complications. Second, do not assume that all type information is erased – the structural type arguments are there, so make use of them, if needed (but don’t be over-reliant on reflection).Reference: On Java Generics and Erasure from our JCG partner Bozhidar Bozhanov at the Bozho’s tech blog blog....

Java 8 Streams: Micro Katas

A programming kata is an exercise which helps a programmer hone his skills through practice and repetition. This article is part of the series Java Tutorial Through Katas. The article assumes that the reader already has experience with Java, that he is familiar with the basics of unit tests and that he knows how to run them from his favorite IDE (mine is IntelliJ IDEA). The idea behind exercises displayed below is to learn Java 8 Streaming using test-driven development approach (write the implementation for the first test, confirm that it passes and move to the next). Each section will start with an objective in form of tests that prove that the implementation will be correct once it’s written. Each of those tests are followed with one possible implementation in Java 7 (or earlier) and Java 8 using Streams. That way reader can compare some of the new features of Java 8 with their equivalents in earlier JDKs. Please try to solve tests without looking at provided solutions. For more information about TDD best practices, please read the Test Driven Development (TDD): Best Practices Using Java Examples. Java 8 map Convert elements of a collection to upper case. Tests package com.technologyconversations.java8exercises.streams;import org.junit.Test;import java.util.List;import static com.technologyconversations.java8exercises.streams.ToUpperCase.*; import static java.util.Arrays.asList; import static org.assertj.core.api.Assertions.assertThat;/* Convert elements of a collection to upper case. */ public class ToUpperCaseSpec {@Test public void transformShouldConvertCollectionElementsToUpperCase() { List<String> collection = asList("My"< "name"< "is"< "John"< "Doe"); List<String> expected = asList("MY"< "NAME"< "IS"< "JOHN"< "DOE"); assertThat(transform(collection)).hasSameElementsAs(expected); }} Java 7 (transform7) and Java8 (transform) Implementations package com.technologyconversations.java8exercises.streams;import java.util.ArrayList; import java.util.List;import static java.util.stream.Collectors.toList;public class ToUpperCase {public static List<String> transform7(List<String> collection) { List<String> coll = new ArrayList<>(); for (String element : collection) { coll.add(element.toUpperCase()); } return coll; }public static List<String> transform(List<String> collection) { return collection.stream() // Convert collection to Stream .map(String::toUpperCase) // Convert each element to upper case .collect(toList()); // Collect results to a new list }} Java 8 filter Filter collection so that only elements with less than 4 characters are returned. Tests package com.technologyconversations.java8exercises.streams;import org.junit.Test;import java.util.List;import static com.technologyconversations.java8exercises.streams.FilterCollection.*; import static java.util.Arrays.asList; import static org.assertj.core.api.Assertions.assertThat;/* Filter collection so that only elements with less then 4 characters are returned. */ public class FilterCollectionSpec {@Test public void transformShouldFilterCollection() { List<String> collection = asList("My", "name", "is", "John", "Doe"); List<String> expected = asList("My", "is", "Doe"); assertThat(transform(collection)).hasSameElementsAs(expected); }} Java 7 (transform7) and Java8 (transform) Implementations package com.technologyconversations.java8exercises.streams;import java.util.ArrayList; import java.util.List;import static java.util.stream.Collectors.toList;public class FilterCollection {public static List<String> transform7(List<String> collection) { List<String> newCollection = new ArrayList<>(); for (String element : collection) { if (element.length() < 4) { newCollection.add(element); } } return newCollection; }public static List<String> transform(List<String> collection) { return collection.stream() // Convert collection to Stream .filter(value -> value.length() < 4) // Filter elements with length smaller than 4 characters .collect(toList()); // Collect results to a new list }} Java 8 flatMap Flatten multidimensional collection. Tests package com.technologyconversations.java8exercises.streams;import org.junit.Test;import java.util.List;import static com.technologyconversations.java8exercises.streams.FlatCollection.*; import static java.util.Arrays.asList; import static org.assertj.core.api.Assertions.assertThat;/* Flatten multidimensional collection */ public class FlatCollectionSpec {@Test public void transformShouldFlattenCollection() { List<List<String>> collection = asList(asList("Viktor", "Farcic"), asList("John", "Doe", "Third")); List<String> expected = asList("Viktor", "Farcic", "John", "Doe", "Third"); assertThat(transform(collection)).hasSameElementsAs(expected); }} Java 7 (transform7) and Java8 (transform) Implementations package com.technologyconversations.java8exercises.streams;import java.util.ArrayList; import java.util.List;import static java.util.stream.Collectors.toList;public class FlatCollection {public static List<String> transform7(List<List<String>> collection) { List<String> newCollection = new ArrayList<>(); for (List<String> subCollection : collection) { for (String value : subCollection) { newCollection.add(value); } } return newCollection; }public static List<String> transform(List<List<String>> collection) { return collection.stream() // Convert collection to Stream .flatMap(value -> value.stream()) // Replace list with stream .collect(toList()); // Collect results to a new list }} Java 8 max and comparator Get oldest person from the collection. Tests package com.technologyconversations.java8exercises.streams;import org.junit.Test;import java.util.List;import static com.technologyconversations.java8exercises.streams.OldestPerson.*; import static java.util.Arrays.asList; import static org.assertj.core.api.Assertions.assertThat;/* Get oldest person from the collection */ public class OldestPersonSpec {@Test public void getOldestPersonShouldReturnOldestPerson() { Person sara = new Person("Sara", 4); Person viktor = new Person("Viktor", 40); Person eva = new Person("Eva", 42); List<Person> collection = asList(sara, eva, viktor); assertThat(getOldestPerson(collection)).isEqualToComparingFieldByField(eva); }} Java 7 (getOldestPerson7) and Java8 (getOldestPerson) Implementations package com.technologyconversations.java8exercises.streams;import java.util.Comparator; import java.util.List;public class OldestPerson {public static Person getOldestPerson7(List<Person> people) { Person oldestPerson = new Person("", 0); for (Person person : people) { if (person.getAge() > oldestPerson.getAge()) { oldestPerson = person; } } return oldestPerson; }public static Person getOldestPerson(List<Person> people) { return people.stream() // Convert collection to Stream .max(Comparator.comparing(Person::getAge)) // Compares people ages .get(); // Gets stream result }} Java 8 sum and reduce Sum all elements of a collection. Tests package com.technologyconversations.java8exercises.streams;import org.junit.Test;import java.util.List;import static com.technologyconversations.java8exercises.streams.Sum.*; import static java.util.Arrays.asList; import static org.assertj.core.api.Assertions.assertThat;/* Sum all elements of a collection */ public class SumSpec {@Test public void transformShouldConvertCollectionElementsToUpperCase() { List<Integer> numbers = asList(1, 2, 3, 4, 5); assertThat(calculate(numbers)).isEqualTo(1 + 2 + 3 + 4 + 5); }} Java 7 (calculate7) and Java8 (calculate) Implementations package com.technologyconversations.java8exercises.streams;import java.util.List;public class Sum {public static int calculate7(List<Integer> numbers) { int total = 0; for (int number : numbers) { total += number; } return total; }public static int calculate(List<Integer> people) { return people.stream() // Convert collection to Stream .reduce(0, (total, number) -> total + number); // Sum elements with 0 as starting value }} Java 8 filter and map Get names of all kids (under age of 18). Tests package com.technologyconversations.java8exercises.streams;import org.junit.Test;import java.util.List;import static com.technologyconversations.java8exercises.streams.Kids.*; import static java.util.Arrays.asList; import static org.assertj.core.api.Assertions.assertThat;/* Get names of all kids (under age of 18) */ public class KidsSpec {@Test public void getKidNameShouldReturnNamesOfAllKidsFromNorway() { Person sara = new Person("Sara", 4); Person viktor = new Person("Viktor", 40); Person eva = new Person("Eva", 42); Person anna = new Person("Anna", 5); List<Person> collection = asList(sara, eva, viktor, anna); assertThat(getKidNames(collection)) .contains("Sara", "Anna") .doesNotContain("Viktor", "Eva"); }} Java 7 (getKidNames7) and Java8 (getKidNames) Implementations package com.technologyconversations.java8exercises.streams;import java.util.*;import static java.util.stream.Collectors.toSet;public class Kids {public static Set<String> getKidNames7(List<Person> people) { Set<String> kids = new HashSet<>(); for (Person person : people) { if (person.getAge() < 18) { kids.add(person.getName()); } } return kids; }public static Set<String> getKidNames(List<Person> people) { return people.stream() .filter(person -> person.getAge() < 18) // Filter kids (under age of 18) .map(Person::getName) // Map Person elements to names .collect(toSet()); // Collect values to a Set }} Java 8 summaryStatistics Get people statistics: average age, count, maximum age, minimum age and sum og all ages. Tests package com.technologyconversations.java8exercises.streams;import org.junit.Test;import java.util.List;import static com.technologyconversations.java8exercises.streams.PeopleStats.*; import static java.util.Arrays.asList; import static org.assertj.core.api.Assertions.assertThat;/* Get people statistics: average age, count, maximum age, minimum age and sum og all ages. */ public class PeopleStatsSpec {Person sara = new Person("Sara", 4); Person viktor = new Person("Viktor", 40); Person eva = new Person("Eva", 42); List<Person> collection = asList(sara, eva, viktor);@Test public void getStatsShouldReturnAverageAge() { assertThat(getStats(collection).getAverage()) .isEqualTo((double)(4 + 40 + 42) / 3); }@Test public void getStatsShouldReturnNumberOfPeople() { assertThat(getStats(collection).getCount()) .isEqualTo(3); }@Test public void getStatsShouldReturnMaximumAge() { assertThat(getStats(collection).getMax()) .isEqualTo(42); }@Test public void getStatsShouldReturnMinimumAge() { assertThat(getStats(collection).getMin()) .isEqualTo(4); }@Test public void getStatsShouldReturnSumOfAllAges() { assertThat(getStats(collection).getSum()) .isEqualTo(40 + 42 + 4); }} Java 7 (getStats7) and Java8 (getStats) Implementations package com.technologyconversations.java8exercises.streams;import java.util.IntSummaryStatistics; import java.util.List;public class PeopleStats {public static Stats getStats7(List<Person> people) { long sum = 0; int min = people.get(0).getAge(); int max = 0; for (Person person : people) { int age = person.getAge(); sum += age; min = Math.min(min, age); max = Math.max(max, age); } return new Stats(people.size(), sum, min, max); }public static IntSummaryStatistics getStats(List<Person> people) { return people.stream() .mapToInt(Person::getAge) .summaryStatistics(); }} Java 8 partitioningBy Partition adults and kids. Tests package com.technologyconversations.java8exercises.streams;import org.junit.Test;import java.util.List; import java.util.Map;import static com.technologyconversations.java8exercises.streams.Partitioning.*; import static java.util.Arrays.asList; import static org.assertj.core.api.Assertions.assertThat;/* Partition adults and kids */ public class PartitioningSpec {@Test public void partitionAdultsShouldSeparateKidsFromAdults() { Person sara = new Person("Sara", 4); Person viktor = new Person("Viktor", 40); Person eva = new Person("Eva", 42); List<Person> collection = asList(sara, eva, viktor); Map<Boolean, List<Person>> result = partitionAdults(collection); assertThat(result.get(true)).hasSameElementsAs(asList(viktor, eva)); assertThat(result.get(false)).hasSameElementsAs(asList(sara)); }} Java 7 (partitionAdults7) and Java8 (partitionAdults) Implementations package com.technologyconversations.java8exercises.streams;import java.util.*; import static java.util.stream.Collectors.*;public class Partitioning {public static Map<Boolean, List<Person>> partitionAdults7(List<Person> people) { Map<Boolean, List<Person>> map = new HashMap<>(); map.put(true, new ArrayList<>()); map.put(false, new ArrayList<>()); for (Person person : people) { map.get(person.getAge() >= 18).add(person); } return map; }public static Map<Boolean, List<Person>> partitionAdults(List<Person> people) { return people.stream() // Convert collection to Stream .collect(partitioningBy(p -> p.getAge() >= 18)); // Partition stream of people into adults (age => 18) and kids }} Java 8 groupingBy Group people by nationality. Tests package com.technologyconversations.java8exercises.streams;import org.junit.Test;import java.util.List; import java.util.Map;import static com.technologyconversations.java8exercises.streams.Grouping.*; import static java.util.Arrays.asList; import static org.assertj.core.api.Assertions.assertThat;/* Group people by nationality */ public class GroupingSpec {@Test public void partitionAdultsShouldSeparateKidsFromAdults() { Person sara = new Person("Sara", 4, "Norwegian"); Person viktor = new Person("Viktor", 40, "Serbian"); Person eva = new Person("Eva", 42, "Norwegian"); List<Person> collection = asList(sara, eva, viktor); Map<String, List<Person>> result = groupByNationality(collection); assertThat(result.get("Norwegian")).hasSameElementsAs(asList(sara, eva)); assertThat(result.get("Serbian")).hasSameElementsAs(asList(viktor)); }} Java 7 (groupByNationality7) and Java8 (groupByNationality) Implementations package com.technologyconversations.java8exercises.streams;import java.util.ArrayList; import java.util.HashMap; import java.util.List; import java.util.Map;import static java.util.stream.Collectors.*;public class Grouping {public static Map<String, List<Person>> groupByNationality7(List<Person> people) { Map<String, List<Person>> map = new HashMap<>(); for (Person person : people) { if (!map.containsKey(person.getNationality())) { map.put(person.getNationality(), new ArrayList<>()); } map.get(person.getNationality()).add(person); } return map; }public static Map<String, List<Person>> groupByNationality(List<Person> people) { return people.stream() // Convert collection to Stream .collect(groupingBy(Person::getNationality)); // Group people by nationality }} Java 8 joining Return people names separated by comma. Tests package com.technologyconversations.java8exercises.streams;import org.junit.Test;import java.util.List;import static com.technologyconversations.java8exercises.streams.Joining.namesToString; import static java.util.Arrays.asList; import static org.assertj.core.api.Assertions.assertThat;/* Return people names separated by comma */ public class JoiningSpec {@Test public void toStringShouldReturnPeopleNamesSeparatedByComma() { Person sara = new Person("Sara", 4); Person viktor = new Person("Viktor", 40); Person eva = new Person("Eva", 42); List<Person> collection = asList(sara, viktor, eva); assertThat(namesToString(collection)) .isEqualTo("Names: Sara, Viktor, Eva."); }} Java 7 (namesToString7) and Java8 (namesToString) Implementations package com.technologyconversations.java8exercises.streams;import java.util.List;import static java.util.stream.Collectors.joining;public class Joining {public static String namesToString7(List<Person> people) { String label = "Names: "; StringBuilder sb = new StringBuilder(label); for (Person person : people) { if (sb.length() > label.length()) { sb.append(", "); } sb.append(person.getName()); } sb.append("."); return sb.toString(); }public static String namesToString(List<Person> people) { return people.stream() // Convert collection to Stream .map(Person::getName) // Map Person to name .collect(joining(", ", "Names: ", ".")); // Join names }} Source Full source is located in the GitHub repo https://github.com/vfarcic/java-8-exercises. Besides tests and implementation, repository includes build.gradle that can be used, among other things, to download AssertJ dependencies and run tests.Reference: Java 8 Streams: Micro Katas from our JCG partner Viktor Farcic at the Technology conversations blog....

Beyond Thread Pools: Java Concurrency is Not as Bad as You Think

Apache Hadoop, Apache Spark, Akka, Java 8 streams and Quasar: The classic use cases to the newest concurrency approaches for Java developers There’s a lot of chatter going around about newer concepts in concurrency, yet many developers haven’t had a chance to wrap their heads around them yet. In this post we’ll go through the things you need to know about Java 8 streams, Hadoop, Apache Spark, Quasar fibers and the Reactive programming approach – and help you stay in the loop, especially if you’re not working with them on a regular basis. It’s not the future, this is happening right now. What are we dealing with here? When talking about concurrency, a good way to characterize the issue at hand is answering a few questions to get better feel for it:Is it a data processing task? If so, can it be broken down to independent pieces of work? What’s the relationship between the OS, the JVM and your code? (Native threads Vs. light-weight threads) How many machines and processors are involved? (Single core Vs. Multicore)Let’s go through each of these and figure out the best use cases to each approach. 1. From Thread Pools to Parallel Streams Data processing on single machines, letting the Java take care of thread handling With Java 8, we’ve been introduced to the new Stream API that allows applying aggregate operations like Filter, Sort or Map on streams of data. Another thing Streams allow are parallel operations on multicore machines when applying .parallelStream() – Splitting the work between threads using the Fork/Join framework introduced in Java 7. An evolution from the Java 6 java.util.concurrency library, where we met the ExecutorService which creates and handles our worker thread pools. Fork/Join is also built on top of the ExecuterService, the main difference from a traditional thread pool is how they distribute the work between threads and thereby multicore machine support. With a simple ExecuterService you’re in full control of the workload distribution between worker threads, determining the size of each task for the threads to handle. With Fork/Join on the other hand, there’s a work-stealing algorithm in place that abstracts workload handling between threads. In a nutshell, this allows large tasks to be divided to smaller ones (forked), and processed in different threads, eventually joining the results – Balancing the the work between threads. However, it’s not a silver bullet. Sometimes Parallel Streams may even slow you down, so you’ll need to think it through. Adding .parallelStream() to your methods can cause bottlenecks and slowdowns (some 15% slower on this benchmark we ran), the fine line goes through the number of threads. Let’s say we’re already running multiple threads and we’re using .parallelStream() in some of them, adding more and more threads to the pool. This could easily turn into more than our cores could handle, and slow everything down due to increased context switching. Bottom line: Parallel Streams abstract handling threads on a single machine in a way that distributes the workload between your cores. However, if you want to use them efficiently it’s critical to keep the hardware in mind not spawn more threads than your machine can handle. 2. Apache Hadoop and Apache Spark Heavy duty lifting: Big data processing across multiple machines Moving on to multiple machines, petabytes of data, and tasks that resemble pulling all tweets that mention Java from twitter or heavy duty machine learning algorithms. When speaking of Hadoop, it’s important to take another step and think of the wider framework and its components: The Hadoop Distributed File System (HDFS), a resource management platform (YARN), the data processing module (MapReduce) and other libraries and utilities needed for Hadoop (Common). On top of these come other optional tools like a database which runs on top of HDFS (HBase), a platform for a querying language (Pig), and a data warehouse infrastructure (Hive) to name a few of the popular ones. This is where Apache Spark steps in as a new data processing module, famous for its in-memory performance and the use of fast performing Resilient Distributed Datasets (RDDs), unlike the Hadoop MapReduce which doesn’t employ in-memory (and on-disk) operations as efficiently. The latest benchmark released by Databricks shows that Spark was 3x faster than Hadoop in sorting a petabyte of data, while using 10x less nodes.The classic use case for Hadoop would be querying data, while Spark is getting famous for its fast runtimes of machine learning algorithms. But this is only the tip of the iceberg, as stated by Databricks: “Spark enables applications in Hadoop clusters to run up to 100x faster in memory, and 10x faster even when running on disk”. Bottom line: Spark is the new rising star in Hadoop’s ecosystem. There’s a common misconception that we’re talking about something unrelated or competing, but I believe that what we’re seeing here is the evolution of the framework. 3. Quasar fibers Breaking native threads to virtual light-weight threads We’ve had the chance to run through the Hadoop, now let’s back to single machines. In fact, let’s zoom in even further than the standard multithreaded Java application and focus on one single thread. As far as we’re concerned, HotSpot JVM threads are the same as native OS threads, holding one thread and running “virtual” threads within it is what fibers are all about. Java doesn’t have a native fibers support, but no worries, Quasar by Parallel Universe got us covered. Quasar is an open source JVM library that supports fibers (Also known as light weight threads), and also acts as an Actor Framework, which I’ll mention later. Context switching is the name of the game here. As we’re limited by the number of cores, once the native thread count grows larger we’re subjected to more and more context switching overhead. One way around this is fibers, using a single thread that supports “multithreading”. Looks like a case of threadcepiton. Fibers can also be seen as an evolution from thread pools, dodging the dangers of thread overload we went through with Parallel Streams. They make it easier to scale threads and allow a significantly larger number of concurrent “light” threads. They’re not intended to replace threads and should be used for code that blocks relatively often, it’s like they’re acting as true async threads. Bottom line: Parallel Universe is offering a fresh approach to concurrency in Java, haven’t reached v1.0 yet but definitely worth checking out. 4. Actors & Reactive Programming A different model for handling concurrency in Java In the Reactive Manifesto, the new movement is described with 4 principles: Responsive, Resilient, Elastic and Message-Driven. Which basically means fast, fault tolerant, scalable and suuports non-blocking communication. Let’s see how Akka Actors support that. To simplify things, think of Actors as people that have a state and a certain behavior, communicating by exchanging messages that go to each other’s mailbox. An Actor system as a whole should be created per application, with a hierarchy that breaks down tasks to smaller tasks so that each actor has only one supervising actor at most. An actor can either take care of the task, break it down event further with delegation to another actor or in case of failure, escalate it to his supervisor. Either way, messages shouldn’t include behavior or share mutable states, each Actor has an isolated stated and behavior of its own. It’s a paradigm shift from the concurrency models most developers are used to. And a bit of an off-shoot from the evolution in the first 3 topics we covered here. Although its roots stem back from the 70’s, its been under the radar just until recent years with a revival to better fit modern application demands. Parallel Universe’s Quasar also supports Actor, based on its light-weight threads. The main difference in implementation lies in the fibers/light-weight threads. Bottom line: Taking on the Actor model takes managing thread pools off your back, leaving it to the toolkit. The revival of interest comes from the kind of problems applications deal with today, highly concurrent systems with much more cores that we can work with. Conclusion We’ve ran through 4 methods to solve problems using concurrent or parallel algorithms with the most interesting approaches to tackle today’s challenges. Hopefully this helped pique your interest and get a better view of the hot topics in concurrency today. Going beyond the thread pools, there’s a trend of delegating this responsibly to the language and its tools – Focusing dev resources on shipping new functionality rather than spending countless hours solving race conditions and locks.Reference: Beyond Thread Pools: Java Concurrency is Not as Bad as You Think from our JCG partner Alex Zhitnitsky at the Takipi blog....
Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy | Contact
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.
Do you want to know how to develop your skillset and become a ...
Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

Get ready to Rock!
You can download the complementary eBooks using the links below: