Featured FREE Whitepapers

What's New Here?


Scala pattern matching: A Case for new thinking?

The 16th President of the United States. Abraham Lincoln once said: ‘As our case is new we must think and act anew’. In software engineering things probably aren’t as dramatic as civil wars and abolishing slavery but we have interesting logical concepts concerning ‘case’. In Java the case statement provides for some limited conditional branching. In Scala, it is possible to construct some very sophisticated pattern matching logic using the case / match construct which doesn’t just bring new possibilities but a new type of thinking to realise new possibilities. Let’s start with a classical 1st year Computer Science homework assignment: a fibonacci series that doesn’t start with 0, 1 but that starts with 1, 1. So the series will look like: 1, 1, 2, 3, 5, 8, 13, … every number is the sum of the previous two. In Java, we could do: public int fibonacci(int i) { if (i < 0) return 0; switch(i) { case 0: return 1; case 1: return 1; default: return fibonacci(i-1) + fibonacci(i - 2); } } All straight forward. If 0 is passed in it counts as the first element in the series so 1 should be returned. Note: to add some more spice to the party and make things a little bit more interesting I added a little bit of logic to return 0 if a negative number is passed in to our fibonacci method. In Scala to achieve the same behaviour we would do: def fibonacci(in: Int): Int = { in match { case n if n <= 0 => 0 case 0 | 1 => 1 case n => fibonacci(n - 1) + fibonacci(n- 2) } } Key points:The return type of the recursive method fibonacci is an Int. Recursive methods must explictly specify the return type (see: Odersky – Programming in Scala – Chapter 2). It is possible to test for multiple values on the one line using the | notation. I do this to return a 1 for both 0 and 1 on line 4 of the example. There is no need for multiple return statements. In Java you must use multiple return statements or multiple break statements. Pattern matching is an expression which always returns something. In this example, I employ a guard to check for a negative number and if it a number is negative zero is returned. In Scala it is also possible to check across different types. It is also possible to use the wildcard _ notation. We didn’t use either in the fibonacci, but just to illustrate these features… def multitypes(in: Any): String = in match { case i:Int => 'You are an int!' case 'Alex' => 'You must be Alex' case s:String => 'I don't know who you are but I know you are a String' case _ => 'I haven't a clue who you are' }Pattern matching can be used with Scala Maps to useful effect. Suppose we have a Map to capture who we think should be playing in each position of the Lions backline for the Lions series in Austrailia. The keys of the map will be the position in the back line and the corresponding value will be the player who we think should be playing there. To represent a Rugby player we use a case class. Now now you Java Heads, think of the case class as an immutable POJO written in extremely concise way – they can be mutable too but for now think immutable. case class RugbyPlayer(name: String, country: String); val robKearney = RugbyPlayer('Rob Kearney', 'Ireland'); val georgeNorth = RugbyPlayer('George North', 'Wales'); val brianODriscol = RugbyPlayer('Brian O'Driscol', 'Ireland'); val jonnySexton = RugbyPlayer('Jonny Sexton', 'Ireland'); val benYoungs = RugbyPlayer('Ben Youngs', 'England');// build a map val lionsPlayers = Map('FullBack' -> robKearney, 'RightWing' -> georgeNorth, 'OutsideCentre' -> brianODriscol, 'Outhalf' -> jonnySexton, 'Scrumhalf' -> benYoungs);// Note: Unlike Java HashMaps Scala Maps can return nulls. This achieved by returing // an Option which can either be Some or None.// So, if we ask for something that exists in the Map like below println(lionsPlayers.get('Outhalf')); // Outputs: Some(RugbyPlayer(Jonny Sexton,Ireland))// If we ask for something that is not in the Map yet like below println(lionsPlayers.get('InsideCentre')); // Outputs: None In this example we have players for every position except inside centre – which we can’t make up our mind about. Scala Maps are allowed to store nulls as values. Now in our case we don’t actually store a null for inside center. So, instead of null being returned for inside centre (as what would happen if we were using a Java HashMap), the type None is returned. For the other positions in the back line, we have matching values and the type Some is returned which wraps around the corresponding RugbyPlayer. (Note: both Some and Option extend from Option). We can write a function which pattern matches on the returned value from the HashMap and returns us something a little more user friendly. def show(x: Option[RugbyPlayer]) = x match { case Some(rugbyPlayerExt) => rugbyPlayerExt.name // If a rugby player is matched return its name case None => 'Not decided yet ?' // } println(show(lionsPlayers.get('Outhalf'))) // outputs: Jonny Sexton println(show(lionsPlayers.get('InsideCentre'))) // Outputs: Not decided yet This example doesn’t just illustrate pattern matching but another concept known as extraction. The rugby player when matched is extracted and assigned to the rugbyPlayerExt. We can then return the value of the rugby player’s name by getting it from rugbyPlayerExt. In fact, we can also add a guard and change around some logic. Suppose we had a biased journalist ( Stephen Jones) who didn’t want any Irish players in the team. He could implement his own biased function to check for Irish players def biasedShow(x: Option[RugbyPlayer]) = x match { case Some(rugbyPlayerExt) if rugbyPlayerExt.country == 'Ireland' => rugbyPlayerExt.name + ', don't pick him.' case Some(rugbyPlayerExt) => rugbyPlayerExt.name case None => 'Not decided yet ?' } println(biasedShow(lionsPlayers.get('Outhalf'))) // Outputs Jonny... don't pick him println(biasedShow(lionsPlayers.get('Scrumhalf'))) // Outputs Ben Youngs Pattern matching Collections Scala also provides some powerful pattern matching features for Collections. Here’s a trivial exampe for getting the length of a list. def length[A](list : List[A]) : Int = list match { case _ :: tail => 1 + length(tail) case Nil => 0 } And suppose we want to parse arguments from a tuple… def parseArgument(arg : String, value: Any) = (arg, value) match { case ('-l', lang) => setLanguage(lang) case ('-o' | '--optim', n : Int) if ((0 < n) && (n <= 3)) => setOptimizationLevel(n) case ('-h' | '--help', null) => displayHelp() case bad => badArgument(bad) } Single Parameter functions Consider a list of numbers from 1 to 10. The filter method takes a single parameter function that returns true or false. The single parameter function can be applied for every element in the list and will return true or false for every element. The elements that return true will be filtered in; the elements that return false will be filtered out of the resultant list. scala> val myList = List(1,2,3,4,5,6,7,8,9,10) myList: List[Int] = List(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)scala> myList.filter(x => x % 2 ==1) res13: List[Int] = List(1, 3, 5, 7, 9) Now now now, listen up and remember this. A pattern can be passed to any method that takes a single parameter function. Instead of passing a single parameter function which always returned true or false we could have used a pattern which always returns true or false. scala> myList.filter { | case i: Int => i % 2 == 1 // odd number will return false | case _ => false // anything else will return false | } res14: List[Int] = List(1, 3, 5, 7, 9) Use it later? Scala compiles patterns to a PartialFunction. This means that not only can Scala pattern expressions be passed to other functions but they can also be stored for later use. scala> val patternToUseLater = : PartialFunction[String, String] = { | case 'Dublin' => 'Ireland' | case _ => 'Unknown' } What this example is saying is patternToUseLater is a partial function that takes a string and returns a string. The last statemeent in a function is returned by default and because the case expression is a partial function it will returned as a partial function and assigned to pattenrToUseLater which of course can use it later. Finally, Johnny Sexton is a phenomenal Rugby player and it is a shame to hear he is leaving Leinster. Obviously, with Sexton’s busy schedule we can’t be sure if Johnny is reading this blog but if he is, Johnny sorry to see you go we wish you all the best and hopefully will see you back one day in the Blue Jersey.   Reference: Scala pattern matching: A Case for new thinking? from our JCG partner Alex Staveley at the Dublin’s Tech Blog blog. ...

Managing the Stream of Features in an Agile Program

One of the challenges in a program is how you manage the checkins, especially if you have continuous integration. I am quite fond of continuous integration, no matter how large your program is. I also like short iterations. (Remember Short is Beautiful?) But imagine a product where you have a platform and layers. I’m separating the GUI and the API for the GUI, so you can see the application and the middleware and the platform. Now, this architecture is different from separate-but-related products that also might be a program. This is an archetype of an architecture, not your architecture. I am sure you have more than 3 middleware components or 4 app layer components. The product I’m thinking about had 12 middleware components, about another 12 platform components and about 50 app layer components. It was a big program with about 200 people working on the program for over 2 years. I wanted to simplify the picture so we could have a conversation. The features cut through the app layers and the middleware layers. The colored lines are the features. Now, multiply these lines by each project team and each feature, and you can see what happens in a program. Imagine if I added colored lines for 25 features for 25 different feature teams. It could be a problem. However, if project teams limit their WIP (work in progress), and swarm around features, integrating as they proceed, they have fewer features in progress. And, if they are networks of people, so they have communities of practice, they have ways of talking to each other, so they don’t have to wait to sync with each other. People talk with each other when they need to. That’s it. When features are small, and they integrate every day, or every other day, people expect to discuss integration issues all the time. And, while I am not a fan of integration teams, even I admit that on a large program, you might need an integration team to help keep things moving. This is in addition to what everyone does of syncing with the main line everyday: taking down the new additions to the main and syncing just before putting new changes up.If you keep a stream of features moving in a program, even with many feature teams, as long as the project teams keep talking to one another, you are okay. You are not okay if someone decides, “I own this code and no one else can touch it.” Now, you might decide that all middleware code or all particular component code has to be reviewed. Or it all has to be smoke tested. Or, it all has some other gate that it has to go through before it can be changed. Or you pair on everything. Or, you have situational code ownership. That’s perfectly okay. You decide on the technical mores for your program. It’s a good idea to have them. But, the larger the program, the less you can have one gatekeeper. Because that person cannot be in one place, holding their fingers in the figurative dike. This is why I don’t like czar-like architects, but I do like embedded architects in the teams for agile programs. When the product reaches a certain level of maturity, you can work on a particular component as a feature, and swap it in or out once you change it as a feature. This takes significant skill. If you want to be agile for a program, you need to be more agile and lean, not less. You need to have smaller stories. You need to work by feature, not by architecture. You need to swarm. Well, that is if you don’t want program bloat. Okay, what’s confusing to you? What have I forgotten? Where do you disagree? Let’s discuss.   Reference: Managing the Stream of Features in an Agile Program from our JCG partner Johanna Rothman at the Managing Product Development blog. ...

Jenkins Description Setter Plugin for Improving Continuous Delivery Visibility

In Continuous Delivery each build is potentially shippable. This fact implies among a lot of other things, to assign a none snapshot version to your components as fast as possible so you can refer them through all the process. I suggest creating a release branch, assign the version to the project and then start the typical pipeline (compile, tests, code quality …) steps to release branch.               If you are using Jenkins, your build job screen will look something like:Note that we have released the project many times, but there is no quick way to know exactly which version has been constructed in build number 40. To avoid this problem and having a quick overview of which version has been executed in each build job instance, we can use Jenkins description setter plugin. This plugin sets the description for each build, based upon a regular expression of the build log file. So your build job screen will look something like:Much better, now we know exactly the result of a build job and which product version has been generated. So first step is installing the plugin by simply going to: Jenkins -> Manage Jenkins -> Manage Plugins -> Available. After installation you can open Build Job configuration screen and add a post-build action called ‘ Set build description‘. Then add a regular expression for extracting the version number. In this case the regular expression is: \[INFO\] from version 0\.0\.1-SNAPSHOT to (.*) Take a look at next fragment of build log file:[INFO] Scanning for projects...[INFO][INFO] ------------------------------------------------------------------------[INFO] Building hello 0.0.1-SNAPSHOT[INFO] ------------------------------------------------------------------------[INFO][INFO] --- versions-maven-plugin:2.0:set (default-cli) @ hello ---[INFO] Searching for local aggregator root...[INFO] Local aggregation root: /jobs/helloworld-inital-build/workspace[INFO] Processing com.lordofthejars.helloworld:hello[INFO] Updating project com.lordofthejars.helloworld:hello[INFO] from version 0.0.1-SNAPSHOT to 1.0.43Props: {project.version=1.0.43, project.artifactId=hello, project.groupId=com.lordofthejars.helloworld} At line 12 we are logging the final version of our product for current pipeline execution, so we create a regular expression which parses that line and the part between brackets are used as decorator. Depending on log traces the regular expression will differ from this one. In this case, we are always using the same SNAPSHOT version in development and only when product is going to be released (this could be 3 times per day or every night) the final version is generated and set. Hope this plugin helps you to make your builds more clear.   Reference: Jenkins Description Setter Plugin for Improving Continuous Delivery Visibility from our JCG partner Alex Soto at the One Jar To Rule Them All blog. ...

Dry parameter names

How often do you see code like this, especially when using dependency injection, single-responsibility principle, and other “good practices”?                   class FriendsTimelineReader(userFinder: UserFinder, userStatusReader: UserStatusReader, statusSecurityFilter: StatusSecurityFilter) { ... } Note that the parameter names are exact copies of the class name. That’s certainly not DRY! In Java writing such code is a bit easier than in Scala, since you write the type first, and then the variable name can be auto-completed (the fact that there’s an auto-complete in IDEs indicates that it’s a common pattern). In Scala there’s more typing, but then, less boilerplate around declaring fields/defining a constructor/assigning to fields. How to avoid that? We can always use cryptic variable names, like “ssf” instead of “statusSecurityFilter“. But then the whole effort of having nicely named classes to make the code readable, which is quite a hard task, goes to waste. Of course, variable names are very useful, for example when we have to distinguish between the 10 String parameters that our method takes, create an Int counter, etc. But the more we use specialised wrapper-types (and now possibly even more, with Scala 2.10 introducing value classes) to make our code more type-safe, the more fine-grained our services – the more often the variable/parameter names will mirror the class name. Even when there are a couple of instances of a class, often the parameter name contains some qualifier + the class name (e.g. given a Person class, personAssociator.createFriends(firstPerson, secondPerson)). It would be interesting to see some statistics on how often variable name is a mirror of the class name or a suffix – I suspect it can be a large percentage. Maybe the next step then in type safety is limiting the complete freedom when naming parameters? Or even, getting rid of the parameter names at all in some cases. For example. In the snippet from the beginning, we just want to get “an instance” of a UserFinder. Most probably there will ever be only one instance of this class in the system, and certainly one will be used at a time by other classes. So why not let the class express that it wants an instance of a class, without having to name it? Let’s use the indefinite article “a” as a keyword (we don’t really care which instance is passed): class FriendsTimelineReader(a UserFinder, a UserStatusReader, a StatusSecurityFilter) Now, how would you use such a variable inside a class? Let’s use the definite article “the” – the instance that was given to the class, for example: val user10 = (the UserFinder).findById(10) Maybe this looks a bit like science-fiction, but wouldn’t it be convenient? Or maybe this problem is already solved by some mechanisms in other languages? Adam EDIT 27/01/2013: Extending the above to handle qualifiers: class Friends(a 'first Person, a 'second Person) { ... (the 'first Person).getName() ... }   Reference: Dry parameter names from our JCG partner Adam Warski at the Blog of Adam Warski blog. ...

Good & Bad Patterns in Development and Operations

As part of my role at a new company I’ve been asked to provide feedback about structuring Dev & Ops as well as what sorts of things work and don’t. I certainly don’t claim to have all the answers, but I’ve seen some very functional and some very dysfunctional organizations. I’ve spent a fair amount of time thinking about what works & why. Below is a cleaned up version of a message I sent to our CEO who asked for my thoughts on what does and doesn’t work. This was intended as scaffolding for further discussion so I didn’t go into deep details. If you want more details on any particular area just throw some comments out there.   I realize not all these issues are black & white to many folks – there are gray areas. My goal with this message was to drive conversation. I figure this is probably review to many folks, but maybe it’ll help someone. First, there are some very simple goals that all these bullets drive toward & they’re somewhat exclusive to SaaS companies:Customers should continuously receive value from Developers as code is incrementally pushed out Developers should get early feedback from customers on changes by enabling features for customers to test We can address problems for customers very quickly – often in a matter of hours We can inspect and understand customer behavior very deeply, gathering exceptional detail about how they use the service. We can swap out components & substantially change the underlying software without the customer knowing (if we do it right) We can measure how happy customers are with changes as we make them based on behavior & feedbackThe lists below are what I feel make that possible (Good) and what inhibit it (Bad) : Culture & Communication Good:Stand ups. Retrospectives Small, self-formed teams (Let folks work on their area of passion) Use Information Radiators whenever possible (Kanban boards, stats on big monitors, etc) Decisions by teams, Leaders facilitate consensus Discovering what doesn’t work is part of finding the right solution, not something to fear. Hackathons allow Developers to do things they are passionate about Hire for personality & team fit first, technical ability second Data driven decisions, strive to have facts to back up decisions. Make the right behavior the easiest thing to do – build a low resistance path to doing the right thing.Bad:Top down decision making Strict role assignments & Silos Fear of not getting it right the first time Hiring for technical ability thinking team fit will come later Creating process out of fear that makes it difficult to do the right thing.Eliminate Manual Processes Good:Continuous Deployment / Delivery Fully automated testing Test Driven Development Fully automated system monitoring, configuration & provisioning Separate Deploy & Release (Feature toggles) Deploy from master, do not branch (Forces particular behaviors)Bad:Manual testing by a QA Team – sometimes it’s necessary, but should be avoided Deploying off a branch, slows things down & allows for other bad behaviors Writing tests after writing code, code isn’t written with testing in mind Developers relying on other teams to perform tasks that could be automated. Processes that are the result of fear rather than necessary business process.If it moves, measure it Good:Collect high resolution metrics about everything you possibly can Developers can add new metrics by pushing new code, do not rely on additional configuration by other teams. Graphs & metrics can be seen by anyone – Developers should rely on these. There should be individuals or teams who are passionate about data visualization & analysis. Dev teams rely on these metrics to make decisions, help identify what metrics are important Developers watch metrics after pushing new code, watch for trend changes (Devs take responsibility for availability)Bad:Operations has to configure new metrics after developers have added support for them (Manual) Operations monitors metrics & asks Dev teams when they think there’s a problem Developers don’t look at metrics unless something is brought to their attention Code doesn’t expose metrics until someone else asks for itAnd here is the long version of all of that… #1 Culture & Communication Above all else I consider these most important. I think most problems in other areas of the business can be overcome if you do well in these areas. Rally has been, by far, the best example of a very successful model that I’ve seen in this area. They aren’t unique – there are other companies with similar models & similar successes. Main pointsStand ups. By far the most effective tool for keeping everyone in touch. As teams grow you have to break them apart, so you have a 2nd standup where teams can bring cross-team items to share. Projects are tackled by relatively small, typically self-formed teams. Get individuals who are interested in working in an area together & they feed on each others passion. Perform retrospectives. This gives individuals & small groups the ability to voice concerns in a way that fosters resolution. There’s an art to facilitating this but it works well when done right. It also allows recognition of things that are done well. Use open information radiators – it should be easy to see what’s going on by looking at status somewhere vs. having to ask for status, go to meetings, etc. Kanban boards are great for this. Leaders exist to facilitate and help drive consensus but decisions are largely made by teams, not leaders. This makes being a leader harder, but it makes the teams more empowered. Accept that things may not work & the team and company will adjust when things do not work. This makes it easy to try new things & easy for people to vocalize when they think it isn’t working. If it’s hard to change process then people are more resistant to try new things. This goes back to retrospectives for keeping things in check. Also important in this are “spikes” or time boxed efforts explicitly designed to explore possibilities. Give developers time to pursue their own projects for the company. Many awesome features have come out of Hackathons where developers spent their own time to build something they were passionate about. Hire for personality fit first. I have seen many awesome people find a special niche in a company because they grew into a role that you couldn’t hire for – but what made that possible was that they worked well with the team as an individual. Hiring for technical skill also means you lose that skill when that person leaves, I would prefer to have cross-functional teams. Data driven decisions. This helps keep emotion and “I think xyz” out of the discussion & focuses on the data we do and do not have. If we don’t have data we either get more or acknowledge we may not be making the right decision but we’re going to move forward. Make the right thing the easiest thing. I’ve seen too many companies put process out there that makes the “right thing” really difficult, so it gets bypassed. The right thing should be an express train to done – very little resistance and very easy to do. It’s when you start wanting to do things differently that it should become harder, more painful.Also, everyone owns the quality of the service. This includes availability, performance, user experience, cost to deliver, etc. At my last company, there was exceptional collaboration between Operations, Engineering and Product (and across engineering teams) on all aspects of the service and there was a strong culture of shared ownership & very little finger pointing. If you want more details on this specific to Rally I wrote a blog post with some more info: Blog Post #2 Obsessively eliminate manual process – let computers do what they are good at. This is so much easier to do up front. There should be as little manual process as possible standing between a developer adding value for customers (writing code) and that code getting into production. There may be business process that controls when that feature is enabled for customers – but the act of deploying & testing that code should not be blocked by manual process. I refer to this as separating “Deploy” from “Release” – those are two very different things. Testing should only be manual to invalidate assumptions, validating assumptions should be automatic When we assume that if x is true then y will occur, there should be a test to validate that this is true. Testers should not manually validate these sorts of things unless there is just no way to automate them (rare). Testers are valuable to invalidate assumptions. Testers should be looking at the assumptions made by Developers and helping identify those assumptions that may not always be correct. Too many organizations rely on manual testing because it’s “easier”, but it has some serious drawbacks:You can only change your system as fast as your team can manually test it – which is very slow. Your testing is done by humans who make mistakes and don’t behave predictably so you get inaccurate results. The # of tests will only grow over time, requiring either more humans or more time, or both. It doesn’t scale.Over time the software quality gets lower, takes longer to test, and the test results become less reliable. This is a death spiral for many companies who eventually find it very hard to make changes due to fear & low confidence in testing. Avoiding this requires developers spend more time up front writing automated tests. This means developers might spend 60-70% of their time developing tests vs. writing code – this is the cost of doing business if you want to produce high quality software. That may seem excessive, but the tradeoffs are significant:Much higher code quality which stays high (those tests are always run, so re-introduced bugs (regressions) get caught) Faster developer on boarding, the tests describe how the code should behave and act as documentation. Refactoring code becomes easier because you know the tests describe what it should do. Each commit to the codebase is fully tested, allowing nearly immediate deployment to production if done right. Problems that make it into product feed back into more tests & continually improve code quality.Much of the time developing tests is spent thinking about how to solve the problem, but you are also writing code with the intent of making it testable. Code is often written differently when the developer knows tests need to pass vs. someone manually testing it. It’s much harder to come along later and write tests for existing code. You will hear me talk about Continuous Deployment & Continuous Integration – I feel these practices are extremely important to driving the above “good” behaviors. If you strive for Continuous Deployment then everything else falls into place without much disagreement because it has to be that way. This has a lot of benefits beyond what’s listed above:Value can be delivered to customers in days or hours instead of weeks or months Developers can get immediate feedback on their change in production New features can be tuned & tweaked while they are fresh in a developers mind You can focus on making it fast to resolve defects, no matter how predictable they are, rather than trying to predict all the ways things might go wrong. Most of the tools and behaviors that enable Continuous Deployment scale to very large teams & very frequent deployments. Amazon is a prime example of this, deploying something, somewhere, about every 11 seconds. Many companies that are in the 30-100 engineer size talk about deploying tens of times per day. This also impacts how you hire QA/Testers. This is a longer discussion, but you want to hire folks who can help during the test planning phase & can help Developers write better tests. Ideally your testers are also developers & work in a way that’s similar to Operations, helping your Developers to be better at their jobs.#3 If it moves, measure it I mentioned above, two big advantages a SaaS organization has are the amount it can learn about how customers use the product & the ability to change things rapidly. Both of these require obsessive measurement of everything that is going on so that you know if things are better or worse. Some of these metrics are about user behavior & experience to understand how the service is being used. Other metrics are about system performance & behavior. The ability to expose some % of your customer base to a new feature & measure their feedback to that is huge. Plenty of companies have perfected the art of A/B testing but at the heart of it is the ability to measure behavior. Similar to testing, the software has to be built in a way which allows this behavior to be measured. System performance similarly requires a lot of instrumentation to identify changes in trends, to identify problems areas in the application & to verify when changes actually improve the situation. I’ve been at too many companies where they simply had no idea how the system was performing today compared to last week to understand if things were better or worse. At my last company I saw a much more mature approach to this measurement which worked pretty well, but it required investment. They had two people fully dedicated to performance analysis & customer behavior analysis.   Reference: Good & Bad Patterns in Development and Operations from our JCG partner Aaron Nichols at the Operation Bootstrap blog. ...

Running unit tests and integration tests separately with Maven Failsafe and TestNG

Recently for my new pet project I decided that I would like to have some tests executed during standard mvn test and some other ones only during different phase, let’s call it integration phase. I googled and googled and nothing seemed to work, so after struggling with making my setup work I’ve decided to write down my findings how I was able to configure TestNG with Maven to run integration and unit tests separately. Basic (not working) setup For integration testing there is a Maven Failsafe plugin that is supposed to do what we want out of the box. Unfortunately, things are not as easy and straightforward as we might expect. Let’s look at basic setup that should work: We have two test classes, one with IT postfix that will indicate that this is our integration test: import org.testng.annotations.Test;import static org.fest.assertions.Assertions.assertThat;public class ExampleUnitTest {@Test public void shouldPass() { assertThat(false).isFalse(); } } and import org.testng.annotations.Test;import static org.fest.assertions.Assertions.assertThat;public class StatusTestIT {@Test public void shouldFail() { assertThat('aaa').isEqualTo(''); } } and now let’s declare plugin mentioned above in our pom.xml: <project><!-- ... --> <build> <plugins> <!-- ... --> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.9</version> </plugin><plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-failsafe-plugin</artifactId> <version>2.13</version> <configuration> <includes> <include>**/*IT.java</include> </includes> </configuration> </plugin> </plugins> </build> </project> After that we should see two things. mvn test should pass as we have a one green test in ExampleUnitTest class, but mvn failsafe:integration-test should fail with StatusTestIT red. But as I said before, this does not work. mvn test looks as expected, but second Maven execution is passing as well showing that no test were run. Plugin seems to omit our completely valid test… Small fix doing its job After testing different approaches I found out that all we need to make this setup work is to add plugin execution to an integration-test phase of maven life cycle. So tiny change, but now our integration tests are executed only when we call mvn integration-test. <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-failsafe-plugin</artifactId> <version>2.13</version> <configuration> <includes> <include>**/*IT.java</include> </includes> </configuration> <executions> <execution> <id>failsafe-integration-tests</id> <phase>integration-test</phase> <goals> <goal>integration-test</goal> </goals> </execution> </executions> </plugin> And that’s all, we could execute integration tests only when we really want to. Of course there are some things to remember when adopting setup described above:our unit tests must have Test postfix, if you use something else, we have to configure surefire plugin using include during integration test unit tests are executed too. This is a small flaw in this setup, but exclude **/*Test.java seem not to work  Reference: Running unit tests and integration tests separately with Maven Failsafe and TestNG from our JCG partner Tomasz Dziurko at the Code Hard Go Pro blog. ...

Design Doesn’t Emerge from Code

I know a lot of people who are transitioning to Agile or already following Agile development methods. Almost all of them are using something based on Scrum at the core, mixed with common XP practices like Continuous Integration and refactoring and automated unit testing – pretty much how Mike Cohn says things should be done in his book Succeeding with Agile. Emergent Design in Scrum and XP But none of them are doing emergent design as Cohn describes it, or as Kent Beck explains how design is done in Extreme Programming: trying to get away without any upfront design and architecture work, coding features right away and relying on   test-first development, refactoring and technical spikes to work out a design on the fly, one week or two weeks at a time.“For the first iteration, pick a set of simple, basic stories that you expect will force you to create the whole architecture. Then narrow your horizon and implement the stories in the simplest way that can possibly work. At the end of this exercise you will have your architecture. It may not be the architecture you expected, but then you will have learned something.” Kent Beck You don’t need upfront architecture and design? Maybe it’s because everyone I know is working at scale – building big enterprise systems and online systems used by lots of customers, systems that have a lot of constraints and dependencies. Many of them are working on brownfield projects where you need to understand the existing system’s design and implementation first, before you can come up with a new design and before you can make any changes. Performance-critical, mission-critical systems in highly-regulated environments. Emergent, incremental design doesn’t work for these cases. And it doesn’t scale to large projects or any project that has to be delivered along with other projects and that has specified integration points and dependencies – which is pretty much every project that I’ve ever worked on. Bob Martin, another one of the people who helped define how Agile development should be done, thinks that this incremental approach to design is, well…“One of the more insidious and persistent myths of agile development is that up-front architecture and design are bad; that you should never spend time up front making architectural decisions. That instead you should evolve your architecture and design from nothing, one test-case at a time. Pardon me, but that’s Horse Shit.”Martin goes on to say that: “there are architectural issues that need to be resolved up front. There are design decisions that must be made early. It is possible to code yourself into a very nasty cul-de-sac that you might avoid with a little forethought.”Architecture and Design in Disciplined Agile Delivery The way that most people that I know approach Agile development is better described by Scott Ambler in Disciplined Agile Delivery, a model for scaling Agile to larger systems, projects and organizations. As Ambler’s research shows, almost all teams (86%) spend at least some time (on average a month or more) on upfront on planning, scoping and architecture envisioning – what he calls the “Inception Phase” (borrowing from Rational’s Unified Process) or what most others call “Sprint 0” or “Iteration 0”. This is time spent to understand the scope of the system at a high-level at least, and the constraints and dependencies that the project needs to work within. Time to model the main chunks of the system and their interfaces, and to choose a technical direction to start with. Upfront architectural and design work doesn’t have to take a lot of time. As Ambler points out, for many teams (except for some startups), a lot of architectural decisions have already been made for you: “In practice, it’s likely you won’t need to do much initial architectural modeling: a large majority of project teams work with technical architecture decisions that were made years earlier. Your organization will most likely already have chosen a network infrastructure, an application-server platform, a database platform, and so on. In these situations your team will need to invest the time to understand those decisions and to consider whether the existing infrastructure build-out is sufficient (and if not, identify potential improvements).”It’s when you have a real greenfield development project, when you don’t have anything to leverage and you’re doing something completely new, that you should spend more time on upfront thinking about design – not less. Can you “be Agile” without Emergent Design? Of course you can. Bob Martin points out that there’s nothing in “Agile Development” that says that you shouldn’t do design upfront – as much design as you need to for the size of the system that you are building and the environment that you are working in. You can and should do iterative, incremental design and development starting with a plan of where you are going and how you think that you are going to get there. As you go along and prove out your design and respond to feedback and deal with changes in requirements, this is where incremental design actually does come into play – handling changes in direction, filling in gaps, correcting misunderstandings. The design will change and maybe become something that you didn’t expect. But you need a place to start from – designs don’t just emerge from code.   Reference: Design Doesn’t Emerge from Code from our JCG partner Jim Bird at the Building Real Software blog. ...

Custom error pages for expired conversations involving CDI and JSF

It’s been a while since I last blogged. I keep thinking of blogging something technical but end up getting busy with other things. This last week there was a very interesting discussion at the coderanch forums. It was even more interesting because it involved JBoss. Developers familiar with Java EE web applications would know that the web application deployment descriptor (web.xml) allows you to specify ‘error pages’ which the container will display when a specific exception (class) or a error code is thrown by the server for a web request. Here’s a quick example of how it looks like:     <web-app> ... <!-- A custom error page for error code == 500 --> <error-page> <error-code>500</error-code> <location>/my-foo-bar-500-page.html</location> </error-page> <!-- A custom error page for exception type org.myapp.foo.bar.MyException --> <error-page> <exception-type>org.myapp.foo.bar.MyException</exception-type> <location>/my-foo-bar-exception-page.html</location> </error-page> ... </web-app> Simple enough – a couple of custom error pages defined for a specific error code and an exception type respectively. All of this works fine. In current days, more and more programming models and frameworks come into picture while developing web applications. CDI and JSF are some of those. CDI has this concept of scopes (ex: request scope, session scope, application scope, conversation scope). We won’t go into the details of what those are and when those are used, but let’s consider conversation scope in this blog since that’s what the discussion was about in the forum thread that prompted this blog. So CDI allows multiple requests to be part of a ‘conversation scope’. A conversation has a ‘start’ and an ‘end’, both of which can be managed by the application. When the application involves JSF, any conversation (id) gets auto-propagated to the JSF request(s). Apart from an explicit start/end demarcation of conversations, a conversation can also timeout. A request which refers to a conversation which has ended or timed out will run into an exception. So we know have a bit of background on CDI conversation scope. So let’s consider a case where the application wants to present a good looking page when the ‘conversation no longer present’ exception is thrown (maybe because of a timeout). We have seen how to write a web.xml for error-page configurations – it would be as simple as: <web-app> ... <!-- A custom error page for exception type org.jboss.weld.context.NonexistentConversationException --> <error-page> <exception-type>org.jboss.weld.context.NonexistentConversationException</exception-type> <location>/my-foo-bar-exception-page.html</location> </error-page> ... </web-app> Simple enough. The org.jboss.weld.context.NonexistentConversationException is the exception class type which gets thrown when the conversation has timed out (note that we are assuming that the web application is relying on Weld as the CDI spec implementation library). The above configuration works fine. The my-foo-bar-exception-page.html gets displayed when the org.jboss.weld.context.NonexistentConversationException is thrown. BUT, let’s now consider that we want to involve JSF in the error page just like the other parts of our application. So let’s point the error-page to a URL pattern which maps to the JSF servlet: ...

On The Illusion of Controlling People’s Time

Software Engineering is going through an identity crisis. The old school has failed because of too much control, moving people away from what they really have to deliver. The new school, or agile fever, is more focused on humanities, directing all the attention to human beings. This approach practically decimated the old school, showing that writing software is not an exact science. It is actually unpredictable and cannot be done without creativity, inspiration and transpiration. Maybe the old school, which insists to exist, got confused by the fact that software execution, data processing, and system and hardware design are engineering stuff, but it doesn’t mean that developing those stuff is also engineering. On the contrary… it’s human and social.   When I see managers trying to measure the time spent on people’s daily activities, I understand they are taking into consideration only the transpiration of their teams and completely ignoring the creativity and inspiration. These two last aspects are so subjective that it’s hard to compare their manifestation on different people. How to compare artists? We may find artists’ comparisons out there, but they heavily rely on who is producing the comparison. Let’s ask ourselves: why on earth would someone use quantitative criteria, such as time and money, to measure and compare subjective matters?Measurements are only useful when we’re capable of making comparisons with them in an objective way. We compare the date of our birth with the calendar to know our age. In times of crisis, we pay attention to our water, gaz and electricity consumption and make comparisons, finding ways to save money. Climate is measured because we need to decide which clothes to wear or which precautions to take by comparing with previous experiences or common sense. Notice we put measured subjects into perspective to understand the real dimension, the scale, the direction, and then we’re able to reason about trends, behaviours, issues, patterns, etc. When it comes to subjective matters, we might be able to make comparisons, but measurements are just useless in this case. Actually, we can come closer with a lot of statistical methods, multi-variable analysis, multi-criteria craziness, but do we really have time and resources to produce all these analyses in a daily basis, and even take the risk of being probabilistically far from what it’s expected? It sounds complicated, isn’t it? That’s because it is. In software development, every activity is different from one another. It’s impracticable to precisely compare two activities even if they look the same and are done by the same person. It’s all subjective, otherwise it would be easy to put the project on schedule by adding more people in the team. We know that since 1975, when Frederick Brooks published ‘The Mythical Man-Month. So, when a manager asks their developers to inform the time spent in every activity, he/she is in fact collecting lies (or non-truths). If he/she prepares a report with those collected times, this is going to be the spread of an even bigger lie (or useless information).But why lies? There are two good explanations for that:most people don’t track exactly how much time they spent on their activities, thus they actually inform safer figures, which are usually higher than reality; time control gives a feeling of surveillance, making developers believe their manager wants to know whether they are really working or not, thus they smartly add some extra time to mask non-productive time.Managers should also be aware that time control also kills creativity. When developers have new ideas, potentially useful for the project, they cannot simply turn them into activities because of scope constraints or just because those ideas are not clear enough to be described. Without activities, developers cannot report their time. Therefore, if they spend too much time working on their ideas, it will look like huge time gaps for their managers. Eventually, managers are moved to implement time control because of visible procrastination. I remember once when a new project was starting, a big room was allocated to a team of 12 people, and the project manager asked the team to choose their places in the room. There were 15 places available, 12 with monitors facing the wall and 3 with monitors facing the door. Guess which places were left available? That’s right! The three ones facing the door. I remember the symphony of mouse clicks and sudden movements when the manager walked through the room. That was so funny! But time control wouldn’t solve that problem because inputing time on activities is also a distraction. Anything that is not building deliverables is a distraction. We may never eliminate them, but we can always reduce them. The solution for procrastination, in a single word, is: Motivation! But, is time control totally out of sense? It has some sense when the work is fully based on time, which is the case of consultants payed hourly. This is about their business model, not software engineering. That’s a good practice for consultants to inform to customers how many hours they had spent because it’s directly related to their payments. Notice that this kind of service is for a short period of time and for highly specialised skills. However, we are talking about software projects that last longer. Finding ways to motivate your team is the best strategy to put them on track and fully productive. Every form of control that threats their feelings will put them down. Motivation doesn’t need to be monetary or entertaining. Sometimes, it’s enough to show their wins, how important is their work for society, how it changed people’s lives, and why it’s important to keep doing what they’re doing.   Reference: http://www.hildeberto.com/2013/01/on-illusion-of-controlling-peoples-time.html from our JCG partner Hildeberto Mendonca at the Hildeberto’s Blog blog. ...

Iterator Pattern and Java

Hello all, in this post we’ll be checking on the Iterator Pattern. A design pattern that I know many of you have already used, but maybe you didn’t realize it was pattern or didn’t know its great value. According to the book Head First Design:The Iterator Pattern provides a way to access the elements of an aggregate object sequentially without exposing its underlaying representation.Whaaaaat? Well, it says that no matter what data structure (arrays, lists, hashtables,etc.) you are using, you can always traverse it in the same way if you implement this pattern. It gives you a uniform way of accessing the elements of your data structures (aggregates), but you don’t have to know what kind of data structure   you are traversing… nice! Also, it sets the responsibility of iteration on the Iterator object not on your data structure, which simplifies the coding in your data structure. Let’s check the classic class diagram for the Iterator pattern:The actual class diagram for the Iterator Pattern has a few changes, specially in the Iterator class (interface), where we now have different methods as we’ll see in a minute, but first, lets review each of the previous classes (or interfaces):Aggregate: This is the base class (or interface) of our data structures, you can think of it as the java.util.Collection interface, which defines a lot of methods for the collection classes. ConcreteAggregate: This is the concrete data structure that we’ll be iterating, for example, java.util.ArrayList, java.util.Vector, etc. Iterator: Base class (or interface) for iterators. You can find one in Java at java.util.Iterator. You can notice that the Java version has different methods that we’ll be discussing later in this post. Here you define de methods you need in order to traverse the data structures. ConcreteIterator: As you want to traverse different data structures you need different iterators. So a concreteIterator is an Iterator for the data structure you want to traverse.Now, let’s take a look at the Java implementation of the Iterator Pattern. The following diagram was generated using Architexa’s Free Tool for Understanding Code and it shows the relations between some classes of the Java Collections Framework where we can see a structure similar to the classic class diagram:The above diagram shows only one implementation of the pattern in Java, there are a lot more, but they always use the java.util.Iterator interface; this is the interface you should use in your implementations of the Iterator Pattern when coding in Java. Let’s compare both diagrams:Classic DiagramJava exampleAggregatejava.util.AbstractListConcreteAggregatejava.util.ArrayList, java.util.VectorIteratorjava.util.IteratorConcreteIteratorPrivate internal class at java.util.AbstractList You can’t see this class in  the JavaDocs, but it is there in the source code: java.util.ItrNotice that the methods of the Iterator object in the Java example are different from the methods in the classic class diagram:There is no +First() method. If you need to go to the first element, you have to instance a new iterator. The +IsDone() method has been renamed to +hasNext(). The +Next() and +CurrentItem() has been merged into +next(). The +remove() method has been added.So, if you ever have to work with different data structures and need a uniform way to traverse them and/or access their items, think about the Iterator Pattern: //... in a class /** * Traverse a list, hashtable, vector, etc. what ever that implements * the Iterator Pattern */ public void traverse(Iterator iter) { while(iter.hasNext()) { System.out.println(iter.next()); } } Of course, you will always have to create the ConcreteIterator class for your data structure, but if you are using classes from the Java Collections Framework, it’s already done. One last thing, remember the most important OO Principle of all: Always use the simplest solution that meets your needs, even if it doesn’t include a pattern. Resources: Freeman Eric and Freeman Elisabeth and Sierra Kathy and Bates Bert (2004). Head First Design Patterns. United States of America: O’Reilly Media, Inc.   Reference: Iterator Pattern and Java from our JCG partner Alexis Lopez at the Java and ME blog. ...
Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy | Contact
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.
Do you want to know how to develop your skillset and become a ...
Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

Get ready to Rock!
You can download the complementary eBooks using the links below: