Featured FREE Whitepapers

What's New Here?


Estimates or #NoEstimates? that is the question

To estimate or not to estimate, to join the #NoEstimates bang-wagon or not, that is the question. Maybe it is a navel gazing exercise for agile-folk but it does seem to be the reoccurring theme. And I can’t get over this feeling that some of my peers think I’m a bit stupid for continuing to support estimates. Complicating matters I’m finding my own work and research is starting to be cited in support of #NoEstimate – Dear Customer (PDF version), my publicity for Vierdort’s Law, Notes on Estimation and More notes on estimation. Add my own #NoProjects / #BeyondProjects logic isn’t far removed from the whole estimates discussion. At Lean Agile Scotland a few weeks ago Seb Rose and I were discussing the subject, in the end Seb said something to the effect: “How can you continue to believe in estimation when your own arguments show it is pointless?” (I’m sure Seb will correct me if my memory is fault.) My reply to Seb was something along the lines: I continue to believe that estimation can be both useful and accurate, however I increasingly believe the conditions under which this holds are beyond most organizations.To which Seb challenged me to list those conditions. Well here is that list. I’ve blogged about this before (well, I’ve mentioned it in lots of blogs, see this one Conclusions and Hypothesis about estimates) and I’ve devoted a large section of the Xanpan book to talking about I see estimates working but I think its worth revisiting the subject. Before continuing I should say: I’m talking about Effort Estimates specifically. There is another discussion which needs to be had around business value/benefit estimation. Here is, a probably incomplete, list of conditions I think are required in order for effort estimates to be accurate:The people and teams who will undertake the work need to do the estimates Estimates go off if not used: estimates only remain valid for a short period of time (days), the longer the elapsed time between making the estimate and doing the work the less accurate they will prove Estimates will only be accurate if the teams are stable Estimates much be calibrated against past performance, i.e. data is needed Together #3 and #4 imply that only teams which have been working in this fashion for a while can produce accurate estimates Teams must have a record of delivering and must be largely able to undertake the work needed, i.e. there are few dependencies on other teams and few “call outs” to elsewhere in the organization Estimates must be used as is, they cannot be adjusted Estimates cannot be used as targets (Goodharts Law will cut if they are) Estimates made in units of time (hours, days, etc.) are not reliable The tracking and measurement process must measure all work done, not just “project” work Financial bonus should not be tied to estimates or work People outside the team should not coerce the team in any wayThere are probably some other conditions which need to be on this list but I haven’t realised. I’m happy to have additional suggestions. Perhaps this list is already so long enough as to be unachievable for most teams. Perhaps meeting this conditions are unachievable for many, even most, organizations. In which case the #NoEstimate’ers are right. So… I believe estimation can be useful, I also believe it can be accurate but I believe there are a lot of factors that can cause effort estimates to go wrong. In fact, I know one team, possibly two, who claim their estimates and planning processes is very accurate. Perhaps I cling to my belief in estimates because I know these teams. When estimates do work I don’t believe they can work far into the future. They are primarily a tool for teams to decide how much work to take on for the next couple of weeks. I don’t expect estimates further out will prove reliable. Estimates for 2 to 12 weeks have some value but beyond the 3 month mark I don’t believe they will prove accurate. So my advice: don’t estimate anything that isn’t likely to happen in the next 3 months, and don’t plan any work based on estimates which extend more than 3 months into the future. Which means: that even if you accept my argument that estimates work they may not tell you what you want to know, they may not have much value to you under these conditions. And to further complicate matters I suspect that for mature teams estimation becomes pointless too. As implied by the list above I would not expect a team new to this (agile) way of working to produce reliable estimates. With experience, and the conditions above, I think they can. One of the ways I think it works is by helping teams break down work into small pieces which flow. As a team get better I would expect the effort estimation to exhibit a very tight distribution. When this happens then simply counting the number of card (tasks, stories, whatever the thing you are estimating is) will have about the same information value for a fraction of the cost. (For example, suppose a team normally do 45 points of work per iteration, if the teams average size estimate is 5 with a standard deviation of 0.5 then they would be expected to accept 9 pieces of work per iteration. If these statistics are stable then estimation works. But at this point simply taking in 9 pieces of work would also prove a reliable guide.) So:Effort estimation doesn’t work for immature teams because they don’t exhibit the conditions above Effort estimation does work for mature teams but Effort estimation is pointless for very mature teamsEven given all this I think estimation is a worthwhile activity for teams of type 1 and 2 because it has other benefits. One benefit is that is promotes discussion – or at least it should. Another is that it forms part of a design activity that helps teams make pieces of work smaller. But there is another reason I want teams to do it: Credibility. Estimation is so enshrined in the way many businesses work that teams and those trying to introduce change/agile risk undermining their own credibility if they remove estimation early. And I don’t just mean credibility with “the business” I think many developers also expect estimation and if asked to adopt a process without it will be skeptical. So its just possible that estimation as we knowing – planning poker, velocity and such – is a placebo. It doesn’t actually help many teams but it helps people feel they are doing something right. In time they may find the placebo actually works or they may find they don’t need it. Another reason why I like developers to think about “how long will the take” is that I believe it helps them set their own deadline. It helps them focus their own work. Thus I keep advocating estimates because I think they are useful to the team, the fact that you might be able to tell when something might be “done” is a side effect. Since I find long range estimates questionable I advocate a cheap approach which might be usefulness or might just be a placebo. However, I do believe, that given the right conditions teams can estimate accurately, and can deliver to those estimates. Increasing I believe very few organizations can provide those conditions to their teams.Reference: Estimates or #NoEstimates? that is the question from our JCG partner Allan Kelly at the Agile, Lean, Patterns blog....

From Legacy Code To Testable Code–Introduction

The word “legacy” has a lot of connotations. Mostly bad ones. We seem to forget that our beautiful code gets to “legacy“ status three days after writing it. Michael Feathers, in his wonderful book “Working Effectively With Legacy Code” defined legacy code as code that doesn’t have tests, and there is truth in that, although it took me a while to fully understand it. Code that doesn’t have tests rots. It rots because we don’t feel confident to touch it, we’re afraid to break the “working” parts. Code rotting means that it doesn’t change, staying the way we first wrote it. I’ll be the first to admit that whenever I write code, it comes in its ugly form. It may not look ugly immediately after I wrote it, but if I wait a couple of days (or a couple of hours), I know I will find many ways to improve it. Without tests I can rely either on the automatic capabilities of refactoring tools, or pure guts (read: stupidity). Most code doesn’t look nice after writing it. But nice doesn’t matter. Because code costs, we’d like it to help us understand it, and minimize debugging time. Refactoring is essential to lower maintenance costs, and therefore tests are essentials. And this is where you start paying The problem of course, is that writing tests for legacy code is hard. Code is convoluted, full of dependencies both near and far, and without proper tests its risky to modify. On the other hand, legacy code is the one that needs tests the most. It is also the most common code out there – most of time we don’t write new code, we add it to an existing code base. We will need to change the code to test it, in most cases. Here are some examples why:We can’t create an instance of the tested object. We can’t decouple it from its dependencies Singletons that are created once, and impact the different scenarios Algorithms that are not exposed through public interface Dependencies in base classes of the tested code.Some tools, like PowerMockito in Java, or Typemock Isolator for C# allow us to bypass some of these problems, although they too come with a price: lower speed and code lock-down. The lower speed come from the way these tools work, which makes them slower compared to other mocking tools. The code lock-down comes as a side effect of extra coupling to the tested code – the more we use the power tools’ capabilities, they know more about the implementation. This leads to coupling between the tests and the code, and therefore make the tests more fragile. Fragile tests carry a bigger maintenance cost, and therefore people try not to change them, and the code. While this looks like a technology barrier, it manifests itself, and therefore can be overcome, by procedure and leadership (e.g., once we have the tests, encourage the developers to improve the code and the tests). Even with the power tools, we’ll be left with some work. We might even want to do some work up front. Some tweaks to the tested code before we write the tests (as long as they are not risky), can simplify the tests. Unless the code was written simple and readable the first time. Yeah, right. We’ll need to do some of the following:Expose interfaces Derive and implement classes Change accessibility Use dependency injection Add accessors Renaming Extract method Extract classSome of these changes to the code is introducing “seams” into it. Through these seams, we can enter probes to check the impact of the code changes.  Other changes just help us make sense of it. We can if these things are refactoring patterns or not. If we apply them wisely, and more important SAFELY, we can prepare the code to be tested more easily and make the tests more robust. In the upcoming posts I’ll look into these with more details.Reference: From Legacy Code To Testable Code–Introduction from our JCG partner Gil Zilberfeld at the Geek Out of Water blog....

Neo4j: COLLECTing multiple values

One of my favourite functions in Neo4j’s cypher query language is COLLECT which allows us to group items into an array for later consumption. However, I’ve noticed that people sometimes have trouble working out how to collect multiple items with COLLECT and struggle to find a way to do so. Consider the following data set:         create (p:Person {name: "Mark"}) create (e1:Event {name: "Event1", timestamp: 1234}) create (e2:Event {name: "Event2", timestamp: 4567})   create (p)-[:EVENT]->(e1) create (p)-[:EVENT]->(e2) If we wanted to return each person along with a collection of the event names they’d participated in we could write the following: $ MATCH (p:Person)-[:EVENT]->(e) > RETURN p, COLLECT(e.name); +--------------------------------------------+ | p | COLLECT(e.name) | +--------------------------------------------+ | Node[0]{name:"Mark"} | ["Event1","Event2"] | +--------------------------------------------+ 1 row That works nicely, but what about if we want to collect the event name and the timestamp but don’t want to return the entire event node? An approach I’ve seen a few people try during workshops is the following: MATCH (p:Person)-[:EVENT]->(e) RETURN p, COLLECT(e.name, e.timestamp) Unfortunately this doesn’t compile: SyntaxException: Too many parameters for function 'collect' (line 2, column 11) "RETURN p, COLLECT(e.name, e.timestamp)" ^ As the error message suggests, the COLLECT function only takes one argument so we need to find another way to solve our problem. One way is to put the two values into a literal array which will result in an array of arrays as our return result: $ MATCH (p:Person)-[:EVENT]->(e) > RETURN p, COLLECT([e.name, e.timestamp]); +----------------------------------------------------------+ | p | COLLECT([e.name, e.timestamp]) | +----------------------------------------------------------+ | Node[0]{name:"Mark"} | [["Event1",1234],["Event2",4567]] | +----------------------------------------------------------+ 1 row The annoying thing about this approach is that as you add more items you’ll forget in which position you’ve put each bit of data so I think a preferable approach is to collect a map of items instead: $ MATCH (p:Person)-[:EVENT]->(e) > RETURN p, COLLECT({eventName: e.name, eventTimestamp: e.timestamp}); +--------------------------------------------------------------------------------------------------------------------------+ | p | COLLECT({eventName: e.name, eventTimestamp: e.timestamp}) | +--------------------------------------------------------------------------------------------------------------------------+ | Node[0]{name:"Mark"} | [{eventName -> "Event1", eventTimestamp -> 1234},{eventName -> "Event2", eventTimestamp -> 4567}] | +--------------------------------------------------------------------------------------------------------------------------+ 1 row During the Clojure Neo4j Hackathon that we ran earlier this week this proved to be a particularly pleasing approach as we could easily destructure the collection of maps in our Clojure code.Reference: Neo4j: COLLECTing multiple values from our JCG partner Mark Needham at the Mark Needham Blog blog....

Spring WebApplicationInitializer and ApplicationContextInitializer confusion

These are two concepts that I mix up occasionally – a WebApplicationInitializer and an ApplicationContextInitializer, and wanted to describe each of them to clarify them for myself. I have previously blogged about WebApplicationInitializer here. It is relevant purely in a Servlet 3.0+ spec compliant servlet container and provides a hook to programmatically configure the servlet context. How does this help – you can have a web application without potentially any web.xml file, typically used in a Spring based web application to describe the root application context and the Spring web front controller called the DispatcherServlet. An example of using WebApplicationInitializer is the following:   public class CustomWebAppInitializer extends AbstractAnnotationConfigDispatcherServletInitializer { @Override protected Class<?>[] getRootConfigClasses() { return new Class<?>[]{RootConfiguration.class}; }@Override protected Class<?>[] getServletConfigClasses() { return new Class<?>[]{MvcConfiguration.class}; }@Override protected String[] getServletMappings() { return new String[]{"/"}; } } Now, what is an ApplicationContextInitializer. It is essentially code that gets executed before the Spring application context gets completely created. A good use case for using an ApplicationContextInitializer would be to set a Spring environment profile programmatically, along these lines: public class DemoApplicationContextInitializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {@Override public void initialize(ConfigurableApplicationContext ac) { ConfigurableEnvironment appEnvironment = ac.getEnvironment(); appEnvironment.addActiveProfile("demo");} } If you have a Spring-Boot based application then registering an ApplicationContextInitializer is fairly straightforward: @Configuration @EnableAutoConfiguration @ComponentScan public class SampleWebApplication { public static void main(String[] args) { new SpringApplicationBuilder(SampleWebApplication.class) .initializers(new DemoApplicationContextInitializer()) .run(args); } } For a non Spring-Boot Spring application though, it is a little more tricky, if it is a programmatic configuration of web.xml, then the configuration is along these lines: public class CustomWebAppInitializer implements WebApplicationInitializer {@Override public void onStartup(ServletContext container) { AnnotationConfigWebApplicationContext rootContext = new AnnotationConfigWebApplicationContext(); rootContext.register(RootConfiguration.class); ContextLoaderListener contextLoaderListener = new ContextLoaderListener(rootContext); container.addListener(contextLoaderListener); container.setInitParameter("contextInitializerClasses", "mvctest.web.DemoApplicationContextInitializer"); AnnotationConfigWebApplicationContext webContext = new AnnotationConfigWebApplicationContext(); webContext.register(MvcConfiguration.class); DispatcherServlet dispatcherServlet = new DispatcherServlet(webContext); ServletRegistration.Dynamic dispatcher = container.addServlet("dispatcher", dispatcherServlet); dispatcher.addMapping("/"); } } If it a normal web.xml configuration then the initializer can be specified this way: <context-param> <param-name>contextInitializerClasses</param-name> <param-value>com.myapp.spring.SpringContextProfileInit</param-value> </context-param><listener> <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class> </listener> So to conclude, except for the Initializer suffix, both WebApplicationInitializer and ApplicationContextInitializer serve fairly different purposes. Whereas the WebApplicationInitializer is used by a Servlet Container at startup of the web application and provides a way for programmatic creating a web application(replacement for a web.xml file), ApplicationContextInitializer provides a hook to configure the Spring application context before it gets fully created.Reference: Spring WebApplicationInitializer and ApplicationContextInitializer confusion from our JCG partner Biju Kunjummen at the all and sundry blog....

Exploring the SwitchYard 2.0.0.Alpha2 Quickstarts

In one of my last posts I explained how you get started with SwitchYard on WildFly 8.1. In the meantime the project was busy and released another Alpha2. A very good opportunity to explore the quickstarts here and refresh your memory about it. Beside the version change, you can still use the earlier blog to setup you local WildFly 8 server with latest Switchyard. As with all frameworks there is plenty of stuff to explore and a prerequisite for doing this is to have a working development environment to make this easier.       Setting up JBoss Developer StudioFirst things first. Download a copy of the latest JBoss Developer Studio (JBDS) 7.1.1.GA for your operating system and install it. You should already have a JDK in place so a simple:               java -jar jbdevstudio-product-eap-universal-7.1.1.GA-v20140314-2145-B688.jar will work. A simply 9 step installer will guide you through the steps necessary. Make sure to select the suitable JDK installation. JBDS works and has been tested with Java SE 6.x and 7.x. If you like to, install the complete EAP but it’s not a requirement for this little how-to. A basic setup without EAP requires roughly 400 MB disc space and shouldn’t take longer than a couple of minutes. If you’re done with that part launch the IDE and go on and configure the tooling. We need the JBoss Tools Integration Stack (JBTIS). Configure them by visiting “Help -> Install New Software” and add a new Update Site with the “Add” button. Call it SY-Development and point it to: “http://download.jboss.org/jbosstools/updates/development/kepler/integration-stack/” Wait for the list to refresh and expand the JBoss Integration and SOA Development and select all three SwitchYard entries. Click your way through the wizards and you’re ready for a re-start.Please make sure to disable Honour all XML schema locations in preferences, XML→XML Files→Validation after installation.  This will prevent erroneous XML validation errors from appearing on switchyard.xml files.That’s it for sure. Go ahead and import the bean-service example from the earlier blog-post (Import -> Maven -> Existing Maven Projects) General Information about SwitchYard Projects Lets find out more about the general SwitchYard project layout before we dive into the bean-service example.  A SwitchYard project is a Maven based project with the following characteristics:a switchyard.xml file in the project’s META-INF folder one or more SwitchYard runtime dependencies declared in the pom.xml file org.switchyard:switchyard-plugin mojo configured in the pom.xml fileGenerally, a SwitchYard project may also contain a variety of other resources used to implement the application, for example: Java, BPMN2, DRL, BPEL, WSDL, XSD, and XML files. The tooling supports you with creating, changing and developing your SY projects. You can also add SY capabilities to existing Maven projects. More details can be found in the documentation for the Eclipse tooling. Exploring the Bean-Service Example The Bean-Service example is one of the more simpler ones to get a first impression about SY. All of the example applications in the Quickstarts repository are included in quickstarts/ directory of your installation and also available on GitHub. The bean-service quickstart demonstrates the usage of the bean component. The scenario is easy: An OrderService, which is provided through the OrderServiceBean, and an InventoryService which is provided through the InventoryServiceBean implementation take care of orders. Orders are submitted through the OrderService.submitOrder, and the OrderService then looks up items in the InventoryService to see if they are in stock and the order can be processed. Up to here it is basically a simple CDI based Java EE application. In this application the simple process is invoked through a SOAP gateway binding (Which is indicated by the little envelope).Let’s dive into the implementation a bit. Looking at the OrderServiceBean reveals some more details. It is the implementation of the OrderService interface which defines the operations. The OrderServiceBean is just a bean class few extra CDI annotations. Most notably is the: @org.switchyard.component.bean.Service(OrderService.class) The @Service annotation allows the SwitchYard CDI Extension to discover your bean at runtime and register it as a service. Every bean service must have an @Service annotation with a value identifying the service interface for the service. In addition to providing a service in SwitchYard, beans can also consume other services. Those references need to be injected. In this example the InventoryService is injected: @Inject @org.switchyard.component.bean.Reference private InventoryService _inventory; Finally, all you need is the switchyard.xml configuration file where your Service, Components, Types and implementations are described. <composite name="orders" > <component name="OrderService"> <implementation.bean class="org.switchyard.quickstarts.bean.service.OrderServiceBean"/> <service name="OrderService"> <interface.java interface="org.switchyard.quickstarts.bean.service.OrderService"/> </service> </component> </composite> That was a very quick rundown. We’ve not touched the webservice endpoints, the WSDL and the Transformer configuration and implementation. Have a look at the SwitchYard tutorial which was published by mastertheboss and take the chance to read more about SY at the following links:SwitchYard Project Documentation SwitchYard Homepage Community Pages on JBoss.org SwitchYard is part of Fuse ServiceWorks, give it a try in a full fledged SOA Suite.Reference: Exploring the SwitchYard 2.0.0.Alpha2 Quickstarts from our JCG partner Markus Eisele at the Enterprise Software Development with Java blog....

Men in Tech

Background Between my partner and I, we have six daughters, and as they have grown I have thought more interested in their long term future, the role of women in society, the way technology will change our lives and in particular the role of women in technology. On the last topic, all the articles I have read have been written my women. In this post, I hope to outline my experiences in this regard. Women face many challenges, most typical of male dominated professions as well as some specific to the technology field. I believe that there will be a time when IT, like teaching and accounting, will have more women than men. I can understand it is frustrating as a woman in technology to be treated as a novelty.  Perhaps my experience can illustrate why that might be. One thing I have learnt over the years is that while I am happy to talk technology all day, there is one time to stop, in social situations when women are present. This doesn’t come from men, but women overwhelmingly. Men in Tech In high school, those interested in computers were not generally the most popular. In fact, I hung out with a large circle of friends which were some of the smartest, creative, kindest people I have known, but being popular was not priority for them. The first time I met a women who was interested in computers was at university. Those I got to know were at a disadvantage, as I saw it, as none of them had programmed a computer before university whereas the male friends of mine had been programming for 4 – 6 years (in my case, an average of 5 days of the week). When I tutored at university, all my students in computing were female, one has retired many years previous and had gone back to university. While they were all A-level students, each had plans other than IT after they left university. My first serious relationships also started at university. Some of my girlfriends studied computer science courses but dropped them after the first year. The C-word When I went out socially with my girlfriend, and I found someone interested in computers, I would happily talk to them about what interesting topics we could think of. At some point it would be made clear that it was time to talk about something else, usually by my significant other who was a bit embarrassed or bored with the topic by this stage. Usually this would start with hints, but as I am not so good with hints, she would make it really clear, resulting in her telling me repeatedly before we got there; Don’t talk about the “C-word” at all. And thus she taught me to not mention Computers when there were ladies present. I am not referred to one woman in particular, but all my other halves preferred I not talk about computers. I can remember a number of occasions when a group of men were talking about IT and a girlfriend or woman would join the group, we would immediately change the subject of conversation. We all understood while our significant others were happy to go out with computer geeks, they didn’t want to hear about it. I was at a conference recently where there was a large proportion of partners attending, joining the lunches, dinners and field trips. Even though it was an IT conference, it was generally understood that we minimise discussion of computers when women were around. I can be confident that if I am asked what I did for a living, this would be a conversation stopper and a change of subject would follow. Most men are not interested in IT, but almost all women I have met have nothing to follow such a revelation. There have been exceptions and a few women were polite enough to carry on talking about it, and from my point have view these conversations have been memorable in terms of their rarity, no matter how brief. I regularly attend conferences and give regular training sessions and I would say I can remember every conversation I had about IT with a woman for the last four years or more, with whom and broadly what we discussed. I have found that being an IT Consultant is more interesting because involves a lot of travel and I can talk about where I have been. Certainly I wouldn’t volunteer that I work in IT as an ice breaker. Conclusion I believe that having more women in technology is essential for the industry and society as a whole. I also believe while it is inevitable, we should try to ensure it happens soon rather than later especially as technology is growing at such a rapid pace. For this to happen, the attitudes of both men and women need to change.Reference: Men in Tech from our JCG partner Peter Lawrey at the Vanilla Java blog....

New programming techniques and the productivity curve

Though I love learning new programming techniques and technologies, I often struggle to make them a part of my normal development processes. For example, it took years before I finally started using regular expressions on a normal basis. The reason? The productivity curve:              You may have seen a chart like this before; the productivity curve is used to show one of the challenges of software delivery. The general idea is that when you first use a new product (or technology), your productivity always plummets in the short term. No matter how much better the new stuff is, you will be less productive using it at first, because you were more familiar with the old stuff. However, as you become more proficient with the new technology, your productivity gradually increases, and eventually your productivity will be greater than it was with the old technology (assuming the new technology is actually better). So the key is to push through that initial dip, and eventually you get to the increased productivity that comes with mastering the new technology. Easy, right? Well, if your company just replaced your HR system, you have little choice in the matter; you’ll get through that productivity dip because you have no other choice. However, if you’ve just decided you want to pick up a new programming technique, it’s a little more difficult. Before I got the hang of regular expressions, it was simply far faster for me to whip together something ugly using things like StringTokenizers and substring. Nobody cared if I was using regular expressions, but they did care if I took 10 times as long to complete a task. On the rare occasions where I needed to make a change to existing regular expressions, I’d end up slogging through references and online tutorials to try and make sense out of it. Each time, after finally figuring it out, I’d tell myself that I was going to remember how these worked next time. But since I hadn’t made regular expressions a part of my standard toolchain, I’d forget everything I learned, and I would be lost again next time I needed to use them. I eventually got the hang of regular expressions through forced practice and forced application. Forced practice was straightforward enough: I dedicated time to reading references, running through tutorials, and solving regex challenges. This practice was an important step in building the skills, but I had tried this to a certain degree before. I’d spend a few hours practicing regular expressions, and feel like I was getting the hang of it. But time would pass, and the next time I needed to fix a regex, I’d have forgotten most of it again. The key for me was forced application of regular expressions. Like the poor HR staff who learn a new HR system because they have no other choice, I forced myself to use regular expressions any time they could be useful, even though it would take me longer than a quick hack with stuff I was already familiar with. Good unit tests helped a lot; though I was not confident in my ability to write bug-free regular expressions, I was confident in my ability to write thorough tests. As the productivity curve predicted, I was definitely less efficient at first. But as I forced myself to use regular expressions instead of hacking something together more quickly, I gradually improved. And before long, I was past the productivity dip, having finally gotten the hang of regular expressions. Now they are an essential part of my development toolchain, and I am a better programmer. I know this isn’t really earth shattering advice; my five year old son has figured out that you get better at something the more you do it. But when you find some new technology or technique you want to make part of your development process, I think it’s important to go into it recognizing that not only will it take time to master it, but you will almost certainly be less productive in the short-term than you were before. And I think that if you are prepared for the productivity dip and are willing to accept it, you can push through the difficult times and master it. In the long run, you will be a better developer for it.Reference: New programming techniques and the productivity curve from our JCG partner Jerry Orr at the Jerry on Java blog....

Using Your RDBMS for Messaging is Totally OK

Controversial database topics are a guaranteed success on reddit, because everyone has an opinion on those topics. More importantly, many people have a dogmatic opinion, which always triggers more debate than pragmatism. So, recently, I posted a link to an older article titled The Database As Queue Anti-Pattern by Mike Hadlow, and it got decent results on /r/programming:Mike’s post was pretty much against all sorts of queueing in the database, which matches the opinions I have heard from a couple of JavaZone speakers in a recent discussion, who all agreed that messaging in the database is “evil.” … and I’m saying: No. Messaging in the database is not an anti-pattern, it is (can be) totally OK. Let’s consider why: KISS and YAGNI First off, if you don’t plan on deploying thousands of message types and millions of messages per hour, then you might have a very simple messaging problem. Since you’re already using an RDBMS, using that RDBMS also for messaging is definitely an option. Obviously, many of you will now think: If all you have is a hammer, everything looks like a nail … and you’re right! But guess what, the reasons for only having a hammer can be any of:You don’t have the time to equip with a sophisticated tool box You don’t have the money to equip with a sophisticated tool box You actually don’t really need the sophisticated tool boxThink about these arguments. Yes, the solution might not be perfect, or even ugly…But we’re engineers and as such, we’re not here to debate perfection. We’re here to deliver value to our customers and if we can get the job done with the hammer, why not just forget our vanity and simply use that hammer to get the job done. Transactional queues In a perfect world, queues are transactional and guarantee (to the extent allowed by underlying theory) atomic message delivery or failure – something that we’ve been taking for granted with JMS forever. At GeeCON Krakow, I had a very interesting discussion with Konrad Malawski regarding his talk about Akka Persistence. If we remove all the hype and buzzwords (e.g. reactive programming, etc.) from Akka, we can see that Akka is just a proprietary alternative to JMS, which looks more modern but is lacking tons of features that we’re used to having in JMS (e.g. like transactional queue persistence). One of the interesting aspects of that discussion with Konrad Malawski is the fact that a 100% message delivery guarantee is a myth (details here). Which leads to the conclusion: Messaging is really hard It is, indeed! So if you really think you need to embed a sophisticated MQ system, beware of the fact that you will have to learn how it really works and how to correctly operate it. If you’re using RDBMS-backed queues, you can get rid of this additional transactional complexity, because your queue operations participate in the transactions that you already have with your database. You get ACID for free! No additional operations efforts What developers very often underestimate (we can’t say this enough) are the costs incurring to your operations team when you add new external systems to yours. Having just one simple RDBMS (and your own application) is a very very lean and simple architecture. Having an RDBMS, an MQ, and your application is already more complex. There are a lot of excellent DBA out there who know what they’re doing when operating productive databases. Finding excellent “MQA” is much harder. If you’re using Oracle: Use Oracle AQ Oracle has a very sophisticated built-in queueing API called Oracle AQ, which can interoperate with JMS. Queues in AQ are essentially just tables that contain a serialised version of your message type. If you’re using jOOQ, we’ve blogged about how to integrate Oracle AQ with jOOQ, recently. RDBMS-centric applications can be much easier We’ve blogged about that before as well: Why Your Boring Data Will Outlast Your Sexy New Technology. Your data might just survive your application. Consider Paypal replacing Java with JavaScript (it could also have gone the other way round). In the end, however, do you think that Paypal also replaced all their databases? I don’t. Migrating from Oracle to DB2 (different vendor), or from Oracle to MongoDB (different DBMS type) is mostly motivated by political decisions rather than technical ones. Specifically, people don’t migrate from RDBMS to NoSQL databases entirely. They usually just implement a specific domain with NoSQL (e.g. document storage, or graph traversal) Assuming that the above really applies to you (it may, of course, not apply): If your RDBMS is in the middle of your system, then running queues in your RDBMS to communicate between system components is quite an obvious choice, isn’t it? All system parts are already connected to the database. Why not keep it that way? Conclusion The arguments listed here are all pretty obvious and pragmatic. At some point, they no longer hold true, as your messaging demands are really big enough to justify the integration with a sophisticated MQ system. But many people have strong opinions about the “hammer / nail” argument. Those opinions may be correct but premature. Very often in software engineering, it is entirely acceptable and sufficient to work with just one tool. The hammer of software: The RDBMS.Reference: Using Your RDBMS for Messaging is Totally OK from our JCG partner Lukas Eder at the JAVA, SQL, AND JOOQ blog....

WORA Is Better Than Native

When “Write Once Run Anywhere” is done right it can produce applications that are “better” than native apps by targeting the highest common denominator. Some would claim that native is the best approach, but that looks at existing WORA tools/communities, which mostly target cost saving. In fact, even native Android/iOS tools produce rather bad results without deep platform familiarity. Native is very difficult to properly maintain in the real world and this is easily noticeable by inspecting the difficulties we have with the ports of Codename One, this problem is getting worse rather better as platforms evolve and fragment. E.g. Some devices crash when you take more than one photo in a row, some devices have complex issues with http headers, and many have issues when editing text fields in the “wrong position”. There are workarounds for everything, but you need to do extensive testing to become aware of the problem in the first place. WORA solutions bring all the workarounds and the “ugly” code into their porting layer, allowing developers to focus on their business logic. This is similar to Spring/Java EE approaches that addressed complexities of application server fragmentation. If you review most user feedback on mobile applications you will quickly notice that a company that releases applications often and fixes issues gets consistent 5 star reviews. The ability to provide a stable app across platforms is a much tougher nut to crack than most developers assume, bugs are remarkably portable even between programming languages. Being able to scale your development team to give fast response times to user requirements is a huge deal. Being able to use Google Play as a beta site for the more conservative (and slower to approve) iTunes is a remarkable benefit for any developer. Codename One is relatively young, so we don’t have a blockbuster, multi-million downloads app that contains a large enough representative segment (we do have some huge customers and great apps so its a matter of time). However WAZE (recently acquired by Google for over 1b USD) was constructed with a WORA solution, has multi-million downloads and 4.5+ star reviews on iOS/Android. I’m not crazy about their UI (as a matter of personal taste) but I think it proves that a HUGE majority of users want a functional, stable, and fast app. The word native and its various interpretations are only a part of the developer lexicon and don’t matter as much to actual users. Using the native platform has a huge conflict of interest problem, e.g. Google is very keen on developers building Android applications. However, it has slowly migrated new API’s into the Market app, which only enables them for Google sanctioned devices. This makes sense since vendors have been very slow at migrating to newer Android OS versions. However, this also means that non-Google devices such as the Kindle (and future Amazon phone etc.) wouldn’t work with those API’s. Thus, we have a situation where the platform vendor wants us to develop to his platform alone while, as developers, we want to have as many users as possible. Yes, iOS users provide better monetization but that might change in the future and as developers we have an interest in agility. As a side-note, Google itself uses a relatively simple WORA solution when targeting iOS named J2ObjC its a great solution for porting libraries from Java to Objective-C. In the past, the conflict of interest between the platform developer and the tool developer pushed many developers from Microsoft tools to Borland tools and later on to Java/Swing. I believe we will see similar trends as mobile platforms migrate towards the enterprise and we are already seeing such development with PhoneGap despite all its faults. The Native Is Easy Myth Often when I bring these things up, I get the strong knee-jerk reaction from the fans of native claiming that native is the best approach. This completely ignores the long-term complexities of building/maintaining complex applications across platforms. For example, a well known Google developer asked in a recent Google IO who in the audience understood the activity lifecycle, he then made fun of the audience members who raised their hands claiming that despite all his years in Google he still doesn’t get the activity lifecycle (I built a few VM’s in my life and I can attest that this is indeed a complex subject with many edge cases). iOS is not much better, the compiler/runtime doesn’t protect you from basic mistakes such as wrong function arguments (function, not message), memory access issues etc. You need to maintain around seven splash screens for a typical universal app, which you need to update with every change to the first view and don’t get me started on provisioning profiles… The main issue isn’t writing the code, it’s maintaining it in the long term, scaling it for additional team members, and working in a corporate environment. As the mobile market matures from app startups, who have relatively small teams of hackers, and moves to the enterprise the need to scale the development becomes far greater. Enterprise developers need the ease of Swing, Visual Basic or Powerbuilder when it comes to mobile development. They also usually prefer vendor independence when possible. Just like Java EE, SOA or ESB took over backend development we will see mobile services take over the client side. Developers who didn’t work in demanding Enterprise environments or worked in a very specific niche in an enterprise sometimes claim, “well they should hire better developers”. But that isn’t really the issue, projects such as these are maintained over years when developers hand them off from one person to the next as manpower/positions change. In such circumstances tools must be very simple to enable efficiency. Speed & Flexibility Performance is a big issue with WORA solutions, unfortunately we are sometimes judged by the performance of a specific application which might not always be under our control. The Pareto principle applies here: by improving a very small section of the code we can improve everything. Most of our rendering code is native and if you have mission critical code you can always write it natively to reach the performance level you would expect from any solution. Unlike HTML5, which is effectively throttled by how fast webkit can reflow, in a proper WORA solution you are throttled only by the performance of the native OS. Since we can invoke native code we can pretty much do anything we want within an application e.g. integrate with the Dropbox API via OAuth while using a native intent to invoke Dropbox on Android. This is actually implemented by some applications to enable easy usage of files within a mobile app, which is a painful experience on all mobile platforms. Final Thoughts My goal here isn’t so much to convince native developers to abandon native programming, but rather to change the basic “WORA as a compromise” attitude where native is considered as the best approach. I think this approach rose from sub par WORA solutions and implementations, such as the much maligned Facebook app. Despite the fact that I think embedding HTML5 (as is done in PhoneGap) isn’t a very good approach, I think that in the right hands it can produce a decent app. As a developer I try not to blame my tools but rather look at how the tool works and try to understand if its limitations are inherent or temporary. I think most of the issues we see in WORA tools are being resolved and we will see far more WORA apps on the market in the coming years. ...

The fastest way of drawing UML class diagrams

A picture is worth a thousand words Understanding a software design proposal is so much easier once you can actually visualize it. While writing diagrams might take you an extra effort, the small time investment will pay off when others will require less time understanding your proposal. Software is a means, not a goal We are writing software to supports other people business requirements. Understanding business goals is the first step towards coming up with an effective design proposal. After gathering input from your product owner, you should write down the business story. Writing it makes you reason more about the business goal and the product owner can validate your comprehension. After the business goals are clear you need to move to technical challenges. A software design proposal is derived from both business and technical requirements. The quality of service may pose certain challenges that are better addressed by a specific design pattern or software architecture. The class diagram drawing hassle My ideal diagram drawing tool will simply transpose my hand-drawing sketches to a digital format. Unfortunately I haven’t yet found such tool, so this is how I do it:I hand draw all concepts and interactions on a piece of paper. That’s the most rapid way of design prototyping. While I could use a UML drawing tool, I prefer the paper-and-pencil approach, because changes require much less effort Once I settle for a design proposal, I start writing down the interfaces and request/response objects in plain Java classes. Changing the classes is pretty easy, thanks to IntelliJ IDEA refactoring tools. When all Java classes are ready, I simply delegate the class diagram drawing to IntelliJ IDEAIn the end, this is what you end up with:Reference: The fastest way of drawing UML class diagrams from our JCG partner Vlad Mihalcea at the Vlad Mihalcea’s Blog blog....
Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy | Contact
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.
Do you want to know how to develop your skillset and become a ...
Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

Get ready to Rock!
You can download the complementary eBooks using the links below: