Featured FREE Whitepapers

What's New Here?

software-development-2-logo

REST Tooling

In section 6.3 of Roy’s dissertation, he explains how REST applies to HTTP. But the implementing a RESTful approach requires painstaking assembly without REST tooling. Java JAX-RS and API Management infrastructure reduce the learning curve, increase API adoption and decrease development effort by simplifying API creation, publication, and consumption. The Java API for RESTful Web Services: JAX-RS JSR 311, JAX-RS, is Java’s RESTful programming model. In JAX-RS, a single class corresponds to a resource. Java annotations are used to specify URI mappings, mime type information, and representation meta-data conforming with REST constraints (see Table 1). Table 1. Mapping REST concepts to JAX-RSREST concept JAX-RS Annotation or class ExamplesAddressability @Path and URI Path Template @Path(“/user/{username}”)Uniform Interface @GET, @PUT, @POST, @DELETE, @HEAD @GET@Produces(“application/json”)public String getUser(String username) { return getUserService(username)); }Self-descriptive messages @Produces, @Consumes @Produces({“application/xml”, application/json”})HATEOAS UriBuilder UriBuilder.fromUri(“http://localhost/”).   path(“{a}”). queryParam(“name”, “{value}”). build(“segment”, “value”);WSO2 Application Server relies on Apache CXF to process JAX-RS annotations and expose a RESTful API.   Your existing Apache CXF code can be readily migrated to WSO2 Application Server. API Management RESTful APIs may be either naked or managed.  A naked API is not wrapped in security, subscription, usage tracking, and service level management.  A managed API increases reliability, availability, security, and operational visibility.   By placing an API gateway in front of your naked RESTful API or service, you can easily gain advanced capabilities (see Figure 1).  The API gateway systematizes the API façade pattern, and enforces authorization, quality of service compliance, and usage monitoring without requiring any back-end API modifications.   Figure 2 demonstrates API facade actions commonly provided by industry leading API gateway products.  WSO2 API Manager can easily integrate with your RESTful system and rapidly add advanced capabilities.  For more information on API management, read the technical evaluation guide.Reference: REST Tooling from our JCG partner Chris Haddad at the IT Business Navigation Charts blog....
apache-cassandra-logo

Getting Started with Apache Cassandra and Java (Part II)

Requirements To follow this tutorial, you should already have a running instance of Cassandra (a small cluster would be good, but not necessary), the Datastax Java driver installed (refer to Part I), and have gone through the 10 minute walkthrough here: http://planetcassandra.org/create-a-keyspace-and-table/.           Try it out For this demo, we’re going to be creating a simple console application, almost identical to the one in Part I, only this time we will be exploring connection policies, prepared statements, and query builder. Open a text editor and create a java file with a “GettingStartedTwo” class and a single main method. public class GettingStartedTwo { public static void main(String[] args) { Cluster cluster; Session session; ResultSet results; Row rows; Then we can connect to our cluster and create a session instance. // Connect to the cluster and keyspace "demo" Cluster cluster = Cluster.builder() .addContactPoint("localhost") .build(); Session session = cluster.connect("demo"); But wait, now that we are running a cluster instead of a single instance, we’ll want to put some safeguards in place incase of a failover. We can do this using a RetryPolicy.The retry policy determines the default behavior to adopt when a request either times out or a node is unavailable. We’re using the DefaultRetryPolicy in this case, which will retry queries either:on a read timeout, when enough replicas have replied but the data wasn’t received. on a write timeout, if we timeout while writing the log used by batch statements.cluster = Cluster .builder() .addContactPoint("192.168.0.30") .withRetryPolicy(DefaultRetryPolicy.INSTANCE) .build(); session = cluster.connect("demo"); A load balancing policy will determine which node it is to run a query. Since a client can read or write to any node, sometimes that can be inefficient. If a node receives a read or write owned on another node, it will coordinate that request for the client. We can use a load balancing policy to control that action. The TokenAwarePolicy ensures that the request will go to the node or replica responsible for the data indicated by the primary key. It is wrapped around DCAwareRoundRobinPolicy to make sure the requests stay in the local datacenter.  This is a good choice for us as, although we only have our one local cluster at the moment, we are already thinking about the next step, expanding to multi-datacenter. cluster = Cluster .builder() .addContactPoint("192.168.0.30") .withRetryPolicy(DefaultRetryPolicy.INSTANCE) .withLoadBalancingPolicy( new TokenAwarePolicy(new DCAwareRoundRobinPolicy())) .build(); session = cluster.connect("demo"); Now that you are connected to the “demo” keyspace, let’s insert a user into the “users” table. This is exactly what we were doing in Part I earlier, but we’re doing it a little differently this time. Using prepared a statement is  more secure and the most performant way to get data into or out of our database. Prepared statements only need to be parsed once by the cluster, and then then values are bound to variables and then we execute the bound statement to read/write data from the cluster. // Insert one record into the users table PreparedStatement statement = session.prepare( "INSERT INTO users" + "(lastname, age, city, email, firstname)" + "VALUES (?,?,?,?,?);"); BoundStatement boundStatement = new BoundStatement(statement); session.execute(boundStatement.bind("Jones", 35, "Austin", "bob@example.com", "Bob")); Using the Java driver, we can easily pull the user back out. In Part I of Getting Started with Apache Cassandra with Java, we used a string representation of CQL. Now (and for the rest of the tutorial), we are going to do the same with Query Builder, which is more secure and saves us from potential CQL injection attacks. // Use select to get the user we just entered Statement select = QueryBuilder.select().all().from("demo", "users") .where(eq("lastname", "Jones")); results = session.execute(select); for (Row row : results) { System.out.format("%s %d \n", row.getString("firstname"), row.getInt("age")); } Since it’s Bob’s birthday, we are going to update his age. // Update the same user with a new age Statement update = QueryBuilder.update("demo", "users") .with(QueryBuilder.set("age", 36)) .where((QueryBuilder.eq("lastname", "Jones"))); session.execute(update); // Select and show the change select = QueryBuilder.select().all().from("demo", "users") .where(eq("lastname", "Jones")); results = session.execute(select); for (Row row : results) { System.out.format("%s %d \n", row.getString("firstname"), row.getInt("age")); Now let’s delete Bob from the table, and print out all the information left in the users table. You’ll notice that Bob’s information no longer comes back after being deleted (others might, if you have inserted users previously). // Delete the user from the users table Statement delete = QueryBuilder.delete().from("users") .where(QueryBuilder.eq("lastname", "Jones")); results = session.execute(delete); // Show that the user is gone select = QueryBuilder.select().all().from("demo", "users"); results = session.execute(select); for (Row row : results) { System.out.format("%s %d %s %s %s\n", row.getString("lastname"), row.getInt("age"), row.getString("city"), row.getString("email"), row.getString("firstname")); } Make sure that the connection closes once you are done. // Clean up the connection by closing it cluster.close(); } } CQL is very similar to SQL, in many cases the same syntax will work.  This makes querying for data very straightforward if you have a background with relational databases.You have just managed to connect to a Cassandra cluster and perform queries against a live (local) database. Hopefully this demonstrates just how easy it is to use Cassandra using the Java driver.  A Gist of the complete console application for this sample is available on GitHub.Reference: Getting Started with Apache Cassandra and Java (Part II) from our JCG partner Rebecca Mills at the Planet Cassandra blog....
agile-logo

Opposite Views of a Product Roadmap

Your product roadmap a view of what you are building right now, in the near future, and in the more distant future.  Or is your roadmap a view of why you are building whatever you’re building right now, in the near future, and in the more distant future? Your roadmap is both – but one is more important than the other – and product managers need to be able to view the roadmap both ways. When you view the spinning cat animation above, you will either see it as rotating clockwise or counter-clockwise.  Everyone has a default.  Because of bi-stable perception, the direction of rotation may reverse for you, but reversing direction is apparently more difficult for some people than others. Depending on your role in your organization you will be biased towards viewing your roadmap either as a view of what the team will be building into your product, or a view of why the team will be building things into your product.  As a product manager, it is imperative that you can view it both ways. I also hope to make the case that one view is more important than the other. What the Team Will Be Building Janna Bastow wrote an excellent article about the importance of flexibility in roadmaps.  The article provides a really good explanation of how product management is not project management, and the evils of having a roadmap which is nothing more than a “glorified Gantt chart.” She provides a great visual depiction of rolling-wave planning.  I describe rolling-wave planning as being precise about the immediate-term future, having less specificity about the near future, and being nonspecific (but still helpful) about the more distant future.  This is because –  from the perspective of someone focusing on what they will be doing – there is too much uncertainty about what will happen between now and the future.  [ BTW, this is where waterfall is wasteful – it assumes incorrectly an ability to predict the future in detail.] The article provides very good explanations about the need for flexibility both in terms of time and scope within a roadmap.  Shifting the delivery of something to an earlier or time frame, or alternately thinking about a particular time frame including more or less deliverable.  This is however, only a “good explanation” when you’re thinking from the point of view of what the team will be building. The key attributes of a product roadmap from this view are that descriptions of what to do are precise right now, less specific in the near term, and very flexible in the future. While I completely agree – given a focus on what is being built – I don’t think about product roadmaps in this way.  I believe my thinking has shifted because I’m primarily creating roadmaps, not consuming them. Intentionality A concept which may help you shift to thinking about a product roadmap in terms of why the team will be building is intentionality. There is a reason why you’re building something. Your team is building features because they are trying to build solutions. More precisely, your team is building a product with a set of capabilities, which you hope your customers will choose (and use) to help with solving their problems. Customers are trying to solve problems, they aren’t trying to use features. As a product manager, your perspective needs to be rooted in the perspective of the problems your customers are trying to solve – the intent driving your roadmap – not the things your team is building in order to solve the problem. Why the Team Will Be Building Where the view of what to build gets fuzzier as you move from the present to the future, the view of why you build gets clearer as you move from the present to the future.  This is the exact opposite of rolling-wave planning, and that’s not only fine, it is good. Choosing a particular group of customers (or market) to serve is a strategic decision. Generally this will not change, and when it does it will not be frequent. This is a long term view – the future for the team will involve building things to support this group of customers.  There is great and powerful clarity here – specifically about the future.  “We are, and will be, building to support customer group X.” Given the decision to provide value for a specific group of customers, the logical next question is “how?”  To avoid ambiguity, the question is really “which of the problems our target customer faces are we going to help them solve?”  The charter for a product team can be described as “Help [a specific set of] customers solve [a specific set of] problems.” In the near-term, there is flexibility in the choice of which problem to address next.  Having that flexibility is imperative, because discovery (from feedback from customers) tells us if we’ve got the right problems, in the right sequence.  So we need to be able to re-prioritize, and add or remove from the list of problems as we manage our roadmaps thematically. In the “right now” the team is testing hypotheses.  Given a customer, and a problem the customer is trying to solve, there is a hypothesis about how best to help.  That hypothesis is a design.  The design may or may not be a good one.  The implementation of the design may be great, adequate, failing, or incomplete.  In this time horizon, the activities of the team building the product are concrete – build, test, learn. These activities are much hazier from the perspective of why the team is building something.  Technically, there is still clarity about the customer and the problem (they aren’t changing).  However, there’s additional explanation required – a description of the hypothesis – to explain why a particular activity is being managed for the current sprint or release.  As an example “We found that forcing users to make decision (X) when solving problem (Y) caused those users to abandon our product.  Based on interviews we’re defaulting decision (X), and we believe this will reduce abandonment by (Z%).” The need for explanation about the specific “what” is how I interpret the reduced clarity about the near term, when focusing on why versus what. I selected the spinning cat image because both points of view are valid.  From one point of view, all of the clarity occurs in the immediate term, and from the other point of view, all of the clarity manifests in the longer-term big picture.  Most people first see the cat spinning in the same direction, and most of the time product managers should seef their roadmap “customer first.” A product manager has to be able to switch between both views – and communicate in either framework – depending on the context of what they are doing in the moment.  Building the roadmap is working backwards from the customer to the feature.  Helping the team execute, and appreciate the context and relevance of what they are working on involves going from the current deliverables up to the intent for building them. Frequency of Change from Learning Another reason I work “outside in” in my thinking about roadmaps is the frequency of change which comes from what we learn as we engage with our market, study our competitors, and adapt to industry changes. Significant and sustained* feedback is needed to tell us we have chosen the wrong customers. How we choose customers is a topic for another article.  When I hear about companies pivoting, I think of it as picking different customers. We start out engaging customers with a hypothesis that they will pay us to help them address a particular set of problems.  We start this process with a point of view of the relative priority of solving each problem.  Feedback from customers – throughout the product creation process – helps us improve this list of problems.  All within the stable context of helping a particular customer group solve their problems. Processes like the structured conversation from Discover to Deliver include these activities as part of the overall product creation process. While working with customers, we find out what it means to satisfice.  We also develop and refine our understanding of the nature of the problems using tools like Kano analysis.  We also have the opportunity to discover when a particular design does not achieve the goal of helping the customer, or when a poor implementation fails to achieve the vision of the designer.  Sometimes, when stories are split to fit within a sprint, the smaller stories don’t actually solve the problem, and additional splits need to be completed before moving forward. *This assumes we did not make colossally bad choices to begin with – opening us up to the debate between rationality and empiricism.  Bad choices may be discovered immediately. Communication and Conclusion Executives (should) want to talk about intent and strategy for investment in the product. Base your conversations on this.  If they need status report type updates about the activities of the team, then talk about features.  But try and shift the conversation back to corporate goals, strategy, and the role your product is intended to fill. Other interested stakeholders will almost always ask about features.  This is perfectly understandable – they are not product managers, and they are focused on the wrong things.  They are focused on problem manifestations, not problems. Help them refocus while you address their concerns. Product managers should drive roadmaps based on “why” not “what.”  We still need to be able to think in the opposite direction, but that approach should be secondary.Reference: Opposite Views of a Product Roadmap from our JCG partner Scott Sehlhorst at the Tyner Blain’s blog blog....
agile-logo

Estimating business value: adding Value poker and Dragons Den to the Agile toolkit

A common piece of advice heard in Agile circles is: “Prioritise by value. Do the highest value first.” Sound advice, easy to say but perhaps harder to do. And if you know me – or just read this blog regularly – you may have heard me say something like: “Estimate the benefit/value expected, measure what is actually delivered and feed this back to your decision making process: calibrate you benefit estimates, do more work where benefit is missing or change direction when it is not possible.”   I’m sure I could find more examples but I’m sure you know what I’m talking about: understand the benefit/value you expect to get – and possibly check it afterwards. Easy really. But there is a problem: How do you know what benefit/value is expected? A good product manager or business analyst might be able to come up with some numbers. Good, but if you dig deep enough you’ll find assumptions or models in these figures which could be questionable. The better your analyst the deeper you will need to dig before any assumptions come to light. As for teams who don’t have a product manager or business analyst, well, they aren’t even going to get that far before they find questionable assumptions. Very often the expected benefit/value is a matter of conjecture and opinion. So let me make a suggestion: Value poker. This is a technique I’ve been using for a while and always teach it in my Agile for BAs courses. Whenever I mention it people get interested. To make it work I adapt a game-show format, specifically: Dragons Den, Sharks’ Tank if you are in the US. Here is how you play… Two teams. One drawn from the people who are planning to build a product. This could be the entire development team, it could be just the product manager or business analyst with the product sponsor/champion. This team play the Entrepreneurs. If need be this could be just one person (a product owner/business analysts/product manager) but it helps if there are two of them and if there is a whole team then bring them along too. The second team is the Dragons/Sharks/Investors Team. This team is probably a bigger. In a training session I usually use two teams from an earlier exercise where they have created user stories but in real life it is business managers from elsewhere in the business, perhaps product managers, analysts, sponsors and champions of other products. It could even be a high level committee – CEO, CFO, CTO, Sales, etc. The Entrepreneurs come armed with a set of story cards – these could be in user story format, use case format or some other format, they could be epics or smaller. Whatever, the team need to believe each of these has business value. Preferably I’d rather these cards did NOT have any effort estimates on them at this stage. Then we set up a Dragons Den setting.Next I ask the Entrepreneurs to pitch their product – the whole thing – to the Dragons. Usually one of the team who is a bit more entrepreneurial steps up. When the pitch is finished the dragons get to ask questions. And we have a discussion back and forth. Then, as moderator, I ask the Entrepreneurs for the lowest value item them have in their deck. I take it from them and I invent a currency. This is usually named after the town I’m in, so I’ve invented Newcastle Shillings, Houston Dollars, Bath Spa Pounds or some such. Its imaginary, lets pretend I’m using London Dollars, L$. I read out the card the Entrepreneurs gave me and make sure everyone understands what it is. If necessary the Dragons can ask some questions. Then I write on the card L$10,000 – ten thousand London Dollars. I tell everyone about the imaginary currency and about London Dollars. I then place the card in full view – on a magnetic whiteboard or blu-tacked to the wall, or somewhere.I hand out the planning poker cards to the Dragons only and tell them the cards are now denominated in thousands of London Dollars. So a 1 card is worth L$1,000 and a 8 card is worth L$8,000, a 21 card is worth L$21,000 and so on. And I ask the Entrepreneurs for the next card. I take it, I read it out. I ask the Entrepreneurs if they want to add anything to what is written. Then we take questions from the Dragons, and the discussion rolls. After a while – sometimes a few minutes, sometimes a lot longer – I move to the vote, planning poker style. I read the card out again and ask choose a card that indicates how many London Dollars this story is worth – relative to the L$10,000 card we already have. I count down, 3, 2, 1 – show me! And the Dragons hold up the cards. I average the answer and write the number on the story card. So, if I have a vote of 11, 21, 65 and 40 the value would be: 137/4 = L$34,000. I usually don’t bother doing any discussion or re-voting, I just average – and I don’t care if the average is a number not on any planning poker card. And we repeat – as a value estimate is assigned to one card we move to the next. Not every story needs to be estimated, the Entrepreneurs may decide to skip some once they see the results of previous rounds. Entrepreneurs may write some new ones as conversations with Dragons reveals new ideas or prompts a rethink. Indeed one of the reasons I like to have more than one entrepreneur in the game is so that one can write new cards while the other is pitching and talking to the Dragons. As each card is estimated it goes on the board with the others relative to the value assigned so everyone can see how the stories stack up. People can really get into their role play, you can see some entrepreneurs really fighting for their product as the Dragons poke holes in the idea. Sometimes – perhaps even most time – the conversations that occur as the game plays out are the most interesting bit. New features and functionality are brought to light. Sometimes the value the entrepreneurs see is not what the dragons see. Sometimes critical pieces of requirement or specification are discovered. During the summer I played this game with a class in Louisiana, the entrepreneurs had created a set of stories around a food-truck locator app. Some of the stories related to the food-truck owner and some to a Hungry Jo. The entrepreneurs saw the value being on the food-truck owner side, so they emphasised this in their pitch and kept offering up stories abut the owner. The dragons kept low-balling these stories, the entrepreneurs got frustrated and argued more, how the dragons didn’t realise what they saw. At my promoting the entrepreneurs offered up a story about the Hungry Jo. To their surprise the dragons went high. This was the story the dragons saw value in. Now you could say that it would be better to test the market – research or lean start-up – and I wouldn’t disagree but even if you do that it can be hard to put value behind stories. Plus, faced with 20 stories which one should you research or try first? This approach applies wisdom of crowds. It gives you a starting point. And as I just said, its just possible that the real value of the technique is not in the value it assigns to the cards – although that is useful – but in the conversation you have in the process. Sure you end up with a fantasy valuation but you do have an idea of relative values, you do let stakeholders have their say, and you have some initial priorities. Much better than Must, Should, Could, etc. Potentially even better than 1, 2, 3, … Maybe, just maybe, one day you might be able to see the value one story actually delivered – a jump in eyeballs, sales, donations or something. And with that you might be able to calculate what L$1 is worth. Two final points before I end. I try to keep effort estimates out of this. It is my (unproven) belief that if the dragons know the effort estimate on a card this will anchor their value estimate. I want value estimates to be made without reference to cost. Second, a twist on this would be to revisit the story cards with a cost of delay dimension. So: value estimate the cards on the basis of “If you had this next month” then revisit then say “Now lets assume the cards aren’t ready for three months” and revote. I haven’t had a chance to do that yet but I think it would be interesting. Finally: if you get a chance to try this technique – or if you have done something similar already – please share, I’d love to heard what other people think of the technique and how it plays out elsewhere.Reference: Estimating business value: adding Value poker and Dragons Den to the Agile toolkit from our JCG partner Allan Kelly at the Agile, Lean, Patterns blog....
java-logo

Java performance tuning survey results (part IV)

This is the last post in series where we analyze the results of the Java Performance Tuning Survey we conducted in October 2014. If you have not read the first posts, I recommend to go through the following first:                Frequency and severity of performance problems Most popular monitoring solutions Tools and techniques used to find the root causeThis post is opening up some interesting correlations found in the data and summarizing the results. Reproducing is a key to quick success When you are in charge of solving a performance issue, you need evidence to find the root cause. To get the evidence, you often need to reproduce the issue. In survey, we asked the respondents whether they were able to reproduce the issue:9% did not need to reproduce, having enough evidence already 27% were unable to reproduce the issue 64% managed to reproduce the issueIn another question we asked “How long does it take to find and fix the issue you were facing”. On average, this took 80 hours. We analyzed, whether to the 27% who were not able to reproduce the issue were struggling more. Results were clear:If the respondent was able to reproduce the issue, then on average it took 65 hours If respondent was unable to reproduce the issue, it took 113 hours, or 74% more time to find the root cause and fix it.The difference is clearly visible. The cause for the difference is hidden inside the troubleshooting process. To fix an issue, you need evidence, typically gathered from various sources, such as log files, thread dumps or heap dumps. But you can only get the evidence if you are able to reproduce the case, preferably at will. If you cannot reproduce the problem, you are left without evidence and the only tool in your arsenal tends to be the good old trial and error. Facing a 100,000+ lines of code, you are doomed to face a lot of failed trials along the way. Some issues are tougher than others. Respondents also gave us the underlying root cause for the performance issue they were solving. We looked into the different issues in order to understand whether some of the issues are harder to solve than others Let us again recall that average time to find and fix the issues was 80 hours. When categorizing by the issue type, we found the following:The easiest issues to find and fix are related to network IO: with 51 hours spent on average. Memory leaks rank exactly average by the time spent: on average it takes 80 hours and 24 minutes to spot and fix one. On the other end of the spectrum are architectural issues – when the underlying cause were related to monolithic architecture and HTTP session bloat, where it took 98 and 105 hours correspondingly. This is around to 100% more time to find and fix the cause.Looking at the extremes, it should not actually be surprising. When your architecture is causing performance issues, then the fix itself tends to be complex and time-consuming, thus requiring more time to fix. And when you tend to abuse the network, it often boils down to a single malicious call, which you can rather easily isolate and fix. Random tools to help Next, we analyzed the tools and techniques used to troubleshoot certain underlying root causes. We noticed that on average, users try out no more, no less than four different tools to gather evidence and find the root cause. Most popular tools and techniques involved log analysis, heap/thread dumps and profilers. When we looked the usage of tools across different underlying problems, we became truly surprised. There was very little correlation in between the underlying problem and tools used to troubleshoot – the same tooling was listed with the same frequency independent of the problem at hand. The best example for this could be thread dump analysis. This is a good way to gather evidence about concurrency issues. Indeed, 52% of the respondents solving a concurrency issue used thread dump analysis as one of the root cause analysis sources. But for example, when the problem at hand was a memory leak, then the very same thread dump analysis was listed on 42% of the occasions. Or when looking from the tooling perspective – independent of the problem type, 41-53% of the respondents used profiler to gather evidence, independent of the symptoms and underlying problem. It is tricky to draw a conclusion from this data, but it appears that the evidence gathering and analysis process is very informal and involves using the tools and techniques this particular person has used or has heard of before. Conclusions This survey was conducted to steer further Plumbr development. Main conclusion for us is built upon four key outcomes of the survey:Average time to find and fix a performance issue is 80 hours For 76% of the cases, most of this time is spent in the vicious “trying to reproduce – gathering evidence – interpreting evidence” cycle. Reproducing is impossible for 27% of the cases. In such cases, finding and fixing the issue took 73% more time. Evidence gathering process is completely informal and involves on average four randomly chosen toolsWe promise to take it from here and offer a solution to the issues listed above. Using Plumbr to monitoring your systems pinpoints you to actual underlying root cause immediately, completely skipping the “trying to reproduce – gathering evidence – interpreting evidence” cycle:Our current offering allows this for thread locks, inefficient GC and memory leaks, but we keep expanding our offering, so that you will have a safety net in place for all the performance issues affecting your JVM.Reference: Java performance tuning survey results (part IV) from our JCG partner Ivo Mägi at the Plumbr Blog blog....
java-interview-questions-answers

Quick Guide: Dissecting JBoss BPM Cross Process Communication

(Article guest authored together with Jey Paulraj, Senior Solution Architect at Red Hat in North America) The weeks tips & tricks article will dive in JBoss BPM Suite and specifically a question around how to communicate between two processes. Before we get into the solution details, let us first constrain the use case we will be talking about.   There could be many interpretations around what communication is between two processes, but we will start here with a simple way for one process to call another process. We will also show this simple usage through the provided RestAPI, which we will leverage to provide a deployable artifact that you can leverage as a custom work handler in any BPM process. The artifact is a class we labeled RestApi.java, it contains the setup and details that make it possible for you to start another process from your existing process. We provide the class in it’s entirety at the end of this article but first we look closer at the various moving parts. The top of every class includes the various imported objects or classes that will be used, we are most interested in the Knowledge Is Everything (KIE) API components and you will find them listed there along with some objects that represent our healthcare examples domain model. package org.jboss.demo.heathcare;import java.net.MalformedURLException; import java.net.URL; import java.util.ArrayList; import java.util.HashMap; import java.util.List; import java.util.Map;// JBoss BPM Suite API import org.kie.api.runtime.KieSession; import org.kie.api.runtime.manager.RuntimeEngine; import org.kie.api.runtime.process.ProcessInstance; import org.kie.services.client.api.RemoteRestRuntimeEngineFactory;// Domain model. import com.redhat.healthcare.CaseContext; import com.redhat.healthcare.Doctor; import com.redhat.healthcare.PatientInfo; import com.redhat.healthcare.Prescription; import com.redhat.healthcare.Rxdetail; Next we will find the actual start of our RestAPI class, where we setup some of the attributes needed to work with the API along with a constructor that will make sure our JBoss BPM Suite server is running. Please note the process deployment ID along with the user and password are fictitious, so any resemblance to actual process data is by chance only. String deploymentId = "com.redhat.healthcare:patients:1.0"; String bpmUrl = "http://localhost:8080/business-central"; String userId = "admin"; String password = "bpmsuite1!"; URL deploymentUrl;// Constructor to check for availability of BPM server. // public RestApi() { super(); try { this.deploymentUrl = new URL(); } catch (MalformedURLException e) { e.printStackTrace(); } } The URL tested is assuming a basic default local installation, so if your installation is in a different setup you will need to adjust for that. The next code snippet highlights a core helper method, the one that gets us a reference to the runtime engine. This is the engine that ties us through a RestAPI to a specific deployment com.redhat.healthcare:patients:1.0 giving us access to start a process in that deployment. // Get a runtime engine based on RestAPI and our deployment. // public RuntimeEngine getRuntimeEngine() { RemoteRestRuntimeEngineFactory restSessionFactory = new RemoteRestRuntimeEngineFactory(deploymentId, deploymentUrl, userId, password);// create REST request RuntimeEngine engine = restSessionFactory.newRuntimeEngine(); return engine; } With a runtime engine we can now access and create our sessions, where we will then be able to start our process instances. The following method is called to start a process instance and contains for the sake of clarity only, the creation of a collection of data for submission into our process. It should be easy for you to see that this can be abstracted out as needed, for the use of process variables being mapped into your class. // Setup our session, fill the data needed for process // instances and starting our process. // public void startProcess() {String taskUserId = userId;// create REST request. RuntimeEngine engine = getRuntimeEngine(); KieSession ksession = engine.getKieSession();// setup data for submission to process instance. Doctor doctor = new Doctor(); doctor.setAddress("3018 winter"); doctor.setCity("madison"); doctor.setGender("M"); doctor.setGroupId("UW1001"); doctor.setHospital("1001"); doctor.setName("jey"); doctor.setState("WI");PatientInfo pat = new PatientInfo(); pat.setAge(12); pat.setName("jey"); pat.setSymbtom("Diabetes Insipidus"); pat.setType("Diabetes");Rxdetail rxdetail = new Rxdetail(); List<rxdetail> details = new ArrayList<rxdetail>(); rxdetail.setDrugName("xx"); rxdetail.setOther("red"); rxdetail.setQty(11); rxdetail.setRxNum(11); details.add(rxdetail);CaseContext cont = new CaseContext(); cont.setApprovalReq("N"); cont.setApprovalReq("Supervisor"); Prescription prescription = new Prescription(); prescription.setDoctor(doctor); prescription.setPatientInfo(pat); prescription.setRxdetails(details);// collect all data in our map. Map<string object=""> params = new HashMap<string object="">(); params.put("prescription", prescription); params.put("caseContext", cont); // start process. ProcessInstance processInstance = ksession.startProcess("healthcare.patientCaseProcess", params);// verify process started. System.out.println("process id " + processInstance.getProcessId()); System.out.println("process id " + processInstance.getId()); } With this method we have setup our doctor, patient and other medical details required by the process, collected them into a map and submitted them to the process instance to kick it all off. Now we can tie this all together so that the main class that is run when this is called will just setup our RestAPI and start a new process instance each time it is called. // Start our process by using RestAPI. // public static void main(String[] ar) {RestApi api = new RestApi(); api.startProcess(); } We hope this simple tour through this medical example gives you an idea of how to use the provided JBoss BPM Suite RestAPI to your advantage. In this case we use it for communicating to a specific process in a specific deployment from any other process deployed on our BPM server. Here is the RestApi class: package org.jboss.demo.heathcare;import java.net.MalformedURLException; import java.net.URL; import java.util.ArrayList; import java.util.HashMap; import java.util.List; import java.util.Map;// JBoss BPM Suite API import org.kie.api.runtime.KieSession; import org.kie.api.runtime.manager.RuntimeEngine; import org.kie.api.runtime.process.ProcessInstance; import org.kie.services.client.api.RemoteRestRuntimeEngineFactory;// Domain model. import com.redhat.healthcare.CaseContext; import com.redhat.healthcare.Doctor; import com.redhat.healthcare.PatientInfo; import com.redhat.healthcare.Prescription; import com.redhat.healthcare.Rxdetail;String deploymentId = "com.redhat.healthcare:patients:1.0"; String bpmUrl = "http://localhost:8080/business-central"; String userId = "admin"; String password = "bpmsuite1!"; URL deploymentUrl;// Constructor to check for availability of BPM server. // public RestApi() { super(); try { this.deploymentUrl = new URL(); } catch (MalformedURLException e) { e.printStackTrace(); } }// Get a runtime engine based on RestAPI and our deployment. // public RuntimeEngine getRuntimeEngine() { RemoteRestRuntimeEngineFactory restSessionFactory = new RemoteRestRuntimeEngineFactory(deploymentId, deploymentUrl, userId, password);// create REST request RuntimeEngine engine = restSessionFactory.newRuntimeEngine(); return engine; }// Setup our session, fill the data needed for process // instances and starting our process. // public void startProcess() {String taskUserId = userId;// create REST request. RuntimeEngine engine = getRuntimeEngine(); KieSession ksession = engine.getKieSession();// setup data for submission to process instance. Doctor doctor = new Doctor(); doctor.setAddress("3018 winter"); doctor.setCity("madison"); doctor.setGender("M"); doctor.setGroupId("UW1001"); doctor.setHospital("1001"); doctor.setName("jey"); doctor.setState("WI");PatientInfo pat = new PatientInfo(); pat.setAge(12); pat.setName("jey"); pat.setSymbtom("Diabetes Insipidus"); pat.setType("Diabetes");Rxdetail rxdetail = new Rxdetail(); List<rxdetail> details = new ArrayList<rxdetail>(); rxdetail.setDrugName("xx"); rxdetail.setOther("red"); rxdetail.setQty(11); rxdetail.setRxNum(11); details.add(rxdetail);CaseContext cont = new CaseContext(); cont.setApprovalReq("N"); cont.setApprovalReq("Supervisor"); Prescription prescription = new Prescription(); prescription.setDoctor(doctor); prescription.setPatientInfo(pat); prescription.setRxdetails(details);// collect all data in our map. Map<string object=""> params = new HashMap<string object="">(); params.put("prescription", prescription); params.put("caseContext", cont); // start process. ProcessInstance processInstance = ksession.startProcess("healthcare.patientCaseProcess", params);// verify process started. System.out.println("process id " + processInstance.getProcessId()); System.out.println("process id " + processInstance.getId()); }// Start our process by using RestAPI. // public static void main(String[] ar) {RestApi api = new RestApi(); api.startProcess(); }Reference: Quick Guide: Dissecting JBoss BPM Cross Process Communication from our JCG partner Eric Schabell at the Eric Schabell’s blog blog....
mandrill-logo

Mandrill: Email infrastructure from MailChimp

Mandrill is an email infrastructure service that started as an idea in 2010. That idea became reality in 2012, when Mandrill cannibalized a crew of MailChimp’s best engineers. Isolated from the rest of the team, they turned the idea from a skunkworks project into a product that outperforms competitor services. Growing fast and innovating faster, Mandrill is now the largest Email as a Service platform on the market, with more than 250,000 active customers. Use Mandrill to send automated one-to-one email like password resets and welcome messages, as well as marketing emails and customized newsletters. Mandrill is quick to set up, easy to use, and ridiculously stable. We made it for developers, who love documentation, integrations, high delivery rates, webhooks, and analytics.Mandrill comes with a beautiful interface, flexible template options, custom tagging, and advanced tracking and reports. Mandrill is the only email infrastructure service with a mobile app that lets you monitor delivery and troubleshoot from wherever you are. It’s also powerful, scalable, and affordable. But you don’t have to take our word for it. This post is sponsored via Syndicate Ads. ...
software-development-2-logo

R: Applying a function to every row of a data frame

In my continued exploration of London’s meetups I wanted to calculate the distance from meetup venues to a centre point in London. I’ve created a gist containing the coordinates of some of the venues that host NoSQL meetups in London town if you want to follow along:           library(dplyr)   # https://gist.github.com/mneedham/7e926a213bf76febf5ed venues = read.csv("/tmp/venues.csv")   venues %>% head() ## venue lat lon ## 1 Skills Matter 51.52482 -0.099109 ## 2 Skinkers 51.50492 -0.083870 ## 3 Theodore Bullfrog 51.50878 -0.123749 ## 4 The Skills Matter eXchange 51.52452 -0.099231 ## 5 The Guardian 51.53373 -0.122340 ## 6 White Bear Yard 51.52227 -0.109804 Now to do the calculation. I’ve chosen the Centre Point building in Tottenham Court Road as our centre point. We can use the distHaversine function in the geosphere library allows us to do the calculation: options("scipen"=100, "digits"=4) library(geosphere)   centre = c(-0.129581, 51.516578) aVenue = venues %>% slice(1) aVenue ## venue lat lon ## 1 Skills Matter 51.52 -0.09911 Now we can calculate the distance from Skillsmatter to our centre point: distHaversine(c(aVenue$lon, aVenue$lat), centre) ## [1] 2302 That works pretty well so now we want to apply it to every row in the venues data frame and add an extra column containing that value. This was my first attempt… venues %>% mutate(distHaversine(c(lon,lat),centre)) ## Error in .pointsToMatrix(p1): Wrong length for a vector, should be 2 …which didn’t work quite as I’d imagined! I eventually found my way to the by function which allows you to ‘apply a function to a data frame split by factors’. In this case I wouldn’t be grouping rows by a factor – I’d apply the function to each row separately. I wired everything up like so: distanceFromCentre = by(venues, 1:nrow(venues), function(row) { distHaversine(c(row$lon, row$lat), centre) })   distanceFromCentre %>% head() ## 1:nrow(venues) ## 1 2 3 4 5 6 ## 2301.6 3422.6 957.5 2280.6 1974.1 1509.5 We can now add the distances to our venues data frame: venuesWithCentre = venues %>% mutate(distanceFromCentre = by(venues, 1:nrow(venues), function(row) { distHaversine(c(row$lon, row$lat), centre) }))   venuesWithCentre %>% head() ## venue lat lon distanceFromCentre ## 1 Skills Matter 51.52 -0.09911 2301.6 ## 2 Skinkers 51.50 -0.08387 3422.6 ## 3 Theodore Bullfrog 51.51 -0.12375 957.5 ## 4 The Skills Matter eXchange 51.52 -0.09923 2280.6 ## 5 The Guardian 51.53 -0.12234 1974.1 ## 6 White Bear Yard 51.52 -0.10980 1509.5 Et voila!Reference: R: Applying a function to every row of a data frame from our JCG partner Mark Needham at the Mark Needham Blog blog....
software-development-2-logo

Go for Java Programmers: Simple Types

Part of the Go for Java Programmers series Before we get into custom types, and Go’s version of object-orientation, let’s cover the simple types that come out of the box.  Full details can be found in the (surprisingly readable) language spec, but here are some highlights that are important from a Java programmer’s perspective.           Booleans Boolean variables are declared with type bool, and their value must be one of two constants, true or false.... var isActive bool fmt.Printf("The value of 'isActive' is %t\n", isActive) ... The output of this program would be: The value of 'isActive' is false Notice that Go’s bool, just like Java’s boolean primitive, initializes to false if you don’t supply any value.  However, unlike Java, Go does not have the separate concepts of “primitive” boolean and “object” Boolean. Go does have the concept of “a type” vs. “a pointer to that type”, and we will have to worry about null pointers when we get there (a couple of articles in the future!).  For now, rest easy that when you’re not dealing with pointers, Go types operate much like Java primitives.  Go initializes them with sane defaults, and in fact the compiler won’t allow you to initialize one with nil (i.e. Go’s version of null). Characters and Strings In Java, the String type is a class… there is no concept of string as a primitive type.  Once again however, the line of demarcation in Go lies between “types” and “pointers-to-types”.  When you initialize a non-pointer type directly, it is similar to working with a Java primitive. So no more null-check hassles with strings!  If you initialize a Go string without assigning a value, then by default it is given the empty-string value (“”), rather than nil. Unicode support Speaking of assigning values, Go strings are quite Unicode-friendly.  The file format for Go source code is UTF-8, and so any string literals declared in your code are likewise stored as UTF-8.  Strictly speaking, a Go string is a series of bytes rather than an array of characters, and so there are edge cases in which a string variable might contain binary data (see this interesting article on the official Go blog for more detail).  However, Go strings in practice are almost always UTF-8.... fmt.Printf("Escaped represention:\t %+q\n", "世界") fmt.Printf("UTF-8 representation:\t %s\n", "\u4e16\u754c") ... The output of this program would be: Escaped represention: "\u4e16\u754c" UTF-8 representation: 世界 Characters and substrings For storing single characters, Go has the data type rune.  The rune type is an alias for int32 (covered in the next section below), and is meant for storing one Unicode character (or “code point”).  You can extract from a string single runes, or a substring, by using a zero-indexed array syntax.  The built-in Go function len(s) gives you the length of a string, to help you avoid the Go equivalent of an ArrayIndexOutOfBoundsException:... testString := "This is just a test string"fmt.Printf("This string has %d characters\n", len(testString)) fmt.Printf("The 3rd character is: %c\n", testString[2])lastRuneIndex := len(testString) - 1 fmt.Printf("The last character is: %c\n", testString[lastRuneIndex])fmt.Printf("The third word in the sentence is: '%s'\n", testString[8:12]) ... The output of this program would be: This string has 26 characters The 3rd character is: i The last character is: g The third word in the sentence is: 'just' The testString[8:12] syntax used to extract a substring will make more sense when we cover arrays and slices in an upcoming article.  For now, just know that the number on the left-hand side of the colon is the starting index for the substring, and the number on the right-hand side is the ending index. Standard Go string functions In Java, the String class contains numerous methods for manipulating, formatting, and searching for characters within a string.  Other core classes contain methods for converting between String and other common data types (e.g. Integer.parseInt(s)). With Go, string is effectively a built-in primitive type, with no methods.  Therefore, the Go standard library contains two packages to fill that gap:strings – Search and manipulation functions.  Supported operations including splitting and joining, finding characters and substrings, converting to lower-case or upper-case, etc. strconv – Functions for converting between string and other data types or output formats.Numeric Types Types vary between platforms One of the tricky things about writing in C is that data types can vary from one platform to the next.  An int might take up 32 bits on one machine, and 64 bits on another.   The Java Virtual Machine, with its “write once, run anywhere” mantra, hides this from the programmer and enforces consistent data types on all platforms. Go is a partial throwback, combining elements of both.  If you need a particular size, then you can declare a specific numeric type (e.g. int32, int64).  However, if you merely declare a variable as int, then its size can vary from 32 to 64 bits depending on the platform. In practice, this doesn’t matter much.  Even in the worst-case, the range of a 32-bit int is sufficient for most normal tasks.  Also, the ambiguous int is the data type used to index arrays and slices (covered in an upcoming article).  So general best practice is to:use an int when you need something that’s at least 32 bit, and you don’t care if it’s bigger use a more specific type only when you really need a larger or smaller number rangeAltogether, these are the integer and floating-point types available in Go, along with the range of possible values for each:Go type Java equivalent Rangeint8 byte -128 to 127int16 short -32,768 to 32,767int32 int -32,768 to 32,767int64 long -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807float32 float 1.40129846432481707e-45 to 3.40282346638528860e+38, positive or negativefloat64 double 4.94065645841246544e-324d to 1.79769313486231570e+308d, positive or negative  Go also provides complex64 and complex128 types, for advanced math operations involving imaginary numbers.  There is no direct equivalent in the Java library, although you can find similar data types in the Apache Commons Math project. Notice that while Go has has an ambiguous int type, which might be 32-bit or 64-bits depending on the platform, there is no generic floating-point type.  You must specify float32 or float64.  Unlike integer types, which differ only in the range of numbers that they can hold, there are subtle differences in the way that 32-bit and 64-bit floats handle precision.  This is why Java developers typically use BigDecimal for financial transactions!  Speaking of which, Go’s standard library offers equivalent types in its “math/big” package.  The Go type Int is similar to Java’s BigInteger, and Go’s Rat (short for “rational”) is similar to Java’s BigDecimal. Go has both signed and unsigned ints All of the integer primitives in Java are signed.  This means that if Java is your only programming language, you may not even be aware of a difference between signed and unsigned numbers!  In essence, a signed integer can be either positive or negative, wheres an unsigned integer can only be zero and up.  So with unsigned types, you can get twice the range out the same memory space, so long you don’t need to store any negative values.Go type Rangeuint8 0 to 255uint16 0 to 65,535uint32 0 to 4,294,967,295uint64 0 to 18,446,744,073,709,551,615  Conclusion There are few quirks that you have to look out for with Go’s primitive types, such as the possibility of a string containing binary data, or the fact that integers vary in size between platforms.  Overall, however, many things about Go primitives might be a breath of fresh air to you as Java programmer.  There is none of Java’s bifurcation between “primitives” and “object wrappers”, and no autoboxing.  Moreover, unless you choose to create a pointer, there are no NullPointerException‘s! In the next article we will look at creating custom types in Go, and how Go’s type system differs from object-orientation in Java.Reference: Go for Java Programmers: Simple Types from our JCG partner Steve Perkins at the steveperkins.net blog....
software-development-2-logo

The “Uncanny Valley” of L3 Cache Contention

While preparing for my talk at QCon SF 2014, I wanted to investigate a theory around how micro-benchmarks are not a useful reflection of how software may behave when run as part of a larger application.  Specifically due contention in the last-level cache (L3* in current Intel CPUs). An Example Last year while working on a system to store market data, I needed a time series data store.  I’d used LevelDB to solve a similar problem in the past and looked at using it again.  The system I was building had a reasonably high throughput requirement (160K ops/sec).  I built a micro-benchmark around the data I was trying to store in order to verify that that LevelDB would be sufficiently fast.  It managed to clock in at around 400K ops/sec, which suggested that it would be a workable solution.  However, when I merged the code into our main application an ran it within our end to end performance testing environment and it was unable to keep up the aforementioned throughput rate. My suspicion is when running the code within the micro-benchmark it had the whole L3 cache to itself.  When running as part of a larger application, with a number of other busy threads running concurrently, the shared L3 cache becomes a point of contention significantly impacting the throughput of the data store.  The problem with a contended L3 is that each thread will be constantly replacing the values in that cache pushing out the cache lines of the other thread, meaning there is a much greater probability of the memory access missing cache and needing to access the data from main memory. CPU Architecture Refresher If you have a general interest in computing performance and haven’t been hiding under a rock for the past 10 years, then you are probably aware that modern CPUs have a multi-layer memory hierarchy; consisting of a series of progressively larger and slower** caches between the processor and main memory.  The diagram below shows the layout of a Intel Nehalem CPU.As you can see each of the individual cores has a dedicated L1 and L2 cache, but the L3 cache is a shared resource.  If 2 threads are running on 2 separate cores and their working set fits cleanly within L1 or L2 and the memory used by each is independent, then the 2 threads can run in parallel without impeding each other.  So the question remains that if the working set is larger and requires L3 how will that impact the performance of individual threads. A Performance Test To see if I could understand the impact I wrote a small performance test.  The test works by holding two integer arrays.  The first holds a set of random values, the second holds indexes into the first array.  The code iterates over the array of indexes, reading values from the first array.  The index array is shuffled resulting in a data access pattern that mixes sequential and random access.  The test measures the how quickly all of the values from the data array are loaded via the index array. The test runs with a varying number of threads (one to five) with a range of working set sizes, ranging from 16 bytes to 128 MB.  The chart below shows the throughput for varying thread counts against the working set size.The throughput of each scenario proceeds in a relatively linear fashion until the size of the working set reaches the size of the L3, which happens to be 8MB on my test machine.  After this point the performance degrades as most of the access stop hitting L3 and have to fetch data from main memory.  What is interesting is the point around 8MB where the throughput of all the cases converge to roughly the same value.  If instead of plotting the throughput we calculated the “speed up” that each of the scenarios had over the baseline single-threaded case, we would see the scalability*** of the code, by working set size.There is clearly a dramatic drop in the “speed up” or scalability when the working set for each thread matches the size of the L3.  This is what I referred to as the “uncanny valley” in the title of the post.  At this point the code is “perfectly contended” or, to quote the story of Mel, “most pessimum”.  This shows very clearly that multiple threads working on completely independent sets of data can contend with each other when running on different cores, but the same socket.  To the point where the performance degrades such that it is equivalent of running the code on a single thread. There are a couple things that can be taken from this:Beware believing the specific numbers from micro-benchmarks as there is a good chance that when running on a busy system that other threads on the same core could be reducing the overall performance of the code that was benchmarked. If writing parallel code, working set size is very important.  In fork/join style applications the point where the code stops subdividing should take into account the size of the working set to avoid contending on the L3.There is one further thing to consider.  The test above is based on a mix of sequential and random access.  If we change the tested so that the index array wasn’t shuffled, so that all memory accesses are sequential, does the “uncanny valley” still make an appearance?Looking at these charts, there is only a small movement in the “speed-up” at 8MB for the 2 Threads case.  However, this could be explained by run-to-run variance, so is not conclusive.  I think that if memory accesses are purely sequential then L3 cache contention is unlikely to have a significant affect on performance and scalability.  So (surprise, surprise) algorithms that are completely (or mostly) sequential with regards to the way they access memory are going to suffer from fewer performance artefacts.  However, there are many algorithms that are random access by their very nature, e.g. the mark phase of a garbage collection system.  In those cases paying attention to working set size and what other threads in the system are doing will be important to ensure that the code runs well. * Some of the latest Haswell CPUs have an L4 cache. ** Specifically higher latency. *** Increase in throughput with additional threads.Reference: The “Uncanny Valley” of L3 Cache Contention from our JCG partner Michael Barker at the Bad Concurrency blog....
Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy | Contact
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.
Do you want to know how to develop your skillset and become a ...
Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

Get ready to Rock!
You can download the complementary eBooks using the links below:
Close