Featured FREE Whitepapers

What's New Here?


Things Great Engineers (almost) Never Say

My job as a recruiter of software engineers and my 12 years as a user group leader have provided me the opportunity to speak with many talented technologists since entering the business in 1998, and through my interactions I’ve been able to notice several behavioral patterns shared by those that are considered to be the most skilled. To develop a profile of a candidate I ask many questions that can lead to a variety of answers, and I pay close attention to responses. Beyond behaviors, there are things that you hear in conversations with less skilled engineers that you don’t hear in talks with the best. Here is a list of quotes I sometimes will hear, but that I almost never hear from the engineers that are most coveted.“I’ve used _____ but I have no idea how it works” – Great engineers gained their skills through probing and curiosity. They go ‘under the hood’ of the products they use just to understand how things work, even if that information will never be very useful to them. It is unclear whether this need to dig deeper is a choice or a compulsion, but it seems that it is a trait of the best talent. “______ works, I just don’t know how to explain it” – Not only do the greats know how things work, but most of them derive genuine pleasure in telling others why and how things work. This is often true even if the other person doesn’t care to know. Over the course of my career I’ve listened to engineers talk for thousands of hours, and in almost all cases my conversations with the most talented go much longer. I actually made a point of scheduling an extra fifteen to twenty minutes on my calendar when I will be speaking to someone who appears to be great on paper, as I know that the person will go deep into some technical details and nuances that will come up in conversation. “I will need ______ (tool/condition) to complete this task” – The masters of development will have the ability to improvise and adjust on the fly to arrive at a solution in non-ideal conditions. When you hear of engineers being compared to MacGyver they are speaking of this very rare skill. Greats will figure out a way based on minimal resources and will be aware of alternatives to their first choice of tool. “I’ve learned all I want/will ever need to know about ________ “ – Continuous learning and improvement is probably the most obvious differentiation between the good and the great. They don’t just get to a high level and then rest. The best engineers understand that industry progress causes them to never stop learning, and if they are not gaining enough new knowledge in their day jobs they will invest the time to learn during off hours. “There is no solution” – The greats will continue looking at a problem different ways to come up with an answer instead of conceding. This characteristic can be both a blessing and a curse. “I hate programming” - At times a great engineer will hate their job or employer, but their love of solving problems with code is what brought them to this skill level. “I’m an expert in _____” (when it’s not true, and even sometimes when it is true) – The strongest engineers have no need or desire to lie or exaggerate the depth of their skills. Much of the great talent will be reluctant to say they are an expert in anything, as they are aware others could know more. “I don’t understand the business” – The stronger engineers will always want to know how their code impacts the employer and is able to describe their contribution and value to the organization. “I don’t pay particular attention to industry trends” – Being both great and relevant (in demand) is only possible if you keep an eye on where the industry is headed. When speaking to talented engineers, the topic of which new technologies are actually viable and which are a passing phase will often come up.If you find yourself using any of these lines on a regular basis, give some thought to why. Quoting one of these lines in a job interview could be a serious mistake. Reference: Things Great Engineers (almost) Never Say from our JCG partner Dave Fecak at the Job Tips For Geeks blog....

Using Hudson/Jenkins to diagnose that intermittent failure

I have been working on one of those intermittent bugs that just won’t reproduce on my machine; but will reproduce intermittently on other machines while they are running automated testing. I filled the code with trace statements, now I suspect the problem is in code that I don’t control and doesn’t appear to have much in the way of diagnostics in the place I am working on. So I did the obvious thing which is to run the tests on a loop on my machine overnight, 12 hours later and 8 test iterations later, no test failures and I am no further forward. Since the tests are failing in the Hudson farm, it make sense to try to connect the debugger up to those jobs; but I don’t want to hang around to attach the remove debugger to each. Thankfully there is a workaround that will allow me to set suitable breakpoints and manage the debugger connection for me. First of all you need to configure you IDE to accept incoming debugger connections, here are some notes on configuring JDeveloper for a listening debugger, in Netbeans you need to use the Debug->Attach menu item and select ‘SocketListen’ as the connector and configure as per JDeveloper. In Eclipse you need to configure the debug type as ‘Socket Listen’. The second step is modifying your build system so that there is a target you can call that will start the test cases in debug mode. This is an example of the parameters for one of our CI jobs that passes in the right information. Note of course the blacked out text the the name of then machine you are trying to connect back to. (The java tests are started with the parameter -agentlib:jdwp=transport=dt_socket,address=xxxx.oracle.com: 5000 ,server=n) Make sure that you don’t have any firewalls running on that machine that will block the in-coming connections.You probably will want to run to run multiple jobs at the same time if you have the nodes available, so consider checking this concurrent build box. Always a good idea to bring cakes / cookies into the office if you are going to tie up all the preflight nodes for the day.And then all that remains is to run a bunch of jobs and wait for your breakpoint to be hit, might take a little while; but it is going be quicker than running these jobs in series on your own machine. And if your farm is Heterogeneous so much the better for reproducing intermittent failures.You can sit back and then wait for your code to fail….. may I suggest some sessions from JavaOne while you wait? Reference: Using Hudson/Jenkins to diagnose that intermittent failure from our JCG partner Gerard Davison at the Gerard Davison’s blog blog....

Bad developer – good boss

While I am typing this, some developer is creating his kick-off application that will make him a millionaire. Well ok, wealthy enough to sustain his small business and motivate him to create more. At this moment some other developer is publishing his revolutionary application or game on Google Play store or Apple’s Appstore or a website out there. High hopes and hard work. Might change their lives. These small ways to make money or become popular do change our lives. Not only from perspective that the new software innovations ease our lives, but also from the perspective of the changer, of those that build things for which people would say “thank you for this!!” or “you saved me so much time!”. “Sliding tackle” beginning But developers at some point have to start somewhere. They have to learn to solve the easy-to-bypass problems the hard way. And most probably they’ll do it in the first company that will give them an offer. And so it begins their professional life as developers. In my experience, it’s been quite a long road so far. I’ve seen much and committed lots of lines of code (out of which only small percentage on distributed VCS). So I figured out that maybe I need to share that experience. Not that I want to warn someone about something or show-off infront of the other developers. Simply, because I miss those days when I was at the beginning of my learning process. Curvy and edgy. Well this might sound a bit old and cliche, but I have to say it again. Software development at home has nothing to do with corporate-driven software development. When the developer goes into corporate environment, he’s simply thrown into the boiling oil. Into the pan. If he does not jump off the pan there is big chance that he will survive the process afterwards. The problem is what comes afterwards and why would most of the developers allow treatment that they do not deserve? Not because they are developers, but because they are humans. When we go behind the cubicle we need to understand at the beginning, there is a big chance that behind those walls are money hungry people that look at you as their resource (or body ). Not much space for love (ok, not in that sense). This sounds bad and aggressive. It seems that it puts every manager, boss or supervisor in the same basket. While I believe that there are pretty nice and caring bosses, I also know that out of 5 companies I have worked with, only 1 had the characteristic to be polite enough so that I felt comfortable whenever I got sick, had to finish something at the bank or had a bike accident. Of course there are good and nice bosses to which we offer our humble appreciations, but also there is other side.Knowing it Not sure if your boss knows it, but you know it. They are doing it wrong. And should you be quiet about it. You go at work, see all those mistakes they make, get your salary, do your thing, go home, eat, sleep, live with the disease until you notice that it has eaten all of your career. Ok. No. Just go and tell them about the problem. They will not agree and there is a fat chance they’ll change their opinion about the calm and obedient employer they talked to yesterday. But would you care? From this perspective I cannot care less. Why? Because when you quit your position in few years, well again there is big chance that nobody will remember you. Maybe painful, but it’s true. Ok, if you didn’t invent the next big thing out there on the web or assembled the best Android tablet the human eye has seen, then you are forgotten in two days. And no one cares. Being honest about what you consider wrong in your environment is crucial and you should always talk to your boss(es). It’s not that personal thing rather than it’s general for the good of the company. The company would benefit, you would also, and what’s most important, the collective will. And those that come after you. People should always point to the mistakes other people do. Not by making fun of them but by talking with them about it. On your way to being the best developer the solar system has known, you’ll be seeing things like bad business strategy, bad logistics within the company, ugly project management and lots of milk in the coffee. And showing your opinion in public is the best tool to change that. It’s much better than complaining on your breaks and not taking any action. ( hitting the drums while jumping on your desk yelling about the problem is not something I also suggest ). While working on one of my first big projects as C++ developer, which happened to be a multi-platform desktop application and my first serious project I worked on as junior developer, I had the feeling that the leading project manager was underestimating the things that everyone else except the team-lead (senior developer) was proposing. Why that behaviour, I might guess, but that didn’t make me be silent. At some points I was trying to organize meetings on which I’d openly mention the problems I thought were hanging over the project. Not always was I right, but as the time passed, it turned out that on some occasions I really was. The thing that the less experienced observer sees is pretty hard for the experienced executor to notice. Always try to appreciate everyone’s opinion just as you’d like yours to be appreciated. If you have an idea on the project, on the business plan or the project management share it even if it’s not considered at all. After all, that way you build your professional inclusion in the decision making for the good of the company. Punishment Punishment is so 2012. BC . Punishment creates rage and rage turns into resentment. Why would you like to make the employees dissatisfied? No idea. But I know why you should make your developers satisfied. Because they are the final key in the chain of production. Keep your camel slaked in the desert. Working for the same company mentioned earlier I was once called at the .. well, let’s say boss’ office and told to remove a retweet about other company’s job opportunity. I even got warned that if I don’t do that I might be suspended or even fired for destroying company’s reputation on the social networks. That, I might add, was one of the last drops in the glass of reasons to quit that job. Few weeks later I resigned. And why? Because I was fed with the ignorance of the people, with their superiority complex and the way they were resolving situations. Never tolerate punishment as a way to force proper way of work. Managers should talk to their employees before doing anything stupid that would make them feel less valuable in the collective. Of course, I don’t say that developers or any other employee does not make mistakes. But guessing we are people, I bet there are much civilized ways to solve similar situations. An estimation is something you create so that you get the general idea behind the project by disassembling the requirements. Also you make use of an estimation to protect the business value of the company’s attempt to finish a certain project successfully by estimating its costs for delivery. That said, the primary purpose of the estimation is to serve the developer and then the company.Estimation Estimation is something that the developer must do and the employer must respect. It’s simple math. If I can walk from point A to point B with a pot full of water in 10 minutes, where the main rule is not to pour more than 10% from the content of the pot, than the two-way walk would most probably take me at about 20 minutes. Of course I can run on the way back and get at the final destination in 12 minutes, but the thing is that most probably I’d lose half of the water running. And that might not be a problem again, if the company doesn’t mind lousy software and spaghetti code, developer should be fine with that. The problem comes when the developer is forced to run with a pot full of water from Washington to Los Angeles in 2 days, without pouring 50% of the water from the pot. That would be the same as when working on delivering a software solution. People tend to cut off developers estimations or even give an estimation for themselves for a project that would not be carried by them which is something the developer, regardless to his level of experience should not allow. Let’s not get this idea wrong and say that there should never be a case when the developer should not hurry and try to accomplish something which is urgent and suddenly popped up as problem (“shoot for the stars and hit the moon”-kind of bullshit). I am trying to highlight that the general practice is not to take into consideration the estimation given by the guy or team that would create the final product. And that unfortunately is commonly seen in most of the companies out there. And in reality the process of estimation and delivery is something totally different. When someone gives you an estimation try to see if there are any overestimates or underestimates, if not try to respect that estimation. Because it’s a way of telling you I can deliver this in x days, anything before that would be forced and with poor quality. Also, when someone gives you an estimation for a project it’s because you asked for it. If you can modify it and adapt the estimation to the business needs then you should’ve never asked for an estimation from technical perspective. What you might need in that case is a project management estimation where the development estimate would be mentioned as time in some range. And after all, you never take things off the estimation. If you remove an estimate, do tell the one that created the estimation. Your need for business value hunting time over quality has nothing to do with the developer. If you want to play it that way, figure out how to do it by yourself. Most often the thing that is taken out of the estimation is the QA time the developer or the QA engineer has estimated. The developer should not allow this. QA is important process and not only helps the delivery of the project it validates the abilities of the developer(s) to deliver bug-free solution, something all developers should aim to.Clients from hell How to handle clients that request from you to build proper software solution but do not give the proper requirements list or specification? Well it’s quite simple and there are two ways to do that. The first would be if you work for a company where the client has been negotiating with your bosses about the nature of the project and the requirements that need to be fulfilled. In that case, you as developer have the buffer between you and the client. And trust me about something when I say that in that case the client should not be aware of who is the lead developer of the project, has his Skype, Email or GTalk contacts. Why? Because the first time some angry client sends feedback about something that you’ve messed up (regardless whose fault it is) would be you. And your bosses in CC. So the first step in keeping your client satisfied in corporate-driven environment is making clear about everything you need to do and getting everything you need before he changes his mind. And document every single communication, every meeting (yes, create MoM and forward it to your boss after you finish) that you have with the client. That way you’ll skip the nonsense you’ll have to go through when the client points his finger at someone. Another advice on avoiding a bad client when you don’t work in a company and do your staff at home as freelancer or company of your own is not choosing a bad client in the first place. Bad clients would occupy your time, would make you lose money (since you lose time) and demotivate you to finish the project properly. And motivation is 90% of the work. The first symptoms for a bad client that your bosses can’t handle is change with the wind directives (just as Mark Berry would say) and not providing specification. One of the clients that comes first in mind when thinking about such situations has delivered a specification to me (on my email to be precise) which was actually a Powerpoint file (.ppt) that had screenshots of another application, photoshopped with red-text explanations. That was the first specification I received in the company where I worked at that time and the first specification of that kind. However, being inexperienced and trying to prove myself in the new environment I got into, I accepted the document as SRS. The initial estimation was month and a half. It took 11 months to finish the project. Because of two things. The development finished right on time, but then begun the fixing (QA) phase where the client would report 10 issues today and 120 tomorrow. At some point his fixing process turned into feature/change-requests where we had to do additional development and then by changing his mind, get everything back to its place just like it used to be before. 11 months for something that I, personally would have made thousands of dollars if I have released it in the first 3 months. I might have received negative feedback from the users, but I like to stick to release early, release often philosophy, delivering period updates with fixes to problems reported by real users and not by the imaginary friends in my client’s head. After all, the whole company, the client and me as developer lost so much time, motivation, money and temper to work further on new projects. Always request specification, features-list or whatever that defines the project.Unhealthy growth Companies grow and get filled with dirt every single day. The expensive entrepreneur to whom you gave this month’s salary would say that is natural process of every company but what I think it is, it’s unhealthy development of the company. Let’s put ourselves in the general idea of a small company. We have 5 developers and 3 project managers, we have 2 bosses (let’s say CTO and CEO). Project managers should do one simple task called measurement. They get the specifications, estimations, risks and measure it all together. They might control the behaviour of the employers that they lead but at no point (if not precisely said) project managers are developer’s bosses. And most of us have the opposite experience, project managers behaving as bosses. And this is not the real problem regarding the growth of the company. The problem comes when the mentioned CTO and CEO decide that it’d be quite easier if they employ 2 more project managers, making them equal with the number of developers. And then the games begin. These 2 managers will be the lead managers, the others are just project coordinators, and yet every developer would have single project manager. Nonsense! Let’s face it. Developers are tough category to handle. You cannot pressure them with your stupid egos and complexes from home. You do what you need to do and don’t bother the others with less important issues. Project managers should stop being proxies between the developers and the top management. Forwarding an email to few more people in the company is something everyone can do. Developers don’t need secretaries to resend their emails, they need managers to handle the changes, risks and constraints in the development process. And yes, development process is quite a magic and shit happens. Putting more project managers in one company makes the things worse. What the project managers commonly do in the software-development world is something 70% of the developers can do themselves. You cannot say the opposite. Don’t forget to share! Reference: Bad developer – good boss (I), Bad developer – good boss (II) from our JCG partner Aleksandar Balalovski at the 2Dwarfs blog....

A Lazy Developers Introduction to Java Concurrency Executors

I would make a fool out of myself if I tell you that util.concurrent APIs kicks cheetah’s ass when the classes are available since 2004. However, there are some cool features which I would like to revisit. Concurrency experts, now is the time for you to close this window. All others, stay tight for the fun ride. Thou shall not forget your rootsExecutor is the root interface with a single execute method. Anything that implements a Runnable interface can passed as a parameter. Silly Executor, however, has no support for Callable though.Good news : ExecutorService interface, which extends Executor, adds support for Callable. Its implementation class is ThreadPoolExecutor. I am going to pretend that the ScheduledExecutorService interface and its implementation class ScheduledThreadPoolExecutor does not exist as they just add scheduling capabilities on top of the ExecutorService and ThreadPoolExecutor. But remember this class when the powerful but boring java.util.Timer is not enough and a full blown external scheduler is just too much. If you are new to concurrency or forgot the difference between Callable and Runnable, you might want to read up a little before reading further. A dummy guide is here The ExecutorService.submit Facts : The three submit variants : Future submit(Callable task) Future submit(Runnable task) Future submit(Runnable task, T result)The submit method of the ExecutorService is overloaded and accepts either a Callable or Runnable. Since the run method of the Runnable returns void, it is no surprise that Future.get always returns a null when the task is complete. Future<?> submit(Runnable task)The other overloaded submit method that accepts a Runnable and a generic returns whatever you passed in as the second parameter as the result. <T> Future<T> submit(Runnable task, T result)In fact, opening up the code (FutureTask), you’ll notice that the RunnableAdapter top level nested class of Executors simply holds the result and returns the same result after the run method is complete. static final class RunnableAdapter<T> implements Callable<T> { final Runnable task; final T result;RunnableAdapter(Runnable task, T result) { this.task = task; this.result = result; }public T [More ...] call() { task.run(); return result; } }RunnableAdapter source In both the cases, if you would like to (you should !) terminate the program instead of your executor thread blocking the program and entering a busy loop, you should call the shutdown method as in executorService.shutdown()shutDown facts You could imagine shutdown as a half-closed door of a mall. No new customers will be let in but the existing customers can leave the mall once they are done. To reiterate,shutdown is a polite method. It does not actually shut down the tasks in the pool immediately. It just says that no new tasks will be accepted. Unless you are executing your tasks using invokeAll, you would need to wait for all tasks in progress to complete. This is achieved by calling the awaitTermination method. (invokeAll and submit examples at the bottom of the post) Once all the current tasks are done, the executor service shuts down.If you are in need of an impolite, intruding method which doesn’t care whether the current threads are done with their tasks, then shutdownNow is your guy. However, there’s no guarantee that the method will shutdown the service on the dot but it is the closest you have to immediate shutdown. On awaitTermination, you could specify the timeout period until which the main thread should wait for the pool threads to complete its tasks. ExecutorService executorService=Executors.newFixedThreadPool(10); … future = executorService.submit(getInstanceOfCallable(count,sum)); … executorService.shutdown();if (executorService.awaitTermination(10, TimeUnit.SECONDS)){ System.out.println('All threads done with their jobs'); }Executors – the factory guyThe classes above are all awesome. But, say, you wanted to create a single thread executor, you would write something like new ThreadPoolExecutor(1, 1,0L, TimeUnit.MILLISECONDS,new LinkedBlockingQueue<Runnable>());Compare that to Executors.newSingleThreadExecutor()So, here you go. Executors is a class with just factory methods for creating various forms of executor service with some commonly used defaults. Note that, other than the awesome factory methods, it doesn’t bring any new features to the table. It is recommended that you have a quick look at the implementation of the factory methods and check if it suits your needs. The invokeAll and the submit The All part of invokeAll method of the ExecutorService gives no surprise. It just says that you need to pass in a Collection of Callables. Again, as expected, the method does not return until all the Threads are done with their tasks. So, for cases when you are interested in the result only after all the jobs are complete, invokeAll is your guy. On the other hand, the submit method returns immediately after the callable is submitted to the executor service. Unless you are doing nothing at all in your call method of your Callable, your worker threads should ideally be running when the submit method returns. The following samples might be useful for your reference. The programs just tries to find the sum of all the natural numbers till 100 (Brute force, of course) package me.rerun.incubator;import java.util.ArrayList; import java.util.List; import java.util.concurrent.Callable; import java.util.concurrent.ExecutionException; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import java.util.concurrent.Future; import java.util.concurrent.atomic.AtomicInteger;public class ExecutorInvokeAll {public void runApp() throws InterruptedException, ExecutionException{//variable to store the sum AtomicInteger sum=new AtomicInteger();//Use our friendly neighbourhood factory method of the Executors. ExecutorService executorService=Executors.newFixedThreadPool(10);List<Callable<AtomicInteger>> callableList=new ArrayList<Callable<AtomicInteger>>();for (int count = 0; count <= 100;count++) { callableList.add(getInstanceOfCallable(count,sum));}//returns only after all tasks are complete List<Future<AtomicInteger>> resultFuture = executorService.invokeAll(callableList);//Prints 5050 all through for (Future<AtomicInteger> future : resultFuture) { //Didn't deliberately put a timeout here for the get method. Remember, the invoke All does not return until the task is done. System.out.println("Status of future : " + future.isDone() +". Result of future : "+future.get().get()); }executorService.shutdown();// You might as well call a resultFuture.get(0).get().get() and that would give you the same //result since all your worker threads hold reference to the same atomicinteger sum. System.out.println("Final Sum : "+sum);} //Adds count to the sum and returns the reference of the sum as the result private Callable<AtomicInteger> getInstanceOfCallable(final int count, final AtomicInteger sum) {Callable<AtomicInteger> clientPlanCall=new Callable<AtomicInteger>(){ public AtomicInteger call() { sum.addAndGet(count); System.out.println("Intermediate sum :"+sum); return sum; } };return clientPlanCall; }public static void main(String[] args) throws ExecutionException {try { new ExecutorInvokeAll().runApp(); } catch (InterruptedException e) { e.printStackTrace(); } }}package me.rerun.incubator;import java.util.concurrent.Callable; import java.util.concurrent.ExecutionException; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import java.util.concurrent.Future; import java.util.concurrent.TimeUnit; import java.util.concurrent.TimeoutException; import java.util.concurrent.atomic.AtomicInteger;public class ExecutorSubmit {public void runApp() throws InterruptedException, ExecutionException{//holder for the total sum AtomicInteger sum=new AtomicInteger();//Use the factory method of Executors ExecutorService executorService=Executors.newFixedThreadPool(10);Future<AtomicInteger> future = null;for (int count = 0; count <= 100; count++) { future = executorService.submit(getInstanceOfCallable(count,sum)); //prints intermediate sum try { System.out.println("Status of future : " + future.isDone() +". Result of future : "+future.get(1000, TimeUnit.MILLISECONDS).get()); } catch (TimeoutException e) { System.out.println("<IGNORE> Timeout exception for count : "+count); //e.printStackTrace(); } //System.out.println("Result of future : "+future.get().get() +".Status of future : " + future.isDone()); }executorService.shutdown();if (executorService.awaitTermination(10, TimeUnit.SECONDS)){ System.out.println("All threads done with their jobs"); } //exec System.out.println("Final Sum : "+sum);}//Adds count to the sum and returns the reference of the sum as the result private Callable<AtomicInteger> getInstanceOfCallable(final int count, final AtomicInteger sum) {Callable<AtomicInteger> clientPlanCall=new Callable<AtomicInteger>(){ public AtomicInteger call() { sum.addAndGet(count); //System.out.println("Intermediate sum :"+sum); return sum; } };return clientPlanCall; }public static void main(String[] args) throws ExecutionException {try { new ExecutorSubmit().runApp(); } catch (InterruptedException e) { e.printStackTrace(); } }}Further reading : Alex Miller’s amazing blog Alex Miller’s concurrency gotchas Vogella’s article on comparison with original API Good introduction to concurrency in general Highly recommended book on Java concurrency Happy coding and don’t forget to share! Reference: A Lazy Developers Introduction to Java Concurrency Executors from our JCG partner Arun Manivannan at the Rerun.me blog....

Bad Things Happen to Good Code

We need to understand what happens to code over time and why, and what a healthy, long-lived code base looks like. What architectural decisions have the most lasting impact, and what decisions made early will make the most difference over the life of a system. Forces of Compromise Most of the discussion around technical debt assumes that code degrades over time because of sloppiness and lazy coding practices and poor management decisions, by programmers who don’t know or don’t care about what they are doing or who are forced to take short-cuts under pressure. But it’s not that simple. Code is subject to all kinds of pressures and design compromises, big and small, in the real world. Performance optimization trade-offs can force you to bend the design and code in ways that were never expected. Dealing with operational dependencies and platform quirks and run-time edge cases also adds complexity. Then there are regulatory requirements – things that don’t fit the design and don’t necessarily make sense but you have to do anyways. And customization: customer-specific interfaces and options and custom workflow variants and custom rules and custom reporting, all to make someone important happy. Integration with other systems and API lock-in and especially maintaining backwards compatibility with previous versions can all make for ugly code. Michael Feathers, who I think is doing the most interesting and valuable work today in understanding what happens to code and what should happen to code over time, has found that code around APIs and other hard boundaries becomes especially messy – because some interfaces are so hard to change, this forces programmers to do extra work (and workarounds) behind the scenes. All of these forces contribute to making a system more complex, harder to understand, harder to change and harder to test over time – and harder to love.Iterative Development is Erosive In Technical Debt, Process and Culture, Feathers explains that “generalized entropy in software systems” is inevitable, the result of constant and normal wear and tear in an organization. As more people work on the same code, the design will naturally deteriorate as each person interprets the design in their own way and makes their own decisions on how to do something. What’s interesting is that the people working with this code can’t see how much of the design has been lost because their familiarity with the code makes it appear to be simpler and clearer than it really is. It’s only when somebody new joins the team that it becomes apparent how bad things have become. Feathers also suggests that highly iterative development accelerates entropy, and that code which is written iteratively is qualitatively different than code in systems where the team spent more time in upfront design. Iterative development and maintenance tend to bias towards the existing structure of the system, meaning that more compromises will end up being made. Iterative design and development involves making a lot of small mistakes, detours and false starts as you work towards the right solution. Testing out new ideas in production through A/B split testing amplifies this effect, creating more options and complexity. As you work this way some of the mistakes and decisions that you make won’t get unmade – you either don’t notice them, or it’s not worth the cost. So you end up with dead abstractions and dead ends, design ideas that aren’t meaningful any more or are harder to work with than they should be. Some of this will be caught and corrected later in the course of refactoring, but the rest of it becomes too pervasive and expensive to justify ripping out.Dealing with Software Sprawl Software, at least software that gets used, gets bigger and more complicated over time – it has to, as you add more features and interfaces and deal with more exceptions and alternatives and respond to changing laws and regulations. Capers Jones analysis shows that the size of the code base for a system under maintenance will increase between 5-10% per year. Our own experience bears this out – the code base for our systems has doubled in size in the last 5 years. As the code gets bigger it also gets more complex – code complexity tends to increase an average of between 1% and 3% per year. Some of this is real, essential complexity – not something that you can wish your way out of. But the rest is due to how changes and fixes are done. Feathers has confirmed by mining code check-in history (Discovering Startling Things from your Version Control System) that most systems have a common shape or “power curve”. Most code is changed only infrequently or not at all, but the small percentage of methods and classes in the system that are changed a lot tend to get bigger and more complex over time. This is because it is easier to add code to an existing method than to add a new method and easier to add another method to an existing class than to add a new class. The key to keeping a code base healthy is disciplined refactoring of this code, taking the time to come up with new and better abstractions, and preventing the code from festering and becoming malignant. There is also one decision upfront that has a critical impact on the future health of a code base. Capers Jones has found that the most important factor in how well a system ages is, not surprisingly, how complex the design was in the beginning: The rate of entropy increase, or the increase in cyclomatic complexity, seems to be proportional to the starting value. Applications that are high in complexity when released will experience much faster rates or entropy or structural decay than applications put into production with low complexity levels. The Economics of Software Quality Systems that were poorly designed only get worse – but Jones has found that systems that were well-designed can actually get better over time. Reference: Bad Things Happen to Good Code from our JCG partner Jim Bird at the Building Real Software blog....

Android: Multi-touch gestures controller

Few of our projects (ours or for clients) required to implement a multi-touch features for manipulation over images with the standard gestures (drag&drop, rotate, zoom). While an implementation of a multi-touch controller is challenging and fun thing to do, it consumes a lot of time and if there is something ready and simply works, why reinvent the wheel? So we decided to make a little research to see which of the open-source projects out there suits best for our needs. Without mentioning the controllers and projects that we didn’t like and didn’t find compelling, I will mention that that our experience with few demo projects we built back then showed that the best multi-touch controller with easy to implement logics and easy to understand syntax was Luke Hutchison‘s Android Multitouch Controller. The sources are available under the terms of the MIT License. What is a multitouch controller? The term is not general. What we refer to as ‘multitouch controller’ is a piece of code that makes the process of implementing manipulative operations over entities which are shown on the screen (views, bitmaps, etc.) convinient. That means that the multi-touch controller wraps the logics behind the user’s input through the touch-screen. So when the user puts down one finger and starts moving it over the touch-screen the controller should ‘point’ to part of the code where the developer is supposed to put his logic on what he wants to happen then. The controller ‘knows’ that it’s one-finger touch, pinch-gesture or rotate-gesture, or all of them at once (wherever possible). To be more precise, the multi-touch controller would not make your graphics move over the screen magically by linking a library or code to your app. Think of it as interface that you need to implement so that you make your objects move, but you’ll need to figure out how to make it. The already mentioned multitouch controller of our choice is, well Android Multitouch Controller. What you need to do first as preparation is create empty Android project and download the MultiTouchController.java file. 90% of what we need for this tutorial is here.Necessities So far the examples for the Android Multitouch Controller are given by using batch of resources (Drawables) which can later be modified over a Canvas. Our idea is to simplify this process and make only one movable object over a canvas. So we’ll go with simple Bitmap that would be drawn over the canvas which can later on be moved/scaled/rotated from its initial position. So we need two things at first, a View that implements the MultiTouchObjectCanvas interface and then a custom object/entity/widget which would contain the logics for its drawing. This object is the ‘manipulative’ one and it contains the Bitmap that it represents it. So when we say the Canvas we think of a custom View that you need to create, that would implement the MultiTouchObjectCanvas interface (where T is the Pinch object that is to be modified). This interface tells us whether we have an object (draggable object) or widget at the point where the user has touched the screen. If so, the implementation will return the relevant object, otherwise null. So to use this interface we need to implement couple of methods without which we cannot achive results. Anyway, the IDE will tell you what you must implement, these are the methods (among the constructors from the parent View class etc.): @Override public T getDraggableObjectAtPoint(PointInfo touchPoint) { return null; } This method as you can see, returns the object of interest from the place where the user has made the touch input. Meaning, if the user touched x:120;y:100 point, this method should check if this point is in the area occupied by the widget. When you see the full implementation you’ll know how this is done. @Override public void getPositionAndScale(T obj, PositionAndScale objPosAndScaleOut) { } This method operates over the PositionAndScale object for the widget that is touched. The widget is passed as first argument, the PositionAndScale as second and that is pretty much self-descriptive: when the ‘obj’ object is touched, apply the ‘objPosAndScaleOut’ attributes for the position, scale and angle. But this is only for the initial position of the screen, what actually makes the motion is the next obligatory method. @Override public boolean setPositionAndScale(T obj, PositionAndScale newObjPosAndScale, PointInfo touchPoint) { return false; } Again, we have the widget as first argument, the new object’s position/scale/angle properties and a helper object from PointInfo which tells is whether it’s multi-touch or single (drag) gesture. @Override public void selectObject(T obj, PointInfo touchPoint) { } This method takes care of informing the controller which object is being selected. The ‘obj’ object has the selected object. So when we implement this methods and we have set the multitouch controller object, we can implement the widget logics which holds the data behind the item(s) that is/are drawn over the canvas. Yes, in that case we need a MultiTouchController object which needs to be defined like this: private MultiTouchController mMultiTouchController = new MultiTouchController (this); And what to do with this? Well, when the touch event happens, we need to pass ‘everything’ to this controller. And that is done by overriding the onTouchEvent for the custom View/Canvas: @Override public boolean onTouchEvent(MotionEvent ev) { return mMultiTouchController.onTouchEvent(ev); } And with this and few other staff, we have the essential Canvas that takes care of the objects it draws. So we need one more thing, the logics for the Pinch widget with its Bitmap, coordinates, etc. So we create a PinchWidget object which is modification from some of the examples that go with the Multitouch Controller. The essence of this object are these two methods: public boolean setPos(PositionAndScale newImgPosAndScale, int uiMode, int uiModeAnisotropic, boolean isMultitouch) { boolean ret = false; float x = newImgPosAndScale.getXOff(); float y = newImgPosAndScale.getYOff(); if(isMultitouch) { x = mCenterX; y = mCenterY; }ret = setPos(x, y, (uiMode & uiModeAnisotropic) != 0 ? newImgPosAndScale.getScaleX() : newImgPosAndScale.getScale(), (uiMode & uiModeAnisotropic) != 0 ? newImgPosAndScale.getScaleY() : newImgPosAndScale.getScale(), newImgPosAndScale.getAngle());return ret; } and private boolean setPos(float centerX, float centerY, float scaleX, float scaleY, float angle) { float ws = (mImage.getWidth() / 2) * scaleX, hs = (mImage.getHeight() / 2) * scaleY; float newMinX = centerX - ws, newMinY = centerY - hs, newMaxX = centerX + ws, newMaxY = centerY + hs; mCenterX = centerX; mCenterY = centerY; mScaleFactor = scaleX; mAngle = angle;mMinX = newMinX; mMinY = newMinY; mMaxX = newMaxX; mMaxY = newMaxY;return true; } These two methods make the movement possible since they give information on the coordinates, scale factor and the angle of the PinchWidget. They are in a way coupled, meaning the first method makes calculations regarding the data it gets from the second one. So now we have the coordinates of the object, its scale and angle. We just need to draw it by using its draw(Canvas) method: public void draw(Canvas canvas) { Paint itemPaint = new Paint(); itemPaint.setAntiAlias(true); itemPaint.setFilterBitmap(true);float dx = (mMaxX + mMinX) / 2; float dy = (mMaxY + mMinY) / 2;canvas.save();canvas.translate(dx, dy); canvas.rotate(mAngle * 180.0f / (float) Math.PI); canvas.translate(-dx, -dy);Rect srcRect = new Rect(0, 0, mImage.getWidth(), mImage.getHeight()); Rect dstRect = new Rect((int) mMinX, (int) mMinY, (int) mMaxX, (int) mMaxY);canvas.drawBitmap(mImage, srcRect, dstRect, null);canvas.restore(); } mImage is the Bitmap of the item/widget we draw. It must be drawn to see something. And to draw it we need its source and destination rects (in this kind of implementation). The source is the size of the image (in our case Rect[0,0,300,300]) and the destination (where to be drawn) which is calculated from the init and setPos methods of PinchWidget. Then the image is drawn the same way any other Bitmap is drawn, by using drawBitmap(…) method. Now a bit back to the MultiTouchView implementation. As mentioned before, having in mind that we have declared this View in XML, we’ll make use of the: public MultiTouchView(Context context, AttributeSet attrs) constructor. There we initialize the Context. Also we have this method: public void setPinchWidget(Bitmap bitmap) { mPinchWidget = new PinchWidget(bitmap); mPinchWidget.init(mContext.getResources()); } This method tells the MultiTouchView what is our PinchWidget. It is where it’s created, and by calling init() (Resources here is passed just to calculate display’s width and height), we call the whole machanism that will draw the widget. And from this View that happens in its onDraw() method: @Override public void onDraw(Canvas canvas) { super.onDraw(canvas); canvas.drawColor(Color.WHITE); mPinchWidget.draw(canvas); } Pretty simple ain’t it? So if everything goes as explained and you get the idea behind this kind of implementation, you’ll see something like this on the screen:Conclusion MultiTouch operations in Android are basically the same math you’ll need to do on other platforms. But when you need time this kind of projects, the Android Multitouch Contoroller is time-saving tool, and when you download it do read the documented methods, do see the elegant and nice code and don’t forget to thank the developer for what he had done. Maybe worth mentioning that our research took almost 1 year (9 months to be precise) about which Multitouch Controller we need to use in our present and future apps. The sources of the sample app are available on our GitHub repository. Next we’ll try to cover what needs to be done to show many Bitmaps over the Canvas view. Or if you figure it out yourself, don’t hesitate to write a tutorial and we’ll be happy to publish it here. Happy coding and don’t forget to share! Reference: Android: Multi-touch gestures controller from our JCG partner Aleksandar Balalovski at the 2dwarfs blog....

Mocking with JodaTime’s DateTime and Google Guava’s Supplier

Introduction If you’re a seasoned unit tester, you’ve learned to take note when you see any code working with time, concurrency, random, persistence and disc I/O. The reason for this is that tests can be very brittle and sometimes down-right impossible to test properly. This post will show how to abstract out ‘time’ by injecting a replacement for it in the consumer. This post will be using Spring 3 as the Dependency Injection container, though Guice, other DI containers or constructor/setters on POJOs would work as well. I will also ignore Locales since the focus is on the injection of the DateTime, not DateTime itself. Existing code You’ve been handed a piece of code to unit test (or you are creating one and this is your first stab at it). Our first piece of code, only one class: (This class is a Spring 3.1 controller and the purpose is to return back the current time as a String) @Controller @RequestMapping(value = '/time') @VisibleForTesting class TimeController {@RequestMapping(value = '/current', method = RequestMethod.GET) @ResponseBody public String showCurrentTime() { // BAD!!! Can't test DateTime dateTime = new DateTime(); return DateTimeFormat.forPattern('hh:mm').print(dateTime); } } Take note that the class does a ‘new DateTime()’ in the class. Here is the corresponding test class: What happens when we run the test? How about assuming we have a very slow machine. You could (and most likely will) end up with your comparison DateTime to be different than the returned DateTime. This is a problem! First thing to do is to remove the dependency, but how are we going to do this? If we make the DateTime a field on the class, we will still have the same problem. Introduce Google Guava’s Supplier interface. Google Guava Supplier The Supplier interface only has one method, ‘get()’ which will return an instance of whatever the supplier is setup for. An example, the supplier will return a user’s first name if they have logged in, and a default one if they have not: public class FirstNameSupplier implements Supplier<String> {private String value; private static final String DEFAULT_NAME = 'GUEST';public FirstNameSupplier() { // Just believe that this goes and gets a User from somewhere String firstName = UserUtilities.getUser().getFirstName(); // more Guava if(isNullOrEmpty(firstName)) { value = DEFAULT_NAME; } else { value = firstName; } }@Override public String get() { return value; } } To your implementing method, you don’t care what the first name is, only that you get one. Refactoring out DateTime Let’s move on. For a much more real example of using a Supplier (and the point of this post) let’s implement a DateTime supplier to give us back the current DateTime. While we’re at it, let’s also create an interface so that we can create mock implementations for testing: public interface DateTimeSupplier extends Supplier<DateTime> { DateTime get(); } and an implementation: public class DateTimeUTCSupplier implements DateTimeSupplier { @Override public DateTime get() { return new DateTime(DateTimeZone.UTC); } } Now we can take the DateTimeUTCSupplier and inject that into our code that needs the current DateTime as the DateTimeSupplier interface: @Controller @RequestMapping(value = '/time') @VisibleForTesting class TimeController {@Autowired @VisibleForTesting // Injected DateTimeSupplier DateTimeSupplier dateTime;@RequestMapping(value = '/current', method = RequestMethod.GET) @ResponseBody public String showCurrentTime() { return DateTimeFormat.forPattern('hh:mm').print(dateTime.get()); } } In order to test this, we’ll need to create a MockDateTimeSupplier and have a controller to pass in the specific instance we want to return: public class MockDateTimeSupplier implements DateTimeSupplier {private final DateTime mockedDateTime;public MockDateTimeSupplier(DateTime mockedDateTime) { this.mockedDateTime = mockedDateTime; }@Override public DateTime get() { return mockedDateTime; } } Notice that the object being saved is passed in via the constructor. This will not get you the current date/time back, but will return back the specific instance you want and finally our test that exercises (slightly) the TimeController we implemented above: public class TimeControllerTest {private final int HOUR_OF_DAY = 12; private final int MINUTE_OF_DAY = 30;@Test public void testShowCurrentTime() throws Exception { TimeController controller = new TimeController(); // Create the mock DateTimeSupplier with our known DateTime controller.dateTime = new MockDateTimeSupplier(new DateTime(2012, 1, 1, HOUR_OF_DAY, MINUTE_OF_DAY, 0, 0));// Call our method String dateTimeString = controller.showCurrentTime();// Using hamcrest for easier to read assertions and condition matchers assertThat(dateTimeString, is(String.format('%d:%d', HOUR_OF_DAY, MINUTE_OF_DAY))); }}Conclusion This post has shown how to use Google Guava’s Supplier interface to abstract out a DateTime object so you can better design your implementations with unit testing in mind! Suppliers are a great way to solve the ‘just give me something’, mind you it’s a known type of something. Happy coding and don’t forget to share! Reference: Mocking with JodaTime’s DateTime and Google Guava’s Supplier from our JCG partner Mike at the Mike’s site blog....

Apache Hive on Windows in 6 easy steps

Note: You need to have cygwin installed to run this tutorial, as Hadoop (needed by Hive) needs cygwin to run on windows. At a minimum, Basic, Net (OpenSSH,tcp_wrapper packages) and Security related Cygwin packages need to be present in the system. Here are the 6 steps: 1. Download WSO2 BAM 2.0.0. It’s free and open source. 2. Extract it to a preferred location. Let’s call it $BAM_HOME. 3.Start the server by executing the wso2server.bat file present in $BAM_HOME/bin. The server would startup on the default port 9443 on the machine’s IP. 4. Log in to the web console at https://localhost:9443 using the default credentials, i.e. username: admin, password: admin and clicking “Sign-In”.WSO2 BAM login screen 5. Navigate to the “Add Analytics” option by clicking the menu item on the left hand menu.WSO2 BAM left hand menu – add Analytics option 6. Now execute your Hive script, by entering the script and clicking execute! Note: Follow this KPI sample document to see a sample working for you in no time, with results appearing on a dashboard. Also, notice that you can schedule the Hive script as well.Execute Apache Hive script I have to thank my colleague Buddhika Chamith, as all this was possible because of some grueling work done by him. Also, I hate the fact Hadoop and Hive makes it so hard to run stuff on Windows, especially since this is a Java application. Read about those concerns here. Don’t forget to share! Reference: HOWTO: Run Apache Hive on Windows in 6 easy steps from our JCG partner Mackie Mathew at the dev_religion blog....

Android: Finding the SD Card Path

Finding the SD Card path in Android is easy right? All you have to do is use Environment.getExternalStoreDirectory(), and you’re good to go! Well, not quite. After all, that’s what StackOverflow says. If you actually try the above method on a Samsung device, life won’t be fun for you. Environment.getExternalStoreDirectory() actually returns the incorrect path on most Samsung devices. That’s how I came across the issue. It turns out, the above method doesn’t actually guarantee that it will return the SD Card directory. According to Android’s API documentation, “In devices with multiple ‘external’ storage directories (such as both secure app storage and mountable shared storage), this directory represents the ‘primary’ external storage that the user will interact with.” So the call doesn’t guarantee that the path returned truly points SD Card. There are a few other ways to get an “external” path on the device where files can be stored, though, like the getExternalFilesDir(). There are also a few other tricks to actually get the path of the SD Card directory. The below code works on most Android devices (Samsung included). It’s a pretty hacky solution, though, and who knows how long this trick will actually work (source). Instead of using the code below, it may be better to ask the question, “do I really need the SD Card directory, or just a path that I can store files to?” File file = new File("/system/etc/vold.fstab"); FileReader fr = null; BufferedReader br = null; try { fr = new FileReader(file); } catch (FileNotFoundException e) { e.printStackTrace(); } try { if (fr != null) { br = new BufferedReader(fr); String s = br.readLine(); while (s != null) { if (s.startsWith("dev_mount")) { String[] tokens = s.split("\\s"); path = tokens[2]; //mount_point if (!Environment.getExternalStorageDirectory().getAbsolutePath().equals(path)) { break; } } s = br.readLine(); } } } catch (IOException e) { e.printStackTrace(); } finally { try { if (fr != null) { fr.close(); } if (br != null) { br.close(); } } catch (IOException e) { e.printStackTrace(); } }Happy coding and don’t forget to share! Reference: Android Tutorial: Finding the SD Card Path from our JCG partner Isaac Taylor at the Programming Mobile blog....

JavaOne 2012: Diagnosing Your Application on the JVM

It was worth attending Staffan Larsen‘s (Oracle Java Serviceability Architect) presentation ‘Diagnosing Your Application on the JVM‘ (Hilton Plaza A/B) just for learning of the new jcmd command-line tool provided with Oracle’s JVM 7. The rest of the presentation was ‘bonus’ for me, which was nice for the last session I attended on Wednesday of JavaOne 2012.The Oracle HotSpot JDK provides jcmd, a command-line tool designed to be both backwards compatible and forward adaptable for future versions of Java. It is designed to support new tools and features that come with new SDKs in a standardized approach. The following screen snapshot shows it used for most basic jps-like functionality (Larsen mentioned jps almost as briefly as I just did and referred to jcmd as ‘like jps but more powerful’).As the above image shows, jcmd can be used like jps. Larsen showed some handy features of the jcmd command. He had some small sample Java applications that helped him to demonstrate jcmd. For my purposes, I’m running jconsole in one terminal on my machine and then I’ll run jcmd commands against that JVM in which jconsole is running. The next screen snapshot shows how the basic (no arguments) jcmd call provides information on that JConsole process.jcmd supports execution against JVM processes either by process ID (pid) or by process name. The next screen snapshot shows running jcmd against the JConsole process by that name and passing it help to see which options can be run against that particular process. Note that I tried unsuccessfully to run this against ‘dustin’ (no existing process) to prove that jcmd is really showing options available for running processes.The feature demonstrated in the last screen snapshot is one of the most compelling reasons for moving from existing command-line tools provided with the Oracle JDK to jcmd. This image shows how jcmd can provide a list of the available options on a per process basis, allowing for ultimate flexibility in terms of supporting past versions or future versions of Java that support different/new commands. Just as jcmd <pid> help (or replace pid with process name) lists the available operations that can be run by jcmd against a particular JVM process, this same help mechanism can be run against any one of those specific listed commands [with syntax jcmd <pid> <command_name> help (or use process name instead of pid)], though I could not get this to work properly on my Windows machine. The next image shows actually running that command against that JVM process rather than simply asking for help on it.In the two screen snapshots immediately above, I ran jcmd against the pid instead of the process name simply to show that it works against both process ID as well as name. The next screen snapshot shows executing jcmd against the JVM process to get VM flags and command-line options from the JVM process (the pid of this instance of JConsole process is 3556).                                Running jcmd‘s Thread.print command against a supporting JVM process makes easy work of viewing the targeted JVM’s threads. The following output is generated from running jcmd JConsole Thread.print against my running JConsole process. 3556: 2012-10-04 23:39:36 Full thread dump Java HotSpot(TM) Client VM (23.2-b09 mixed mode, sharing):'TimerQueue' daemon prio=6 tid=0x024bf000 nid=0x1194 waiting on condition [0x069af000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x23cf2db0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.DelayQueue.take(DelayQueue.java:209) at javax.swing.TimerQueue.run(TimerQueue.java:171) at java.lang.Thread.run(Thread.java:722)'DestroyJavaVM' prio=6 tid=0x024be400 nid=0x1460 waiting on condition [0x00000000] java.lang.Thread.State: RUNNABLE'AWT-EventQueue-0' prio=6 tid=0x024bdc00 nid=0x169c waiting on condition [0x0525f000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x291a90b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.awt.EventQueue.getNextEvent(EventQueue.java:521) at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:213) at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:163) at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:151) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:147) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:139) at java.awt.EventDispatchThread.run(EventDispatchThread.java:97)'Thread-2' prio=6 tid=0x024bd800 nid=0x4a8 in Object.wait() [0x04bef000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x2917ed80> (a java.io.PipedInputStream) at java.io.PipedInputStream.read(PipedInputStream.java:327) - locked <0x2917ed80> (a java.io.PipedInputStream) at java.io.PipedInputStream.read(PipedInputStream.java:378) - locked <0x2917ed80> (a java.io.PipedInputStream) at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177) - locked <0x29184e28> (a java.io.InputStreamReader) at java.io.InputStreamReader.read(InputStreamReader.java:184) at java.io.BufferedReader.fill(BufferedReader.java:154) at java.io.BufferedReader.readLine(BufferedReader.java:317) - locked <0x29184e28> (a java.io.InputStreamReader) at java.io.BufferedReader.readLine(BufferedReader.java:382) at sun.tools.jconsole.OutputViewer$PipeListener.run(OutputViewer.java:109)'Thread-1' prio=6 tid=0x024bd000 nid=0x17dc in Object.wait() [0x047af000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x29184ee8> (a java.io.PipedInputStream) at java.io.PipedInputStream.read(PipedInputStream.java:327) - locked <0x29184ee8> (a java.io.PipedInputStream) at java.io.PipedInputStream.read(PipedInputStream.java:378) - locked <0x29184ee8> (a java.io.PipedInputStream) at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177) - locked <0x2918af80> (a java.io.InputStreamReader) at java.io.InputStreamReader.read(InputStreamReader.java:184) at java.io.BufferedReader.fill(BufferedReader.java:154) at java.io.BufferedReader.readLine(BufferedReader.java:317) - locked <0x2918af80> (a java.io.InputStreamReader) at java.io.BufferedReader.readLine(BufferedReader.java:382) at sun.tools.jconsole.OutputViewer$PipeListener.run(OutputViewer.java:109)'AWT-Windows' daemon prio=6 tid=0x024bc800 nid=0x16e4 runnable [0x0491f000] java.lang.Thread.State: RUNNABLE at sun.awt.windows.WToolkit.eventLoop(Native Method) at sun.awt.windows.WToolkit.run(WToolkit.java:299) at java.lang.Thread.run(Thread.java:722)'AWT-Shutdown' prio=6 tid=0x024bc400 nid=0x157c in Object.wait() [0x04c6f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x2918b098> (a java.lang.Object) at java.lang.Object.wait(Object.java:503) at sun.awt.AWTAutoShutdown.run(AWTAutoShutdown.java:287) - locked <0x2918b098> (a java.lang.Object) at java.lang.Thread.run(Thread.java:722)'Java2D Disposer' daemon prio=10 tid=0x024bbc00 nid=0x3b8 in Object.wait() [0x0482f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x2918b128> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135) - locked <0x2918b128> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151) at sun.java2d.Disposer.run(Disposer.java:145) at java.lang.Thread.run(Thread.java:722)'Service Thread' daemon prio=6 tid=0x024bb800 nid=0x1260 runnable [0x00000000] java.lang.Thread.State: RUNNABLE'C1 CompilerThread0' daemon prio=10 tid=0x024c6400 nid=0x120c waiting on condition [0x00000000] java.lang.Thread.State: RUNNABLE'Attach Listener' daemon prio=10 tid=0x024bb000 nid=0x1278 waiting on condition [0x00000000] java.lang.Thread.State: RUNNABLE'Signal Dispatcher' daemon prio=10 tid=0x024bac00 nid=0xe3c runnable [0x00000000] java.lang.Thread.State: RUNNABLE'Finalizer' daemon prio=8 tid=0x024a9c00 nid=0x15c4 in Object.wait() [0x046df000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x2918b358> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135) - locked <0x2918b358> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:177)'Reference Handler' daemon prio=10 tid=0x024a4c00 nid=0xe40 in Object.wait() [0x0475f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x2917e9c0> (a java.lang.ref.Reference$Lock) at java.lang.Object.wait(Object.java:503) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133) - locked <0x2917e9c0> (a java.lang.ref.Reference$Lock)'VM Thread' prio=10 tid=0x024a3800 nid=0x164c runnable'VM Periodic Task Thread' prio=10 tid=0x024e7c00 nid=0xcf0 waiting on conditionJNI global references: 563 Larsen showed how to use thread information provided by jcmd to resolve a deadlock. Larsen showed getting a class histogram from the running JVM process with jcmd. This is done using the command jcmd <pid> GC.class_histogram. A very small subset of its output is shown next (the pid of this JConsole process is 4080 this time). 4080:num #instances #bytes class name ---------------------------------------------- 1: 1730 3022728 [I 2: 5579 638168 3: 5579 447072 4: 645 340288 5: 4030 337448 [C 6: 645 317472 7: 602 218704 8: 942 167280 [B 9: 826 97720 java.lang.Class 10: 3662 87888 java.lang.String 11: 2486 79552 javax.swing.text.html.parser.ContentModel 12: 3220 77280 java.util.Hashtable$Entry 13: 1180 67168 [S 14: 2503 60072 java.util.HashMap$Entry 15: 181 59368 16: 971 43584 [Ljava.lang.Object; 17: 1053 41160 [[I 18: 206 29040 [Ljava.util.HashMap$Entry; 19: 111 27880 [Ljava.util.Hashtable$Entry; 20: 781 18744 java.util.concurrent.ConcurrentHashMap$HashEntry 21: 1069 17104 java.lang.Integer 22: 213 9816 [Ljava.util.concurrent.ConcurrentHashMap$HashEntry; 23: 202 9696 java.util.HashMap 24: 201 9280 [Ljava.lang.String; 25: 24 8416 [[I Larsen also demonstrated jstat and several of its useful functions. He demonstrated use of jstat -gcnew (new generation behavior), jstat -precompilation (compilation method statistics), and jstat -options (displays options). During the course of his presentation, Larsen needed to convert a decimal number (pid?) to its hexadecimal representation for comparing it to the output of another tool. He used the handy printf '%x\n' <pid> command to get the hexadecimal representation of the pid. Larsen demonstrated use of VisualVM to compare two heap dumps and browse a heap dump. He also demonstrated the VisualVM Profiler. Larsen moved from the tools previously covered aimed at running JVMs to tools that can be used to analyze JVM core files. He returned to jstack to analyze contents of the core file. Larsen talked about remotely accessing JVM information via JMX and tools like jconsole and jvisualvm. He demonstrated that jcmd can be used to start JMX exposure as well: ManagementServer.start ‘with a bunch of parameters.’ Larsen feels that VisualVM and JConsole would use ManagementServer.start rather than Attach API if implemented today. jstat can also connect to daemon remotely through use of jstatd. There is no encryption or authentication with jstatd. jps and jcmd find what’s running on system using ‘well-known file for each JVM': /hsperfdata_<user>/<pod> This file is created on JVM startup and deleted on JVM shutdown. Unused previous files are deleted on startup, so jps and jcmd, as Java programs themselves, will clean these old ones up. The Attach API ‘allows sending ‘commands’ for executionin the JVM,’ but only works on local machine and for current/same user. This is what jcmd and jstack use. Larsen then went onto explain the different mechanics of using the Attach API for Linux/BSD/Solaris (uses temporary file creation) versus Windows (uses code injection). I employed the Attach API in my post Groovy, JMX, and the Attach API. Diagnostic commands are ‘helper routines inside the JVM’ that produce ‘text output.’ They can be executed via the jcmd utility (and soon via JMX). They each have a self-describing facility: jcmd PerfCounter.print to see the raw contents. Larsen showed an informative table comparing ‘communicating with the JVM’ approaches: attach, jvmstat, JMX, jstatd, and Serviceability Agent (SA). The SA ‘should be used as a last resort (‘typically for a JVM that is hung’)’ and uses a ‘debugger to read information.’ Larsen transitioned to talk of future tools. He started this portion of the presentation with coverage of Java Flight Recorder. Java Flight Recorder is a ‘JVM-built-in profiler and tracer’ with ‘low overhead’ and is ‘always on.’ Other coming tools are Java Mission Control (‘graphical tool providing very detailed runtime monitoring details’), more diagnostic commands for jcmd (‘eventually replacing jstack, jmap, jinfo’ for various reasons), JMX 2.0 (‘something we’re picking up again; it was started a very long time ago’), improved logging for JVM (JVM Enhancement Proposal [JEP] 158), and Java Discovery Protocol (anticipating forthcoming JEP for this). One question asked was if one could see MBeans in VisualVM as can be done in JConsole. As I’ve blogged on, there is a VisualVM plug-in for doing just that. Although I felt somewhat comfortable with the Oracle HotSpot JDK command line tools, I was unfamiliar with jcmd and appreciated Larsen’s coverage of it. I learned some other things along the way as well. My only complaint is that Larsen’s presentation (especially the demonstration) was so rapid fire and so content-rich that I wish I could see it again. A related (but older) presentation with some of the same content is available at http://www.oracle.com/javaone/lad-en/session-presentations/corejava/22260-enok-1439100.pdf   Reference: JavaOne 2012: Diagnosing Your Application on the JVM from our JCG partner Dustin Marx at the Inspired by Actual Events blog. ...
Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy | Contact
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.
Do you want to know how to develop your skillset and become a ...
Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

Get ready to Rock!
You can download the complementary eBooks using the links below: