What's New Here?

java-logo

Top 50 Java Thread Interview Questions Answers for Freshers, Experienced Programmers

You go to any Java interview, senior or junior, experience or freshers,  you are bound to see couple of questions from thread, concurrency and multi-threading. In fact this built-in concurrency support is one of the strongest point of Java programming language and helped it to gain popularity among enterprise world and programmers equally. Most of lucrative Java developer position demands excellent core Java multi-threading skills and experience on developing, debugging and tuning high performance low latency concurrent Java applications. This is the reason, it is one of the most sought after skill on interviews. In a typical Java interview, Interviewer slowly starts from basic concepts of Thread by asking questions like, why you need threads, how to create threads, which one is better way to create threads e.g. by extending thread class or implementing Runnable and then slowly goes into Concurrency issues, challenges faced during development of concurrent Java applications, Java memory model, higher order concurrency utilities introduced in JDK 1.5, principles and design patterns of concurrent Java applications, classical multi-threading problems e.g. producer consumer, dining philosopher, reader writer or simply bounded buffer problems. Since its not enough just to know basics of threading, you must know how to deal with concurrency problems e.g. deadlock, race conditions, memory inconsistency and various thread safety related issues. These skills are thoroughly get tested by presenting various multi-threading and concurrency problems. Many Java developers are used to only look and read interview questions before going for interview, which is not bad but you should not be too far away. Also collecting questions and going through same exercise is too much time consuming, that’s why I have created this list of top 50 Java multi-threading and concurrency related questions, collected from various interviews. I am only going to add new and recent interview questions as and when I am going to discover them. By the way, I have not provided answers of this questions here, Why? because I expect most of Java developer to know the answers of this question and if not, also answers are widely available by using Google. If you don’t find answer of any particular question, you can always ask me in comments section. You can even find answers of few questions on the link provided or my earlier post Top 12 Java Thread Questions with Answers. 50 Interview questions from Java Multi-threading and Concurrency Here is our list of top questions from Java thread, concurrency and multi-threading. You can use this list to prepare well for your Java interview.What is Thread in Java? Thread is an independent path of execution. It’s way to take advantage of multiple CPU available in a machine. By employing multiple threads you can speed up CPU bound task. For example, if one thread takes 100 millisecond to do a job, you can use 10 thread to reduce that task into 10 millisecond. Java provides excellent support for multi-threading at language level, and its also one of strong selling point. For more details see here.Difference between Thread and Process in Java? Thread is subset of Process, in other words one process can contain multiple threads. Two process runs on different memory space, but all threads share same memory space. Don’t confuse this with stack memory, which is different for different thread and used to store local data to that thread. For more detail see this answer.How do you implement Thread in Java? At language level, there are two ways to implement Thread in Java. An instance of java.lang.Thread represent a thread but it need a task to execute, which is instance of interface java.lang.Runnable. Since Thread class itself implement Runnable, you can override run() method either by extending Thread class or just implementing Runnable interface. For detailed answer and discussion see this article.When to use Runnable vs Thread in Java? This is follow-up of previous multi-threading interview question. As we know we can implement thread either by extending Thread class or implementing Runnable interface, question arise, which one is better and when to use one? This question will be easy to answer, if you know that Java programming language doesn’t support multiple inheritance of class, but it allows you to implement multiple interface. Which means, its better to implement Runnable than extends Thread, if you also want to extend another class e.g. Canvas or CommandListener. For more points and discussion you can also refer this post.Difference between start() and run() method of Thread class?  One of trick Java question from early days, but still good enough to differentiate between shallow understanding of Java threading model start() method is used to start newly created thread, while start() internally calls run() method, there is difference calling run() method directly. When you invoke run() as normal method, its called in the same thread, no new thread is started, which is the case when you call start() method. Read this answer for much more detailed discussion.Difference between Runnable and Callable in Java? Both Runnable and Callable represent task which is intended to be executed in separate thread. Runnable is there from JDK 1.0, while Callable was added on JDK 1.5. Main difference between these two is that Callable’s call() method can return value and throw Exception, which was not possible with Runnable’s run() method. Callable return Future object, which can hold result of computation. See my blog post on same topic for more in-depth answer of this question.Difference between CyclicBarrier and CountDownLatch in Java?  Though both CyclicBarrier and CountDownLatch wait for number of threads on one or more events, main difference between them is that you can not re-use CountDownLatch once count reaches to zero, but you can reuse same CyclicBarrier even after barrier is broken.  See this answer for few more points and sample code example.What is Java Memory model? Java Memory model is set of rules and guidelines which allows Java programs to behave deterministically across multiple memory architecture, CPU, and operating system. It’s particularly important in case of multi-threading. Java Memory Model provides some guarantee on which changes made by one thread should be visible to others, one of them is happens-before relationship. This relationship defines several rules which allows programmers to anticipate and reason behaviour of concurrent Java programs. For example, happens-before relationship guarantees :Each action in a thread happens-before every action in that thread that comes later in the program order, this is known as program order rule. An unlock on a monitor lock happens-before every subsequent lock on that same monitor lock, also known as Monitor lock rule. A write to a volatile field happens-before every subsequent read of that same field, known as Volatile variable rule. A call to Thread.start on a thread happens-before any other thread detects that thread has terminated, either by successfully return from Thread.join() or by Thread.isAlive() returning false, also known as Thread start rule. A thread calling interrupt on another thread happens-before the interrupted thread detects the interrupt( either by having InterruptedException thrown, or invoking isInterrupted or interrupted), popularly known as Thread Interruption rule. The end of a constructor for an object happens-before the start of the finalizer for that object, known as Finalizer rule. If A happens-before B, and B happens-before C, then A happens-before C, which means happens-before guarantees Transitivity.I strongly suggest to read Chapter 16 of Java Concurrency in Practice to understand Java Memory model in more detail.What is volatile variable in Java? volatile is a special modifier, which can only be used with instance variables. In concurrent Java programs, changes made by multiple threads on instance variables is not visible to other in absence of any synchronizers e.g. synchronized keyword or locks. Volatile variable guarantees that a write will happen before any subsequent read, as stated “volatile variable rule” in previous question. Read this answer to learn more about volatile variable and when to use them.What is thread-safety? is Vector a thread-safe class? (Yes, see details) Thread-safety is a property of an object or code which guarantees that if executed or used by multiple thread in any manner e.g. read vs write it will behave as expected. For example, a thread-safe counter object will not miss any count if same instance of that counter is shared among multiple threads. Apparently, you can also divide collection classes in two category, thread-safe and non-thread-safe. Vector is indeed a thread-safe class and it achieves thread-safety by synchronizing methods which modifies state of Vector, on the other hand, its counterpart ArrayList is not thread-safe.What is race condition in Java? Given one example?  Race condition are cause of some subtle programming bugs when Java programs are exposed to concurrent execution environment. As name suggests, race condition occurs due to race between multiple threads, if a thread which is supposed to execute first lost the race and executed second, behaviour of code changes, which surface as non-deterministic bugs. This is one of the hardest bugs to find and re-produce because of random nature of racing between threads. One example of race condition is out-of-order processing, see this answer for some more example of race conditions in Java programs.How to stop thread in Java? I always said that Java provides rich APIs for everything but ironically Java doesn’t provide a sure shot way of stopping thread. There was some control methods in JDK 1.0 e.g. stop(), suspend() and resume() which was deprecated in later releases due to potential deadlock threats, from then Java API designers has not made any effort to provide a consistent, thread-safe and elegant way to stop threads. Programmers mainly rely on the fact that thread stops automatically as soon as they finish execution of run() or call() method. To manually stop, programmers either take advantage of volatile boolean variable and check in every iteration if run method has loops or interrupt threads to abruptly cancel tasks. See this tutorial for sample code of stopping thread in Java.What happens when an Exception occurs in a thread? This is one of the good tricky Java question I have seen on interviews. In simple words, If not caught thread will die, if an uncaught exception handler is registered then it will get a call back. Thread.UncaughtExceptionHandler is an interface, defined as nested interface for handlers invoked when a Thread abruptly terminates due to an uncaught exception. When a thread is about to terminate due to an uncaught exception the Java Virtual Machine will query the thread for its UncaughtExceptionHandler using Thread.getUncaughtExceptionHandler() and will invoke the handler’s uncaughtException() method, passing the thread and the exception as arguments.How do you share data between two thread in Java? You can share data between threads by using shared object, or concurrent data-structure like BlockingQueue. See this tutorial to learn inter thread communication in Java. It implements Producer consumer pattern using wait and notify methods, which involves sharing objects between two threads.Difference between notify and notifyAll in Java? This is another tricky questions from core Java interviews, since multiple threads can wait on single monitor lock, Java API designer provides method to inform only one of them or all of them, once waiting condition changes, but they provide half implementation. There notify() method doesn’t provide any way to choose a particular thread, that’s why its only useful when you know that there is only one thread is waiting. On the other hand, notifyAll() sends notification to all threads and allows them to compete for locks, which ensures that at-least one thread will proceed further. See my blog post on similar topic for more detailed answer and code example.Why wait, notify and notifyAll are not inside thread class?  This is a design related question, which checks what candidate thinks about existing system or does he ever thought of something which is so common but looks in-appropriate at first. In order to answer this question, you have to give some reasons why it make sense for these three method to be in Object class, and why not on Thread class. One reason which is obvious is that Java provides lock at object level not at thread level. Every object has lock, which is acquired by thread. Now if thread needs to wait for certain lock it make sense to call wait() on that object rather than on that thread. Had wait() method declared on Thread class, it was not clear that for which lock thread was waiting. In short, since wait, notify and notifyAll operate at lock level, it make sense to defined it on object class because lock belongs to object. You can also see this article for more elaborate answer of this question.What is ThreadLocal variable in Java?  ThreadLocal variables are special kind of variable available to Java programmer. Just like instance variable is per instance, ThreadLocal variable is per thread. It’s a nice way to achieve thread-safety of expensive-to-create objects, for example you can make SimpleDateFormat thread-safe using ThreadLocal. Since that class is expensive, its not good to use it in local scope, which requires separate instance on each invocation. By providing each thread their own copy, you shoot two birds in one arrow. First, you reduce number of instance of expensive object by reusing fixed number of instances, and Second, you achieve thread-safety without paying cost of synchronization or immutability. Another good example of thread local variable is ThreadLocalRandom class, which reduces number of instances of expensive-to-create Random object in multi-threading environment. See this answer to learn more about thread local variables in Java.What is FutureTask in Java? FutureTask represents a cancellable asynchronous computation in concurrent Java application. This class provides a base implementation of Future, with methods to start and cancel a computation, query to see if the computation is complete, and retrieve the result of the computation. The result can only be retrieved when the computation has completed; the get methods will block if the computation has not yet completed. A FutureTask object can be used to wrap a Callable or Runnable object. Since FutureTask also implements Runnable, it can be submitted to an Executor for execution.Difference between interrupted and isInterrupted method in Java? Main difference between interrupted() and isInterrupted() is that former clears the interrupt status while later does not. The interrupt mechanism in Java multi-threading is implemented using an internal flag known as the interrupt status. Interrupting a thread by calling Thread.interrupt() sets this flag. When interrupted thread checks for an interrupt by invoking the static method Thread.interrupted(), interrupt status is cleared. The non-static isInterrupted() method, which is used by one thread to query the interrupt status of another, does not change the interrupt status flag. By convention, any method that exits by throwing an InterruptedException clears interrupt status when it does so. However, it’s always possible that interrupt status will immediately be set again, by another thread invoking interrupt.Why wait and notify method are called from synchronized block? Main reason for calling wait and notify method from either synchronized block or method is that it made mandatory by Java API. If you don’t call them from synchronized context, your code will throw IllegalMonitorStateException. A more subtle reason is to avoid race condition between wait and notify calls. To learn more about this, check my similarly titled post here.Why you should check condition for waiting in a loop? Its possible for a waiting thread to receive false alerts and spurious wake up calls, if it doesn’t check the waiting condition in loop, it will simply exit even if condition is not met. As such, when a waiting thread wakes up, it cannot assume that the state it was waiting for is still valid. It may have been valid in the past, but the state may have been changed after the notify() method was called and before the waiting thread woke up. That’s why it always better to call wait() method from loop, you can even create template for calling wait and notify in Eclipse. To learn more about this question, I would recommend you to read Effective Java items on thread and synchronization.Difference between synchronized and concurrent collection in Java? Though both synchronized and concurrent collection provides thread-safe collection suitable for multi-threaded and concurrent access, later is more scalable than former. Before Java 1.5, Java programmers only had synchronized collection which becomes source of contention if multiple thread access them concurrently, which hampers scalability of system. Java 5 introduced concurrent collections like ConcurrentHashMap, which not only provides thread-safety but also improves scalability by using modern techniques like lock stripping and partitioning internal table. See this answer for more differences between synchronized and concurrent collection in Java.Difference between Stack and Heap in Java? Why do someone this question as part of multi-threading and concurrency? because Stack is a memory area which is closely associated with threads. To answer this question, both stack and heap are specific memories in Java application. Each thread has their own stack, which is used to store local variables, method parameters and call stack. Variable stored in one Thread’s stack is not visible to other. On other hand, heap is a common memory area which is shared by all threads. Objects whether local or at any level is created inside heap. To improve performance thread tends to cache values from heap into their stack, which can create problems if that variable is modified by more than one thread, this is where volatile variables comes in picture. volatile suggest threads to read value of variable always from main memory. See this article to learn more about stack and heap in Java to answer this question in greater detail.What is thread pool? Why should you thread pool in Java?  Creating thread is expensive in terms of time and resource. If you create thread at time of request processing it will slow down your response time, also there is only a limited number of threads a process can create. To avoid both of these issue, a pool of thread is created when application starts-up and threads are reused for request processing. This pool of thread is known as “thread pool” and threads are known as worker thread. From JDK 1.5 release, Java API provides Executor framework, which allows you to create different types of thread pools e.g. single thread pool, which process one task at a time, fixed thread pool (a pool of fixed number of thread) or cached thread pool (an expandable thread pool suitable for applications with many short lived tasks). See this article to learn more about thread pools in Java to prepare detailed answer of this question.Write code to solve Producer Consumer problem in Java? Most of the threading problem you solved in real world are of category of Producer consumer pattern, where one thread is producing task and other thread is consuming that. You must know how to do inter thread communication to solve this problem. At lowest level, you can use wait and notify to solve this problem, and at high level you can leverage Semaphore or BlockingQueue to implement Producer consumer pattern, as shown in this tutorial.How do you avoid deadlock in Java? Write Code?Deadlock is a condition in which two threads wait for each other to take action which allows them to move further. It’s a serious issue because when it happen your program hangs and doesn’t do the task it is intended for. In order for deadlock to happen, following four condition must be true :Mutual Exclusion : At least one resource must be held in a non-shareable mode. Only one process can use the resource at any given instant of time. Hold and Wait : A process is currently holding at least one resource and requesting additional resources which are being held by other processes. No Pre-emption : The operating system must not de-allocate resources once they have been allocated; they must be released by the holding process voluntarily. Circular Wait : A process must be waiting for a resource which is being held by another process, which in turn is waiting for the first process to release the resource.Easiest way to avoid deadlock is to prevent Circular wait, and this can be done by acquiring locks in a particular order and releasing them in reverse order, so that a thread can only proceed to acquire a lock if it held the other one. Check this tutorial for actual code example and detailed discussion on techniques of avoiding deadlock in Java.Difference between livelock and deadlock in Java? This question is extension of previous interview question. A livelock is similar to a deadlock, except that the states of the threads or processes involved in the livelock constantly change with regard to one another, without any one progressing further. Livelock is a special case of resource starvation. A real-world example of livelock occurs when two people meet in a narrow corridor, and each tries to be polite by moving aside to let the other pass, but they end up swaying from side to side without making any progress because they both repeatedly move the same way at the same time. In short, main difference between livelock and deadlock is that in former state of process change but no progress is made.How do you check if a Thread holds a lock or not? I didn’t even know that you can check if a Thread already holds lock before this question hits me in a telephonic round of Java interview. There is a method called holdsLock() on java.lang.Thread, it returns true if and only if the current thread holds the monitor lock on the specified object. You can also check this article for more detailed answer.How do you take thread dump in Java? There are multiple ways to take thread dump of Java process depending upon operating system. When you take thread dump, JVM dumps state of all threads in log files or standard error console. In windows you can use Ctrl + Break key combination to take thread dump, on Linux you can use kill -3 command for same. You can also use a tool called jstack for taking thread dump, it operate on process id, which can be found using another tool called jps.Which JVM parameter is used to control stack size of thread? This is the simple one, -Xss parameter is used to control stack size of Thread in Java. You can see this list of JVM options to learn more about this parameter.Difference between synchronized and ReentrantLock in Java? There were days when only way to provide mutual exclusion in Java was via synchronized keyword, but it has several shortcomings e.g. you can not extend lock beyond a method or block boundary, you can not give up trying for a lock etc. Java 5 solves this problem by providing more sophisticated control via Lock interface. ReentrantLock is a common implementation of Lock interface and provides re-entrant mutual exclusion Lock with the same basic behaviour and semantics as the implicit monitor lock accessed using synchronized methods and statements, but with extended capabilities. See this article to learn about those capabilities and some more differences between synchronized vs ReentrantLock in Java.There are three threads T1, T2 and T3? How do you ensure sequence T1, T2, T3 in Java? Sequencing in multi-threading can be achieved by different means but you can simply use join() method of thread class to start a thread when another one is finished its execution. To ensure three threads execute you need to start the last one first e.g. T3 and then call join methods in reverse order e.g. T3 calls T2. join, and T2 calls T1.join, this ways T1 will finish first and T3 will finish last. To learn more about join method, see this tutorial.What does yield method of Thread class do? Yield method is one way to request current thread to relinquish CPU so that other thread can get chance to execute. Yield is a static method and only guarantees that current thread will relinquish the CPU but doesn’t say anything about which other thread will get CPU. Its possible for same thread to get CPU back and start its execution again. See this article to learn more about yield method and to answer this question better.What is concurrence level of ConcurrentHashMap in Java? ConcurrentHashMap achieves it’s scalability and thread-safety by partitioning actual map into number of sections. This partitioning is achieved using concurrency level. It’s optional parameter of ConcurrentHashMap constructor and it’s default value is 16. The table is internally partitioned to try to permit the indicated number of concurrent updates without contention. To learn more about concurrency level and internal resizing, see my post How ConcurrentHashMap works in Java.What is Semaphore in Java? Semaphore in Java is a new kind of synchronizer. It’s a counting semaphore. Conceptually, a semaphore maintains a set of permits. Each acquire() blocks if necessary until a permit is available, and then takes it. Each release() adds a permit, potentially releasing a blocking acquirer. However, no actual permit objects are used; the Semaphore just keeps a count of the number available and acts accordingly. Semaphore is used to protect expensive resource which is available in fixed number e.g. database connection in pool. See this article to learn more about counting Semaphore in Java.What happens if you submit task, when queue of thread pool is already fill? This is another tricky question in my list. Many programmer will think that it will block until a task is cleared but its true. ThreadPoolExecutor’s submit() method throws RejectedExecutionException if the task cannot be scheduled for execution.Difference between submit() and execute() method thread pool in Java? Both method are ways to submit task to thread pools but there is slight difference between them. execute(Runnable command) is defined in Executor interface and executes given task in future, but more importantly it does not return anything. It’s return type is void. On other hand submit() is overloaded method, it can take either Runnable or Callable task and can return Future object which can hold pending result of computation. This method is defined on ExecutorService interface, which extends Executor interface, and every other thread pool class e.g. ThreadPoolExecutor or ScheduledThreadPoolExecutor gets these methods. To learn more about thread pools you can check this article.What is blocking method in Java? A blocking method is a method which blocks until task is done, for example accept() method of ServerSocket blocks until a client is connected. here blocking means control will not return to caller until task is finished. On the other hand there are asynchronous or non-blocking method which returns even before task is finished. To learn more about blocking method see this answer.Is Swing thread-safe? What do you mean by Swing thread-safe? You can simply this question as No, Swing is not thread-safe, but you have to explain what you mean by that even if interviewer doesn’t ask about it. When we say swing is not thread-safe we usually refer its component, which can not be modified in multiple threads. All update to GUI components has to be done on AWT thread, and Swing provides synchronous and asynchronous callback methods to schedule such updates. You can also read my article to learn more about swing and thread-safety to better answer this question. Even next two questions are also related to this concept.Difference between invokeAndWait and invokeLater in Java? These are two methods Swing API provides Java developers to update GUI components from threads other than Event dispatcher thread. InvokeAndWait() synchronously update GUI component, for example a progress bar, once progress is made, bar should also be updated to reflect that change. If progress is tracked in a different thread, it has to call invokeAndWait() to schedule an update of that component by Event dispatcher thread. On other hand, invokeLater() is asynchronous call to update components. You can also refer this answer for more points.Which method of Swing API are thread-safe in Java? This question is again related to swing and thread-safety, though components are not thread-safe there are certain method which can be safely call from multiple threads. I know about repaint(), and revalidate() being thread-safe but there are other methods on different swing components e.g. setText() method of JTextComponent, insert() and append() method of JTextArea class.How to create Immutable object in Java? This question might not look related to multi-threading and concurrency, but it is. Immutability helps to simplify already complex concurrent code in Java. Since immutable object can be shared without any synchronization its very dear to Java developers. Core value object, which is meant to be shared among thread should be immutable for performance and simplicity. Unfortunately there is no @Immutable annotation in Java, which can make your object immutable, hard work must be done by Java developers. You need to keep basics like initializing state in constructor, no setter methods, no leaking of reference, keeping separate copy of mutable object to create Immutable object. For step by step guide see my post, how to make an object Immutable in Java. This will give you enough material to answer this question with confidence.What is ReadWriteLock in Java? In general, read write lock is result of lock stripping technique to improve performance of concurrent applications. In Java, ReadWriteLock is an interface which was added in Java 5 release. A ReadWriteLock maintains a pair of associated locks, one for read-only operations and one for writing. The read lock may be held simultaneously by multiple reader threads, so long as there are no writers. The write lock is exclusive. If you want you can implement this interface with your own set of rules, otherwise you can use ReentrantReadWriteLock, which comes along with JDK and supports a maximum of 65535 recursive write locks and 65535 read locks.What is busy spin in multi-threading? Busy spin is a technique which concurrent programmers employ to make a thread wait on certain condition. Unlike traditional methods e.g. wait(), sleep() or yield() which all involves relinquishing CPU control, this method does not relinquish CPU, instead it just runs empty loop. Why would someone do that? to preserve CPU caches. In multi core system, its possible for a paused thread to resume on different core, which means rebuilding cache again. To avoid cost of rebuilding cache, programmer prefer to wait for much smaller time doing busy spin. You can also see this answer to learn more about this question.Difference between volatile and atomic variable in Java? This is an interesting question for Java programmer, at first, volatile and atomic variable look very similar, but they are different. Volatile variable provides you happens-before guarantee that a write will happen before any subsequent write, it doesn’t guarantee atomicity. For example count++ operation will not become atomic just by declaring count variable as volatile. On the other hand AtomicInteger class provides atomic method to perform such compound operation atomically e.g. getAndIncrement() is atomic replacement of increment operator. It can be used to atomically increment current value by one. Similarly you have atomic version for other data type and reference variable as well.What happens if a thread throws an Exception inside synchronized block? This is one more tricky question for average Java programmer, if he can bring the fact about whether lock is released or not is key indicator of his understanding. To answer this question, no matter how you exist synchronized block, either normally by finishing execution or abruptly by throwing exception, thread releases the lock it acquired while entering that synchronized block. This is actually one of the reason I like synchronized block over lock interface, which requires explicit attention to release lock, generally this is achieved by releasing lock in finally block.What is double checked locking of Singleton? This is one of the very popular question on Java interviews, and despite its popularity, chances of candidate answering this question satisfactory is only 50%. Half of the time, they failed to write code for double checked locking and half of the time they failed how it was broken and fixed on Java 1.5. This is actually an old way of creating thread-safe singleton, which tries to optimize performance by only locking when Singleton instance is created first time, but because of complexity and the fact it was broken for JDK 1.4,  I personally don’t like it. Anyway, even if you not prefer this approach its good to know from interview point of view. Since this question deserve a detailed answer, I have answered in a separate post, you can read my post how double checked locking on Singleton works to learn more about it.How to create thread-safe Singleton in Java? This question is actually follow-up of previous question. If you say you don’t like double checked locking then Interviewer is bound to ask about alternative ways of creating thread-safe Singleton class. There are actually man, you can take advantage of class loading and static variable initialization feature of JVM to create instance of Singleton, or you can leverage powerful enumeration type in Java to create Singleton. I actually preferred that way, you can also read this article to learn more about it and see some sample code.List down 3 multi-threading best practice you follow? This is my favourite question, because I believe that you must follow certain best practices while writing concurrent code which helps in performance, debugging and maintenance. Following are three best practices, I think an average Java programmer should follow:Always give meaningful name to your threadThis goes a long way to find a bug or trace an execution in concurrent code. OrderProcessor, QuoteProcessor or TradeProcessor is much better than Thread-1. Thread-2 and Thread-3. Name should say about task done by that thread. All major framework and even JDK follow this best practice. Avoid locking or Reduce scope of Synchronization Locking is costly and context switching is even more costlier. Try to avoid synchronization and locking as much as possible and at bare minimum, you should reduce critical section. That’s why I prefer synchronized block over synchronized method, because it gives you absolute control on scope of locking. Prefer Synchronizers over wait and notify Synchronizers like CountDownLatch, Semaphore, CyclicBarrier or Exchanger simplifies coding. It’s very difficult to implement complex control flow right using wait and notify. Secondly, these classes are written and maintained by best in business and there is good chance that they are optimized or replaced by better performance code in subsequent JDK releases. By using higher level synchronization utilities, you automatically get all these benefits. Prefer Concurrent Collection over Synchronized Collection This is another simple best practice which is easy to follow but reap good benefits. Concurrent collection are more scalable than their synchronized counterpart, that’s why its better to use them while writing concurrent code. So next time if you need map, think about ConcurrentHashMap before thinking Hashtable. See my article Concurrent Collections in Java, to learn more about modern collection classes and how to make best use of them.How do you force start a Thread in Java? This question is like how do you force garbage collection in Java, their is no way, though you can make request using System.gc() but its not guaranteed. On Java multi-threading their is absolute no way to force start a thread, this is controlled by thread scheduler and Java exposes no API to control thread schedule. This is still a random bit in Java.What is fork join framework in Java? The fork join framework, introduced in JDK 7 is a powerful tool available to Java developer to take advantage of multiple processors of modern day servers. It is designed for work that can be broken into smaller pieces recursively. The goal is to use all the available processing power to enhance the performance of your application. One significant advantage of The fork/join framework is that it uses a work-stealing algorithm. Worker threads that run out of things to do can steal tasks from other threads that are still busy. See this article for much more detailed answer of this question.What is difference between calling wait() and sleep() method in Java multi-threading? Though both wait and sleep introduce some form of pause in Java application, they are tool for different needs. Wait method is used for inter thread communication, it relinquish lock if waiting condition is true and wait for notification when due to action of another thread waiting condition becomes false. On the other hand sleep() method is just to relinquish CPU or stop execution of current thread for specified time duration. Calling sleep method doesn’t release the lock held by current thread. You can also take look at this article to answer this question with more details.That’s all on this list of top 50 Java multi-threading and concurrency interview questions. I have not shared answers of all the questions but provided enough hints and links to explore further and find answers by yourselves. As I said, let me know if you don’t find answer of any particular question and I will add answer here. You can use this list to not only to prepare for your core Java and programming interviews but also to check your knowledge about basics of threads, multi-threading, concurrency, design patterns and threading issues like race conditions, deadlock and thread safety problems. My intention is to make this list of question as mother of all list of Java Multi-threading questions, but this can not be done without your help. You can also share any question with us, which has been asked to you or any question for which you yet to find an answer. This master list is equally useful to Java developers of all levels of experience. You can read through this list even if you have 2 to 3 years of working experience as junior developer or 5 to 6 years as senior developer. It’s even useful for freshers and beginners to expand their knowledge. I will add new and latest multi-threading question as and when I come across, and I request you all to ask, share and answer questions via comments to keep this list relevant for all Java programmers.Reference: Top 50 Java Thread Interview Questions Answers for Freshers, Experienced Programmers from our JCG partner Javin Paul at the Javarevisited blog....
apache-zookeeper-logo

ZooKeeper, Curator and How Microservices Load Balancing Works

How Zookeeper makes sure that every worker happily gets some stuff to do from job delegating manager. Apache ZooKeeper is a tool to register, manage and discover services working on different machines. It is an indispensable member in technology stack when we have to deal with distributed system with many nodes which need to know where their dependencies are launched.      However ZooKeeper is pretty low level and even standard use-cases require many lines of code. That is why Apache Curator was born – a much more friendly and easier to use wrapper library over ZooKeeper. Using Curator we can deliver more with less code and in a much cleaner way.“Guava is to Java what Curator is to ZooKeeper” – Patrick Hunt, ZooKeeper committerLoad balancing microservices with ZooKeeper We are accustomed to situation when there is a load balancer deployed in front of our application. Its role is to make sure that every single node gets more or less same amount of traffic. In a microservices world situation is the same but we have a significant advantage over a monolithic approach: when we need to scale application we don’t have to duplicate whole system and deploy it on a server powerful enough to run it smoothly. We can concentrate only on a small module that needs to be scaled so cost of scaling is much lower (both in terms of server cost and development needed to prepare module for deployment in many instances). But as our system grows, we end up with many scaled modules, each of them requiring separate load balancer. This seems troublesome as our infrastructure is very complex even without them. Luckily if we use ZooKeeper as a service orchestration and discovery tool we can use built-in load balancing capability without introducing any additional complexity into our microservice architecture. To present how out of the box load-balancing works in ZooKeeper we need two services: worker which will be deployed multiple times and manager delegating tasks to registered workers. Simple worker Let’s start with creating a simple worker that will listen on a given port and return some result when asked to perform its job. To implement this tiny microservice we will use Groovy, Undertow light-weight servlet container and of course ZooKeeper and Curator. Our worker will consist of one, small class with main method that does three things: class Main {static final void main(String[] args) { // Step 1: Extract name and port number to launch worker // Step 2: Configure and start Rest server on given port // Step 3: Register worker in ZooKeeper } } For a brevity I will omit steps 1 and 2 here, you can check complete source code on GitHub project. Our worker will have only a single endpoint GET /work which returns a response with name of the called worker: @Path("/") class Main {@GET @Path("/work") public String work() { String response = "Work done by $workerName" println response return response }} Step 3: Register worker in ZooKeeper is the most interesting here so I will explain it in more detail: private static void registerInZookeeper(int port) { CuratorFramework curatorFramework = CuratorFrameworkFactory.newClient("localhost:2181", new RetryNTimes(5, 1000)) curatorFramework.start() ServiceInstance<Void> serviceInstance = ServiceInstance.builder() .uriSpec(new UriSpec("{scheme}://{address}:{port}")) .address('localhost') .port(port) .name("worker") .build()ServiceDiscoveryBuilder.builder(Void) .basePath("load-balancing-example") .client(curatorFramework) .thisInstance(serviceInstance) .build() .start() }Lines 2-3: we create and start CuratorFramework client wrapping all operations we want to perform on ZooKeeper instance. For a simplicity we use localhost with default port (in general it should be the URL to the running instance of ZooKeeper) Lines 4-9 create ServiceInstance representing our worker. We pass all “contact details” required to call this worker from other microservices Lines 11-16 register our instance in a ZooKeeper represented by CuratorFramework client.Starting worker Worker is now ready so we can create fat jar (with Gradle fatJar task) and then launch it using: java -jar simple-worker/build/libs/simple-worker-1.0-shadow.jar Worker_1 18005 Before starting worker remember that you need ZooKeeper instance running on default 2181 port! To check that worker is running you should open browser on http://localhost:18005/work and see “Work done by Worker_1” text there. To verify that worker has properly registered itself in ZooKeeper, launch command line client: cd /bin ./zkCli.sh and then execute ls command to see one node registered under /load-balancing-example/worker path: [zk: localhost:2181(CONNECTED) 1] ls /load-balancing-example/worker <enter> [f69545e8-8466-40c0-93e9-f493eb7496b4] Simple manager Now, as we have worker listening on /work for requests, we can create simple manager service delegating tasks to its subordinates. Main method looks quite similar to one in simple-worker project, the main difference is that we do not register in ZooKeeper, we only create ServiceProvider which role is to (surprise, surprise) provide us with instance of worker. So basic workflow is:Wait for requests on /delegate Get instance of worker service from ZooKeeper’s ServiceProvider Call worker’s /work and return results of its executionTo create ServiceProvider we have to create CuratorFramework client, connect to ZooKeeper and then fetch ServiceProvider for service with given name, worker in our case: CuratorFramework curatorFramework = CuratorFrameworkFactory.newClient("localhost:2181", new RetryNTimes(5, 1000)) curatorFramework.start()ServiceDiscovery<Void> serviceDiscovery = ServiceDiscoveryBuilder .builder(Void) .basePath("load-balancing-example") .client(curatorFramework).build() serviceDiscovery.start()serviceProvider = serviceDiscovery.serviceProviderBuilder().serviceName("worker").build() serviceProvider.start()Lines 1-2, create ZooKeeper client, same way as in simple-worker Lines 4-8, create ServiceDiscovery which will be able to give us service providers. Lines 10-11. create and start ServiceProvider which will be used to fetch working instance of worker node.Finally, we need a Rest endpoint on which manager waits for tasks to delegate: @GET @Path("/delegate") public String delegate() { def instance = serviceProvider.getInstance() String address = instance.buildUriSpec() String response = (address + "/work").toURL().getText() println response return response } Starting manager Manager is ready and after fatJar task we can launch it using: java -jar simple-manager/build/libs/simple-manager-1.0-shadow.jar 18000 To verify it is working we can open (http://localhost:18000/delegate) in browser to see message “Work done by Worker_1”. Manager doesn’t know anything about its workers. Only thing he know is that service is registered under specific path in ZooKeeper. And it is that simple no matter if we have multiple workers launched locally or spread across different servers in different countries. Out of the box load-balancing with ZooKeeper Imagine situation that manager gets so many tasks from his CEO that he needs more than one worker to delegate job. In a standard case we will be forced to scale workers and place load balancer in front of them. But ZooKeeper gives us this feature without any additional work. Let’s add some more workers listening on different ports: java -jar simple-worker/build/libs/simple-worker-1.0-shadow.jar Worker_1 18005 & java -jar simple-worker/build/libs/simple-worker-1.0-shadow.jar Worker_2 18006 &java -jar simple-worker/build/libs/simple-worker-1.0-shadow.jar Worker_3 18007 & java -jar simple-worker/build/libs/simple-worker-1.0-shadow.jar Worker_4 18008 & The trick is that all workers are registered under the same path in ZooKeeper, so when we list nodes under /load-balancing-example/worker we will see four instances: [zk: localhost:2181(CONNECTED) 0] ls /load-balancing-example/worker <enter> [d5bc4eb9-8ebb-4b7c-813e-966a25fdd843, 13de9196-bfeb-4c1a-b632-b8b9969b9c0b, 85cd1387-2be8-4c08-977a-0798017379b1, 9e07bd1d-c615-430c-8dcb-bf228e9b56fc] What is most important here is that to utilize all these four new workers, manager doesn’t require any changes in the code. We can launch new worker instances as traffic increases or shut them down when there is nothing to do. Manager is decoupled from these actions, it still calls ServiceProvider to get instance of worker and passes job to him. So now when we open http://localhost:18000/delegate and hit refresh several times we will see: Work done by Worker_1 Work done by Worker_2 Work done by Worker_3 Work done by Worker_4 Work done by Worker_1 Work done by Worker_2 Work done by Worker_3 How is it implemented under the hood? By default ServiceProvider uses Round-robin ProviderStrategy implementation which rotates instances available under given path so each gets some job to do. Of course we can implement our custom strategy if default mechanism doesn’t suit our needs. Summary That’s all for today. As you can see in by using Apache ZooKeeper and Curator we can live without separate load balancers that need to be deployed, monitored and managed. Infrastructure in a microservices architecture is pretty complicated even without them.Reference: ZooKeeper, Curator and How Microservices Load Balancing Works from our JCG partner Tomasz Dziurko at the Code Hard Go Pro blog....
spring-logo

SpringBoot: Introducing SpringBoot

SpringBoot…there is a lot of buzz about SpringBoot nowadays. So what is SpringBoot? SpringBoot is a new spring portfolio project which takes opinionated view of building production-ready Spring applications by drastically reducing the amount of configuration required. Spring Boot is taking the convention over configuration style to the next level by registering the default configurations automatically based on the classpath libraries available at runtime.   Well.. you might have already read this kind of introduction to SpringBoot on many blogs. So let me elaborate on what SpringBoot is and how it helps developing Spring applications more quickly. Spring framework was created by Rod Johnson when many of the Java developers are struggling with EJB 1.x/2.x for building enterprise applications. Spring framework makes developing the business components easy by using Dependency Injection and Aspect Oriented Programming concepts. Spring became very popular and many more Spring modules like SpringSecurity, Spring Batch, Spring Data etc become part of Spring portfolio. As more and more features added to Spring, configuring all the spring modules and their dependencies become a tedious task. Adding to that Spring provides atleast 3 ways of doing anything! Some people see it as flexibility and some others see it as confusing. Slowly, configuring all the Spring modules to work together became a big challenge. Spring team came up with many approaches to reduce the amount of configuration needed by introducing Spring XML DSLs, Annotations and JavaConfig. In the very beginning I remember configuring a big pile of jar version declarations in section and lot of declarations. Then I learned creating maven archetypes with basic structure and minimum required configurations. This reduced lot of repetitive work, but not eliminated completely. Whether you write the configuration by hand or generate by some automated ways, if there is code that you can see then you have to maintain it. So whether you use XML or Annotations or JavaConfig, you still need to configure (copy-paste) the same infrastructure setup one more time. On the other hand, J2EE (which is dead long time ago) emerged as JavaEE and since JavaEE6 it became easy (compared to J2EE and JavaEE5) to develop enterprise applications using JavaEE platform. Also JavaEE7 released with all the cool CDI, WebSockets, Batch, JSON support etc things became even more simple and powerful as well. With JavaEE you don’t need so much XML configuration and your war file size will be in KBs (really??? for non-helloworld/non-stageshow apps also!) Naturally this “convention over configuration” and “you no need to glue APIs together appServer already did it” arguments became the main selling points for JavaEE over Spring. Then Spring team addresses this problem with SpringBoot! Now its time to JavaEE to show whats the SpringBoot’s counterpart in JavaEE land JBoss Forge?? I love this Spring vs JavaEE thing which leads to the birth of powerful tools which ultimately simplify the developers life! Many times we need similar kind of infrastructure setup using same libraries. For example, take a web application where you map DispatcherServlet url-pattern to “/”, implement RESTFul webservices using Jackson JSON library with Spring Data JPA backend. Similarly there could be batch or spring integration applications which needs similar infrastructure configuration. SpringBoot to the rescue. SpringBoot look at the jar files available to the runtime classpath and register the beans for you with sensible defaults which can be overridden with explicit settings. Also SpringBoot configure those beans only when the jars files available and you haven’t define any such type of bean. Altogether SpringBoot provides common infrastructure without requiring any explicit configuration but lets the developer overrides if needed. To make things more simpler, SpringBoot team provides many starter projects which are pre-configured with commonly used dependencies. For example Spring Data JPA starter project comes with JPA 2.x with Hibernate implementation along with Spring Data JPA infrastructure setup. Spring Web starter comes with Spring WebMVC, Embedded Tomcat, Jackson JSON, Logback setup. Aaah..enough theory..lets jump onto coding. I am using latest STS-3.5.1 IDE which provides many more starter project options like Facebbok, Twitter, Solr etc than its earlier version. Create a SpringBoot starter project by going to File -> New -> Spring Starter Project -> select Web and Actuator and provide the other required details and Finish. This will create a Spring Starter Web project with the following pom.xml and Application.java <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion><groupId>com.sivalabs</groupId> <artifactId>hello-springboot</artifactId> <version>1.0-SNAPSHOT</version> <packaging>jar</packaging><name>hello-springboot</name> <description>Spring Boot Hello World</description><parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>1.1.3.RELEASE</version> <relativePath/> </parent><dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies><properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <start-class>com.sivalabs.springboot.Application</start-class> <java.version>1.7</java.version> </properties><build> <plugins> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> </plugin> </plugins> </build></project> package com.sivalabs.springboot;import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.EnableAutoConfiguration; import org.springframework.context.annotation.ComponentScan; import org.springframework.context.annotation.Configuration;@Configuration @ComponentScan @EnableAutoConfiguration public class Application {public static void main(String[] args) { SpringApplication.run(Application.class, args); } } Go ahead and run this class as a standalone Java class. It will start the embedded Tomcat server on 8080 port. But we haven’t added any endpoints to access, lets go ahead and add a simple REST endpoint. @Configuration @ComponentScan @EnableAutoConfiguration @Controller public class Application {public static void main(String[] args) { SpringApplication.run(Application.class, args); } @RequestMapping(value="/") @ResponseBody public String bootup() { return "SpringBoot is up and running"; } } Now point your browser to http://localhost:8080/ and you should see the response “SpringBoot is up and running”. Remember while creating project we have added Actuator starter module also. With Actuator you can obtain many interesting facts about your application. Try accessing the following URLs and you can see lot of runtime environment configurations that are provided by SpringBoot.http://localhost:8080/beans http://localhost:8080/metrics http://localhost:8080/trace http://localhost:8080/env http://localhost:8080/mappings http://localhost:8080/autoconfig http://localhost:8080/dumpSpringBoot actuator deserves a dedicated blog post to cover its vast number of features, I will cover it in my upcoming posts. I hope this article provides some basic introduction to SpringBoot and how it simplifies the Spring application development. More on SpringBoot in upcoming articles.Reference: SpringBoot: Introducing SpringBoot from our JCG partner Siva Reddy at the My Experiments on Technology blog....
jboss-hornetq-logo

Evaluating persistent, replicated message queues

Message queues are useful in a number of situations; for example when we want to execute a task asynchronously, we enqueue it and some executor eventually completes it. Depending on the use case, the queues can give various guarantees of message persistence and delivery. For some use-cases, it is enough to have an in-memory message queue. For others, we want to be sure that once the message send completes, it is persistently enqueued and will be eventually delivered, despite node or system crashes. To be really sure that messages are not lost, we will be looking at queues which:    persist messages to disk replicate messages across the networkIdeally, we want to have 3 identical, replicated nodes containing the message queue. There is a number of open-source messaging projects available, but only a handful supports both persistence and replication. We’ll evaluate the performance and characteristics of 4 message queues:Amazon SQS Mongo DB RabbitMq HornetQWhile SQS isn’t an open-source messaging system, it matches the requirements and I’ve recently benchmarked it, so it will be interesting to compare self-hosted solutions with an as-a-service one. MongoDB isn’t a queue of course, but a document-based NoSQL database, however using some of its mechanisms it is very easy to implement a message queue on top of it. By no way this aims to be a comprehensive overview, just an evaluation of some of the projects. If you know of any other messaging systems, which provide durable, replicated queues, let me know! Update: As many readers pointed out (thx @tlockney, @Evanfchan, @conikeec and others), Kafka is missing! While it works a bit differently (consumers are stateful – clustered – they keep their offset), Kafka supports point-to-point messaging where each message is consumed by one consumer, so stay tuned for an updated version of the blog, this time with Kafka! Testing methodology All sources for the tests are available on GitHub. The tests run a variable number of nodes (1-8); each node either sends or receives messages, using a variable number of threads (1-25), depending on the concrete test setup. Each Sender thread tries to send the given number of messages as fast as possible, in batches of random size between 1 and 10 messages. For some queues, we’ll also evaluate larger batches, up to 100 or 1000 messages. After sending all messages, the sender reports the number of messages sent per second. The Receiver tries to receive messages (also in batches), and after receiving them, acknowledges their delivery (which should cause the message to be removed from the queue). When no messages are received for a minute, the receiver thread reports the number of messages received per second.The queues have to implement the Mq interface. The methods should have the following characteristics:send should be synchronous, that is when it completes, we want to be sure (what sure means exactly may vary) that the messages are sent receive should receive messages from the queue and block them; if the node crashes, the messages should be returned to the queue and re-delivered ack should acknowledge delivery and processing of the messages. Acknowledgments can be asynchronous, that is we don’t have to be sure that the messages really got deleted.The model above describes an at-least-once message delivery model. Some queues offer also other delivery models, but we’ll focus on that one to compare possibly similar things. We’ll be looking at how fast (in terms of throughput) we can send and receive messages using a single 2 or 3 node message queue cluster. Mongo Mongo has two main features which make it possible to easily implement a durable, replicated message queue on top of it: very simple replication setup (we’ll be using a 3-node replica set), and various document-level atomic operations, like find-and-modify. The implementation is just a handful of lines of code; take a look at MongoMq. We are also able to control the guarantees which send gives us by using an appropriate write concern when writing new messages:WriteConcern.ACKNOWLEDGED (previously SAFE) ensures that once a send completes, the messages have been written to disk (though it’s not a 100% durability guarantee, as the disk may have its own write caches) WriteConcern.REPLICA_ACKNOWLEDGED ensures that a message is written to the majority of the nodes in the clusterThe main downside of the Mongo-based queue is that:messages can’t be received in bulk – the find-and-modify operation only works on a single document at a time when there’s a lot of connections trying to receive messages, the collection will encounter a lot of contention, and all operations are serialised.And this shows in the results: sends are faster then receives. But overall the performance is quite good! A single-thread, single-node setup achieves 7 900 msgs/s sent and 1 900 msgs/s received. The maximum send throughput with multiple thread/nodes that I was able to achieve is about 10 500 msgs/s, while the maximum receive rate is 3 200 msgs/s, when using the “safe” write concern.Threads Nodes Send msgs/s Receive msgs/s1 1 7 968,60 1 914,055 1 9 903,47 3 149,0025 1 10 903,00 3 266,831 2 9 569,99 2 779,875 2 10 078,65 3 112,5525 2 7 930,50 3 014,00  If we wait for the replica to acknowledge the writes (instead of just one node), the send throughput falls to 6 500 msgs/s, and the receive to about 2 900 msgs/s.Threads Nodes Send msgs/s Receive msgs/s1 1 1 489,21 1 483,691 2 2 431,27 2 421,015 2 6 333,10 2 913,9025 2 6 550,00 2 841,00  In my opinion, not bad for a very straightforward queue implementation on top of Mongo. SQS SQS is pretty well covered in my previous blog, so here’s just a short recap. SQS guarantees that if a send completes, the message is replicated to multiple nodes. It also provides at-least-once delivery guarantees. We don’t really know how SQS is implemented, but it most probably spreads the load across many servers, so including it here is a bit of an unfair competition: the other systems use a single replicated cluster, while SQS can employ multiple replicated clusters and route/balance the messages between them. But since we have the results, let’s see how it compares. A single thread on single node achieves 430 msgs/s sent and the same number of msgs received. These results are not impressive, but SQS scales nicely both when increasing the number of threads, and the number of nodes. On a single node, with 50 threads, we can send up to 14 500 msgs/s, and receive up to 4 200 msgs/s. On an 8-node cluster, these numbers go up to 63 500 msgs/s sent, and 34 800 msgs/s received.RabbitMQ RabbitMQ is one of the leading open-source messaging systems. It is written in Erlang, implements AMQP and is a very popular choice when messaging is involved. It supports both message persistence and replication, with well documented behaviour in case of e.g. partitions. We’ll be testing a 3-node Rabbit cluster. To be sure that sends complete successfully, we’ll be using publisher confirms, a Rabbit extension to AMQP. The confirmations are cluster-wide, so this gives us pretty strong guarantees: that messages will be both written to disk, and replicated to the cluster (see the docs). Such strong guarantees probably explain the poor performance. A single-thread, single-node gives us 310 msgs/s sent&received. This scales nicely as we add nodes, up to 1 600 msgs/s:The RabbitMq implementation of the Mq interface is again pretty straightforward. We are using the mentioned publisher confirms, and setting the quality-of-service when receiving so that at most 10 messages are delivered unconfirmed. Interestingly, increasing the number of threads on a node doesn’t impact the results. It may be because I’m incorrectly using the Rabbit API, or maybe it’s just the way Rabbit works. With 5 sending threads on a single node, the throughput increases just to 410 msgs/s. Things improve if we send messages in batches up to 100 or 1000, instead of 10. In both cases, we can get to 3 300 msgs/s sent&received, which seems to be the maximum that Rabbit can achieve. Results for batches up to 100:Threads Nodes Send msgs/s Receive msgs/s1 1 1 829,63 1 811,141 2 2 626,16 2 625,851 4 3 158,46 3 124,921 8 3 261,36 3 226,40And for batches up to 1000:Threads Nodes Send msgs/s Receive msgs/s1 1 3 181,08 2 549,451 2 3 307,10 3 278,291 4 3 566,72 3 533,921 8 3 406,72 3 377,68HornetQ HornetQ, written by JBoss and part of the JBossAS (implements JMS) is a strong contender. Since some time it supports over-the-network replication using live-backup pairs. I tried setting up a 3-node cluster, but it seems that data is replicated only to one node. Hence here we will be using a two-node cluster. This raises a question on how partitions are handled; by default the backup server won’t automatically fail-over, the operator must do that (turn the backup server into a live one). That’s certainly a valid way of handling partitions, but usually not the preferred one. It is possible to add configuration to automatically detect that the primary died, but then we can easily end up with two live servers, and that rises the question what happens with the data on both primaries when the connection is re-established. Overall, the replication support and documentation is worse than for Mongo and Rabbit. Also, as far as I understand the documentation (but I think it isn’t stated clearly anywhere), replication is asynchronous, meaning that even though we send messages in a transaction, once the transaction commits, we can be sure that messages are written only on the primary node’s journal. That is a weaker guarantee than in Rabbit, and corresponds to Mongo’s safe write concern. The HornetMq implementation uses the core Hornet API. For sends, we are using transactions, for receives we rely on the internal receive buffers and turn off blocking confirmations (making them asynchronous). Interestingly, we can only receive one message at a time before acknowledging it, otherwise we get exceptions on the server. But this doesn’t seem to impact performance. Speaking of performance, it is very good! A single-node, single-thread setup achieves 1 100 msgs/s. With 25 threads, we are up to 12 800 msgs/s! And finally, with 25 threads and 4 nodes, we can achieve 17 000 msgs/s.Threads Nodes Send msgs/s Receive msgs/s1 1 1 108,38 1 106,685 1 4 333,13 4 318,2525 1 12 791,83 12 802,421 2 2 095,15 2 029,995 2 7 855,75 7 759,4025 2 14 200,25 13 761,751 4 3 768,28 3 627,025 4 11 572,10 10 708,7025 4 17 402,50 16 160,50  One final note: when trying to send messages using 25 threads in bulks of up to 1000, I once got into a situation where the backup considered the primary dead even though it was working, and another time when the sending failed because the “address was blocked” (in other words, queue was full and couldn’t fit in memory), even though the receivers worked all the time. Maybe that’s due to GC? Or just the very high load? In summary As always, which message queue you choose depends on specific project requirements. All of the above solutions have some good sides:SQS is a service, so especially if you are using the AWS cloud, it’s an easy choice: good performance and no setup required if you are using Mongo, it is easy to build a replicated message queue on top of it, without the need to create and maintain a separate messaging cluster if you want to have high persistence guarantees, RabbitMQ ensures replication across the cluster and on disk on message send. finally, HornetQ has the best performanceWhen looking only at the throughput, HornetQ is a clear winner (unless we include SQS with multiple nodes, but as mentioned, that would be unfair):There are of course many other aspects besides performance, which should be taken into account when choosing a message queue, such as administration overhead, partition tolerance, feature set regarding routing, etc. Do you have any experiences with persistent, replicated queues? Or maybe you are using some other messaging solutions?Reference: Evaluating persistent, replicated message queues from our JCG partner Adam Warski at the Blog of Adam Warski blog....
java-logo

If BigDecimal is the answer, it must have been a strange question

Overview Many developers have determined that BigDecimal is the only way to deal with money.  Often they site that by replacing double with BigDecimal, they fixed a bug or ten.  What I find unconvincing about this is that perhaps they could have fixed the bug in the handling of double and that the extra overhead of using BigDecimal. My comparison, when asked to improve the performance of a financial application, I know at some time we will be removing BigDecimal if it is there. (It is usually not the biggest source of delays, but as we fix the system it moves up to the worst offender).   BigDecimal is not an improvement BigDecimal has many problems, so take your pick, but an ugly syntax is perhaps the worst sin.BigDecimal syntax is an unnatural. BigDecimal uses more memory BigDecimal creates garbage BigDecimal is much slower for most operations (there are exceptions)The following JMH benchmark demonstrates two problems with BigDecimal, clarity and performance. The core code takes an average of two values. The double implementation looks like this. Note: the need to use rounding. mp[i] = round6((ap[i] + bp[i]) / 2); The same operation using BigDecimal is not only long, but there is lots of boiler plate code to navigatemp2[i] = ap2[i].add(bp2[i]) .divide(BigDecimal.valueOf(2), 6, BigDecimal.ROUND_HALF_UP);Does this give you different results? double has 15 digits of accuracy and the numbers are far less than 15 digits. If these prices had 17 digits, this would work, but nor work the poor human who have to also comprehend the price (i.e. they will never get incredibly long). Performance If you have to incurr coding overhead, usually this is done for performance reasons, but this doesn’t make sense here.Benchmark Mode Samples Score Score error Unitso.s.MyBenchmark.bigDecimalMidPrice thrpt 20 23638.568 590.094 ops/so.s.MyBenchmark.doubleMidPrice thrpt 20 123208.083 2109.738 ops/sConclusion If you don’t know how to use round in double, or your project mandates BigDecimal, then use BigDecimal. But if you have choice, don’t just assume that BigDecimal is the right way to go. The codeimport org.openjdk.jmh.annotations.Benchmark; import org.openjdk.jmh.annotations.Scope; import org.openjdk.jmh.annotations.State; import org.openjdk.jmh.runner.Runner; import org.openjdk.jmh.runner.RunnerException; import org.openjdk.jmh.runner.options.Options; import org.openjdk.jmh.runner.options.OptionsBuilder;import java.math.BigDecimal; import java.util.Random;@State(Scope.Thread) public class MyBenchmark { static final int SIZE = 1024; final double[] ap = new double[SIZE]; final double[] bp = new double[SIZE]; final double[] mp = new double[SIZE];final BigDecimal[] ap2 = new BigDecimal[SIZE]; final BigDecimal[] bp2 = new BigDecimal[SIZE]; final BigDecimal[] mp2 = new BigDecimal[SIZE];public MyBenchmark() { Random rand = new Random(1); for (int i = 0; i < SIZE; i++) { int x = rand.nextInt(200000), y = rand.nextInt(10000); ap2[i] = BigDecimal.valueOf(ap[i] = x / 1e5); bp2[i] = BigDecimal.valueOf(bp[i] = (x + y) / 1e5); } doubleMidPrice(); bigDecimalMidPrice(); for (int i = 0; i < SIZE; i++) { if (mp[i] != mp2[i].doubleValue()) throw new AssertionError(mp[i] + " " + mp2[i]); } }@Benchmark public void doubleMidPrice() { for (int i = 0; i < SIZE; i++) mp[i] = round6((ap[i] + bp[i]) / 2); }static double round6(double x) { final double factor = 1e6; return (long) (x * factor + 0.5) / factor; }@Benchmark public void bigDecimalMidPrice() { for (int i = 0; i < SIZE; i++) mp2[i] = ap2[i].add(bp2[i]) .divide(BigDecimal.valueOf(2), 6, BigDecimal.ROUND_HALF_UP); }public static void main(String[] args) throws RunnerException { Options opt = new OptionsBuilder() .include(".*" + MyBenchmark.class.getSimpleName() + ".*") .forks(1) .build();new Runner(opt).run(); } }Reference: If BigDecimal is the answer, it must have been a strange question from our JCG partner Peter Lawrey at the Vanilla Java blog....
android-logo

Android RecyclerView

RecyclerView is one of the two UI widgets introduced by the support library in Android L. In this post I will describe how we can use it and what’s the difference between this widget and the “classic” ListView. This new widget is more flexible than the ListView but it introduces some complexities. As we are used RecyclerView introduces a new Adapter that must be used to represent the underlying data in the widget. This new adapter is called RecyclerView.Adapter. To use this widget you have to add latest v7 support library.       Introduction We know already that in the ListView to increase the performance we have to use the ViewHolder pattern. This is simply a java class that holds the references to the widget in the row layout of the ListView (for example TextView, ImageView and so on). Using this pattern we avoid to call several times findById method to get the UI widget reference making the ListView scrolling smoother. Even if this pattern was suggested as best-practice we could implement our Adapter without using this pattern. RecyclerView enforces this pattern making it the core of this UI widget and we have to use it in our Adapter. The Adapter: RecyclerView.Adapter If we want to show the information in a ListView or in the new RecyclerView we have to use an Adapter. This component stands behind the UI widget and determines how the rows, in the ListView, have to be rendered and what information to show. Also in the RecyclerView we have to use an Adapter: public class MyRecyclerAdapter extends RecyclerView.Adapter<MyRecyclerAdapter.MyHolder> { .... } where MyHolder is our implementation of ViewHolder pattern. We can suppose, we have a simple row layout in our RecyclerView: <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" android:orientation="vertical" android:layout_width="match_parent" android:layout_height="match_parent"><TextView android:layout_width="wrap_content" android:layout_height="wrap_content" android:id="@+id/txt1"/><TextView android:layout_width="wrap_content" android:layout_height="wrap_content" android:id="@+id/txt2"/></LinearLayout> so our ViewHolder patter implementation is: public static class MyHolder extends RecyclerView.ViewHolder { protected TextView txt1; protected TextView txt2;private MyHolder(View v) { super(v); this.txt1 = (TextView) v.findViewById(R.id.txt1); this.txt2 = (TextView) v.findViewById(R.id.txt2); } } As you can notice, the lookup process (findViewById) is made in the view holder instead of in getView method. Now we have to implement some important method in our adapter to make it working properly. There are two important methods we have to override:onCreateViewHolder(ViewGroup viewGroup, int i) onBindViewHolder(MyHolder myHolder, int i)OnCreateViewHolder is called whenever a new instance of View Holder class must be instantiated, so this method becomes: @Override public MyHolder onCreateViewHolder(ViewGroup viewGroup, int i) { numCreated++; Log.d("RV", "OncreateViewHolder ["+numCreated+"]"); View v = LayoutInflater.from(viewGroup.getContext()).inflate(R.layout.row_layout, null); MyHolder mh = new MyHolder(v); return mh; } as you can notice, the method returns an instance of our View holder implementation, while the second method is called when the SO binds the view with the data, so we set the UI widget content to the data values: @Override public void onBindViewHolder(MyHolder myHolder, int i) { Log.d("RV", "OnBindViewHolder"); Item item = itemList.get(i); myHolder.txt1.setText(item.name); myHolder.txt2.setText(item.descr); } Notice that we don’t make a lookup but we simply use the UI widget reference stored in our view holder. RecyclerView Now we have our adapter we can create our RecyclerView: RecyclerView rv = (RecyclerView) findViewById(R.id.my_recycler_view); rv.setLayoutManager(new LinearLayoutManager(this)); MyRecyclerAdapter adapter = new MyRecyclerAdapter(createList()); rv.setAdapter(adapter); LinearLayoutManager is the “main” layout manager used to dispose items inside the RecyclerView. We can extend or implement our layout manager. Final considerations Running the example we obtain:The most interesting aspect is how many times the onCreateViewHolder is called compared to the number of items shown. If you look at the log you will find that the object created is 1/3 of the total number displayed.Reference: Android RecyclerView from our JCG partner Francesco Azzola at the Surviving w/ Android blog....
javaone-logo

Gearing up for JavaOne 2014 !

Hold that thought! Yeah…I wish I was presenting at Java One 2014 – but I am only worthy of doing that in my dreams right now! But nothing is stopping me from following Java One and tracking sessions/talks about my favorite topics. I am hoping Oracle would make the 2014 talks available online for mortals like us just like it did for the 2013 edition. I have already made my list of talks (see below) which I am going to pounce upon (once they are available)…Have You ?     Lessons Learned from Real-World Deployments of Java EE 7 – Arun Gupta talks dissects live Java EE 7 applications and shares his thoughts and learnings. 50 EJB 3 Best Practices in 50 Minutes – need I say more…? eBay, Connecting Buyers and Sellers Globally via JavaServer Faces Applied Domain-Driven Design Blueprints for Java EE - Have you been following and learning from the Cargo Tracker Java EE application already? You are going to love this talk by Reza and Vijay. JPA Gotchas and Best Practices: Lessons from Overstock.com – nothing like listening to a ‘real’ world example. RESTful Microservices – Never heard of this one! It’s about leveraging Jersey 2.0 to achieve what you haven’t until now! Java API for JSON Binding: Introduction and Update – Potential Java EE 8 candidate and JAXB’s better half ! Is Your Code Parallel-Ready? – Love hacking on Java 8 Stream API? Yes.. this one is for you! Java EE Game Changers – Tomitribe’s David Blevins will talk about the journey from J2EE to Java EE 7, provide insights into Java EE 8 candidates and much more! Programming with Streams in Java 8 – I don’t think I need to comment. Unorthodox Enterprise Practices – Thoughts on Java EE design choices with live coding.. By one of my favorites…Adam Bien. Inside the CERT Oracle Secure Coding Standard for Java RESTing on Your Laurels Will Get You Pwned – When the rest of the world in going ga-ga about REST, this talk will attempt to look at the dark side and empower us with skills to mitigate REST related vulnerabilities. Java EE 7 Batch Processing in the Real World – another live coding session using Java EE 7 standards! Scaling a Mobile Startup Through the Cloud: A True Story – The folks at CodenameOne reveal their secrets. The Anatomy of a Secure Web Application Using Java – developing and deploying secure Java EE web application in the cloud.. now who does not want to do that?! Dirty Data: Why XML and JSON Don’t Cut It and What You Can Do About It – I haven’t thought beyond XML and JSON, but apparently some guys have and you must listen to them! API Design Checklist - no comments needed. Java EE 7 and Spring 4: A Shootout — I have been waiting for this! The Path to CDI 2.0 – Antoine, the CDI guy at Red Hat talks about…of course…CDI ! This includes discussion about 1.1/1.2 along with what’s coming up in CDI 2.0. Five Keys for Securing Java Web Apps – looks like I am repeating myself here.. but too much security can never be bad.. right? Java EE 7 Recipes – Yeah! cooked up by Josh Juneau. There is still time…Pick and gear up for your favorite tracks. More details on the official JavaOne website. Until then…Happy Learning!Reference: Gearing up for JavaOne 2014 ! from our JCG partner Abhishek Gupta at the Object Oriented.. blog....
software-development-2-logo

You Probably Don’t Need a Message Queue

I’m a minimalist, and I don’t like to complicate software too early and unnecessarily. And adding components to a software system is one of the things that adds a significant amount of complexity. So let’s talk about message queues. Message Queues are systems that let you have fault-tolerant, distributed, decoupled, etc, etc. architecture. That sounds good on paper. Message queues may fit in several use-cases in your application. You can check this nice article about the benefits of MQs of what some use-cases might be. But don’t be hasty in picking an MQ because “decoupling is good”, for example. Let’s use an example – you want your email sending to be decoupled from your order processing.   So you post a message to a message queue, then the email processing system picks it up and sends the emails. How would you do that in a monolithic, single classpath application? Just make your order processing service depend on an email service, and call sendEmail(..) rather than sendToMQ(emailMessage). If you use MQ, you define a message format to be recognized by the two systems; if you don’t use an MQ you define a method signature. What is the practical difference? Not much, if any. But then you probably want to be able to add another consumer that does additional thing with a given message? And that might happen indeed, it’s just not for the regular project out there. And even if it is, it’s not worth it, compared to adding just another method call. Coupled – yes. But not inconveniently coupled. What if you want to handle spikes? Message queues give you the ability to put requests in a persistent queue and process all of them. And that is a very useful feature, but again it’s limited based on several factors – are your requests processed in the UI background, or require immediate response? The servlet container thread pool can be used as sort-of queue – response will be served eventually, but the user will have to wait (if the thread acquisition timeout is too small, requests will be dropped, though). Or you can use an in-memory queue for the heavier requests (that are handled in the UI background). And note that by default your MQ might not be highly-availably. E.g. if an MQ node dies, you lose messages. So that’s not a benefit over an in-memory queue in your application node. Which leads us to asynchronous processing – this is indeed a useful feature. You don’t want to do some heavy computation while the user is waiting. But you can use an in-memory queue, or simply start a new thread (a-la spring’s @Async annotation). Here comes another aspect – does it matter if a message is lost? If you application node, processing the request, dies, can you recover? You’ll be surprised how often it doesn’t actually matter, and you can function properly without guaranteeing all messages are processed. So, just asynchronously handling heavier invocations might work well. Even if you can’t afford to lose messages, the use-case when a message is put into a queue in order for another component to process it, there’s still a simple solution – the database. You put a row with a processed=false flag in the database. A scheduled job runs, picks all unprocessed ones and processes them asynchronously. Then, when processing is finished, set the flag to true. I’ve used this approach a number of times, including large production systems, and it works pretty well. And you can still scale your application nodes endlessly, as long as you don’t have any persistent state in them. Regardless of whether you are using an MQ or not. (Temporary in-memory processing queues are not persistent state). Why I’m trying to give alternatives to common usages of message queues? Because if chosen for the wrong reason, an MQ can be a burden. They are not as easy to use as it sounds. First, there’s a learning curve. Generally, the more separate integrated components you have, the more problems may arise. Then there’s setup and configuration. E.g. when the MQ has to run in a cluster, in multiple data centers (for HA), that becomes complex. High availability itself is not trivial – it’s not normally turned on by default. And how does your application node connect to the MQ? Via a refreshing connection pool, using a short-lived DNS record, via a load balancer? Then your queues have tons of configurations – what’s their size, what’s their behaviour (should consumers explicitly acknowledge receipt, should they explicitly acknowledge failure to process messages, should multiple consumers get the same message or not, should messages have TTL, etc.). Then there’s the network and message transfer overhead – especially given that people often choose JSON or XML for transferring messages. If you overuse your MQ, then it adds latency to your system. And last, but not least – it’s harder to track the program flow when analyzing problems. You can’t just see the “call hierarchy” in your IDE, because once you send a message to the MQ, you need to go and find where it is handled. And that’s not always as trivial as it sounds. You see, it adds a lot of complexity and things to take care of. Certainly MQs are very useful in some contexts. I’ve been using them in projects where they were really a good fit – e.g. we couldn’t afford to lose messages and we needed fast processing (so pinging the database wasn’t an option). I’ve also seen it being used in non-trivial scenarios, where we are using to for consuming messages on a single application node, regardless which node posts the message (pub/sub). And you can also check this stackoverflow question. And maybe you really need to have multiple languages communicate (but don’t want an ESB), or maybe your flow is getting so complex, that adding a new method call instead of a new message consumer is an overkill. So all I’m trying to say here is the trite truism “you should use the right tool for the job”. Don’t pick a message queue if you haven’t identified a real use for it that can’t be easily handled in a different, easier to setup and maintain manner. And don’t start with an MQ “just in case” – add it whenever you realize the actual need for it. Because probably, in the regular project out there, a message queue is not needed.Reference: You Probably Don’t Need a Message Queue from our JCG partner Bozhidar Bozhanov at the Bozho’s tech blog blog....
software-development-2-logo

Step By Step Path to Becoming a Great Software Developer

I get quite a few emails that basically say “how do I become a good / great software developer?” These kinds of emails generally tick me off, because I feel like when you ask this kind of question, you are looking for some magical potion you can take that will suddenly make you into a super developer. I suspect that very few people who email me asking this question really want to know how to become a great software developer, but are instead looking for a quick fix or an easy answer.   On the other hand, I think there are some genuinely sincere developers that just don’t even know how to formulate the right questions they need to ask to get to their desired future. I think those developers–especially the ones starting out–are looking for a step-by-step guide to becoming a great developer.I thought I would make an attempt, from my experience and the best of my knowledge, to offer up that step-by-step guide. Now, of course, I realize that there is no magical formula and that there are multiple paths to success, but I think what follows is a reasonable outline of steps someone starting out could take to reach a pretty high level of proficiency and be generally regarded as a good–perhaps even great–developer. Step 1: Pick one language, learn the basics Before we can run, we have to learn to walk. You walk by learning how to program in a single programming language. You don’t learn to walk by trying to learn 50 million things all at once and spreading yourself way too thin. Too many beginning programmers try and jump into everything all at once and don’t have the patience to learn a single programming language before moving forward. They think that they have to know all the hot new technologies in order to get a programming job. While it is true that you need to know more than just the basics of a single programming language, you have to start here, so you might as well focus. Pick a single programming language that you think you would be likely to base your career around. The programming language itself doesn’t matter all that much, since you should be thinking for the long term here. What I mean is you shouldn’t try and learn an “easy” programming language to start. Just learn whatever language you are interested in and could see yourself programming in for the next few years. You want to pick something that will have some lasting value. Once you’ve picked the programming language you are going to try and learn, try and find some books or tutorials that isolate that programming language. What I mean by this is that you don’t want to find learning materials that will teach you too much all at once. You want to find beginner materials that focus on just the language, not a full technology stack. As you read through the material or go through the tutorial you have picked out, make sure you actually write code. Do exercises if you can. Try out what you learned. Try to put things together and use every concept you learn about. Yes, this is a pain. Yes, it is easier to read a book cover-to-cover, but if you really want to learn you need to do. When you are writing code, try to make sure you understand what every line of code you write does. The same goes for any code you read. If you are exposed to code, slow down and make sure you understand it. Whatever you don’t understand, look up. Take the time to do this and you will not feel lost and confused all the time. Finally, expect to go through a book or tutorial three times before it clicks. You will not get “programming” on the first try–no one ever does. You need repeated exposure before you start to finally get it and can understand what is going on. Until then you will feel pretty lost, that is ok, it is part of the process. Just accept it and forge ahead. Step 2: Build something small Now that you have a basic understanding of a single programming language, it’s time to put that understanding to work and find out where your gaps are. The best way to do this is to try and build something. Don’t get too ambitious at this point–but also don’t be too timid. Pick an idea for an application that is simple enough that you can do it with some effort, but nothing that will take months to complete. Try to confine it to just the programming language as much as possible. Don’t try to do something full stack (meaning, using all the technologies from user interfaces all the way to databases)–although you’ll probably need to utilize some kind of existing framework or APIs. For your first real project you might want to consider copying something simple that already exists. Look for a simple application, like a To-Do list app and straight out try to copy it. Don’t let your design skills stand in the way of learning to code.I’d recommend creating a mobile application of some kind, since most mobile applications are small and pretty simple. Plus, learning mobile development skills is very useful as more and more companies are starting to need mobile applications. Today, you can build a mobile application in just about any language. There are many solutions that let you build an app for the different mobile OSes using a wide variety of programming languages. You could also build a small web application, but just try to not get too deep into a complex web development stack. I generally recommend starting with a mobile app, because web development has a higher cost to entry. To develop a web application you’ll need to at least know some HTML, probably some back-end framework and JavaScript. Regardless of what you choose to build, you are probably going to have to learn a little bit about some framework–this is good, just don’t get too bogged down into the details. For example, you can write a pretty simple Android application without having to really know a lot about all of the Android APIs and how Android works, just by following some simple tutorials. Just don’t waste too much time trying to learn everything about a framework. Learn what you need to know to get your project done. You can learn the details later. Oh, and this is supposed to be difficult. That is how you learn. You struggle to figure out how to do something, then you find the answer. Don’t skip this step. You’ll never reach a point as a software developer where you don’t have to learn things on the spot and figure things out as you go along. This is good training for your future. Step 3: Learn a framework Now it’s time to actually focus on a framework. By now you should have a decent grasp of at least one programming language and have some experience working with a framework for mobile or web applications. Pick a single framework to learn that will allow you to be productive in some environment. What kind of framework you choose to learn will be based on what kind of developer you want to become. If you want to be a web developer, you’ll want to learn a web development framework for whatever programming language you are programming in. If you want to become a mobile developer, you’ll need to learn a mobile os and the framework that goes with it. Try to go deep with your knowledge of the framework. This will take time, but invest the time to learn whatever framework you are using well. Don’t try to learn multiple frameworks right now–it will only split your focus. Think about learning the skills you need for a very specific job that you will get that will use that framework and the programming language you are learning. You can always expand your skills later. Step 4: Learn a database technology Most software developers will need to know some database technology as most series applications have a back-end database. So, make sure you do not neglect investing in this area. You will probably see the biggest benefit if you learn SQL–even if you plan on working with NoSQL database like MongoDB or Raven, learning SQL will give you a better base to work from. There are many more jobs out there that require knowledge of SQL than NoSQL. Don’t worry so much about the flavor of SQL. The different SQL technologies are similar enough that you shouldn’t have a problem switching between them if you know the basics of one SQL technology. Just make sure you learn the basics about tables, queries, and other common database operations. I’d recommend getting a good book on the SQL technology of your choice and creating a few small sample projects, so you can practice what you are learning–always practice what you are learning. You have sufficient knowledge of SQL when you can:Create tables Perform basics queries Join tables together to get data Understand the basics of how indexes work Insert, update and delete dataIn addition, you will want to learn some kind of object relational mapping technology (ORM). Which one you learn will depend on what technology stack you are working with. Look for ORM technologies that fit the framework you have learned. There might be a few options, so you best bet is to try to pick the most popular one. Step 5: Get a job supporting an existing system Ok, now you have enough skills and knowledge to get a basic job as a software developer. If you could show me that you understand the basics of a programming language, can work with a framework, understand databases and have built your own application, I would certainly want to hire you–as would many employers.The key here is not too aim to high and to be very specific. Don’t try and get your dream job right now–you aren’t qualified. Instead, try and get a job maintaining an existing code base that is built using the programming language and framework you have been learning. You might not be able to find an exact match, but the more specific you can be the better. Try to apply for jobs that are exactly matched to the technologies you have been learning. Even without much experience, if you match the skill-set exactly and you are willing to be a maintenance programmer, you should be able to find a job. Yes, this kind of job might be a bit boring. It’s not nearly as exciting as creating something new, but the purpose of this job is not to have fun or to make money, it is to learn and gain experience. Working on an existing application, with a team of developers, will help you to expand your skills and see how large software systems are structured. You might be fixing bugs and adding small features, but you’ll be learning and putting your skills into action. Pour your heart into this job. Learn everything you can. Do the best work possible. Don’t think about money, raises and playing political games–all that comes later–for now, just focus on getting as much meaningful productive work done as possible and expanding your skills. Step 6: Learn structural best practices Now it’s time to start becoming better at writing code. Don’t worry too much about design at this point. You need to learn how to write good clean code that is easy to understand and maintain. In order to do this, you’ll need to read a lot and see many examples of good code. Beef up your library with the following books:Code Complete Clean Code Refactoring Working Effectively With Legacy Code Programming Pearls - (do the exercises)Language specific structural books like:JavaScript: The Good Parts Effective Java Effective C#At this point you really want to focus your learning on the structural process of writing good code and working with existing systems. You should strive to be able to easily implement an algorithm in your programming language of choice and to do it in a way that is easy to read and understand. Step 7: Learn a second language At this point you will likely grow the most by learning a second programming language really well. You will no doubt, at this point, have been exposed to more than one programming language, but now you will want to focus on a new language–ideally one that is significantly different than the one you know. This might seem like an odd thing to do, but let me explain why this is so important. When you know only one programming language very well, it is difficult to understand what concepts in software development are unique to your programming language and which ones transcend a particular language or technology. If you spend time in a new language and programming environment, you’ll begin to see things in a new way. You’ll start to learn practicality rather than convention. As a new programmer, you are very likely to do things in a particular way without knowing why you are doing them that way. Once you have a second language and technology stack under your belt, you’ll start to see more of the why. Trust me, you’ll grow if you do this. Especially if you pick a language you hate. Make sure you build something with this new language. Doesn’t have to be anything large, but something of enough complexity to force you to scratch your head and perhaps bang it against the wall–gently. Step 8: Build something substantial Allright, now comes the true test to prove your software development abilities. Can you actually build something substantial on your own? If you are going to move on and have the confidence to get a job building, and perhaps even designing something for an employer, you need to know you can do it. There is no better way to know it than to do it. Pick a project that will use the full stack of your skills. Make sure you incorporate database, framework and everything else you need to build a complete application. This project should be something that will take you more than a week and require some serious thinking and design. Try to make it something you can charge money for so that you take it seriously and have some incentive to keep working on it. Just make sure you don’t drag it out. You still don’t want to get too ambitious here. Pick a project that will challenge you, but not one that will never be completed. This is a major turning point in your career. If you have the follow-through to finish this project, you’ll go far, if you don’t… well, I can’t make any promises. Step 9: Get a job creating a new system Ok, now it’s time to go job hunting again. By this point, you should have pretty much maxed out the benefit you can get from your current job–especially if it still involves only doing maintenance.It’s time to look for a job that will challenge you–but not too much. You still have a lot to learn, so you don’t want to get in too far over your head. Ideally, you want to find a job where you’ll get the opportunity to work on a team building something new. You might not be the architect of the application, but being involved in the creation of an application will help you expand your skills and challenge you in different ways than just maintaining an existing code base. You should already have some confidence with creating a new system, since you’ll have just finished creating a substantial system yourself, so you can walk into interviews without being too nervous and with the belief you can do the job. This confidence will make it much more likely that you’ll get whatever job you apply for. Make sure you make your job search focused again. Highlight a specific set of skills that you have acquired. Don’t try to impress everyone with a long list of irrelevant skills. Focus on the most important skills and look for jobs that exactly match them–or at least match them as closely as possible. Step 10: Learn design best practices Now it’s time to go from junior developer to senior developer. Junior developers maintain systems, senior developers build and design them. (This is a generalization, obviously. Some senior developers maintain systems.) You should be ready to build systems by now, but now you need to learn how to design them. You should focus your studies on design best practices and some advanced topics like:Design patterns Inversion of Control (IOC) Test Driven Development (TDD) Behavior Driven Development (BDD) Software development methodologies like: Agile, SCRUM, etc Message buses and integration patternsThis list could go on for quite some time–you’ll never be done learning and improving your skills in these areas. Just make sure you start with the most important things first–which will be highly dependent on what interests you and where you would like to take your career. Your goal here is to be able to not just build a system that someone else has designed, but to form your own opinions about how software should be designed and what kinds of architectures make sense for what kinds of problems. Step 11: Keep going At this point you’ve made it–well, you’ll never really “make it,” but you should be a pretty good software developer–maybe even “great.” Just don’t let it go to your head, you’ll always have something to learn. How long did it take you to get here? I have no idea. It was probably at least a few years, but it might have been 10 or more–it just depends on how dedicated you were and what opportunities presented themselves to you. A good shortcut is to try and always surround yourself with developers better than you are. What to do along the way There are a few things that you should be doing along the way as you are following this 10 step process. It would be difficult to list them in every step, so I’ll list them all briefly here: Teach - The whole time you are learning things, you should be teaching them as well. It doesn’t matter if you are a beginner or expert, you have something valuable to teach and besides, teaching is one of the best ways to learn. Document your process and journey, help others along the way. Market yourself - I think this is so important that I built an entire course around the idea. Learn how to market yourself and continually do it throughout your career. Figure out how to create a personal brand for yourself and build a reputation for yourself in the industry and you’ll never be at want for a job. You’ll get decide your own future if you learn to market yourself. It takes some work, but it is well worth it. You are reading this post from my effort to do it. Read - Never stop learning. Never stop reading. Always be working your way through a book. Always be improving yourself. Your learning journey is never done. You can’t ever know it all. If you constantly learn during your career, you’ll constantly surpass your peers. Do - Every stop along the way, don’t just learn, but do. Put everything you are learning into action. Set aside time to practice your skills and to write code and build things. You can read all the books on golfing that you want, but you’ll never be Tiger Woods if you don’t swing a golf club.Reference: Step By Step Path to Becoming a Great Software Developer from our JCG partner John Sonmez at the Making the Complex Simple blog....
gradle-logo

Getting Started with Gradle: Dependency Management

It is challenging, if not impossible, to create real life applications which don’t have any external dependencies. That is why dependency management is a vital part of every software project. This blog post describes how we can manage the dependencies of our projects with Gradle. We will learn to configure the used repositories and the required dependencies. We will also apply this theory to practice by implementing a simple example application. Let’s get started.   Additional Reading:Getting Started with Gradle: Introduction helps you to install Gradle, describes the basic concepts of a Gradle build, and describes how you can add functionality to your build by using Gradle plugins. Getting Started with Gradle: Our First Java Project describes how you can create a Java project by using Gradle and package your application to an executable jar file.Introduction to Repository Management Repositories are essentially dependency containers, and each project can use zero or more repositories. Gradle supports the following repository formats:Ivy repositories Maven repositories Flat directory repositoriesLet’s find out how we can configure each repository type in our build. Adding Ivy Repositories to Our Build We can add an Ivy repository to our build by using its url address or its location in the local file system. If we want to add an Ivy repository by using its url address, we have to add the following code snippet to the build.gradle file: repositories { ivy { url "http://ivy.petrikainulainen.net/repo" } } If we want to add an Ivy repository by using its location in the file system, we have to add the following code snippet to the build.gradle file: repositories { ivy { url "../ivy-repo" } } If you want to get more information about configuring Ivy repositories, you should check out the following resources:Section 50.6.6 Ivy Repositories of the Gradle User Guide The API documentation of the IvyArtifactRepositoryLet’s move on and find out how we can add Maven repositories to our build. Adding Maven Repositories to Our Build We can add a Maven repository to our build by using its url address or its location in the local file system. If we want to add a Maven repository by using its url, we have to add the following code snippet to the build.gradle file: repositories { maven { url "http://maven.petrikainulainen.net/repo" } } If we want to add a Maven repository by using its location in the file system, we have to add the following code snippet to the build.gradle file: repositories { maven { url "../maven-repo" } } Gradle has three “aliases” which we can use when we are adding Maven repositories to our build. These aliases are:The mavenCentral() alias means that dependencies are fetched from the central Maven 2 repository. The jcenter() alias means that dependencies are fetched from the Bintray’s JCenter Maven repository. The mavenLocal() alias means that dependencies are fetched from the local Maven repository.If we want to add the central Maven 2 repository in our build, we must add the following snippet to our build.gradle file: repositories { mavenCentral() } If you want to get more information about configuring Maven repositories, you should check out section 50.6.4 Maven Repositories of the Gradle User Guide. Let’s move on and find out how we can add flat directory repositories to our build. Adding Flat Directory Repositories to Our Build If we want to use flat directory repositories, we have to add the following code snippet to our build.gradle file: repositories { flatDir { dirs 'lib' } } This means that dependencies are searched from the lib directory. Also, if we want to, we can use multiple directories by adding the following snippet to the build.gradle file: repositories { flatDir { dirs 'libA', 'libB' } } If you want get more information about flat directory repositories, you should check out the following resources:Section 50.6.5 Flat directory repository of the Gradle User Guide Flat Dir Repository post to the gradle-user mailing listLet’s move on and find out how we can manage the dependencies of our project with Gradle. Introduction to Dependency Management After we have configured the repositories of our project, we can declare its dependencies. If we want to declare a new dependency, we have to follow these steps:Specify the configuration of the dependency. Declare the required dependency.Let’s take a closer look at these steps. Grouping Dependencies into Configurations In Gradle dependencies are grouped into a named set of dependencies. These groups are called configurations, and we use them to declare the external dependencies of our project. The Java plugin specifies several dependency configurations which are described in the following:The dependencies added to the compile configuration are required when our the source code of our project is compiled. The runtime configuration contains the dependencies which are required at runtime. This configuration contains the dependencies added to the compile configuration. The testCompile configuration contains the dependencies which are required to compile the tests of our project. This configuration contains the compiled classes of our project and the dependencies added to the compile configuration. The testRuntime configuration contains the dependencies which are required when our tests are run. This configurations contains the dependencies added to compile, runtime, and testCompile configurations. The archives configuration contains the artifacts (e.g. Jar files) produced by our project. The default configuration group contains the dependencies which are required at runtime.Let’s move on and find out how we can declare the dependencies of our Gradle project. Declaring the Dependencies of a Project The most common dependencies are called external dependencies which are found from an external repository. An external dependency is identified by using the following attributes:The group attribute identifies the group of the dependency (Maven users know this attribute as groupId). The name attribute identifies the name of the dependency (Maven users know this attribute as artifactId). The version attribute specifies the version of the external dependency (Maven users know this attribute as version).These attributes are required when you use Maven repositories. If you use other repositories, some attributes might be optional. For example, if you use a flat directory repository, you might have to specify only name and version. Let’s assume that we have to declare the following dependency:The group of the dependency is ‘foo’. The name of the dependency is ‘foo’. The version of the dependency is 0.1. The dependency is required when our project is compiled.We can declare this dependency by adding the following code snipped to the build.gradle file: dependencies { compile group: 'foo', name: 'foo', version: '0.1' } We can also declare the dependencies of our project by using a shortcut form which follows this syntax: [group]:[name]:[version]. If we want to use the shortcut form, we have to add the following code snippet to the build.gradle file: dependencies { compile 'foo:foo:0.1' } We can also add multiple dependencies to the same configuration. If we want to use the “normal” syntax when we declare our dependencies, we have to add the following code snippet to the build.gradle file: dependencies { compile ( [group: 'foo', name: 'foo', version: '0.1'], [group: 'bar', name: 'bar', version: '0.1'] ) } On the other hand, if we want to use the shortcut form, the relevant part of the build.gradle file looks as follows: dependencies { compile 'foo:foo:0.1', 'bar:bar:0.1' } It is naturally possible to declare dependencies which belong to different configurations. For example, if we want to declare dependencies which belong to the compile and testCompile configurations, we have to add the following code snippet to the build.gradle file: dependencies { compile group: 'foo', name: 'foo', version: '0.1' testCompile group: 'test', name: 'test', version: '0.1' } Again, it is possible to use the shortcut form. If we want to declare the same dependencies by using the shortcut form, the relevant part of the build.gradle file looks as follows: dependencies { compile 'foo:foo:0.1' testCompile 'test:test:0.1' } You can get more information about declaring your dependencies by reading the section 50.4 How to declare your dependencies of Gradle User Guide. We have now learned the basics of dependency management. Let’s move on and implement our example application. Creating the Example Application The requirements of our example application are described in thefollowing:The build script of the example application must use the Maven central repository. The example application must write the received message to log by using Log4j. The example application must contain unit tests which ensure that the correct message is returned. These unit tests must be written by using JUnit. Our build script must create an executable jar file.Let’s find out how we can fulfil these requirements. Configuring the Repositories of Our Build One of the requirements of our example application was that its build script must use the Maven central repository. After we have configured our build script to use the Maven central repository, its source code looks as follows (The relevant part is highlighted): apply plugin: 'java'repositories { mavenCentral() }jar { manifest { attributes 'Main-Class': 'net.petrikainulainen.gradle.HelloWorld' } } Let’s move on and declare the dependencies of our example application. Declaring the Dependencies of Our Example Application We have to declare two dependencies in the build.gradle file:Log4j (version 1.2.17) is used to write the received message to the log. JUnit (version 4.11) is used to write unit tests for our example application.After we have declared these dependencies, the build.gradle file looks as follows (the relevant part is highlighted): apply plugin: 'java'repositories { mavenCentral() }dependencies { compile 'log4j:log4j:1.2.17' testCompile 'junit:junit:4.11' }jar { manifest { attributes 'Main-Class': 'net.petrikainulainen.gradle.HelloWorld' } } Let’s move on and write some code. Writing the Code In order to fulfil the requirements of our example application, “we have to over-engineer it”. We can create the example application by following these steps:Create a MessageService class which returns the string ‘Hello World!’ when its getMessage() method is called. Create a MessageServiceTest class which ensures that the getMessage() method of the MessageService class returns the string ‘Hello World!’. Create the main class of our application which obtains the message from a MessageService object and writes the message to log by using Log4j. Configure Log4j.Let’s go through these steps one by one. First, we have to create a MessageService class to the src/main/java/net/petrikainulainen/gradle directory and implement it. After we have do this, its source code looks as follows: package net.petrikainulainen.gradle;public class MessageService {public String getMessage() { return "Hello World!"; } } Second, we have create a MessageServiceTest to the src/main/test/net/petrikainulainen/gradle directory and write a unit test to the getMessage() method of the MessageService class. The source code of the MessageServiceTest class looks as follows: package net.petrikainulainen.gradle;import org.junit.Before; import org.junit.Test;import static org.junit.Assert.assertEquals;public class MessageServiceTest {private MessageService messageService;@Before public void setUp() { messageService = new MessageService(); }@Test public void getMessage_ShouldReturnMessage() { assertEquals("Hello World!", messageService.getMessage()); } } Third, we have create a HelloWorld class to the src/main/java/net/petrikainulainen/gradle directory. This class is the main class of our application. It obtains the message from a MessageService object and writes it to a log by using Log4j. The source code of the HelloWorld class looks as follows: package net.petrikainulainen.gradle;import org.apache.log4j.Logger;public class HelloWorld {private static final Logger LOGGER = Logger.getLogger(HelloWorld.class);public static void main(String[] args) { MessageService messageService = new MessageService();String message = messageService.getMessage(); LOGGER.info("Received message: " + message); } } Fourth, we have to configure Log4j by using the log4j.properties which is found from the src/main/resources directory. The log4j.properties file looks as follows: log4j.appender.Stdout=org.apache.log4j.ConsoleAppender log4j.appender.Stdout.layout=org.apache.log4j.PatternLayout log4j.appender.Stdout.layout.conversionPattern=%-5p - %-26.26c{1} - %m\nlog4j.rootLogger=DEBUG,Stdout That is it. Let’s find out how we can run the tests of our example application. Running the Unit Tests We can run our unit test by using the following command: gradle test When our test passes, we see the following output: > gradle test :compileJava :processResources :classes :compileTestJava :processTestResources :testClasses :testBUILD SUCCESSFULTotal time: 4.678 secs However, if our unit test would fail, we would see the following output (the interesting section is highlighted): > gradle test :compileJava :processResources :classes :compileTestJava :processTestResources :testClasses :testnet.petrikainulainen.gradle.MessageServiceTest > getMessage_ShouldReturnMessageFAILED org.junit.ComparisonFailure at MessageServiceTest.java:221 test completed, 1 failed :test FAILEDFAILURE: Build failed with an exception.* What went wrong: Execution failed for task ':test'. > There were failing tests. See the report at: file:///Users/loke/Projects/Java/Blog/gradle-examples/dependency-management/build/reports/tests/index.html* Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.BUILD FAILEDTotal time: 4.461 secs As we can see, if our unit tests fails, the describes:which tests failed. how many tests were run and how many tests failed. the location of the test report which provides additional information about the failed (and passed) tests.When we run our unit tests, Gradle creates test reports to the following directories:The build/test-results directory contains the raw data of each test run. The build/reports/tests directory contains a HTML report which describes the results of our tests.The HTML test report is very useful tool because it describes the reason why our test failed. For example, if our unit test would expect that the getMessage() method of the MessageService class returns the string ‘Hello Worl1d!’, the HTML test report of that test case would look as follows:Let’s move on and find out how we can package and run our example application. Packaging and Running Our Example Application We can package our application by using one of these commands: em>gradle assembly or gradle build. Both of these commands create the dependency-management.jar file to the build/libs directory. When run our example application by using the command java -jar dependency-management.jar, we see the following output: > java -jar dependency-management.jar Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/log4j/Logger at net.petrikainulainen.gradle.HelloWorld.<clinit>(HelloWorld.java:10) Caused by: java.lang.ClassNotFoundException: org.apache.log4j.Logger at java.net.URLClassLoader$1.run(URLClassLoader.java:372) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:360) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 1 more The reason for this exception is that the Log4j dependency isn’t found from the classpath when we run our application. The easiest way to solve this problem is to create a so called “fat” jar file. This means that we will package the required dependencies to the created jar file. After we have followed the instructions given in the Gradle cookbook, our build script looks as follows (the relevant part is highlighted): apply plugin: 'java'repositories { mavenCentral() }dependencies { compile 'log4j:log4j:1.2.17' testCompile 'junit:junit:4.11' }jar { from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } } manifest { attributes 'Main-Class': 'net.petrikainulainen.gradle.HelloWorld' } } We can now run the example application (after we have packaged it) and as we can see, everything is working properly: > java -jar dependency-management.jar INFO - HelloWorld - Received message: Hello World! That is all for today. Let’s summarize what we learned from this blog post. Summary This blog post has taught us four things:We learned how we can configure the repositories used by our build. We learned how we can declare the required dependencies and group these dependencies into configurations. We learned that Gradle creates a HTML test report when our tests are run. We learned how we can create a so called “fat” jar file.If you want to play around with the example application of this blog post, you can get it from Github.Reference: Getting Started with Gradle: Dependency Management from our JCG partner Petri Kainulainen at the Petri Kainulainen blog....
Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.

Sign up for our Newsletter

20,709 insiders are already enjoying weekly updates and complimentary whitepapers! Join them now to gain exclusive access to the latest news in the Java world, as well as insights about Android, Scala, Groovy and other related technologies.

As an extra bonus, by joining you will get our brand new e-books, published by Java Code Geeks and their JCG partners for your reading pleasure! Enter your info and stay on top of things,

  • Fresh trends
  • Cases and examples
  • Research and insights
  • Two complimentary e-books