Featured FREE Whitepapers

What's New Here?

java-logo

It’s All About Tests – Part 1

This post is the first of a series of three.Mindset of testing Techniques Tools and TipsThe Mindset Testing code is something that needs to be learned. It takes time to absorb how to do it well. It’s a craft that one should always practice and improve.   Back in the old days, developers did not test, they checked their code. Here’s a nice twit about it: Checking: code does what the coder intends it to do. Testing: code does what the customer needs it to do. #agile #tdd #bdd — Neil Killick (@neil_killick) November 7, 2014Today we have many tools and techniques to work with. XUnit frameworks, mock frameworks, UI automation, TDD, XP… But I believe that testing starts with the mind. State of mind. Why Testing Should I really answer that? Tests are your code harness and security for quality. Tests tell the story of your code. They prove that something works. They give immediate feedback if something went wrong. Working with tests correctly makes you more efficient and effective. You debug less and probably have less bugs, therefore you have more time to do actual work. Your design will be better (more about it later) and maintainable. You feel confident changing your code (refactor). More about it later. It reduces stress, as you are more confident with your code. What to Test I say everything. Perhaps you will skip the lowest parts of your system. The parts that reads/writes to the file system or the DB or communicate some external service. But even these parts can be tested. And they should. In following blogs I will describe some techniques how to do that. Test even the smallest thing. For example, if you have a DTO and you decide that a certain field will be initialized with some value, then make a test that only instantiate this class and then verify (assert) the expected value (and yes, I know, some parts really cannot be tested. but they should remain minimal). SRP Single Responsibility Principle. This is how I like to refer to the point that a test needs to check one thing. If it’s a unit test, then it should test one behavior of your method / class. Different behavior should be tested in a different test. If it’s a higher level of test (integration, functional, UI), then the same principle applies. Test one flow of the system. Test a click. Test adding elements to DB correctly, but not deleting in the same test. Isolation Isolated test helps us understand exactly what went wrong. Developing isolated test helps us concentrate on one problem at a time. One aspect of isolation is related to the SRP. When you test something, isolate the tested code from other part (dependencies). That way you test only that part of the code. If the test fails, you know were it was. If you have many dependencies in the test, it is much harder to understand what the actual cause of failure was. But isolation means other things as well. It means that no test would interfere another. It means that the running order of the tests doesn’t matter. For a unit test, it means that you don’t need a DB running (or internet connection for that matter). It means that you can run your tests concurrently without one interfere the other (maven allows exactly this). If you can’t do it (example: DB issues), then your tests are not isolated. Test Smells When the test is too hard to understand / maintain, don’t get mad on it! Say:thank you very much, my dear test, for helping me improve the codeIf it is too complicated to setup environment for the test, then probably the unit being tested has too many dependencies. If after running a method under test, you need to verify many aspects (verify, assert, etc.), the method probably does too much. The test can be your best friend for code improvement. Usually a really complicated test code means less structured production code. I usually see correlation between complicated test and code that doesn’t follow the SRP, or any other DOLID principles. Testable Code This is one of my favorites. Whenever I do code review I ask the other person: “How are you going to test it?”, “How do you know it works?” Whenever I code, I ask myself the same question. “How can I test this piece of code?” In my experience, thinking always on how to create testable code, yields much better design. The code “magically” has more patterns, less duplication, better OOD and behaves SOLIDly. Forcing yourself to constantly test your code, makes you think. It helps divide big, complicated problem into many (or few) smaller, more trivial ones. If your code is testable and tested, you have more confident on it. Confident on the behavior and confident to change it. Refactor it. Refactoring This item can be part of the why. It can be also part of the techniques. But I decided to give it special attention. Refactoring is part of the TDD cycle (but not only). When you have tests, you can be confident doing refactoring. I think that you need to “think about refactoring” while developing. Similar to “think how to produce testable code”. When thinking refactoring, testing comes along. Refactoring is also state of mind. Ask yourself: “Is the code I produced clean enough? Can I improve it?” (BTW, know when to stop…) This was the first post of a series of posts about testing. The following post will be about some techniques and approaches for testing.Reference: It’s All About Tests – Part 1 from our JCG partner Eyal Golan at the Learning and Improving as a Craftsman Developer blog....
apache-lucene-logo

Apache Lucene 5.0.0 is coming!

At long last, after a strong series of 4.x feature releases, most recently 4.10.2, we are finally working towards another major Apache Lucene release! There are no promises for the exact timing (it’s done when it’s done!), but we already have a volunteer release manager (thank you Anshum!). A major release in Lucene means all deprecated APIs (as of 4.10.x) are dropped, support for 3.x indices is removed while the numerous 4.x index formats are still supported for index backwards compatibility, and the 4.10.x branch becomes our bug-fix only release series (no new features, no API changes). 5.0.0 already contains a number of exciting changes, which I describe below, and they are still rolling in with ongoing active development. Stronger index safety Many of the 5.0.0 changes are focused on providing stronger protection against index corruption. All file access now uses Java’s NIO.2 APIs, giving us better error handling (e.g., Files.delete returns a meaningful exception) along with atomic rename for safer commits, reducing the risk of hideous “your entire index is gone” bugs like this doozie. Lucene’s replication module, along with distributed servers on top of Lucene such as Elasticsearch or Solr, must copy index files from one place to another. They do this for backup purposes (e.g., snapshot and restore), for migrating or recovering a shard from one node to another or when adding a new replica. Such replicators try to be incremental, so that if the same file name is present, with the same length and checksum, it will not be copied again. Unfortunately, these layers sometimes have subtle bugs (they are complex!). Thanks to checksums (added in 4.8.0), Lucene already detects if the replicator caused any bit-flips while copying, and this revealed a long standing nasty bug in the compression library Elasticsearch uses. With 5.0.0 we take this even further and now detect if whole files were copied to the wrong file name, by assigning a unique id to every segment and commit (segments_N file). Each index file now records the segment id in its header, and then these ids are cross-checked when the index is opened. The new Lucene50Codec also includes further index corruption detection. Even CorruptIndexException itself is improved! It will now always refer to the file or resource where the corruption was detected, as this is now a required argument to its constructors. When corruption is detected higher up (e.g., a bad field number in the field infos file), the resulting CorruptIndexException will now state whether there was also a checksum mismatch in the file, helping to narrow the possible source of the corruption. Finally, during merge, IndexWriter now always checks the incoming segments for corruption before merging. This can mean, on upgrading to 5.0.0 that merging may uncover long-standing latent corruption in an older 4.x index. Reduced heap usage 5.0.0 also includes several changes to reduce heap usage during indexing and searching. If your index has 1B docs, then caching a single FixedBitSet-based filter in 4.10.2 costs a non-trivial 125 MB of heap! But with 5.0.0, Lucene now supports random-writable and advance-able sparse bitsets (RoaringDocIdSet and SparseFixedBitSet), so the heap required is in proportion to how many bits are set, not how many total documents exist in the index. These bitsets also greatly simplify how MultiTermQuery is rewritten (no more CONSTANT_SCORE_AUTO_REWRITE_METHOD), and they provide faster advance implementations than FixedBitSet‘s linear scan. Finally, they provide a more accurate cost() implementation, allowing Lucene to make better choices about how to drive the intersection at query time. Heap usage during IndexWriter merging is also much lower with the new Lucene50Codec, since doc values and norms for the segments being merged are no longer fully loaded into heap for all fields; now they are loaded for the one field currently being merged, and then dropped. The default norms format now uses sparse encoding when appropriate, so indices that enable norms for many sparse fields will see a large reduction in required heap at search time. An explain API for heap usage If you still find Lucene using more heap than you expected, 5.0.0 has a new API to print a tree structure showing a recursive breakdown of which parts are using how much heap. This is analogous to Lucene’s explain API, used to understand why a document has a certain relevance score, but applied to heap usage instead. It produces output like this: _cz(5.0.0):C8330469: 28MB postings [...]: 5.2MB ... field 'latitude' [...]: 678.5KB term index [FST(nodes=6679, ...)]: 678.3KB This is a much faster way to see what is using up your heap than trying to stare at a Java heap dump. Further changes There is a long tail of additional 5.0.0 changes; here are some of them:Old experimental postings formats (Sep/Fixed/VariableIntPostingsFormat) have been removed. PulsingPostingsFormat has also been removed, since the default postings format already pulses unique terms. FieldCache is gone (moved to a dedicated UninvertingReader in the misc module). This means when you intend to sort on a field, you should index that field using doc values, which is much faster and less heap consuming than FieldCache. Tokenizers and Analyzers no longer require Reader on init. NormsFormat now gets its own dedicated NormsConsumer/Producer Simplifications to FieldInfo (Lucene’s “low schema”): no more normType (it is always a DocValuesType.NUMERIC), no more isIndexed (just check IndexOptions) Compound file handling is simpler, and is now under codec control. SortedSetSortField, used to sort on a multi-valued field, is promoted from sandbox to Lucene’s core PostingsFormat now uses a “pull” API when writing postings, just like doc values. This is powerful because you can do things in your postings format that require making more than one pass through the postings such as iterating over all postings for each term to decide which compression format it should use. Version is no longer required on init to classes like IndexWriterConfig and analysis components.The changes I’ve described here are just a snapshot of what we have lined up today for a 5.0.0 release. 5.0.0 is still under active development (patches welcome!) so this list will change by the time the actual release is done.Reference: Apache Lucene 5.0.0 is coming! from our JCG partner Michael Mc Candless at the Changing Bits blog....
jcg-logo

Java Code Geeks sponsor Philadelphia Java Users’ Group November 2014 Meeting

Here at Java Code Geeks, we love interacting with fellow geeks and we always strive to provide as much value as possible to our great audience. In our continuous effort to support the community and further evangelize Java all over the world, we accepted a recent proposal to sponsor the Philadelphia Java Users’ Group November 2014 Meeting. Dave Fecak, a JCG partner of ours, founder and president of the Philadelphia JUG, reached out to us and kindly asked us if we are willing to sponsor their upcoming event. We gladly accepted the invitation and provided the financial support for it. Dave took over the execution part of the event. The event was a great success and the speaker, Frederic Jambukeswaran, exceeded all expectations with his great presentation on Java Application Deployment (slides here).Unfortunately, we were not able to participate in the meeting (we are in a different continent), but we were there mentally and we were extremely happy (and a bit touched) to see, firsthand, people enjoying our assistance and help, even if that was purely monetary. We would like to thank Dave and all the User Group. Keep coding guys! ...
junit-logo

Unit Testing exercise with FizzBuzz and JUnitParams

I sometimes use FizzBuzz to demonstrate the basics of unit testing to newbies. Although FizzBuzz is really simple problem, it can also be used to demonstrate more advanced unit testing techniques like implementing parametrized tests. One of the possible solutions to FizzBuzz is:             public class FizzBuzz {private static final int THREE = 3; private static final int FIVE = 5;public String calculate(int number) { if (isDivisibleBy(number, THREE) && isDivisibleBy(number, FIVE)) { return "FizzBuzz"; }if (isDivisibleBy(number, THREE)) { return "Fizz"; }if (isDivisibleBy(number, FIVE)) { return "Buzz"; }return String.valueOf(number); }private boolean isDivisibleBy(int dividend, int divisor) { return dividend % divisor == 0; } } The above example is ideal to show parametrized tests with JUnitParams. We could create 4 test methods, each for different FizzBuzz case: @Test @Parameters({"1", "2", "4", "7", "11", "13", "14"}) public void returnsNumberForNumberNotDivisibleByThreeAndFive(int number) { assertThat(fizzBuzz.calculate(number)).isEqualTo("" + number); }@Test @Parameters({"3", "6", "9", "12", "18", "21", "24"}) public void returnFizzForNumberDivisibleByThree(int number) { assertThat(fizzBuzz.calculate(number)).isEqualTo("Fizz"); }@Test @Parameters({"5", "10", "20", "25", "35", "40", "50"}) public void returnBuzzForNumberDivisibleByFive(int number) { assertThat(fizzBuzz.calculate(number)).isEqualTo("Buzz"); }@Test @Parameters({"15", "30", "45", "60"}) public void returnsFizzBuzzForNumberDivisibleByThreeAndFive(int number) { assertThat(fizzBuzz.calculate(number)).isEqualTo("FizzBuzz"); } But we could also cover all the cases in one parametrized test method: @Test @Parameters public void fizzBuzz(int given, String expected) { assertThat(fizzBuzz.calculate(given)).isEqualTo(expected); }public Object[] parametersForFizzBuzz() { return $( $(1, "1"), $(2, "2"), $(3, "Fizz"), $(4, "4"), $(5, "Buzz"), $(6, "Fizz"), $(7, "7"), $(10, "Buzz"), $(15, "FizzBuzz"), $(30, "FizzBuzz") ); } Enjoy JunitParams!Have a look at unit-testing-demo project presenting different aspects of unit testing, including parametrized tests: https://github.com/kolorobot/unit-testing-demoReference: Unit Testing exercise with FizzBuzz and JUnitParams from our JCG partner Rafal Borowiec at the Codeleak.pl blog....
software-development-2-logo

Don’t Miss out on Awesome SQL Power with FIRST_VALUE(), LAST_VALUE(), LEAD(), and LAG()

If you’re using a commercial database or PostgreSQL / Firebird / CUBRID, you will be able to take advantage of the full power of window functions. We’ve blogged about window functions’ awesomeness a couple of times, in particular about ROW_NUMBER(), RANK(), DENSE_RANK(). Today, we’re going to look into some awesome window functions that produce values of other rows that are positioned before or after the current row.         Setting up the test data We’re going to do some interesting statistics today using publicly available data from the World Bank. To keep things simple, we’ll only do analyses for the G8 countries:Canada (CA) France (FR) Germany (DE) Italy (IT) Japan (JP) Russian Federation (RU) United Kingdom (GB) United States (US)And for those countries, let’s consider the following data points for the years 2009-2012: GDP per capita (current US$) 2009 2010 2011 2012 CA 40,764 47,465 51,791 52,409 DE 40,270 40,408 44,355 42,598 FR 40,488 39,448 42,578 39,759 GB 35,455 36,573 38,927 38,649 IT 35,724 34,673 36,988 33,814 JP 39,473 43,118 46,204 46,548 RU 8,616 10,710 13,324 14,091 US 46,999 48,358 49,855 51,755 Central government debt, total (% of GDP) 2009 2010 2011 2012 CA 51.3 51.4 52.5 53.5 DE 47.6 55.5 55.1 56.9 FR 85.0 89.2 93.2 103.8 GB 71.7 85.2 99.6 103.2 IT 121.3 119.9 113.0 131.1 JP 166.8 174.8 189.5 196.5 RU 8.7 9.1 9.3 9.4 US 76.3 85.6 90.1 93.8 Let’s put all that data into a fact table like so (PostgreSQL syntax): CREATE TABLE countries ( code CHAR(2) NOT NULL, year INT NOT NULL, gdp_per_capita DECIMAL(10, 2) NOT NULL, govt_debt DECIMAL(10, 2) NOT NULL );INSERT INTO countries VALUES ('CA', 2009, 40764, 51.3), ('CA', 2010, 47465, 51.4), ('CA', 2011, 51791, 52.5), ('CA', 2012, 52409, 53.5), ('DE', 2009, 40270, 47.6), ('DE', 2010, 40408, 55.5), ('DE', 2011, 44355, 55.1), ('DE', 2012, 42598, 56.9), ('FR', 2009, 40488, 85.0), ('FR', 2010, 39448, 89.2), ('FR', 2011, 42578, 93.2), ('FR', 2012, 39759,103.8), ('GB', 2009, 35455,121.3), ('GB', 2010, 36573, 85.2), ('GB', 2011, 38927, 99.6), ('GB', 2012, 38649,103.2), ('IT', 2009, 35724,121.3), ('IT', 2010, 34673,119.9), ('IT', 2011, 36988,113.0), ('IT', 2012, 33814,131.1), ('JP', 2009, 39473,166.8), ('JP', 2010, 43118,174.8), ('JP', 2011, 46204,189.5), ('JP', 2012, 46548,196.5), ('RU', 2009, 8616, 8.7), ('RU', 2010, 10710, 9.1), ('RU', 2011, 13324, 9.3), ('RU', 2012, 14091, 9.4), ('US', 2009, 46999, 76.3), ('US', 2010, 48358, 85.6), ('US', 2011, 49855, 90.1), ('US', 2012, 51755, 93.8); Start the querying fun People who are used to SQL-92 syntax will be able to quickly find the highest GDP per capita or the highest debt from the table. It’s an easy query like this one: SELECT MAX(gdp_per_capita), MAX(govt_debt) FROM countries; Which will return: 52409.00 196.50 But that’s not interesting. We don’t even know what countries and what years these values are associated with. A standard SQL-92 (and also a standard relational) query to return all of these values would look something like this: SELECT 'highest gdp per capita' AS what, c1.* FROM countries c1 WHERE NOT EXISTS ( SELECT 1 FROM countries c2 WHERE c1.gdp_per_capita < c2.gdp_per_capita ) UNION ALL SELECT 'highest government debt' AS what, c1.* FROM countries c1 WHERE NOT EXISTS ( SELECT 1 FROM countries c2 WHERE c1.govt_debt < c2.govt_debt ) In essence, we select those rows for which there doesn’t exist any other row with a higher value for either gdp_per_capita (first subselect) or govt_debt (second subselect). Trick! Use quantified comparison predicates! If your database supports quantified comparison predicates, then you can write this a bit more concisely like this: SELECT 'highest gdp per capita' AS what, countries.* FROM countries WHERE gdp_per_capita >= ALL ( SELECT gdp_per_capita FROM countries ) UNION ALL SELECT 'highest government debt' AS what, countries.* FROM countries WHERE govt_debt >= ALL ( SELECT govt_debt FROM countries ) Which is essentially the same as… SELECT 'highest gdp per capita' AS what, countries.* FROM countries WHERE gdp_per_capita = ( SELECT MAX(gdp_per_capita) FROM countries ) UNION ALL SELECT 'highest government debt' AS what, countries.* FROM countries WHERE govt_debt = ( SELECT MAX(govt_debt) FROM countries ) The output would be: what code year gdp debt ---------------------------------------------------- highest gdp per capita CA 2012 52409.00 53.50 highest government debt JP 2012 46548.00 196.50 That’s a lot of SQL for only little analysis capability, and somehow, it just doesn’t feel entirely right to query the same table four times with all these subselects! FIRST_VALUE() and LAST_VALUE() This is where window functions come into play, and in this particular case, FIRST_VALUE() or LAST_VALUE(). For now, let’s focus on calculating the maximum GDP per capita from the data set: SELECT countries.*, FIRST_VALUE (code) OVER (w_gdp) AS max_gdp_code, FIRST_VALUE (year) OVER (w_gdp) AS max_gdp_year, FIRST_VALUE (gdp_per_capita) OVER (w_gdp) AS max_gdp_gdp, FIRST_VALUE (govt_debt) OVER (w_gdp) AS max_gdp_debt FROM countries WINDOW w_gdp AS (ORDER BY gdp_per_capita DESC) ORDER BY code, year Notice how we make use of the SQL standard WINDOW clause, which is only currently supported by PostgreSQL and Sybase SQL Anywhere. If you’re using Oracle or any other commercial database, you can simply substitute the window reference w_gdp into the various OVER() clauses to achieve equivalent behaviour – or you can use jOOQ’s WINDOW clause support and let jOOQ do the same for you.The above query will not produce any aggregates, but it will add the values for the country / year with the highest GDP per capita to every row in the table: each country highest per year ----------------------------------------------- CA 2009 40764.00 51.30 CA 2012 52409.00 53.50 CA 2010 47465.00 51.40 CA 2012 52409.00 53.50 CA 2011 51791.00 52.50 CA 2012 52409.00 53.50 CA 2012 52409.00 53.50 CA 2012 52409.00 53.50 This is extremely interesting because the data is not yet aggregated – the original data set remains unchanged, enriched with new computed columns. You can then further process things, e.g. compare each country / year with the highest GDP per capita and with the highest debt per GDP of that country / year: SELECT countries.*, TO_CHAR(100 * gdp_per_capita / FIRST_VALUE (gdp_per_capita) OVER (w_gdp) , '999.99 %') gdp_rank, TO_CHAR(100 * govt_debt / FIRST_VALUE (govt_debt) OVER (w_debt), '999.99 %') debt_rank FROM countries WINDOW w_gdp AS (PARTITION BY year ORDER BY gdp_per_capita DESC), w_debt AS (PARTITION BY year ORDER BY govt_debt DESC) ORDER BY code, year Notice how I’ve added PARTITION BY to the window definitions of the WINDOW clause. I’ve done this because I want to partition the data set by year, in order to find the highest GDP / debt values for each year, not for the whole data set. The outcome of the above query can then be seen here: country percentages ------------------------------------------ CA 2009 40764 51.3 86.73% 30.76% CA 2010 47465 51.4 98.15% 29.41% CA 2011 51791 52.5 100.00% 27.70% CA 2012 52409 53.5 100.00% 27.23% DE 2009 40270 47.6 85.68% 28.54% DE 2010 40408 55.5 83.56% 31.75% DE 2011 44355 55.1 85.64% 29.08% DE 2012 42598 56.9 81.28% 28.96% FR 2009 40488 85.0 86.15% 50.96% FR 2010 39448 89.2 81.57% 51.03% FR 2011 42578 93.2 82.21% 49.18% FR 2012 39759 103.8 75.86% 52.82% GB 2009 35455 121.3 75.44% 72.72% GB 2010 36573 85.2 75.63% 48.74% GB 2011 38927 99.6 75.16% 52.56% GB 2012 38649 103.2 73.74% 52.52% IT 2009 35724 121.3 76.01% 72.72% IT 2010 34673 119.9 71.70% 68.59% IT 2011 36988 113.0 71.42% 59.63% IT 2012 33814 131.1 64.52% 66.72% JP 2009 39473 166.8 83.99% 100.00% JP 2010 43118 174.8 89.16% 100.00% JP 2011 46204 189.5 89.21% 100.00% JP 2012 46548 196.5 88.82% 100.00% RU 2009 8616 8.7 18.33% 5.22% RU 2010 10710 9.1 22.15% 5.21% RU 2011 13324 9.3 25.73% 4.91% RU 2012 14091 9.4 26.89% 4.78% US 2009 46999 76.3 100.00% 45.74% US 2010 48358 85.6 100.00% 48.97% US 2011 49855 90.1 96.26% 47.55% US 2012 51755 93.8 98.75% 47.74% We could say that among the G8 countries, Canada has really improved the most in the last years, decreasing their debt compared to the GDP on a global comparison, while at the same time increasing their GDP per capita on a global comparison. Instead of partitioning the data set by year, we could also partition it by country, and find the best / worst year for each country over the years: SELECT countries.*, TO_CHAR(100 * gdp_per_capita / FIRST_VALUE (gdp_per_capita) OVER (w_gdp), '999.99 %') gdp_rank, TO_CHAR(100 * govt_debt / FIRST_VALUE (govt_debt) OVER (w_debt), '999.99 %') debt_rank FROM countries WINDOW w_gdp AS (PARTITION BY code ORDER BY gdp_per_capita DESC), w_debt AS (PARTITION BY code ORDER BY govt_debt DESC) ORDER BY code, year The result would now look quite different: country percentages ------------------------------------------ CA 2009 40764 51.3 77.78% 95.89% CA 2010 47465 51.4 90.57% 96.07% CA 2011 51791 52.5 98.82% 98.13% CA 2012 52409 53.5 100.00% 100.00% DE 2009 40270 47.6 90.79% 83.66% DE 2010 40408 55.5 91.10% 97.54% DE 2011 44355 55.1 100.00% 96.84% DE 2012 42598 56.9 96.04% 100.00% FR 2009 40488 85.0 95.09% 81.89% FR 2010 39448 89.2 92.65% 85.93% FR 2011 42578 93.2 100.00% 89.79% FR 2012 39759 103.8 93.38% 100.00% GB 2009 35455 121.3 91.08% 100.00% GB 2010 36573 85.2 93.95% 70.24% GB 2011 38927 99.6 100.00% 82.11% GB 2012 38649 103.2 99.29% 85.08% IT 2009 35724 121.3 96.58% 92.52% IT 2010 34673 119.9 93.74% 91.46% IT 2011 36988 113.0 100.00% 86.19% IT 2012 33814 131.1 91.42% 100.00% JP 2009 39473 166.8 84.80% 84.89% JP 2010 43118 174.8 92.63% 88.96% JP 2011 46204 189.5 99.26% 96.44% JP 2012 46548 196.5 100.00% 100.00% RU 2009 8616 8.7 61.15% 92.55% RU 2010 10710 9.1 76.01% 96.81% RU 2011 13324 9.3 94.56% 98.94% RU 2012 14091 9.4 100.00% 100.00% US 2009 46999 76.3 90.81% 81.34% US 2010 48358 85.6 93.44% 91.26% US 2011 49855 90.1 96.33% 96.06% US 2012 51755 93.8 100.00% 100.00% As you can see, most countries have now generally performed better in terms of GDP per capita over the years, and also most countries have almost strictly increased their own debt per GDP (except for Germany, France and Italy), except for the (United Kingdom). Russia and Canada have seen the most growth. In the above examples, we’ve been mainly using FIRST_VALUE(). LAST_VALUE() is almost the opposite function with respect to ordering, much like MAX() is the opposite function of MIN(). I’m saying almost because there is a caveat when using LAST_VALUE() with ORDER BY, because a window definition that uses ORDER BY is implicitly equivalent to a window definition that uses ORDER BY with a so-called “frame clause”: -- Find the "last" year over the complete data set -- This may not behave as expected, so always provide -- an explicit ORDER BY clause LAST_VALUE (year) OVER()-- These two are implicitly equivalent. We're not -- looking for the "last" year in the complete data -- set, but only in the frame that is "before" the -- current row. In other words, the current row is -- always the "last value"! LAST_VALUE (year) OVER(ORDER BY year) LAST_VALUE (year) OVER( ORDER BY year ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW )-- Find the "last" year in the complete data set with -- explicit ordering LAST_VALUE (year) OVER( ORDER BY year ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING ) LEAD() and LAG() The previous functions were about comparing values with the maximum / minimum (FIRST_VALUE() and LAST_VALUE()) within a data set. But using window functions, you can also compare things with the next / previous value. Or with the second next / second previous, etc. The functions used for this are called LEAD() (for the next value) and LAG() (for the previous value). This is best explained by example: -- Use this view as a data source containing -- all the distinct years: 2009-2012 WITH years AS ( SELECT DISTINCT year FROM countries ) SELECT FIRST_VALUE (year) OVER w_year AS first, LEAD (year, 2) OVER w_year AS lead2, LEAD (year) OVER w_year AS lead1, year, LAG (year) OVER w_year AS lag1, LAG (year, 2) OVER w_year AS lag2, LAST_VALUE (year) OVER w_year AS last FROM years WINDOW w_year AS ( ORDER BY year DESC ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING ) ORDER BY year The result is now simply: first lead2 lead1 year lag1 lag2 last ---------------------------------------------- 2012 2009 2010 2011 2009 2012 2009 2010 2011 2012 2009 2012 2009 2010 2011 2012 2009 2012 2010 2011 2012 2009 LEAD() and LAG() are really the best window functions to help understand the whole concept of window functions. For each year, you can see immediately how the previous and next year in the same window and frame can be generated using very simple function calls. This could be used, for instance, to find the “neighboring” countries in terms of GDP per capita for every country / year: SELECT year, code, gdp_per_capita, LEAD (code) OVER w_gdp AS runner_up_code, LEAD (gdp_per_capita) OVER w_gdp AS runner_up_gdp, LAG (code) OVER w_gdp AS leader_code, LAG (gdp_per_capita) OVER w_gdp AS leader_gdp FROM countries WINDOW w_gdp AS (PARTITION BY year ORDER BY gdp_per_capita DESC) ORDER BY year DESC, gdp_per_capita DESC Which returns: year country runner-up leader ------------------------------------------ 2012 CA 52409 US 51755 2012 US 51755 JP 46548 CA 52409 2012 JP 46548 DE 42598 US 51755 2012 DE 42598 FR 39759 JP 46548 2012 FR 39759 GB 38649 DE 42598 2012 GB 38649 IT 33814 FR 39759 2012 IT 33814 RU 14091 GB 38649 2012 RU 14091 IT 338142011 CA 51791 US 49855 2011 US 49855 JP 46204 CA 51791 2011 JP 46204 DE 44355 US 49855 2011 DE 44355 FR 42578 JP 46204 2011 FR 42578 GB 38927 DE 44355 2011 GB 38927 IT 36988 FR 42578 2011 IT 36988 RU 13324 GB 38927 2011 RU 13324 IT 369882010 US 48358 CA 47465 2010 CA 47465 JP 43118 US 48358 2010 JP 43118 DE 40408 CA 47465 2010 DE 40408 FR 39448 JP 43118 2010 FR 39448 GB 36573 DE 40408 2010 GB 36573 IT 34673 FR 39448 2010 IT 34673 RU 10710 GB 36573 2010 RU 10710 IT 346732009 US 46999 CA 40764 2009 CA 40764 FR 40488 US 46999 2009 FR 40488 DE 40270 CA 40764 2009 DE 40270 JP 39473 FR 40488 2009 JP 39473 IT 35724 DE 40270 2009 IT 35724 GB 35455 JP 39473 2009 GB 35455 RU 8616 IT 35724 2009 RU 8616 GB 35455 If you want to do more fancy analyses, you could now compare percentages between leaders and runner-ups, etc. Another great use-case for LEAD() and LAG() can be seen in this article. Conclusion Window functions are an incredibly powerful feature that is available from all major commercial databases, and also from a couple of Open Source databases like PostgreSQL, Firebird, and CUBRID. There has essentially been SQL before window functions, and SQL after window functions. With jOOQ, you can leverage window functions on a type safe level like anything else related to SQL. The last query we’ve seen can be written simply like this: // Static import the generated tables and all // of jOOQ's functions from DSL import static org.jooq.example.db.postgres.Tables.*; import static org.jooq.impl.DSL.*;// Shorten the table reference by aliasing Countries c = COUNTRIES;// Specifiy a window definition WindowDefinition w_gdp = name("w_gdp").as( partitionBy(c.YEAR) .orderBy(c.GDP_PER_CAPITA.desc() ) );// Write the query as if it were native SQL System.out.println( DSL.using(conn) .select( c.YEAR, c.CODE, c.GDP_PER_CAPITA, lead(c.CODE) .over(w_gdp).as("runner_up_code"), lead(c.GDP_PER_CAPITA).over(w_gdp).as("runner_up_gdp"), lag (c.CODE) .over(w_gdp).as("leader_code"), lag (c.GDP_PER_CAPITA).over(w_gdp).as("leader_gdp") ) .from(c) .window(w_gdp) .orderBy(c.YEAR.desc(), c.GDP_PER_CAPITA.desc()) .fetch() ); The above program will output +----+----+--------------+--------------+-------------+-----------+----------+ |year|code|gdp_per_capita|runner_up_code|runner_up_gdp|leader_code|leader_gdp| +----+----+--------------+--------------+-------------+-----------+----------+ |2012|CA | 52409.00|US | 51755.00|{null} | {null}| |2012|US | 51755.00|JP | 46548.00|CA | 52409.00| |2012|JP | 46548.00|DE | 42598.00|US | 51755.00| |2012|DE | 42598.00|FR | 39759.00|JP | 46548.00| |2012|FR | 39759.00|GB | 38649.00|DE | 42598.00| |2012|GB | 38649.00|IT | 33814.00|FR | 39759.00| |2012|IT | 33814.00|RU | 14091.00|GB | 38649.00| |2012|RU | 14091.00|{null} | {null}|IT | 33814.00| |2011|CA | 51791.00|US | 49855.00|{null} | {null}| |2011|US | 49855.00|JP | 46204.00|CA | 51791.00| |2011|JP | 46204.00|DE | 44355.00|US | 49855.00| |2011|DE | 44355.00|FR | 42578.00|JP | 46204.00| |2011|FR | 42578.00|GB | 38927.00|DE | 44355.00| |2011|GB | 38927.00|IT | 36988.00|FR | 42578.00| |2011|IT | 36988.00|RU | 13324.00|GB | 38927.00| |2011|RU | 13324.00|{null} | {null}|IT | 36988.00| |2010|US | 48358.00|CA | 47465.00|{null} | {null}| |2010|CA | 47465.00|JP | 43118.00|US | 48358.00| |2010|JP | 43118.00|DE | 40408.00|CA | 47465.00| |2010|DE | 40408.00|FR | 39448.00|JP | 43118.00| |2010|FR | 39448.00|GB | 36573.00|DE | 40408.00| |2010|GB | 36573.00|IT | 34673.00|FR | 39448.00| |2010|IT | 34673.00|RU | 10710.00|GB | 36573.00| |2010|RU | 10710.00|{null} | {null}|IT | 34673.00| |2009|US | 46999.00|CA | 40764.00|{null} | {null}| |2009|CA | 40764.00|FR | 40488.00|US | 46999.00| |2009|FR | 40488.00|DE | 40270.00|CA | 40764.00| |2009|DE | 40270.00|JP | 39473.00|FR | 40488.00| |2009|JP | 39473.00|IT | 35724.00|DE | 40270.00| |2009|IT | 35724.00|GB | 35455.00|JP | 39473.00| |2009|GB | 35455.00|RU | 8616.00|IT | 35724.00| |2009|RU | 8616.00|{null} | {null}|GB | 35455.00| +----+----+--------------+--------------+-------------+-----------+----------+No matter whether you’re using jOOQ for your database integration, or just plain SQL – start using window functions today.Reference: Don’t Miss out on Awesome SQL Power with FIRST_VALUE(), LAST_VALUE(), LEAD(), and LAG() from our JCG partner Lukas Eder at the JAVA, SQL, AND JOOQ blog....
java-logo

Multithreading and Concurrency Interview Questions and Answers – The ULTIMATE List (PDF Download)

EDITORIAL NOTE: Concurrency is always a challenge for developers and writing concurrent programs can be extremely hard. There is a number of things that could potentially blow up and the complexity of systems rises considerably when concurrency is introduced. However, the ability to write robust concurrent programs is a great tool in a developer’s belt and can help build sophisticated, enterprise level applications. In this article we will discuss different types of questions that can be used in a programming interview in order to assess a candidate’s understanding of concurrency and multithreading. The questions are not only Java specific, but revolve around general programming principles. Enjoy!Programming Interview Coming Up?Subscribe to our newsletter and download the Ultimate Multithreading and Concurrency interview questions and answers collection right now! In order to get you prepared for your next Programming Interview, we have compiled a huge list of relevant Questions and their respective Answers. Besides studying them online you may download the eBook in PDF format!Email address:Given email address is already subscribed, thank you!Oops. Something went wrong. Please try again later.Please provide a valid email address.Thank you, your sign-up request was successful! Please check your e-mail inbox.Please complete the CAPTCHA.Please fill in the required fields. Tip Are you looking for career opportunities in Java?   Visit our Job Board to search and review a hand-picked selection of Java jobs for IT professionals around the world.   Additionally you can add your resume to our Resumes Board promoting your skills to a wide range of Java job recruiters around the world.Table Of Contents1. What do we understand by the term concurrency? 2. What is the difference between processes and threads? 3. In Java, what is a process and a thread? 4. What is a scheduler? 5. How many threads does a Java program have at least? 6. How can a Java application access the current thread? 7. What properties does each Java thread have? 8. What is the purpose of thread groups? 9. What states can a thread have and what is the meaning of each state? 10. How do we set the priority of a thread? 11. How is a thread created in Java? 12. How do we stop a thread in Java? 13. Why should a thread not be stopped by calling its method stop()? 14. Is it possible to start a thread twice? 15. What is the output of the following code? 16. What is a daemon thread? 17. Is it possible to convert a normal user thread into a daemon thread after it has been started? 18. What do we understand by busy waiting? 19. How can we prevent busy waiting? 20. Can we use Thread.sleep() for real-time processing? 21. How can a thread be woken up that has been put to sleep before using Thread.sleep()? 22. How can a thread query if it has been interrupted? 23. How should an InterruptedException be handled? 24. After having started a child thread, how do we wait in the parent thread for the termination of the child thread? 25. What is the output of the following program? 26. What happens when an uncaught exception leaves the run() method? 27. What is a shutdown hook? 28. For what purposes is the keyword synchronized used? 29. What intrinsic lock does a synchronized method acquire? 30. Can a constructor be synchronized? 31. Can primitive values be used for intrinsic locks? 32. Are intrinsic locks reentrant? 33. What do we understand by an atomic operation? 34. Is the statement c++ atomic? 35. What operations are atomic in Java? 36. Is the following implementation thread-safe? 37. What do we understand by a deadlock? 38. What are the requirements for a deadlock situation? 39. Is it possible to prevent deadlocks at all? 40. Is it possible to implement a deadlock detection? 41. What is a livelock? 42. What do we understand by thread starvation? 43. Can a synchronized block cause thread starvation? 44. What do we understand by the term race condition? 45. What do we understand by fair locks? 46. Which two methods that each object inherits from java.lang.Object can be used to implement a simple producer/consumer scenario? 47. What is the difference between notify() and notifyAll()? 48. How it is determined which thread wakes up by calling notify()? 49. Is the following code that retrieves an integer value from some queue implementation correct? 50. Is it possible to check whether a thread holds a monitor lock on some given object? 51. What does the method Thread.yield() do? 52. What do you have to consider when passing object instances from one thread to another? 53. Which rules do you have to follow in order to implement an immutable class? 54. What is the purpose of the class java.lang.ThreadLocal? 55. What are possible use cases for java.lang.ThreadLocal? 56. Is it possible to improve the performance of an application by the usage of multi-threading? Name some examples. 57. What do we understand by the term scalability? 58. Is it possible to compute the theoretical maximum speed up for an application by using multiple processors? 59. What do we understand by lock contention? 60. Which techniques help to reduce lock contention? 61. Which technique to reduce lock contention can be applied to the following code? 62. Explain by an example the technique lock splitting. 63. What kind of technique for reducing lock contention is used by the SDK class ReadWriteLock? 64. What do we understand by lock striping? 65. What do we understand by a CAS operation? 66. Which Java classes use the CAS operation? 67. Provide an example why performance improvements for single-threaded applications can cause performance degradation for multi-threaded applications. 68. Is object pooling always a performance improvement for multi-threaded applications? 69. What is the relation between the two interfaces Executor and ExecutorService? 70. What happens when you submit() a new task to an ExecutorService instance whose queue is already full? 71. What is a ScheduledExecutorService? 72. Do you know an easy way to construct a thread pool with 5 threads that executes some tasks that return a value? 73. What is the difference between the two interfaces Runnable and Callable? 74. Which are use cases for the class java.util.concurrent.Future? 75. What is the difference between HashMap and Hashtable particularly with regard to thread-safety? 76. Is there a simple way to create a synchronized instance of an arbitrary implementation of Collection, List or Map? 77. What is a semaphore? 78. What is a CountDownLatch? 79. What is the difference between a CountDownLatch and a CyclicBarrier? 80. What kind of tasks can be solved by using the Fork/Join framework? 81. Is it possible to find the smallest number within an array of numbers using the Fork/Join-Framework? 82. What is the difference between the two classes RecursiveTask and RecursiveAction? 83. Is it possible to perform stream operations in Java 8 with a thread pool? 84. How can we access the thread pool that is used by parallel stream operations?  1. What do we understand by the term concurrency? Concurrency is the ability of a program to execute several computations simultaneously. This can be achieved by distributing the computations over the available CPU cores of a machine or even over different machines within the same network. 2. What is the difference between processes and threads? A process is an execution environment provided by the operating system that has its own set of private resources (e.g. memory, open files, etc.). Threads, in contrast to processes, live within a process and share their resources (memory, open files, etc.) with the other threads of the process. The ability to share resources between different threads makes thread more suitable for tasks where performance is a significant requirement. 3. In Java, what is a process and a thread? In Java, processes correspond to a running Java Virtual Machine (JVM) whereas threads live within the JVM and can be created and stopped by the Java application dynamically at runtime. 4. What is a scheduler? A scheduler is the implementation of a scheduling algorithm that manages access of processes and threads to some limited resource like the processor or some I/O channel. The goal of most scheduling algorithms is to provide some kind of load balancing for the available processes/threads that guarantees that each process/thread gets an appropriate time frame to access the requested resource exclusively. 5. How many threads does a Java program have at least? Each Java program is executed within the main thread; hence each Java application has at least one thread. 6. How can a Java application access the current thread? The current thread can be accessed by calling the static method currentThread() of the JDK class java.lang.Thread: public class MainThread {public static void main(String[] args) { long id = Thread.currentThread().getId(); String name = Thread.currentThread().getName(); ... } }7. What properties does each Java thread have? Each Java thread has the following properties:an identifier of type long that is unique within the JVM a name of type String a priority of type int a state of type java.lang.Thread.State a thread group the thread belongs to8. What is the purpose of thread groups? Each thread belongs to a group of threads. The JDK class java.lang.ThreadGroup provides some methods to handle a whole group of Threads. With these methods we can, for example, interrupt all threads of a group or set their maximum priority. 9. What states can a thread have and what is the meaning of each state?NEW: A thread that has not yet started is in this state. RUNNABLE: A thread executing in the Java virtual machine is in this state. BLOCKED: A thread that is blocked waiting for a monitor lock is in this state. WAITING: A thread that is waiting indefinitely for another thread to perform a particular action is in this state. TIMED_WAITING: A thread that is waiting for another thread to perform an action for up to a specified waiting time is in this state. TERMINATED: A thread that has exited is in this state.10. How do we set the priority of a thread? The priority of a thread is set by using the method setPriority(int). To set the priority to the maximum value, we use the constant Thread.MAX_PRIORITY and to set it to the minimum value we use the constant Thread.MIN_PRIORITY because these values can differ between different JVM implementations. 11. How is a thread created in Java? Basically, there are two ways to create a thread in Java. The first one is to write a class that extends the JDK class java.lang.Thread and call its method start(): public class MyThread extends Thread {public MyThread(String name) { super(name); }@Override public void run() { System.out.println("Executing thread "+Thread.currentThread().getName()); }public static void main(String[] args) throws InterruptedException { MyThread myThread = new MyThread("myThread"); myThread.start(); } }The second way is to implement the interface java.lang.Runnable and pass this implementation as a parameter to the constructor of java.lang.Thread: public class MyRunnable implements Runnable {public void run() { System.out.println("Executing thread "+Thread.currentThread().getName()); }public static void main(String[] args) throws InterruptedException { Thread myThread = new Thread(new MyRunnable(), "myRunnable"); myThread.start(); } }12. How do we stop a thread in Java? To stop a thread one can use a volatile reference pointing to the current thread that can be set to null by other threads to indicate the current thread should stop its execution: private static class MyStopThread extends Thread { private volatile Thread stopIndicator;public void start() { stopIndicator = new Thread(this); stopIndicator.start(); }public void stopThread() { stopIndicator = null; }@Override public void run() { Thread thisThread = Thread.currentThread(); while(thisThread == stopIndicator) { try { Thread.sleep(1000); } catch (InterruptedException e) { } } } }13. Why should a thread not be stopped by calling its method stop()? A thread should not be stopped by using the deprecated methods stop() of java.lang.Thread, as a call of this method causes the thread to unlock all monitors it has acquired. If any object protected by one of the released locks was in an inconsistent state, this state gets visible to all other threads. This can cause arbitrary behavior when other threads work this this inconsistent object. 14. Is it possible to start a thread twice? No, after having started a thread by invoking its start() method, a second invocation of start() will throw an IllegalThreadStateException. 15. What is the output of the following code? public class MultiThreading {private static class MyThread extends Thread {public MyThread(String name) { super(name); }@Override public void run() { System.out.println(Thread.currentThread().getName()); } }public static void main(String[] args) { MyThread myThread = new MyThread("myThread"); myThread.run(); } }The code above produces the output “main” and not “myThread”. As can be seen in line two of the main() method, we invoke by mistake the method run() instead of start(). Hence, no new thread is started, but the method run() gets executed within the main thread. 16. What is a daemon thread? A daemon thread is a thread whose execution state is not evaluated when the JVM decides if it should stop or not. The JVM stops when all user threads (in contrast to the daemon threads) are terminated. Hence daemon threads can be used to implement for example monitoring functionality as the thread is stopped by the JVM as soon as all user threads have stopped: public class Example {private static class MyDaemonThread extends Thread {public MyDaemonThread() { setDaemon(true); }@Override public void run() { while (true) { try { Thread.sleep(1); } catch (InterruptedException e) { e.printStackTrace(); } } } }public static void main(String[] args) throws InterruptedException { Thread thread = new MyDaemonThread(); thread.start(); } }The example application above terminates even though the daemon thread is still running in its endless while loop. 17. Is it possible to convert a normal user thread into a daemon thread after it has been started? A user thread cannot be converted into a daemon thread once it has been started. Invoking the method thread.setDaemon(true) on an already running thread instance causes a IllegalThreadStateException. 18. What do we understand by busy waiting? Busy waiting means implementations that wait for an event by performing some active computations that let the thread/process occupy the processor although it could be removed from it by the scheduler. An example for busy waiting would be to spend the waiting time within a loop that determines the current time again and again until a certain point in time is reached: Thread thread = new Thread(new Runnable() { @Override public void run() { long millisToStop = System.currentTimeMillis() + 5000; long currentTimeMillis = System.currentTimeMillis(); while (millisToStop > currentTimeMillis) { currentTimeMillis = System.currentTimeMillis(); } } });19. How can we prevent busy waiting? One way to prevent busy waiting is to put the current thread to sleep for a given amount of time. This can be done by calling the method java.lang.Thread.sleep(long) by passing the number of milliseconds to sleep as an argument. 20. Can we use Thread.sleep() for real-time processing? The number of milliseconds passed to an invocation of Thread.sleep(long) is only an indication for the scheduler how long the current thread does not need to be executed. It may happen that the scheduler lets the thread execute again a few milliseconds earlier or later depending on the actual implementation. Hence an invocation of Thread.sleep() should not be used for real-time processing. 21. How can a thread be woken up that has been put to sleep before using Thread.sleep()? The method interrupt() of java.lang.Thread interrupts a sleeping thread. The interrupted thread that has been put to sleep by calling Thread.sleep() is woken up by an InterruptedException: public class InterruptExample implements Runnable {public void run() { try { Thread.sleep(Long.MAX_VALUE); } catch (InterruptedException e) { System.out.println("["+Thread.currentThread().getName()+"] Interrupted by exception!"); } }public static void main(String[] args) throws InterruptedException { Thread myThread = new Thread(new InterruptExample(), "myThread"); myThread.start();System.out.println("["+Thread.currentThread().getName()+"] Sleeping in main thread for 5s..."); Thread.sleep(5000);System.out.println("["+Thread.currentThread().getName()+"] Interrupting myThread"); myThread.interrupt(); } }22. How can a thread query if it has been interrupted? If the thread is not within a method like Thread.sleep() that would throw an InterruptedException, the thread can query if it has been interrupted by calling either the static method Thread.interrupted() or the method isInterrupted() that it has inherited from java.lang.Thread. 23. How should an InterruptedException be handled? Methods like sleep() and join() throw an InterruptedException to tell the caller that another thread has interrupted this thread. In most cases this is done in order to tell the current thread to stop its current computations and to finish them unexpectedly. Hence ignoring the exception by catching it and only logging it to the console or some log file is often not the appropriate way to handle this kind of exception. The problem with this exception is, that the method run() of the Runnable interface does not allow that run() throws any exceptions. So just rethrowing it does not help. This means the implementation of run() has to handle this checked exception itself and this often leads to the fact that it its caught and ignored. 24. After having started a child thread, how do we wait in the parent thread for the termination of the child thread? Waiting for a thread’s termination is done by invoking the method join() on the thread’s instance variable: Thread thread = new Thread(new Runnable() { @Override public void run() {} }); thread.start(); thread.join();25. What is the output of the following program? public class MyThreads {private static class MyDaemonThread extends Thread {public MyDaemonThread() { setDaemon(true); }@Override public void run() { try { Thread.sleep(1000); } catch (InterruptedException e) { } } }public static void main(String[] args) throws InterruptedException { Thread thread = new MyDaemonThread(); thread.start(); thread.join(); System.out.println(thread.isAlive()); } }The output of the above code is “false”. Although the instance of MyDaemonThread is a daemon thread, the invocation of join() causes the main thread to wait until the execution of the daemon thread has finished. Hence calling isAlive() on the thread instance reveals that the daemon thread is no longer running. 26. What happens when an uncaught exception leaves the run() method? I can happen that an unchecked exception escapes from the run() method. In this case the thread is stopped by the Java Virtual Machine. It is possible to catch this exception by registering an instance that implements the interface UncaughtExceptionHandler as an exception handler. This is either done by invoking the static method Thread.setDefaultUncaughtExceptionHandler(Thread.UncaughtExceptionHandler), which tells the JVM to use the provided handler in case there was no specific handler registerd on the thread itself, or by invoking setUncaughtExceptionHandler(Thread.UncaughtExceptionHandler) on the thread instance itself. 27. What is a shutdown hook? A shutdown hook is a thread that gets executed when the JVM shuts down. It can be registered by invoking addShutdownHook(Runnable) on the Runtime instance: Runtime.getRuntime().addShutdownHook(new Thread() { @Override public void run() {} });28. For what purposes is the keyword synchronized used? When you have to implement exclusive access to a resource, like some static value or some file reference, the code that works with the exclusive resource can be embraced with a synchronized block: synchronized (SynchronizedCounter.class) { counter++; }29. What intrinsic lock does a synchronized method acquire? A synchronized method acquires the intrinsic lock for that method’s object and releases it when the method returns. Even if the method throws an exception, the intrinsic lock is released. Hence a synchronized method is equal to the following code: public void method() { synchronized(this) { ... } }30. Can a constructor be synchronized? No, a constructor cannot be synchronized. The reason why this leads to an syntax error is the fact that only the constructing thread should have access to the object being constructed. 31. Can primitive values be used for intrinsic locks? No, primitive values cannot be used for intrinsic locks. 32. Are intrinsic locks reentrant? Yes, intrinsic locks can be accessed by the same thread again and again. Otherwise code that acquires a lock would have to pay attention that it does not accidently tries to acquire a lock it has already acquired. 33. What do we understand by an atomic operation? An atomic operation is an operation that is either executed completely or not at all. 34. Is the statement c++ atomic? No, the incrementation of an integer variable consist of more than one operation. First we have to load the current value of c, increment it and then finally store the new value back. The current thread performing this incrementation may be interrupted in-between any of these three steps, hence this operation is not atomic. 35. What operations are atomic in Java? The Java language provides some basic operations that are atomic and that therefore can be used to make sure that concurrent threads always see the same value:Read and write operations to reference variables and primitive variables (except long and double) Read and write operations for all variables declared as volatile36. Is the following implementation thread-safe? public class DoubleCheckedSingleton { private DoubleCheckedSingleton instance = null;public DoubleCheckedSingleton getInstance() { if(instance == null) { synchronized (DoubleCheckedSingleton.class) { if(instance == null) { instance = new DoubleCheckedSingleton(); } } } return instance; } }The code above is not thread-safe. Although it checks the value of instance once again within the synchronized block (for performance reasons), the JIT compiler can rearrange the bytecode in a way that the reference to instance is set before the constructor has finished its execution. This means the method getInstance() returns an object that may not have been initialized completely. To make the code thread-safe, the keyword volatile can be used since Java 5 for the instance variable. Variables that are marked as volatile get only visible to other threads once the constructor of the object has finished its execution completely. 37. What do we understand by a deadlock? A deadlock is a situation in which two (or more) threads are each waiting on the other thread to free a resource that it has locked, while the thread itself has locked a resource the other thread is waiting on: Thread 1: locks resource A, waits for resource B Thread 2: locks resource B, waits for resource A 38. What are the requirements for a deadlock situation? In general the following requirements for a deadlock can be identified:Mutual exclusion: There is a resource which can be accessed only by one thread at any point in time. Resource holding: While having locked one resource, the thread tries to acquire another lock on some other exclusive resource. No preemption: There is no mechanism, which frees the resource if one thread holds the lock for a specific period of time. Circular wait: During runtime a constellation occurs in which two (or more) threads are each waiting on the other thread to free a resource that it has locked.39. Is it possible to prevent deadlocks at all? In order to prevent deadlocks one (or more) of the requirements for a deadlock has to be eliminated:Mutual exclusion: In some situation it is possible to prevent mutual exclusion by using optimistic locking. Resource holding: A thread may release all its exclusive locks, when it does not succeed in obtaining all exclusive locks. No preemption: Using a timeout for an exclusive lock frees the lock after a given amount of time. Circular wait: When all exclusive locks are obtained by all threads in the same sequence, no circular wait occurs.40. Is it possible to implement a deadlock detection? When all exclusive locks are monitored and modelled as a directed graph, a deadlock detection system can search for two threads that are each waiting on the other thread to free a resource that it has locked. The waiting threads can then be forced by some kind of exception to release the lock the other thread is waiting on. 41. What is a livelock? A livelock is a situation in which two or more threads block each other by responding to an action that is caused by another thread. In contrast to a deadlock situation, where two or more threads wait in one specific state, the threads that participate in a livelock change their state in a way that prevents progress on their regular work. An example would be a situation in which two threads try to acquire two locks, but release a lock they have acquired when they cannot acquire the second lock. It may now happen that both threads concurrently try to acquire the first thread. As only one thread succeeds, the second thread may succeed in acquiring the second lock. Now both threads hold two different locks, but as both want to have both locks, they release their lock and try again from the beginning. This situation may now happen again and again. 42. What do we understand by thread starvation? Threads with lower priority get less time for execution than threads with higher priority. When the threads with lower priority performs a long enduring computations, it may happen that these threads do not get enough time to finish their computations just in time. They seem to “starve” away as threads with higher priority steal them their computation time. 43. Can a synchronized block cause thread starvation? The order in which threads can enter a synchronized block is not defined. So in theory it may happen that in case many threads are waiting for the entrance to a synchronized block, some threads have to wait longer than other threads. Hence they do not get enough computation time to finish their work in time. 44. What do we understand by the term race condition? A race condition describes constellations in which the outcome of some multi-threaded implementation depends on the exact timing behavior of the participating threads. In most cases it is not desirable to have such a kind of behavior, hence the term race condition also means that a bug due to missing thread synchronization leads to the differing outcome. A simple example for a race condition is the incrementation of an integer variable by two concurrent threads. As the operation consists of more than one single and atomic operation, it may happen that both threads read and increment the same value. After this concurrent incrementation the amount of the integer variable is not increased by two but only by one. 45. What do we understand by fair locks? A fair lock takes the waiting time of the threads into account when choosing the next thread that passes the barrier to some exclusive resource. An example implementation of a fair lock is provided by the Java SDK: java.util.concurrent.locks.ReentrantLock. If the constructor with the boolean flag set to true is used, the ReentrantLock grants access to the longest-waiting thread. 46. Which two methods that each object inherits from java.lang.Object can be used to implement a simple producer/consumer scenario? When a worker thread has finished its current task and the queue for new tasks is empty, it can free the processor by acquiring an intrinsic lock on the queue object and by calling the method wait(). The thread will be woken up by some producer thread that has put a new task into the queue and that again acquires the same intrinsic lock on the queue object and calls notify() on it. 47. What is the difference between notify() and notifyAll()? Both methods are used to wake up one or more threads that have put themselves to sleep by calling wait(). While notify() only wakes up one of the waiting threads, notifyAll() wakes up all waiting threads. 48. How it is determined which thread wakes up by calling notify()? It is not specified which threads will be woken up by calling notify() if more than one thread is waiting. Hence code should not rely on any concrete JVM implementation. 49. Is the following code that retrieves an integer value from some queue implementation correct? public Integer getNextInt() { Integer retVal = null; synchronized (queue) { try { while (queue.isEmpty()) { queue.wait(); } } catch (InterruptedException e) { e.printStackTrace(); } } synchronized (queue) { retVal = queue.poll(); if (retVal == null) { System.err.println("retVal is null"); throw new IllegalStateException(); } } return retVal; }Although the code above uses the queue as object monitor, it does not behave correctly in a multi-threaded environment. The reason for this is that it has two separate synchronized blocks. When two threads are woken up in line 6 by another thread that calls notifyAll(), both threads enter one after the other the second synchronized block. It this second block the queue has now only one new value, hence the second thread will poll on an empty queue and get null as return value. 50. Is it possible to check whether a thread holds a monitor lock on some given object? The class java.lang.Thread provides the static method Thread.holdsLock(Object) that returns true if and only if the current thread holds the lock on the object given as argument to the method invocation. 51. What does the method Thread.yield() do? An invocation of the static method Thread.yield() gives the scheduler a hint that the current thread is willing to free the processor. The scheduler is free to ignore this hint. As it is not defined which thread will get the processor after the invocation of Thread.yield(), it may even happen that the current thread becomes the “next” thread to be executed. 52. What do you have to consider when passing object instances from one thread to another? When passing objects between threads, you will have to pay attention that these objects are not manipulated by two threads at the same time. An example would be a Map implementation whose key/value pairs are modified by two concurrent threads. In order to avoid problems with concurrent modifications you can design an object to be immutable. 53. Which rules do you have to follow in order to implement an immutable class?All fields should be final and private. There should be not setter methods. The class itself should be declared final in order to prevent subclasses to violate the principle of immutability. If fields are not of a primitive type but a reference to another object:There should not be a getter method that exposes the reference directly to the caller. Don’t change the referenced objects (or at least changing these references is not visisble to clients of the object).54. What is the purpose of the class java.lang.ThreadLocal? As memory is shared between different threads, ThreadLocal provides a way to store and retrieve values for each thread separately. Implementations of ThreadLocal store and retrieve the values for each thread independently such that when thread A stores the value A1 and thread B stores the value B1 in the same instance of ThreadLocal, thread A later on retrieves value A1 from this ThreadLocal instance and thread B retrieves value B1. 55. What are possible use cases for java.lang.ThreadLocal? Instances of ThreadLocal can be used to transport information throughout the application without the need to pass this from method to method. Examples would be the transportation of security/login information within an instance of ThreadLocal such that it is accessible by each method. Another use case would be to transport transaction information or in general objects that should be accessible in all methods without passing them from method to method. 56. Is it possible to improve the performance of an application by the usage of multi-threading? Name some examples. If we have more than one CPU core available, the performance of an application can be improved by multi-threading if it is possible to parallelize the computations over the available CPU cores. An example would be an application that should scale all images that are stored within a local directory structure. Instead of iterating over all images one after the other, a producer/consumer implementation can use a single thread to scan the directory structure and a bunch of worker threads that perform the actual scaling operation. Another example would be an application that mirrors some web page. Instead of loading one HTML page after the other, a producer thread can parse the first HTML page and issue the links it found into a queue. The worker threads monitor the queue and load the web pages found by the parser. While the worker threads wait for the page to get loaded completely, other threads can use the CPU to parse the already loaded pages and issue new requests. 57. What do we understand by the term scalability? Scalability means the ability of a program to improve the performance by adding further resources to it. 58. Is it possible to compute the theoretical maximum speed up for an application by using multiple processors? Amdahl’s law provides a formula to compute the theoretical maximum speed up by providing multiple processors to an application. The theoretical speedup is computed by S(n) = 1 / (B + (1-B)/n) where n denotes the number of processors and B the fraction of the program that cannot be executed in parallel. When n converges against infinity, the term (1-B)/n converges against zero. Hence the formula can be reduced in this special case to 1/B. As we can see, the theoretical maximum speedup behaves reciprocal to the fraction that has to be executed serially. This means the lower this fraction is, the more theoretical speedup can be achieved. 59. What do we understand by lock contention? Lock contention occurs, when two or more threads are competing in the acquisition of a lock. The scheduler has to decide whether it lets the thread, which has to wait sleeping and performs a context switch to let another thread occupy the CPU, or if letting the waiting thread busy-waiting is more efficient. Both ways introduce idle time to the inferior thread. 60. Which techniques help to reduce lock contention? In some cases lock contention can be reduced by applying one of the following techniques:The scope of the lock is reduced. The number of times a certain lock is acquired is reduced (lock splitting). Using hardware supported optimistic locking operations instead of synchronization. Avoid synchronization where possible. Avoid object pooling.61. Which technique to reduce lock contention can be applied to the following code? synchronized (map) { UUID randomUUID = UUID.randomUUID(); Integer value = Integer.valueOf(42); String key = randomUUID.toString(); map.put(key, value); }The code above performs the computation of the random UUID and the conversion of the literal 42 into an Integer object within the synchronized block, although these two lines of code are local to the current thread and do not affect other threads. Hence they can be moved out of the synchronized block: UUID randomUUID = UUID.randomUUID(); Integer value = Integer.valueOf(42); String key = randomUUID.toString(); synchronized (map) { map.put(key, value); }62. Explain by an example the technique lock splitting. Lock splitting may be a way to reduce lock contention when one lock is used to synchronize access to different aspects of the same application. Suppose we have a class that implements the computation of some statistical data of our application. A first version of this class uses the keyword synchronized in each method signature in order to guard the internal state before corruption by multiple concurrent threads. This also means that each method invocation may cause lock contention as other threads may try to acquire the same lock simultaneously. But it may be possible to split the lock on the object instance into a few smaller locks for each type of statistical data within each method. Hence thread T1 that tries to increment the statistical data D1 does not have to wait for the lock while thread T2 simultaneously updates the data D2. 63. What kind of technique for reducing lock contention is used by the SDK class ReadWriteLock? The SDK class ReadWriteLock uses the fact that concurrent threads do not have to acquire a lock when they want to read a value when no other thread tries to update the value. This is implemented by a pair of locks, one for read-only operations and one for writing operations. While the read-only lock may be obtained by more than one thread, the implementation guarantees that all read operation see an updated value once the write lock is released. 64. What do we understand by lock striping? In contrast to lock splitting, where we introduce different locks for different aspects of the application, lock striping uses multiple locks to guard different parts of the same data structure. An example for this technique is the class ConcurrentHashMap from JDK’s java.util.concurrent package. The Map implementation uses internally different buckets to store its values. The bucket is chosen by the value’s key. ConcurrentHashMap now uses different locks to guard different hash buckets. Hence one thread that tries to access the first hash bucket can acquire the lock for this bucket, while another thread can simultaneously access a second bucket. In contrast to a synchronized version of HashMap this technique can increase the performance when different threads work on different buckets. 65. What do we understand by a CAS operation? CAS stands for compare-and-swap and means that the processor provides a separate instruction that updates the value of a register only if the provided value is equal to the current value. CAS operations can be used to avoid synchronization as the thread can try to update a value by providing its current value and the new value to the CAS operation. If another thread has meanwhile updated the value, the thread’s value is not equal to the current value and the update operation fails. The thread then reads the new value and tries again. That way the necessary synchronization is interchanged by an optimistic spin waiting. 66. Which Java classes use the CAS operation? The SDK classes in the package java.util.concurrent.atomic like AtomicInteger or AtomicBoolean use internally the CAS operation to implement concurrent incrementation. public class CounterAtomic { private AtomicLong counter = new AtomicLong();public void increment() { counter.incrementAndGet(); }public long get() { return counter.get(); } }67. Provide an example why performance improvements for single-threaded applications can cause performance degradation for multi-threaded applications. A prominent example for such optimizations is a List implementation that holds the number of elements as a separate variable. This improves the performance for single-threaded applications as the size() operation does not have to iterate over all elements but can return the current number of elements directly. Within a multi-threaded application the additional counter has to be guarded by a lock as multiple concurrent threads may insert elements into the list. This additional lock can cost performance when there are more updates to the list than invocations of the size() operation. 68. Is object pooling always a performance improvement for multi-threaded applications? Object pools that try to avoid the construction of new objects by pooling them can improve the performance of single-threaded applications as the cost for object creation is interchanged by requesting a new object from the pool. In multi-threaded applications such an object pool has to have synchronized access to the pool and the additional costs of lock contention may outweigh the saved costs of the additional construction and garbage collection of the new objects. Hence object pooling may not always improve the overall performance of a multi-threaded application. 69. What is the relation between the two interfaces Executor and ExecutorService? The interface Executor only defines one method: execute(Runnable). Implementations of this interface will have to execute the given Runnable instance at some time in the future. The ExecutorService interface is an extension of the Executor interface and provides additional methods to shut down the underlying implementation, to await the termination of all submitted tasks and it allows submitting instances of Callable. 70. What happens when you submit() a new task to an ExecutorService instance whose queue is already full? As the method signature of submit() indicates, the ExecutorService implementation is supposed to throw a RejectedExecutionException. 71. What is a ScheduledExecutorService? The interface ScheduledExecutorService extends the interface ExecutorService and adds method that allow to submit new tasks to the underlying implementation that should be executed a given point in time. There are two methods to schedule one-shot tasks and two methods to create and execute periodic tasks. 72. Do you know an easy way to construct a thread pool with 5 threads that executes some tasks that return a value? The SDK provides a factory and utility class Executors whose static method newFixedThreadPool(int nThreads) allows the creation of a thread pool with a fixed number of threads (the implementation of MyCallable is omitted): public static void main(String[] args) throws InterruptedException, ExecutionException { ExecutorService executorService = Executors.newFixedThreadPool(5); Future<Integer>[] futures = new Future[5]; for (int i = 0; i < futures.length; i++) { futures[i] = executorService.submit(new MyCallable()); } for (int i = 0; i < futures.length; i++) { Integer retVal = futures[i].get(); System.out.println(retVal); } executorService.shutdown(); }73. What is the difference between the two interfaces Runnable and Callable? The interface Runnable defines the method run() without any return value whereas the interface Callable allows the method run() to return a value and to throw an exception. 74. Which are use cases for the class java.util.concurrent.Future? Instances of the class java.util.concurrent.Future are used to represent results of asynchronous computations whose result are not immediately available. Hence the class provides methods to check if the asynchronous computation has finished, canceling the task and to the retrieve the actual result. The latter can be done with the two get() methods provided. The first get() methods takes no parameter and blocks until the result is available whereas the second get() method takes a timeout parameter that lets the method invocation return if the result does not get available within the given timeframe. 75. What is the difference between HashMap and Hashtable particularly with regard to thread-safety? The methods of Hashtable are all synchronized. This is not the case for the HashMap implementation. Hence Hashtable is thread-safe whereas HashMap is not thread-safe. For single-threaded applications it is therefore more efficient to use the “newer” HashMap implementation. 76. Is there a simple way to create a synchronized instance of an arbitrary implementation of Collection, List or Map? The utility class Collections provides the methods synchronizedCollection(Collection), synchronizedList(List) and synchronizedMap(Map) that return a thread-safe collection/list/map that is backed by the given instance. 77. What is a semaphore? A semaphore is a data structure that maintains a set of permits that have to be acquired by competing threads. Semaphores can therefore be used to control how many threads access a critical section or resource simultaneously. Hence the constructor of java.util.concurrent.Semaphore takes as first parameter the number of permits the threads compete about. Each invocation of its acquire() methods tries to obtain one of the available permits. The method acquire() without any parameter blocks until the next permit gets available. Later on, when the thread has finished its work on the critical resource, it can release the permit by invoking the method release() on an instance of Semaphore. 78. What is a CountDownLatch? The SDK class CountDownLatch provides a synchronization aid that can be used to implement scenarios in which threads have to wait until some other threads have reached the same state such that all thread can start. This is done by providing a synchronized counter that is decremented until it reaches the value zero. Having reached zero the CountDownLatch instance lets all threads proceed. This can be either used to let all threads start at a given point in time by using the value 1 for the counter or to wait until a number of threads has finished. In the latter case the counter is initialized with the number of threads and each thread that has finished its work counts the latch down by one. 79. What is the difference between a CountDownLatch and a CyclicBarrier? Both SDK classes maintain internally a counter that is decremented by different threads. The threads wait until the internal counter reaches the value zero and proceed from there on. But in contrast to the CountDownLatch the class CyclicBarrier resets the internal value back to the initial value once the value reaches zero. As the name indicates instances of CyclicBarrier can therefore be used to implement use cases where threads have to wait on each other again and again. 80. What kind of tasks can be solved by using the Fork/Join framework? The base class of the Fork/Join Framework java.util.concurrent.ForkJoinPool is basically a thread pool that executes instances of java.util.concurrent.ForkJoinTask. The class ForkJoinTask provides the two methods fork() and join(). While fork() is used to start the asynchronous execution of the task, the method join() is used to await the result of the computation. Hence the Fork/Join framework can be used to implement divide-and-conquer algorithms where a more complex problem is divided into a number of smaller and easier to solve problems. 81. Is it possible to find the smallest number within an array of numbers using the Fork/Join-Framework? The problem of finding the smallest number within an array of numbers can be solved by using a divide-and-conquer algorithm. The smallest problem that can be solved very easily is an array of two numbers as we can determine the smaller of the two numbers directly by one comparison. Using a divide-and-conquer approach the initial array is divided into two parts of equal length and both parts are provided to two instances of RecursiveTask that extend the class ForkJoinTask. By forking the two tasks they get executed and either solve the problem directly, if their slice of the array has the length two, or they again recursively divide the array into two parts and fork two new RecursiveTasks. Finally each task instance returns its result (either by having it computed directly or by waiting for the two subtasks). The root tasks then returns the smallest number in the array. 82. What is the difference between the two classes RecursiveTask and RecursiveAction? In contrast to RecursiveTask the method compute() of RecursiveAction does not have to return a value. Hence RecursiveAction can be used when the action works directly on some data structure without having to return the computed value. 83. Is it possible to perform stream operations in Java 8 with a thread pool? Collections provide the method parallelStream() to create a stream that is processed by a thread pool. Alternatively you can call the intermediate method parallel() on a given stream to convert a sequential stream to a parallel counterpart. 84. How can we access the thread pool that is used by parallel stream operations? The thread pool used for parallel stream operations can be accessed by ForkJoinPool.commonPool(). This way we can query its level of parallelism with commonPool.getParallelism(). The level cannot be changed at runtime but it can be configured by providing the following JVM parameter: -Djava.util.concurrent.ForkJoinPool.common.parallelism=5. Ok, so now you are ready for your interview! Don’t forget to check our FREE Academy course Java Concurrency Essentials! If you enjoyed this, then subscribe to our newsletter to enjoy weekly updates and complimentary whitepapers! Also, check out JCG Academy for more advanced training! You are welcome to contribute with your comments and we will include them in the article! ...
devoxx-logo

What is your “x”? The technical details about the Devoxx Keynote Demo

If everything went fine, I just got off the stage and ended the demo part of Mike’s (@mpiech) demo.We talked a lot about xPaaS. The different pieces of the puzzle that we integrated and about our newest acquisition FeedHenry. I think, that all this is worth a more detailed blog post about the inner workings of the demo. Background RedHat released a new version of OpenShift Enterprise 2.2 on the 10/10/14. This version also adds support for the private integration solutions (iPaaS) Fuse. Beside this, we recently acquired FeedHenry (mPaaS) and it was about time to show how those technologies actually integrate and help customers build solutions for tomorrows applications. I’ve been using the following overview in my latest talks to outline what a decent integration with all the different PaaS platforms might look like.On a very high level it contains exactly the parts and setup we used for today’s demo. Which is the following:I have to admit that it is far more complicated, than what we came up with in the demo on stage, but let’s not complain and get over the different parts in more details. Mobile App The mobile part of the demo was build using Cordova this is very well supported by FeedHenry, from their application console you can create builds for iOS and Android, but also easily test it locally. We used the FeedHenry client side javascript library which beside many other things allows us to generate statistics for the application management console, this can be very helpful when you have a production issue and try to diagnose the problem, for instance. Using Cordova together with Node is really nice, no need to switch paradigms and you can rapidly try out new ideas. FeedHenry Mobile Application Platform FeedHenry is a cloud-based mobile application platform to design, develop, deploy and mobile applications. The platform provides specific services for security, notification and data synchronization. You can build hybrid apps for mobile devices and it covers the complete development lifecycle. You can think of it as an Xcode IDE in the cloud which has three different components for applications. Obviously the mobile app, a server backend which you can built on top of Node.js and so-called mPaaS services which can be re-used across different applications. The interesting part of the demo are the two services. One which connects to JBoss A-MQ for xPaaS running on OSE via TCP/STOMP protocol and the other one connecting to an AeroGear unified Push Server instance running on OpenShift Online via REST. The new AeroGear MBaaS Integration Services leverages the  Node.js module already developed by the AeroGear team and provides a simple, secure way to integrate Push Notifications into your FeedHenry apps. The service itself is quick and painless to set up – simply provide the details of your AeroGear installation and credentials for the AeroGear app you want to use and you are ready to go. Using the service is just as easy – as a standard FeedHenry MBaaS service, you can call it in exactly the same way as you would any other MBaaS service – with a clean, crisp FeedHenry API call. AeroGear UnifiedPush Server on Openshift Online The AeroGear project is a one stop solution for all your Push Notification needs – covering Native iOS & Android, Cordova Hybrid as well as Simple Push for Web. It is available to use right now from the OpenShift Marketplace – so feel free to give it a whirl. JBoss Fuse and A-MQ for xPaaS on OpenShift Enterprise JBoss Fuse for xPaaS and JBoss A-MQ for xPaaS are based on Red Hat’s traditional on-premise integration and messaging offerings, Red Hat JBoss Fuse and Red Hat JBoss A-MQ. The latest versions of both products, announced at Red Hat Summit 2014, introduced features such as full support for AMQP 1.0, a vast library of connectors, improved high availability, and the ability to manage processes. In this particular example both instances running are managed via the Fuse Fabric. And the deployment of the Camel route to receive the twitter stream of tweets actually was done via a profile on the fabric. Distributing those applications and infrastructures gets insanely easy by doing it like that. At the end, the Camel rout isn’t very magical. Just a few lines of code for a log and a little conversion to JSON to make it easier for the FeedHenry A-MQ endpoint to consume it on the Node.js end. This screenshot was taken before the demo happened, I hope that we have some higher numbers after the keynote. The A-MQ side is even more simple. A basic standalone broker setup with just one queue called “tweets” like you already might have guessed. We are using two different client connectors. The Camel instance pushes messages via OpenWire and the FeedHenry service listens to them using STOMP. We’re actually not sending around binary content here, so this was the easiest setup. To be clear at this point: The Twitter integration is a nice showcase which we used to have a contact point with the audience. In real life scenarios you’re going to connect heavyweight stuff with Fuse. Like SAP, Oracle EBS, you name them. Takeaway The beauty of this is the overly complex architecture. We could have taken many shorter ways to make it work, but on the other hand, it was a very good exercise. In less than two weeks the RedHat and FeedHenry teams made both technical integrations possible. And we are very proud to have integrated this first chunk of services and help to better understand for what the different products are meant to be used. My Thank You’s Even if I had the pleasure to be on stage for the demo, I only did the very simple backend part. There are a bunch of folks, I’d like to mention here: – John Frizelle and Conor O’Neill for being our 24/7 contact into FeedHenry. There’s not a single thing those two couldn’t solve. – Jay Balunas, Erik-Jan De Wit, Sebastien Blanc and Matthias Wessendorf for developing the mobile bit’s and pieces and writing the Node.js service that we now have available in FeedHenry – Ben Parees, Grant Shipley, Marek Jelen, Hiram Chirino for all their efforts aroun OpenShift Online AND the Enterprise deployment that we used for the demo – Mike and Arun for all their support around the demo and their patience because we didn’t have it ready until the very last minute. – Christian Posta for holding my hand with all kinds of stupid Fuse questions Further Readings If you are curious, you can just get started with reading some more about what we did. Please keep in mind, that the acquisition is quite new and we still don’t have a publicly available version of FeedHenry ready for you to play with. But this is in the works. Keep your eyes open.FeedHenry and Red Hat Pushing ahead with Integrations New JBoss xPaaS offerings help developers integrate SaaS, PaaS, and on-premise applications FeedHenry meets AeroGear UnifiedPush Server!Reference: What is your “x”? The technical details about the Devoxx Keynote Demo from our JCG partner Markus Eisele at the Enterprise Software Development with Java blog....
java-interview-questions-answers

OSGi Testsuite: Introducing Classname Filters

OSGi Testsuite is a JUnit test-runner that collects dynamically test classes for execution. It has been published by my fellow Rüdiger about a year ago and proven useful in some projects already. However for gonsole we had to use an ugly patch because version 1.0 only supported .*Test postfix matching for test class names. I solved this problem with version 1.1 by introducing an annotation @ClassnameFilters that uses regular expressions to match arbitrary name patterns. This post explains in short how it works.       OSGi Testsuite OSGi Testsuite provides a JUnit test runner BundleTestSuite that can be used to run all tests within a given number of OSGi bundles. To use it annotate a class with @RunWith(BundleTestSuite.class) and specify the bundles with @TestBundles({"bundle.1", ...}). When run JUnit will process all classes in the listed bundles, which have a name ending with 'Test'. @RunWith( BundleTestSuite.class ) @TestBundles( { "org.example.bundle1", "org.example.bundle2" } ) public class MasterTestSuite {} Unfortunately the Test postfix fixation has turned out to be a bit too inflexible. Within gonsole we use different postfixes for unit and integration tests. And we do not want the unit tests to be executed within the OSGi Testsuite run. But this distinction is not possible with version 1.0. ClassnameFilters Inspired by ClasspathSuite (which works similar to OSGi Testsuite on plain JUnit tests) I introduced an annotation @ClassnameFilters. This allows to define filters based on regular expressions to match arbitrary test name patterns: @RunWith( BundleTestSuite.class ) @TestBundles( { "org.example.bundle1", "org.example.bundle2" } ) @ClassnameFilters( { ".*ITest" } ) public class IntegrationTestSuite {} Processing the example would include all the tests of classes in the listed bundles, which have a name ending with the 'ITest' postfix. Note that classes with the simple 'Test' postfix would not be processed. Furthermore it is possible to specify exclusion patterns using a leading '!': @RunWith( BundleTestSuite.class ) @TestBundles( { "org.example.bundle1", "org.example.bundle2" } ) @ClassnameFilters( { ".*ITest", "!.*FooITest" } ) public class IntegrationTestSuite {} The given example would now execute all the tests of classes in the listed bundles, which have a name ending with 'ITest' postfix except for classes whose names end with ‘FooITest’. Simple enough, isn’t it? Conclusion OSGi Testsuite has been enhanced with a filter mechanism for dynamic execution of test classes that matches arbitrary name patterns. Filter specification is done easily using the ClassnameFilters annotation and regular expressions. The code is available under the Eclipse Public License and hosted on GitHub: https://github.com/rherrmann/osgi-testsuite The latest stable version can be obtained from this p2 repository: http://rherrmann.github.io/osgi-testsuite/repositoryReference: OSGi Testsuite: Introducing Classname Filters from our JCG partner Frank Appel at the Code Affine blog....
agile-logo

Agile Through a Matrix Lens

“Agile” is something most teams do wrong*, without realizing they’re doing it wrong.  A good 2×2 matrix acts as a lens, helping to convert information into insight.  Let’s apply this lens to agile as applied within a company, and see if it helps people decide to do things differently.       When You Say Agile, What Do You Mean? There may be as many definitions of agile as there are teams practicing agile development.  Generally, people are talking about iterating in what they do.  Instead of having a long, throw it over the wall process, out of which emerges a deliverable; a team will have a series of shorter iterations where they engage stakeholders and otherwise rethink what they are doing to course-correct and “get smarter” about what they are doing.  The Wikipedia page on agile is pretty comprehensive. Most teams think about agility in terms of how their development teams manage their process.  When “going agile” is the only thing you do, your product does not magically become more successful.  Some teams** think about what it means to be agile when determining what the development team should be doing in the first place. My epiphany was in realizing that these are two separate decisions an organization can make. A 2×2 Matrix of Agile When an organization can make two discrete decisions about being agile in how they create products, it results in four possible outcomes.  A 2×2 matrix can act as a powerful lens for exploring these decisions.  Our first step is to define our two axes. Requirements – how are they treated within the organization / by the team?Requirements and expectations are immutable – this is the typical expectation within a large bureaucracy; someone built a business case, got funding, and allocated a team to deliver the product as-defined. Requirements continually revisited – this is what we see nimble teams doing – at different levels of granularity, context, and relevance; at a low level, this is A|B testing and at a high level this is a pivot.Development process cadence – how frequently does the team deliver***?Infrequent delivery – there is no one size fits all measure to define infrequent vs. frequent; some companies will have fast-moving competitors, customers with rapidly evolving expectations, and significant influence from evolving technology – others will not (for now). Frequent delivery – the precise delineation from infrequent to frequent delivery is contextually dependent.With these two axes, we can draw a matrix.A subordinate message that I couldn’t resist putting into the matrix is that it is harder to do your job in an agile way.  I think you could pedantically argue that agile is easier – by saying it is easier to deliver the equivalent results when your process is agile.  And that’s true.  The point is to deliver a more successful product, which is harder than delivering a less successful product.  An agile approach makes that task easier.  Maybe another way to think about it – if your team is not capable of delivering a good product, going agile will make that more obvious, faster. Living in Boxes Everyone can map their team into one of the four boxes.  That’s the power of this sort of abstraction. Here’s where I can use your help: What are better names for these boxes?  I have satisficed with these names, but they could be better.  Please comment below with proposed alternatives, because I’ll be incorporating this lens into other aspects of my work, and I want it to be better than it currently is. Waterfall as PracticedWhile there are some teams which consciously choose agile because of the planning benefits or perceived risks to quality, I believe that most waterfall teams are still waterfall either because they haven’t chosen to revisit their process choices, or they tried and failed.  Perhaps their instructors weren’t good, perhaps the team was not equipped to make the shift.  My guess is that their organizations were unwilling or unable to support any change in the bureaucratic status quo, effectively making it impossible for the teams to succeed. BUFD & BUFR (Buffed and Buffer)BUFR is an acronym for big up-front requirements, and BUFD is the equivalent for big up-front design.  Both of them are labels assigned as part of the decade-old war between interaction design and extreme programming.  Conceptually, the battle is between rationalists and empiricists.  In a nutshell, the requirements are Defined (capital Defined), then the team applies an agile development methodology (mostly) to incrementally build the product according to the requirements. This is another area where can explore more – what are requirements, what is design, who owns what? My main point is that the developers, while going through the agile motions – even when getting feedback – are only realizing some of the benefits of agile.  Yes, they can improve the effectiveness of their particular design or implementation at solving the intended problem.  Yes, they can avoid the death-march. The problem is that the requirements are set in stone metaphorically. At the end of the day, the team is empowered to rapidly iterate on, and change how they choose to solve the target (market) problems.  The team is not empowered to rapidly change their minds about which market problems to target. When agile is being introduced to a larger organization, as a grass-roots initiative starting with a development team, this is corner the team will find themselves in. Req’s Churn or Glacial DevI struggle for the right way to describe the situation where the people responsible for determining the requirements are getting market feedback and changing their requirements, which the people responsible for creating the product are unwilling or unable to accept changes from the initial plan. From the development team’s point of view, “the product manager can’t make up his mind – we are just churning, without getting anything done!” From the product manager’s point of view, “the development team is too slow, or intransigent, and can’t seem to keep up.” There’s only one environment where this approach is somewhat rational – outsourced development with limited trust.  When the relationship between the product management / design team, and the product creation / test team is defined by the contract, or the two teams do not trust each other, the only reasonable way to make things work is to establish explicit expectations up front, and then deliver to those specifications.  Note that the specifications typically include a change-management process, which facilitates reaching an agreement to change the plan.  The right way to make this type of relationship work is to change it, but if you’re stuck with it – this is your box. Agile as IntendedAh, the magic fourth box. Where rapid delivery leads to rapid learning which leads to rapid changes in the plan.  The success of agile is predicated on the assumption that as we get feedback from the market, we get smarter; as we get smarter, we make better choices about what to do next. This is what enables a sustainable competitive advantage, by enabling you to sustainably differentiate your product from competition, rapidly adapt to changing customer expectations and market conditions.  Effectively, you are empowered to change what you choose to do, as well as how you choose to do it.  This is what agile product management is about – enabling business agility. A winning strategy involves selecting an attractive market, developing a strategy for how you will compete within that market, then developing a product (or portfolio) roadmap which manifests the strategy, while embodying the vision of the company.  It is possible to do this in any corner of the matrix (except the upper left, in my opinion).  The less willing you are to rely on your ability to predict the future accurately, the more you will want to be in the upper right corner. Conclusion There isn’t a particularly strong argument against operating your team in the upper right hand corner of the box, Agile as Intended.  The best argument is really just “we aren’t there yet.”  From conversations I’ve had with many team leaders, they seemed to think that getting to the lower right corner was the right definition of “done.”  They thought they were “doing agile” and that there wasn’t anything left to change, organizationally.  And they wondered why their teams weren’t delivering on the promise of agile.  It’s because they weren’t there yet. Hopefully this visual will help drive the conversation forward for some of you out there.  Let me know if it helps light bulbs go off. Attributions and Clarifications Special thanks to Prabhakar Gopalan, who introduced me to The Power of the 2×2 Matrix, a really good book for framing ideas, in his craftsman product management training class. *Agile isn’t a noun really something you do, agile is an adverb describing how you do something.  English is a funny language, and “doing agile” is generally used to reflect developing a product in an agile manner.  Sometimes it is important to point this out – particularly when you’re trying to help people focus on the product and not the process (like here), but for this article, I didn’t want to dilute the other messages.  As a bonus, the people who would be tweaked by the use of agile as a noun are generally people who “get it” and I like the idea that they read the whole article, just to see this caveat.  Thanks for reading this! **This is based on anecdata (thanks Prabhakar for the great word), but my impression is that small companies commonly do this – think Lean Start Up – and large companies rarely do this.  I suspect this is more about the challenge of managing expectations and otherwise navigating a bureaucracy built to reward execution against a predetermined plan. ***Definitions of “deliver” make for a great devil is in the details discussion too – do you deliver to end-customers or internal stakeholders?  What if your existing customers refuse to update every month?  How many versions of your product do you want in the field?  Another great topic – but not the focus of this article?Reference: Agile Through a Matrix Lens from our JCG partner Scott Sehlhorst at the Tyner Blain’s blog blog....
software-development-2-logo

Don’t Migrate to MariaDB just yet. MySQL is Back!

Now that I have your attention, I’d like to invite you to a critical review of where we’re at in the MySQL vs. MariaDB debate. Around one month ago, I visited Oracle Open World 2014, and I’ve met with Morgan Tocker, the MySQL community manager at Oracle to learn about where MySQL is heading. Who “is” MySQL An interesting learning for myself is the fact that according to Morgan, there had been quite a few former MySQL AB employees that stayed with MySQL when Sun acquired the database, and that are still there now (perhaps after a short break), now that Oracle owns Sun and thus MySQL. In fact, right now as we speak, Oracle is pushing hard to get even more people on board of the MySQL team. When this article was written, there were 21 open MySQL jobs on this blog post dating February 25, 2014. More details about Oracle’s plans to promote and push MySQL can be seen in this presentation by Morten Andersen:So, if you’re still contemplating a migration to MariaDB, maybe give this all a second thought. MySQL is still the database that is being used by most large companies, including Facebook and LinkedIn. MySQL won’t go away and it will get much better in the next couple of years – maybe even to a point where it has a similar amount of features as PostgreSQL? We can only hope. Need another convincing argument? You can now use Oracle Enterprise Manager for both Oracle and MySQL databases. How awesome is that!Stay tuned for a couple of additional blog posts on this blog, about what’s new and what’s on the MySQL roadmap in the next years. See all of Morten Andersen’s slides here: Introduction & News from OOW and MySQL Central from Morten AndersenReference: Don’t Migrate to MariaDB just yet. MySQL is Back! from our JCG partner Lukas Eder at the JAVA, SQL, AND JOOQ blog....
Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy | Contact
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.
Do you want to know how to develop your skillset and become a ...
Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

Get ready to Rock!
You can download the complementary eBooks using the links below:
Close