Featured FREE Whitepapers

What's New Here?

jool-logo-black

Let’s Stream a Map in Java 8 with jOOλ

I wanted to find an easy way to stream a Map in Java 8. Guess what? There isn’t! What I would’ve expected for convenience is the following method:                   public interface Map<K, V> {default Stream<Entry<K, V>> stream() { return entrySet().stream(); } } But there’s no such method. There are probably a variety of reasons why such a method shouldn’t exist, e.g.:There’s no “clear” preference for entrySet() being chosen over keySet() or values(), as a stream source Map isn’t really a collection. It’s not even an Iterable That wasn’t the design goal The EG didn’t have enough timeWell, there is a very compelling reason for Map to have been retrofitted to provide both an entrySet().stream() and to finally implement Iterable<Entry<K, V>>. And that reason is the fact that we now have Map.forEach(): default void forEach( BiConsumer<? super K, ? super V> action) { Objects.requireNonNull(action); for (Map.Entry<K, V> entry : entrySet()) { K k; V v; try { k = entry.getKey(); v = entry.getValue(); } catch(IllegalStateException ise) { // this usually means the entry is no longer in the map. throw new ConcurrentModificationException(ise); } action.accept(k, v); } } forEach() in this case accepts a BiConsumer that really consumes entries in the map. If you search through JDK source code, there are really very few references to the BiConsumer type outside of Map.forEach() and perhaps a couple of CompletableFuture methods and a couple of streams collection methods. So, one could almost assume that BiConsumer was strongly driven by the needs of this forEach() method, which would be a strong case for making Map.Entry a more important type throughout the collections API (we would have preferred the type Tuple2, of course). Let’s continue this line of thought. There is also Iterable.forEach(): public interface Iterable<T> { default void forEach(Consumer<? super T> action) { Objects.requireNonNull(action); for (T t : this) { action.accept(t); } } } Both Map.forEach() and Iterable.forEach() intuitively iterate the “entries” of their respective collection model, although there is a subtle difference:Iterable.forEach() expects a Consumer taking a single value Map.forEach() expects a BiConsumer taking two values: the key and the value (NOT a Map.Entry!)Think about it this way:This makes the two methods incompatible in a “duck typing sense”, which makes the two types even more differentBummer! Improving Map with jOOλ We find that quirky and counter-intuitive. forEach() is really not the only use-case of Map traversal and transformation. We’d love to have a Stream<Entry<K, V>>, or even better, a Stream<Tuple2<T1, T2>>. So we implemented that in jOOλ, a library which we’ve developed for our integration tests at jOOQ. With jOOλ, you can now wrap a Map in a Seq type (“Seq” for sequential stream, a stream with many more functional features): Map<Integer, String> map = new LinkedHashMap<>(); map.put(1, "a"); map.put(2, "b"); map.put(3, "c");assertEquals( Arrays.asList( tuple(1, "a"), tuple(2, "b"), tuple(3, "c") ),Seq.seq(map).toList() ); What you can do with it? How about creating a new Map, swapping keys and values in one go: System.out.println( Seq.seq(map) .map(Tuple2::swap) .toMap(Tuple2::v1, Tuple2::v2) );System.out.println( Seq.seq(map) .toMap(Tuple2::v2, Tuple2::v1) ); Both of the above will yield: {a=1, b=2, c=3} Just for the record, here’s how to swap keys and values with standard JDK API: System.out.println( map.entrySet() .stream() .collect(Collectors.toMap( Map.Entry::getValue, Map.Entry::getKey )) ); It can be done, but the every day verbosity of standard Java API makes things a bit hard to read / write.Reference: Let’s Stream a Map in Java 8 with jOOλ from our JCG partner Lukas Eder at the JAVA, SQL, AND JOOQ blog....
software-development-2-logo

Stop Claiming that you’re Using a Schemaless Database

One of MongoDB’s arguments when evangelising MongoDB is the fact that MongoDB is a “schemaless” database: Why Schemaless? MongoDB is a JSON-style data store. The documents stored in the database can have varying sets of fields, with different types for each field.     And that’s true. But it doesn’t mean that there is no schema. There are in fact various schemas:The one in your head when you designed the data structures The one that your database really implemented to store your data structures The one you should have implemented to fulfill your requirementsEvery time you realise that you made a mistake (see point three above), or when your requirements change, you will need to migrate your data. Let’s review again MongoDB’s point of view here: With a schemaless database, 90% of the time adjustments to the database become transparent and automatic. For example, if we wish to add GPA to the student objects, we add the attribute, resave, and all is well — if we look up an existing student and reference GPA, we just get back null. Further, if we roll back our code, the new GPA fields in the existing objects are unlikely to cause problems if our code was well written. Everything above is true as well. “Schema-less” vs. “Schema-ful” But let’s translate this to SQL (or use any other “schema-ful” database instead): ALTER TABLE student ADD gpa VARCHAR(10); And we’re done! Gee, we’ve added a column, and we’ve added it to ALL rows. It was transparent. It was automatic. We “just get back null” on existing students. And we can even “roll back our code”: ALTER TABLE student DROP gpa; Not only are the existing objects unlikely to cause problems, we have actually rolled back our code AND database. Let’s summarise:We can do exactly the same in “schema-less” databases as we can in “schema-ful” ones We guarantee that a migration takes place (and it’s instant, too) We guarantee data integrity when we roll back the changeWhat about more real-world DDL? Of course, at the beginning of projects, when they still resemble the typical cat/dog/pet-shop, book/author/library sample application, we’ll just be adding columns. But what happens if we need to change the student-teacher 1:N relationship into a student-teacher M:N relationship? Suddenly, everything changes, and not only will the relational data model prove far superior to a hierarchical one that just yields tons of data duplication, it’ll also be moderately easy to migrate, and the outcome is guaranteed to be correct and tidy! CREATE TABLE student_to_teacher AS SELECT id AS student_id, teacher_id FROM student;ALTER TABLE student DROP teacher_id; … and we’re done! (of course, we’d be adding constraints and indexes) Think about the tedious task that you’ll have transforming your JSON to the new JSON. You don’t even have XSLT or XQuery for the task, only JavaScript! Let’s face the truth Schemalessness is about a misleading term as much as NoSQL is: History of NoSQL according to @markmadsen #strataconf pic.twitter.com/XHXMJsXHjV — Edd Dumbill (@edd) November 12, 2013And again, MongoDB’s blog post is telling the truth (and an interesting one, too): Generally, there is a direct analogy between this “schemaless” style and dynamically typed languages. Constructs such as those above are easy to represent in PHP, Python and Ruby. What we are trying to do here is make this mapping to the database natural. When you say “schemaless”, you actually say “dynamically typed schema” – as opposed to statically typed schemas as they are available from SQL databases. JSON is still a completely schema free data structure standard, as opposed to XML which allows you to specify XSD if you need, or operate on document-oriented, “schema-less” (i.e. dynamically typed) schemas. (And don’t say there’s json-schema. That’s a ridiculous attempt to mimick XSD) This is important to understand! You always have a schema, even if you don’t statically type it. If you’re writing JavaScript, you still have types, which you have to be fully aware of in your mental model of the code. Except that there’s no compiler (or IDE) that can help you infer the types with 100% certainty. An example: LOL JavaScript: > null * 1 0 > null == 0 false pic.twitter.com/Hc2NR2tsMP — Lukas Eder (@lukaseder) October 15, 2014… and more:So, there’s absolutely nothing that is really easier with “schemaless” databases than with “schemaful” ones. You just defer the inevitable work of sanitising your schema to some other later time, a time when you might care more than today, or a time when you’re lucky enough to have a new job and someone else does the work for you. You might have believed MongoDB, when they said that “objects are unlikely to cause problems”. But let me tell you the ugly truth: Anything that can possibly go wrong, does – Murphy We wish you good luck with your dynamically typed languages and your dynamically typed database schemas – while we’ll stick with type safe SQL.Reference: Stop Claiming that you’re Using a Schemaless Database from our JCG partner Lukas Eder at the JAVA, SQL, AND JOOQ blog....
software-development-2-logo

ConEmu – Windows console emulator with tabs

After switching to Git some time ago, I started working more and more with Git Bash on Windows. Git Bash is pretty cool as it provides (apart from Git) Bash supported with basic Unix tools including curl or ssh. Git Bash in Windows has some limitation though including limited customization options and lack of good copy & paste options supported with keyboard shortcuts. Fortunately, there is ConEmu that not only fills that gap but adds various features that make working with console applications more productive and more enjoyable for me.         Introduction ConEmu is a Windows console emulator with tabs, which presents multiple consoles and simple GUI applications as one customizable GUI window with various features. And not only working with Git Bash is far better with ConEmu, but with other tools I use too:Far Manager – a program for managing files and archives in Windows – handy Notepad++ – source code editor and Notepad replacement – naturally! cmd (Windows command prompt) – I still use it, rarely but stillPractically, running any tool should not be a problem. Let’s say I want to run my favorite password manager in ConEmu I can execute the following command: $ <KeePassHome>/keepass.exe -new_console -new_console switch instructs ConEmu to start an application in a new console. Working with tabs Controlling tabs I disabled most confirmation on tabs actions (Settings > Main > Confirm), so now I can fully control tab creation, closing and switching between them with shortcuts without additional confirmations. Most commonly used shortcuts for working with tabs:Win + N – show New Console Dialog (e.g. for running tasks with no shortcuts assigned) Win + X – new cmd console Win + Delete – for closing active tab Win + <Num> – switch between tabs (alternative Ctrl+Tab)Split Screen ConEmu may split any tab into several panes:The most common shortcuts to work with Split Screen:Win+N – show New Console Dialog and select Split Screen options Ctrl+Shift+O – duplicate shell from active pane and split horizontally Ctrl+Shift+E – duplicate shell from active pane and split verticallyYou navigate between screens in Split Screen mode just like you navigate between tabs. Tasks Git Bash is my favorite shell on Windows, therefore I made it a startup task in ConEmu:In addition I added Far Manager and Notepad++ tasks and I associated hot keys for them:Win+B,F,P – Git Bash, FAR and Notepad++.Even if you choose shortcuts that are used by Windows, ConEmu will intercept them (once active). Working with text Highlighting, copying & pasting with mouse and keyboard shortcuts makes it really convenient. This is one of the features I appreciate most in ConEmu.Shortcuts:copying current selection with Ctrl + C pasting with Shift+Insert, Ctrl+V (only first line) or with right mouse click, selecting text Shift+Arrow Keys/Home/End or with right click and dragAdditionally, scrolling buffer is also easy withCtrl+Up/Down/PgUp/PgDown shortcuts. Notepad++ Notepad++ is one of my favorite editors for Windows. ConEmu can run Notepad++ in a tab with no problem. I created a task for Notepad++ so I can start it in a new tab whenever I want. In addition, I made possible to run it in console with the file loaded that is passed as argument. This is very easy with Git Bash. Make sure Notepad++ is in the PATH and create an alias: alias edit="notepad++ -new_console" Now, edit filename will run Notepad++ with filename loaded in a new tab.In case you want this alias to be always available, create .bashrc file in your home directory (if does not exist) and add the alias so it is automatically loaded on Git Bash startup. Summary I’ve been using ConEmu for several weeks now and I am far from knowing everything about it, but I already can’t imagine my Windows without it! With ConEmu I can use my favorite tools like Git Bash, cmd, Far Manager, and Notepad++ in one application with great tabbing experience supported with shortcuts. Font anti-aliasing, transparency (can be configured for active and inactive window separately), full screen, split screen and great mark, highlight, copy & paste options makes ConEmu is a complete application and a great choice for developers looking for improved productivity in Windows. I truly recommend ConEmu to every professional! References Project home page: https://code.google.com/p/conemu-maximus5Reference: ConEmu – Windows console emulator with tabs from our JCG partner Rafal Borowiec at the Codeleak.pl blog....
neo4j-logo

Neo4j: Cypher – Avoiding the Eager

  Although I love how easy Cypher’s LOAD CSV command makes it to get data into Neo4j, it currently breaks the rule of least surprise in the way it eagerly loads in all rows for some queries even those using periodic commit. This is something that my colleague Michael noted in the second of his blog posts explaining how to use LOAD CSV successfully:  The biggest issue that people ran into, even when following the advice I gave earlier, was that for large imports of more than one million rows, Cypher ran into an out-of-memory situation. That was not related to commit sizes, so it happened even with PERIODIC COMMIT of small batches.I recently spent a few days importing data into Neo4j on a Windows machine with 4GB RAM so I was seeing this problem even earlier than Michael suggested. Michael explains how to work out whether your query is suffering from unexpected eager evaluation:If you profile that query you see that there is an “Eager” step in the query plan. That is where the “pull in all data” happens.You can profile queries by prefixing the word ‘PROFILE’. You’ll need to run your query in the console of /webadmin in your web browser or with the Neo4j shell. I did this for my queries and was able to identify query patterns which get evaluated eagerly and in some cases we can work around it. We’ll use the Northwind data set to demonstrate how the Eager pipe can sneak into our queries but keep in mind that this data set is sufficiently small to not cause issues. This is what a row in the file looks like: $ head -n 2 data/customerDb.csv OrderID,CustomerID,EmployeeID,OrderDate,RequiredDate,ShippedDate,ShipVia,Freight,ShipName,ShipAddress,ShipCity,ShipRegion,ShipPostalCode,ShipCountry,CustomerID,CustomerCompanyName,ContactName,ContactTitle,Address,City,Region,PostalCode,Country,Phone,Fax,EmployeeID,LastName,FirstName,Title,TitleOfCourtesy,BirthDate,HireDate,Address,City,Region,PostalCode,Country,HomePhone,Extension,Photo,Notes,ReportsTo,PhotoPath,OrderID,ProductID,UnitPrice,Quantity,Discount,ProductID,ProductName,SupplierID,CategoryID,QuantityPerUnit,UnitPrice,UnitsInStock,UnitsOnOrder,ReorderLevel,Discontinued,SupplierID,SupplierCompanyName,ContactName,ContactTitle,Address,City,Region,PostalCode,Country,Phone,Fax,HomePage,CategoryID,CategoryName,Description,Picture 10248,VINET,5,1996-07-04,1996-08-01,1996-07-16,3,32.38,Vins et alcools Chevalier,59 rue de l'Abbaye,Reims,,51100,France,VINET,Vins et alcools Chevalier,Paul Henriot,Accounting Manager,59 rue de l'Abbaye,Reims,,51100,France,26.47.15.10,26.47.15.11,5,Buchanan,Steven,Sales Manager,Mr.,1955-03-04,1993-10-17,14 Garrett Hill,London,,SW1 8JR,UK,(71) 555-4848,3453,\x,"Steven Buchanan graduated from St. Andrews University, Scotland, with a BSC degree in 1976. Upon joining the company as a sales representative in 1992, he spent 6 months in an orientation program at the Seattle office and then returned to his permanent post in London. He was promoted to sales manager in March 1993. Mr. Buchanan has completed the courses ""Successful Telemarketing"" and ""International Sales Management."" He is fluent in French.",2,http://accweb/emmployees/buchanan.bmp,10248,11,14,12,0,11,Queso Cabrales,5,4,1 kg pkg.,21,22,30,30,0,5,Cooperativa de Quesos 'Las Cabras',Antonio del Valle Saavedra,Export Administrator,Calle del Rosal 4,Oviedo,Asturias,33007,Spain,(98) 598 76 54,,,4,Dairy Products,Cheeses,\x MERGE, MERGE, MERGE The first thing we want to do is create a node for each employee and each order and then create a relationship between them. We might start with the following query: USING PERIODIC COMMIT 1000 LOAD CSV WITH HEADERS FROM "file:/Users/markneedham/projects/neo4j-northwind/data/customerDb.csv" AS row MERGE (employee:Employee {employeeId: row.EmployeeID}) MERGE (order:Order {orderId: row.OrderID}) MERGE (employee)-[:SOLD]->(order) This does the job but if we profile the query like so… PROFILE LOAD CSV WITH HEADERS FROM "file:/Users/markneedham/projects/neo4j-northwind/data/customerDb.csv" AS row WITH row LIMIT 0 MERGE (employee:Employee {employeeId: row.EmployeeID}) MERGE (order:Order {orderId: row.OrderID}) MERGE (employee)-[:SOLD]->(order) …we’ll notice an ‘Eager’ lurking on the third line: ==> +----------------+------+--------+----------------------------------+-----------------------------------------+ ==> | Operator | Rows | DbHits | Identifiers | Other | ==> +----------------+------+--------+----------------------------------+-----------------------------------------+ ==> | EmptyResult | 0 | 0 | | | ==> | UpdateGraph(0) | 0 | 0 | employee, order, UNNAMED216 | MergePattern | ==> | Eager | 0 | 0 | | | ==> | UpdateGraph(1) | 0 | 0 | employee, employee, order, order | MergeNode; :Employee; MergeNode; :Order | ==> | Slice | 0 | 0 | | { AUTOINT0} | ==> | LoadCSV | 1 | 0 | row | | ==> +----------------+------+--------+----------------------------------+-----------------------------------------+ You’ll notice that when we profile each query we’re stripping off the periodic commit section and adding a ‘WITH row LIMIT 0′. This allows us to generate enough of the query plan to identify the ‘Eager’ operator without actually importing any data. We want to split that query into two so it can be processed in a non eager manner: USING PERIODIC COMMIT 1000 LOAD CSV WITH HEADERS FROM "file:/Users/markneedham/projects/neo4j-northwind/data/customerDb.csv" AS row WITH row LIMIT 0 MERGE (employee:Employee {employeeId: row.EmployeeID}) MERGE (order:Order {orderId: row.OrderID}) ==> +-------------+------+--------+----------------------------------+-----------------------------------------+ ==> | Operator | Rows | DbHits | Identifiers | Other | ==> +-------------+------+--------+----------------------------------+-----------------------------------------+ ==> | EmptyResult | 0 | 0 | | | ==> | UpdateGraph | 0 | 0 | employee, employee, order, order | MergeNode; :Employee; MergeNode; :Order | ==> | Slice | 0 | 0 | | { AUTOINT0} | ==> | LoadCSV | 1 | 0 | row | | ==> +-------------+------+--------+----------------------------------+-----------------------------------------+ Now that we’ve created the employees and orders we can join them together: USING PERIODIC COMMIT 1000 LOAD CSV WITH HEADERS FROM "file:/Users/markneedham/projects/neo4j-northwind/data/customerDb.csv" AS row MATCH (employee:Employee {employeeId: row.EmployeeID}) MATCH (order:Order {orderId: row.OrderID}) MERGE (employee)-[:SOLD]->(order) ==> +----------------+------+--------+-------------------------------+-----------------------------------------------------------+ ==> | Operator | Rows | DbHits | Identifiers | Other | ==> +----------------+------+--------+-------------------------------+-----------------------------------------------------------+ ==> | EmptyResult | 0 | 0 | | | ==> | UpdateGraph | 0 | 0 | employee, order, UNNAMED216 | MergePattern | ==> | Filter(0) | 0 | 0 | | Property(order,orderId) == Property(row,OrderID) | ==> | NodeByLabel(0) | 0 | 0 | order, order | :Order | ==> | Filter(1) | 0 | 0 | | Property(employee,employeeId) == Property(row,EmployeeID) | ==> | NodeByLabel(1) | 0 | 0 | employee, employee | :Employee | ==> | Slice | 0 | 0 | | { AUTOINT0} | ==> | LoadCSV | 1 | 0 | row | | ==> +----------------+------+--------+-------------------------------+-----------------------------------------------------------+ Not an Eager in sight! MATCH, MATCH, MATCH, MERGE, MERGE If we fast forward a few steps we may now have refactored our import script to the point where we create our nodes in one query and the relationships in another query. Our create query works as expected: USING PERIODIC COMMIT 1000 LOAD CSV WITH HEADERS FROM "file:/Users/markneedham/projects/neo4j-northwind/data/customerDb.csv" AS row MERGE (employee:Employee {employeeId: row.EmployeeID}) MERGE (order:Order {orderId: row.OrderID}) MERGE (product:Product {productId: row.ProductID}) ==> +-------------+------+--------+----------------------------------------------------+--------------------------------------------------------------+ ==> | Operator | Rows | DbHits | Identifiers | Other | ==> +-------------+------+--------+----------------------------------------------------+--------------------------------------------------------------+ ==> | EmptyResult | 0 | 0 | | | ==> | UpdateGraph | 0 | 0 | employee, employee, order, order, product, product | MergeNode; :Employee; MergeNode; :Order; MergeNode; :Product | ==> | Slice | 0 | 0 | | { AUTOINT0} | ==> | LoadCSV | 1 | 0 | row | | ==> +-------------+------+--------+----------------------------------------------------+------------------------------------------------------------ We’ve now got employees, products and orders in the graph. Now let’s create relationships between the trio: USING PERIODIC COMMIT 1000 LOAD CSV WITH HEADERS FROM "file:/Users/markneedham/projects/neo4j-northwind/data/customerDb.csv" AS row MATCH (employee:Employee {employeeId: row.EmployeeID}) MATCH (order:Order {orderId: row.OrderID}) MATCH (product:Product {productId: row.ProductID}) MERGE (employee)-[:SOLD]->(order) MERGE (order)-[:PRODUCT]->(product) If we profile that we’ll notice Eager has sneaked in again! ==> +----------------+------+--------+-------------------------------+-----------------------------------------------------------+ ==> | Operator | Rows | DbHits | Identifiers | Other | ==> +----------------+------+--------+-------------------------------+-----------------------------------------------------------+ ==> | EmptyResult | 0 | 0 | | | ==> | UpdateGraph(0) | 0 | 0 | order, product, UNNAMED318 | MergePattern | ==> | Eager | 0 | 0 | | | ==> | UpdateGraph(1) | 0 | 0 | employee, order, UNNAMED287 | MergePattern | ==> | Filter(0) | 0 | 0 | | Property(product,productId) == Property(row,ProductID) | ==> | NodeByLabel(0) | 0 | 0 | product, product | :Product | ==> | Filter(1) | 0 | 0 | | Property(order,orderId) == Property(row,OrderID) | ==> | NodeByLabel(1) | 0 | 0 | order, order | :Order | ==> | Filter(2) | 0 | 0 | | Property(employee,employeeId) == Property(row,EmployeeID) | ==> | NodeByLabel(2) | 0 | 0 | employee, employee | :Employee | ==> | Slice | 0 | 0 | | { AUTOINT0} | ==> | LoadCSV | 1 | 0 | row | | ==> +----------------+------+--------+-------------------------------+-----------------------------------------------------------+ In this case the Eager happens on our second call to MERGE as Michael identified in his post:The issue is that within a single Cypher statement you have to isolate changes that affect matches further on, e.g. when you CREATE nodes with a label that are suddenly matched by a later MATCH or MERGE operation.We can work around the problem in this case by having separate queries to create the relationships: LOAD CSV WITH HEADERS FROM "file:/Users/markneedham/projects/neo4j-northwind/data/customerDb.csv" AS row MATCH (employee:Employee {employeeId: row.EmployeeID}) MATCH (order:Order {orderId: row.OrderID}) MERGE (employee)-[:SOLD]->(order) ==> +----------------+------+--------+-------------------------------+-----------------------------------------------------------+ ==> | Operator | Rows | DbHits | Identifiers | Other | ==> +----------------+------+--------+-------------------------------+-----------------------------------------------------------+ ==> | EmptyResult | 0 | 0 | | | ==> | UpdateGraph | 0 | 0 | employee, order, UNNAMED236 | MergePattern | ==> | Filter(0) | 0 | 0 | | Property(order,orderId) == Property(row,OrderID) | ==> | NodeByLabel(0) | 0 | 0 | order, order | :Order | ==> | Filter(1) | 0 | 0 | | Property(employee,employeeId) == Property(row,EmployeeID) | ==> | NodeByLabel(1) | 0 | 0 | employee, employee | :Employee | ==> | Slice | 0 | 0 | | { AUTOINT0} | ==> | LoadCSV | 1 | 0 | row | | ==> +----------------+------+--------+-------------------------------+-----------------------------------------------------------+ USING PERIODIC COMMIT 1000 LOAD CSV WITH HEADERS FROM "file:/Users/markneedham/projects/neo4j-northwind/data/customerDb.csv" AS row MATCH (order:Order {orderId: row.OrderID}) MATCH (product:Product {productId: row.ProductID}) MERGE (order)-[:PRODUCT]->(product) ==> +----------------+------+--------+------------------------------+--------------------------------------------------------+ ==> | Operator | Rows | DbHits | Identifiers | Other | ==> +----------------+------+--------+------------------------------+--------------------------------------------------------+ ==> | EmptyResult | 0 | 0 | | | ==> | UpdateGraph | 0 | 0 | order, product, UNNAMED229 | MergePattern | ==> | Filter(0) | 0 | 0 | | Property(product,productId) == Property(row,ProductID) | ==> | NodeByLabel(0) | 0 | 0 | product, product | :Product | ==> | Filter(1) | 0 | 0 | | Property(order,orderId) == Property(row,OrderID) | ==> | NodeByLabel(1) | 0 | 0 | order, order | :Order | ==> | Slice | 0 | 0 | | { AUTOINT0} | ==> | LoadCSV | 1 | 0 | row | | ==> +----------------+------+--------+------------------------------+--------------------------------------------------------+ MERGE, SET I try to make LOAD CSV scripts as idempotent as possible so that if we add more rows or columns of data to our CSV we can rerun the query without having to recreate everything. This can lead you towards the following pattern where we’re creating suppliers: USING PERIODIC COMMIT 1000 LOAD CSV WITH HEADERS FROM "file:/Users/markneedham/projects/neo4j-northwind/data/customerDb.csv" AS row MERGE (supplier:Supplier {supplierId: row.SupplierID}) SET supplier.companyName = row.SupplierCompanyName We want to ensure that there’s only one Supplier with that SupplierID but we might be incrementally adding new properties and decide to just replace everything by using the ‘SET’ command. If we profile that query, the Eager lurks: ==> +----------------+------+--------+--------------------+----------------------+ ==> | Operator | Rows | DbHits | Identifiers | Other | ==> +----------------+------+--------+--------------------+----------------------+ ==> | EmptyResult | 0 | 0 | | | ==> | UpdateGraph(0) | 0 | 0 | | PropertySet | ==> | Eager | 0 | 0 | | | ==> | UpdateGraph(1) | 0 | 0 | supplier, supplier | MergeNode; :Supplier | ==> | Slice | 0 | 0 | | { AUTOINT0} | ==> | LoadCSV | 1 | 0 | row | | ==> +----------------+------+--------+--------------------+----------------------+ We can work around this at the cost of a bit of duplication using ‘ON CREATE SET’ and ‘ON MATCH SET': USING PERIODIC COMMIT 1000 LOAD CSV WITH HEADERS FROM "file:/Users/markneedham/projects/neo4j-northwind/data/customerDb.csv" AS row MERGE (supplier:Supplier {supplierId: row.SupplierID}) ON CREATE SET supplier.companyName = row.SupplierCompanyName ON MATCH SET supplier.companyName = row.SupplierCompanyName ==> +-------------+------+--------+--------------------+----------------------+ ==> | Operator | Rows | DbHits | Identifiers | Other | ==> +-------------+------+--------+--------------------+----------------------+ ==> | EmptyResult | 0 | 0 | | | ==> | UpdateGraph | 0 | 0 | supplier, supplier | MergeNode; :Supplier | ==> | Slice | 0 | 0 | | { AUTOINT0} | ==> | LoadCSV | 1 | 0 | row | | ==> +-------------+------+--------+--------------------+----------------------+ With the data set I’ve been working with I was able to avoid OutOfMemory exceptions in some cases and reduce the amount of time it took to run the query by a factor of 3 in others. As time goes on I expect all of these scenarios will be addressed but as of Neo4j 2.1.5 these are the patterns that I’ve identified as being overly eager. If you know of any others do let me know and I can add them to the post or write a second part.Reference: Neo4j: Cypher – Avoiding the Eager from our JCG partner Mark Needham at the Mark Needham Blog blog....
akka-logo

Akka Notes – Child Actors and ActorPath – 6

Actors are completely hierarchical. Whatever Actors that you create HAS to be a child of some other Actor. Let’s analyze that a bit : Path Say, we create an ActorRef using ActorSystem.actorOf and try to print it’s path.       val actorSystem=ActorSystem("SupervisionActorSystem") val actorRef=actorSystem.actorOf(Props[BasicLifecycleLoggingTeacherActor]) println (actorRef.path) // (prints) akka://SupervisionActorSystem/user/$a As you see, a path looks very similar to a file path in a file system.akka here is fixed because all these are addresses of Akka Actors – more like file:// or http:// prefix (nothing to do with protocol though). The SupervisionActorSystem is just the name of your ActorSystem that you created. We’ll talk about the user in the next section. The $a is the name of your Actor that the system generated for you. How would you like if your operating system generated random file names for your files? You’d obviously hate it because you would want to refer to that name in future. So, let’s give it a proper meaningful name to it :val actorRef=actorSystem.actorOf(Props[BasicLifecycleLoggingTeacherActor], "teacherActor") println (actorRef.path) // (prints) akka://SupervisionActorSystem/user/teacherActor That’s it. Now, the path actually makes sense.Child Actors Similar to top level actors which we create out of ActorSystem, we could also create child actors out of the ActorContext. In fact, the power of Actors’s fault tolerance primarily lie within leveraging the Actor hierarchy and the ability of a parent to manage the life of child actors. Assume you have a TeacherSupervisor and you would like to create a TeacherActor to be a child of the Supervisor, you do a ActorContext.actorOf instead of a ActorSystem.actorOf: class TeacherSupervisor extends Actor with ActorLogging { val teacherActor=context.actorOf(Props[TeacherActor], "teacherActor") ... ... Frankly, in any application, you will be creating a whole lot of child actors than top level actors – which means that you’ll be calling a lot more actorContext.actorOf than actorSystem.actorOf.You’ll notice that the path of the child actor is akka://SupervisionActorSystem/user/teacherSupervisor/teacherActor, which is very similar to the path you get when you create a child folder within a parent folder. When do you create child Actors? You generally create child actors when a particular task is composed of a subtask or multiple subtasks. You also create a child actor when a particular task to be executed by the parent is error-prone and you would want to isolate it (so that if it fails, you could recover). When there is no parent child relationship between tasks, then you DON’T create child actors. Also, there’s nothing which is stopping a child actor from creating children to delegate its subtasks. Actors and their creation are really cheap but the power that comes with it is amazing (we’ll see about this while we talk about supervision). Now what is that user in the path? For lack of a creativity, let me compare the ActorSystem to a Unix file system – with a / root folder and all those /etc, /usr, /bin and various other folders. ActorSystem are much like that. It creates a few top level Actors – the most important being the root Actor with the path /, the user Actor with the path /user and a system Actor with the path /system. (there’s also a /deadLetters that represent the DeadLetterActorRef. We saw this in our previous post) Codewise, the ActorSystem composes three Actors inside it (via ActorRefProvider). Those are the root for ALL the Actors that gets created under the ActorSystem.systemGuardian actor – root of all actors under /system guardian actor – root of all actors under /user and rootGuardian Actor – root of both the systemGuardian and the userGuardian actors./** * Reference to the supervisor of guardian and systemGuardian; .... */ def rootGuardian: InternalActorRef/** * Reference to the supervisor used for all top-level user actors. */ def guardian: LocalActorRef/** * Reference to the supervisor used for all top-level system actors. */ def systemGuardian: LocalActorRef/user (aka) user guardian Any Actor that you create in your program like the StudentActor or the TeacherActor using the ActorSystem‘s actorOf method would directly fall under /user. That’s the reason your teacherActor in the first part of this write-up had the path slug /user/teacherActor. /system (aka) system guardian The system guardian shuts itself down when it notices that the userGuardian is dead. Makes sense considering if the userGuardian is down, then all the business actors under it is also down and therefore all administrative actors needs to be gone too. We could see two distinct places where System Actors are being created – I mean actors under the /system hierarchy.Like we saw earlier, any message that you send to an Actor that is terminated gets forwarded to the mailbox of an internal Actor called DeadLetterActor. The DeadLetter Actor wraps each message as a DeadLetter and publishes it to the EventStream. One other Actor called DeadLetterListener consumes all DeadLetters and publishes that as a log message. Now, the DeadLetterListener is a system Actor with path /system/deadLetterListener. Remember the TestEventListener that we created in our previous write-up to subscribe to log messages in the EventStream? They are System actors too. In fact, all akka.loggers are created as System actors.class TeacherTest extends TestKit(ActorSystem("UniversityMessageSystem", ConfigFactory.parseString("""akka.loggers = ["akka.testkit.TestEventListener"]"""))) ... ... The documentation here says that any Actor that is configured in the configuration files and created and deployed into the ActorSystem while it starts, falls under the /system umbrella. Let me update this post when I find something interesting around this / (aka) root guardian As we saw earlier, the / Actor is the parent of the user and the system guardians. TRIVIA Technically, there’s a superfluous parent for the root actor too. This Actor’s only job is to shut down the entire ActorSystem if the root actor fails. Since it’s strictly not considered within the Actor hierarchy, the Akka team calls it : private[akka] val theOneWhoWalksTheBubblesOfSpaceTime: InternalActorRef = new MinimalActorRef { ...Reference: Akka Notes – Child Actors and ActorPath – 6 from our JCG partner Arun Manivannan at the Rerun.me blog....
java-logo

Why to use String

Recently I was tutoring juniors during a training session. One of the task was to write a class that can dwarwle maps based on some string key. The result one of the juniors created contained the following method:                 void dwarwle(HashMap<String,Dwarwable> mapToDwarwle, String dwarwleKey){ for( final Entry<String, Dwarwable> entry : mapToDwarwle.entrySet()){ dwarwle(entry.getKey(),entry.getValue(),dwarwleKey); } } The code was generally ok. The method to dwarwle an individual dwarwable entry using the actual key it is assigned to in the hash map and the dwarwle key is factored to a separate method. It is so simple that I do not list here. The variable names are also meaningful so long as long you know what actually dwarwling is. The method is short and readable, but the argument list expects a HashMap instead of a Map. Why do we want to restrict the caller to use a HashMap? What if the caller has a TreeMap and for good reason. Do we want to have separate method that can dwarwle TreeMap? Certainly not.Expect the interface, pass the implementation.The junior changed the code replacing HashMap to Map but after five minutes or so this clever lady raised her hand and had the following question: “If we changed HashMap to Map, why did not we change String to CharSequence?” It is not so easy to answer a question like that when it comes out of the blue. The first thing that came up in my mind is that the reason is that we usually do it that way and that is why. But that is not a real argument, at least I would not accept anything like that and I also except my students not to accept such answer. It would be very dictator style anyway. The real answer is that the parameter is used as a key in a map and the key of a map should be immutable (at least mutation should be resilient to equals and hashcode calculation). CharSequence is an interface and an interface in Java (unfortunately) can not guarantee immutability. Only implementation can. String is a good, widely known and well tested implementation of this interface and therefore can be a good choice. There is a good discussion about it on stackoverflow. In this special case we expect the implementation because we need something immutable and we “can not” trust the caller to pass a character sequence implementation that is immutable. Or: we can, but it has a price. If a StringBuilder is passed and modified afterwards our dwarwling library may not work and a blame war may start. When we design an API and a library we should also think about not only the possible but also about the actual, average use.A library is as good as it is used and not as it could be used.This can also be applied to other products, not only libraries but this may lead too far (physics and weapon).Reference: Why to use String from our JCG partner Peter Verhas at the Java Deep blog....
java-logo

Java Extension Mechanism Loads All JARs

The Java Extension Mechanism is described in the Java Tutorial as a “standard, scalable way to make custom APIs available to all applications running on the Java platform.” As described in Understanding Extension Class Loading, “the extension framework makes use of the class-loading delegation mechanism” with extension classes loaded after the bootstrap classes in rt.jar (and related JARs) but before the classes loaded from the typical classpath. The extension directory works a bit like the classpath in that its part of the class loading mechanism and classes available within JARs in the extension directory are made available to Java applications. There are some key, differences, however, and some of these are highlighted next.  Characteristic Classpath Extension Mechanism (Optional Packages)Scope Typically Application-Specific-classpath/-cp java.class.path Executable JAR Manifest’s Class-PathPotentially All JREs on HostCLASSPATH environment variableAll JVMs Running in Specific JREjava.ext.dirsAll Host’s JREsSolaris: /usr/jdk/packages/lib/ext Linux: /usr/java/packages/lib/ext Windows: %SystemRoot%\Sun\Java\lib\extHow Specified .jar FilesExplicitly specified by name (including .jar) Wildcard (*) matching all all JAR files with .jar extensions.class FilesDirectory containing .class files specifiedAll JAR files (even if extension other than .jar or no extension at all) in designated directories are loaded.Class Loading Order After bootstrap and extensions loading. After bootstrap but before classpath.One of the most significant observations worth some more emphasis is that the extension mechanism will pick up all JAR format files in the extension directory even if the file does not have a .jar extension. The implication of this is that while one can change the name of a JAR located in a classpath directory to have an extension other than .jar so that the wildcard does not pick it up, this technique will not work with the extension directory. I’m going to use some simple examples in this post to demonstrate some of these differences. The next two code listings are for a very simple HelloWorld class and a main application class called Main that uses the HelloWorld class. HelloWorld.java public class HelloWorld { @Override public String toString() { return "Hello, World!"; } } Main.java import static java.lang.System.out;public class Main { public static void main(final String[] arguments) { out.println(new HelloWorld()); } } To demonstrate a primary difference between classpath and the extension mechanism (optional packages), I will archive the compiled HelloWorld.class file into a JAR called HelloWorld.jar and place it in a different directory than the compiled Main.class file. To demonstrate the use of the traditional classpath, I place the HelloWorld.jar file in a directory called C:\hello and will access that JAR via wildcard (*) for Main to use. This is demonstrated in the next two screen snapshots.The two previous images demonstrate that the Java Main application can still load the HelloWorld.class file even though I had deleted it from the current directory because the Java launcher was explicitly told (via the -classpath option) to look for it in C:\hello. Using the extensions mechanism (optional packages), it is possible to have the class loaded without it being in the same directory and without explicit classpath specification. This is shown in the next screen snapshot.The previous screen snapshot demonstrates that the Java launcher doesn’t even need the HelloWorld.class in the same directory or specified on its classpath when that class is inside a JAR that is in the extensions (optional packages) directory. This is often cited as a benefit of using the extensions mechanism because all applications using that JRE (or potentially all applications on the host) can see the same classes without need to explicitly specify them on the classpath. With the traditional classpath approach of instructing an application to load classes from JARs, the JAR file containing the .class file needs to end with the .jar extension. The next screen snapshot demonstrates what happens when the HelloWorld.jar is renamed HelloWorld.backup in the same classpath-referenced directory.The last image demonstrates that a NoClassDefFoundError is encountered when the JAR file in the classpath-referenced directory does not have a .jar extension. Perhaps a bit surprisingly, the extensions (optional packages) mechanism does not work the same way. Instead, all JAR files in the extensions specified directory are loaded regardless of their extension and regardless of even if they have a file extension. This is demonstrated in the next screen image.The previous image demonstrates that renaming the JAR file that resides within the extensions directory to not have any file extension whatsoever does not prevent the classloader from loading the classes of that JAR. In other words, the classloading mechanism loads all JAR files in the specified extensions directory based on file type rather than on file name or extension. As the Optional Packages Overview summarizes, “There is nothing special about any particular JAR file itself or the classes it contains that makes it an installed optional package. It is an installed optional package by virtue of its location in jre/lib/ext.” There are some risks and downsides associated with placing too many class definitions in JARs inside the extensions directory. It can be maddening to wonder why NoSuchMethodErrors, for example, are occurring when one can see that a class specified explicitly on the classpath has the method in question. I have previously written about one of the many potential causes of NoSuchMethodError, but forgotten outdated and obsolete class definitions residing inside of JAR files in the extensions directory are another potential cause. This is demonstrated next. The next two code listings show revised versions of Main.java and HelloWorld.java. In particular, HelloWorld has an all-new method that the new version of Main invokes. In this case, I’m going to leave the newly compiled HelloWorld.class file in the same directory when I run the Main to demonstrate that the old, busted version of HelloWorld.class in the JAR in the extensions directory takes precedence over the new hotness HelloWorld.class in the current directory. Revised Hello World.java (New Method) public class HelloWorld { @Override public String toString() { return "Hello, World!"; }public String directedHello(final String name) { return "Hello, " + name; } } Revised Main.java import static java.lang.System.out;public class Main { public static void main(final String[] arguments) { final HelloWorld helloWorld = new HelloWorld(); out.println(helloWorld); out.println(helloWorld.directedHello("Dustin")); } }The last image demonstrates that the now obsolete class definition of HelloWorld in the extensions directory takes precedence over the new class definition of HelloWorld in the same directory. Even when I specify the current directory on the classpath, the old version in the extensions directory takes precedence. This is shown in the next screen snapshot, which also shows that the JAR in the extensions directory that is “hiding” the newer JAR and its class’s newer method is still not even named with a .jar extension.The example just demonstrated is not even the most difficult situation a forgotten JAR in the specified extensions directory (or directories) can cause. In that example, at least I had a NoSuchMethodError to alert me to a problem. A potentially even more difficult situation to debug can exist when the old class definition has the same method signature but has an outdated method implementation. In such cases, there may be no error, exception, or throwable of any kind, but the application logic will simply not work correctly or as expected. The old functionality could exist in the code base for some time before it’s even recognized as an issue, especially if unit tests and other testing is lacking. Use of the extensions directory can make things easier on developers because classes in JAR files residing in the extensions directory (or directories) are available to all applications in the JRE associated with the extensions directory (or with all JREs on the host if the operating system-based host-wide extensions directory is used). However, there are definite risks associated with too liberal use of the directory. It can be easy to forget that outdated class definitions residing in JARs in that directory are preventing classloaders from loading the newer and seemingly obvious versions of the class definitions. When this happens, the very extensions (optional packages) mechanism that made developers’ lives easier now make it more difficult. Elliotte Rusty Harold provides a warning about use of the extensions (optional packages) mechanism, “While this seems convenient, it is also a long-term mistake… Sooner or later (probably sooner), you’ll load the wrong version of a class from a place you aren’t even thinking about and waste hours debugging.” The Java Tutorial also recommends caution (I added the emphasis), “Since this mechanism extends the platform’s core API, its use should be judiciously applied. Most commonly it is used for well standardized interfaces such as those defined by the Java Community Process, although it may also be appropriate for site wide interfaces.” Although the extensions (optional packages) mechanism is similar to the classpath mechanism and both are used as part of class loading, the differences between the two are important to note. In particular, it is important to remember that all JAR files (even if they don’t have .jar file extensions) that reside in the directory referenced as an extension directory will be loaded. Renaming these JARs and even changing their file extension will not be sufficient to have the classloading ignore them. With classpath, on the other hand, renaming the JAR is sufficient to prevent loading when the classpath specifies individual JAR files explicitly and changing the file extension is typically sufficient to prevent loading even when the classpath uses the wildcard (*) to specify all JARs in a directory. There are situations when the extensions (optional packages) mechanism is the appropriate choice, but these seem fairly rare. It is also important to keep in mind the extensions (optional packages) mechanism in mind when dealing with unexplained NoSuchMethodErrors so that one can check it out to see if the offender lives in that directory or directories.Reference: Java Extension Mechanism Loads All JARs from our JCG partner Dustin Marx at the Inspired by Actual Events blog....
docker-logo

Docker RabbitMQ cluster

I have been trying to create a Docker based RabbitMQ cluster on and off for sometime and got it working today – fairly basic and flaky but could be a good starting point for others to improve on. This is how the sample cluster looks on my machine, this is a typical cluster described in the RabbitMQ clustering guide available here -https://www.rabbitmq.com/clustering.html. As recommended at the site, there are 2 disk based nodes and 1 RAM based node here.        To quickly replicate this, you only need to have fig in your machine, just create a fig.yml file with the following entry: rabbit1: image: bijukunjummen/rabbitmq-server hostname: rabbit1 ports: - "5672:5672" - "15672:15672"rabbit2: image: bijukunjummen/rabbitmq-server hostname: rabbit2 links: - rabbit1 environment: - CLUSTERED=true - CLUSTER_WITH=rabbit1 - RAM_NODE=truerabbit3: image: bijukunjummen/rabbitmq-server hostname: rabbit3 links: - rabbit1 - rabbit2 environment: - CLUSTERED=true - CLUSTER_WITH=rabbit1 and in the folder holding this file, run: fig up That is it!, the entire cluster should come up. If you need more nodes, just modify the fig.yml file. The docker files for creating the dockerized rabbitmq-server is available at my github repo here: https://github.com/bijukunjummen/docker-rabbitmq-cluster and the “rabbitmq-server” image itself is here at the docker hub. ReferencesThe base rabbitmq image is somewhat based on cthulhuology’s docker rabbitmq image: https://github.com/cthulhuology/docker-rabbitmq Docker file: https://github.com/bijukunjummen/docker-rabbitmq-cluster Docker image: https://registry.hub.docker.com/u/bijukunjummen/rabbitmq-server/Reference: Docker RabbitMQ cluster from our JCG partner Biju Kunjummen at the all and sundry blog....
scala-logo

Akka Notes – Actor Lifecycle – Basic – 5

(Please note that this lifecycle write-up does not cover the preRestart or the postRestart methods. We’ll talk about them when we discuss supervision) The basic Actor lifecycle is very much intuitive. You could actually compare the basic Actor lifecycle with a Java servlet lifecycle with one special difference.            Just like any other regular class, we have a Constructor The preStart method gets called back next. Here, you could initialize resources that you would like to clean-up in postStop The “servicing” or the message handling by the receive method occupies the major chunk of time and that happens in between.Let’s look at a simple actor which prints the lifecycle. Dumb Lifecycle Actor package me.rerun.akkanotes.lifecycleimport akka.actor.{ActorLogging, Actor} import akka.event.LoggingReceiveclass BasicLifecycleLoggingActor extends Actor with ActorLogging{log.info ("Inside BasicLifecycleLoggingActor Constructor") log.info (context.self.toString()) override def preStart() ={ log.info("Inside the preStart method of BasicLifecycleLoggingActor") }def receive = LoggingReceive{ case "hello" => log.info ("hello") }override def postStop()={ log.info ("Inside postStop method of BasicLifecycleLoggingActor") }} App The LifecycleApp just initiates, sends a message to the Actor and shuts down the ActorSystem. import akka.actor.{ActorSystem, Props}object LifecycleApp extends App{val actorSystem=ActorSystem("LifecycleActorSystem") val lifecycleActor=actorSystem.actorOf(Props[BasicLifecycleLoggingActor],"lifecycleActor")lifecycleActor!"hello"//wait for a couple of seconds before shutdown Thread.sleep(2000) actorSystem.shutdown()} Output Inside BasicLifecycleLoggingActor ConstructorActor[akka://LifecycleActorSystem/user/lifecycleActor#-2018741361]Inside the preStart method of BasicLifecycleLoggingActorhelloInside postStop method of BasicLifecycleLoggingActor What’s that special difference between Servlets and the basic Actor lifecycle? That there is no difference between constructor and preStart in Actor lifecycle – more or less. The reason why I printed the context.self in the constructor is this – unlike Servlets, Actors have access to the ActorContext even inside the constructor. The difference between the preStart and the constructor then becomes very subtle. We’ll revisit the difference while we talk about supervision but if you are curious – calling the preStart when the Actor restarts (in case of failure) could be controlled. With constructor, that isn’t possible. When is postStop called? As we saw from the program, the postStop gets called when the ActorSystem shuts down. There are a couple of other times when the callback gets invoked too. 1. ActorSystem.stop() We could stop an Actor using the stop method of the ActorSystem and the ActorContext object LifecycleApp extends App{val actorSystem=ActorSystem("LifecycleActorSystem") val lifecycleActor=actorSystem.actorOf(Props[BasicLifecycleLoggingActor],"lifecycleActor")actorSystem.stop(lifecycleActor);... ...} 2. ActorContext.stop 1) Either by way of a message (externally or passing a message to itself) class BasicLifecycleLoggingActor extends Actor with ActorLogging{ ... ... def receive = LoggingReceive{ case "hello" => log.info ("hello") case "stop" => context.stop(self) } and object LifecycleApp extends App{val actorSystem=ActorSystem("LifecycleActorSystem") val lifecycleActor=actorSystem.actorOf(Props[BasicLifecycleLoggingActor],"lifecycleActor")lifecycleActor!"stop" ... ... 2) OR kill itself for no reason (this is just for fun. No Actor with an ambition would do this) class BasicLifecycleLoggingActor extends Actor with ActorLogging{log.info ("Inside BasicLifecycleLoggingActor Constructor") log.info (context.self.toString()) context.stop(self) ... ... 3. PoisonPill In the previous example, we passed a message called stop from the LifecycleApp to the Actor. The Actor received that message and killed itself using a context.stop. We could achieve the same thing by passing a PoisonPill message to the target actor. Please note that the PoisonPill message, just like the previous stop message gets enqueued in the regular mailbox and will be processed when it turn comes up. object LifecycleApp extends App{val actorSystem=ActorSystem("LifecycleActorSystem") val lifecycleActor=actorSystem.actorOf(Props[BasicLifecycleLoggingActor],"lifecycleActor")lifecycleActor!PoisonPill ... ... TRIVIA What do I mean regular mailbox? Is there a “special” mailbox too? Yup. There is. And we’ll talk about it when we talk about it when we discuss supervision and system messages. Termination Once the Actor is stopped, it is said to enter into a Terminated state. The immediate question that would come up to your mind is what would happen to the messages that is sent to an Actor which is already terminated? Let’s see that : App object LifecycleApp extends App{val actorSystem=ActorSystem("LifecycleActorSystem") val lifecycleActor=actorSystem.actorOf(Props[BasicLifecycleLoggingActor],"lifecycleActor")lifecycleActor!"hello" lifecycleActor!"stop" lifecycleActor!"hello" //Sending message to an Actor which is already stopped} Actor – Just as before class BasicLifecycleLoggingActor extends Actor with ActorLogging{def receive = LoggingReceive{ case "hello" => log.info ("hello") case "stop" => context.stop(self)} } Output BasicLifecycleLoggingActor - helloakka.actor.RepointableActorRef - Message from Actor[akka://LifecycleActorSystem/deadLetters] to Actor[akka://LifecycleActorSystem/user/lifecycleActor#-569230546] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. From the logs, you see that there are some references of deadletters. Any message that you send to an Actor that is terminated gets forwarded to the mailbox of an internal Actor called DeadLetterActor.CURIOUS AS TO WHAT HAPPENS NEXT? The DeadLetter Actor processes the messages in its mailbox, wraps each message as a DeadLetter and publishes it to theEventStream. One other Actor called DeadLetterListener consumes all DeadLetters and publishes that as a log message. Check this out. Remember, when we talked about logging, we saw that all log messages gets published to the EventStream and that we are free to subscribe to that EventStream – just that the subscriber also needs to be an Actor. Let’s try that now. For our example, we’ll subscribe to the EventStream and watch out for all DeadLetter messages and will print to the console (so much for creativity??!!). Frankly, we are free to do anything from generating an alert, storing it into a database or even feeding into analytics. SUBSCRIBING TO DEADLETTERS IN EVENTSTREAM import akka.actor.ActorSystem import akka.actor.Props import akka.actor.PoisonPill import akka.actor.DeadLetter import akka.actor.Actorobject LifecycleApp extends App {val actorSystem = ActorSystem("LifecycleActorSystem") val lifecycleActor = actorSystem.actorOf(Props[BasicLifecycleLoggingActor], "lifecycleActor")val deadLetterListener = actorSystem.actorOf(Props[MyCustomDeadLetterListener]) actorSystem.eventStream.subscribe(deadLetterListener, classOf[DeadLetter])lifecycleActor ! "hello" lifecycleActor ! "stop" lifecycleActor ! "hello"}class MyCustomDeadLetterListener extends Actor { def receive = { case deadLetter: DeadLetter => println(s"FROM CUSTOM LISTENER $deadLetter") } } Output 164 [LifecycleActorSystem-akka.actor.default-dispatcher-4] INFO BasicLifecycleLoggingActor - hello167 [LifecycleActorSystem-akka.actor.default-dispatcher-4] INFO akka.actor.RepointableActorRef - Message from Actor[akka://LifecycleActorSystem/deadLetters] to Actor[akka://LifecycleActorSystem/user/lifecycleActor#-782937925] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.FROM CUSTOM LISTENER DeadLetter(hello,Actor[akka://LifecycleActorSystem/deadLetters],Actor[akka://LifecycleActorSystem/user/lifecycleActor#-782937925])Reference: Akka Notes – Actor Lifecycle – Basic – 5 from our JCG partner Arun Manivannan at the Rerun.me blog....
agile-logo

Agile tester part 2, questions and answers

Warning: The opinions expressed in this post are mine only, please do not use them against any other group of people, but only against me, that is Augusto Evangelisti a.k.a. Gus. After writing my most recent blog post “The Agile Tester, a Curious and Empathetic Animal” I received quite a lot of feedback for which I am very grateful. Feedback in the form of a conversation is the true fuel for learning and improvement. To all of you that have mailed me, tweeted about my post, responded using the comments in my blog and talked to me face to face about it, thank you so much for helping me learn! When i looked at the feedback I received, I saw 2 quite different trends. Looking closer I saw that the different feedback types came from different schools of thought. Let me give you 2 examples for some context. This one from twitter: Well there are too many great quotes in @augeva's post to tweet 'em all, just go read it! :-> — lisacrispin (@lisacrispin) October 13, 2014This one from one comment to my blog post:Michael Bolton: Something about this post troubles me, Gus. In order to test what was troubling me, I took the content and replaced each instance of “tester” with “programmer”. The result made perfect sense, of course, but it left me wondering: if I can replace “tester” with “programmer”, what distinguishes the tester from anyone else on the project? What is the special role, the particular set of activities, that the tester performs? Is there a difference between the programmer’s mindset and the tester’s mindset? What is the mission, the distinctive reason for having a tester on the team—whether that person has the title “tester” or something else? What is it that distinguishes testing work from all the other work? What testing-specific skills do testers bring to the table? What is testing? I have answers of my own, of course. But I’m wondering what your answers might be.I am using Lisa’s and Michael’s as examples of the feedback received because the former is a recognisable exponent of the agile testing community and the latter is a recognisable exponent of the context driven testing community. The other feedback that i received from people close to each of the 2 communities is extremely similar to theirs. If a group of people finds troubling what the other group finds great, I smell something interesting and an opportunity for learning. I’ll try answering Michael questions and look forward to his very own answers to learn something: Question#1:  Something about this post troubles me, Gus. In order to test what was troubling me, I took the content and replaced each instance of “tester” with “programmer”. The result made perfect sense, of course, but it left me wondering: if I can replace “tester” with “programmer”, what distinguishes the tester from anyone else on the project? What is the special role, the particular set of activities, that the tester performs? Michael, the distinction between different roles in agile teams is becoming more and more blurry. Agile teams value competencies more than roles. As an agile tester in my team I have core competencies that i use to support my team. These include but are not limited  to the ability to evaluate a product by learning about it through experimentation. I also use such competencies to coach and lead other members of my team that are not that strong in that area and help them grow towards an ideal form of generalising specialist. Finally I also perform tasks outside my main competency to support the team. In some cases I will need guidance from another member of the team whose core competency includes the ability of performing such task. I believe that this blurring of the roles increases agile team members accountability, in fact nobody in an agile team (developer, tester, business analyst, operations specialist, UX expert, et cetera) should ever say, “I’m X, doing Y is not my job” but they should instead ask their colleagues “how can I help you?”. Shared activities are key to learning and gaining competencies. The main goal for any agile team member, regardless of his role is delivering customer value. Question#2: Is there a difference between the programmer’s mindset and the tester’s mindset? Yes, there is a difference, I don’t think it constitutes an insurmountable obstacle. One of the teams I have worked with, knowing that I wouldn’t be available for a period of time, suggested they needed the tester’s mindset for certain activities and decided that to make sure they were focusing on the right things, they would wear a big red hat. This simple change helped them keep their focus and their mindset in the right place. It might not be prefect but it worked. It works also because I speak to them about the testers mindset when we work together, I give them examples of what I am thinking at specific times and why it is important to think about such things. Developers are very smart people, once they understand a practice has value and receive sufficient amount of coaching, they can learn to do almost anything. Just to avoid misunderstandings, our developers wear the testers hat when performing testing activities with other developers, i.e. they don’t use the hat for their own code as maybe this could be asking too much.  Question#3: What is the mission, the distinctive reason for having a tester on the team—whether that person has the title “tester” or something else? The mission is to provide the team with the testing competencies it needs so that customer value can be delivered. The secondary purpose is to train and coach the team so that they can gain some of the competencies to support test activities. Coaching and training can be also formal but it is mainly delivered by working in pairs.  Question#4: What is it that distinguishes testing work from all the other work? Every activity the team engages in, is performed to deliver customer value, including test activities, but I am not sure I understood your question completely, could you please rephrase? Question#5: What testing-specific skills do testers bring to the table? See answer to question#1 re. competencies. My main competencies are exploration and learning. Question#6: What is testing? Testing to me is exploration and learning. I test ideas to prevent defects, software to detect them and product owners assumptions to reduce waste. I hope I have clarified some of your doubts and I am curious to hear your answers to your own questions.Reference: Agile tester part 2, questions and answers from our JCG partner Augusto Evangelisti at the mysoftwarequality blog....
Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy | Contact
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.
Do you want to know how to develop your skillset and become a ...
Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

Get ready to Rock!
You can download the complementary eBooks using the links below:
Close