Featured FREE Whitepapers

What's New Here?


Student Questions about Scala, Part 1

Preface I’m currently teaching a course on Applied Text Analysis and am using Scala as the programming language taught and used in the course. Rather than creating more tutorials, I figured I’d take a page from Brian Dunning’s playbook on his Skeptoid podcast (highly recommended) when he takes student questions. So, I had the students in the course submit questions about Scala that they had, based on the readings and assignments thus far. This post covers over half of them — the rest will be covered in a follow up post. I start with some of the more basic questions, and the questions and/or answers progressively get into more intermediate level topics. Suggestions and comments to improve any of the answers are very welcome! Basic Questions Q. Concerning addressing parts of variables: To address individual parts of lists, the numbering of the items is (List 0,1,2 etc.) That is, the first element is called “0?. It seems to be the same for Arrays and Maps, but not for Tuples- to get the first element of a Tuple, I need to use Tuple._1. Why is that? A. It’s just a matter of convention — tuples have used a 1-based index in other languages like Haskell, and it seems that Scala has adopted the same convention/tradition. See: http://stackoverflow.com/questions/6241464/why-are-the-indexes-of-scala-tuples-1-based Q. It seems that Scala doesn’t recognize the “b” boundary character as a regular expression. Is there something similar in Scala? A. Scala does recognize boundary characters. For example, the following REPL session declares a regex that finds “the” with boundaries, and successfully retrieves the three tokens of “the” in the example sentence. scala> val TheRE = """\bthe\b""".r TheRE: scala.util.matching.Regex = \bthe\bscala> val sentence = "She think the man is a stick-in-the-mud, but the man disagrees." sentence: java.lang.String = She think the man is a stick-in-the-mud, but the man disagrees.scala> TheRE.findAllIn(sentence).toList res1: List[String] = List(the, the, the)Q. Why doesn’t the method “split” work on args? Example: val arg = args.split(” “). Args are strings right, so split should work? A. The args variable is an Array, so split doesn’t work on them. Arrays are, in effect, already split. Q. What is the major difference between foo.mapValues(x=>x.length) and foo.map(x=>x.length). Some places one works and one does not. A. The map function works on all sequence types, including Seqs and Maps (note that Maps can be seen as sequences of Tuple2s). The mapValues function, however, only works on Maps. It is essentially a convenience function. As an example, let’s start with a simple Map from Ints to Ints. scala> val foo = List((1,2),(3,4)).toMap foo: scala.collection.immutable.Map[Int,Int] = Map(1 -> 2, 3 -> 4)Now consider the task of adding 2 to each value in the Map. This can be done with the map function as follows. scala> foo.map { case(key,value) => (key,value+2) } res5: scala.collection.immutable.Map[Int,Int] = Map(1 -> 4, 3 -> 6)So, the map function iterates over key/value pairs. We need to match both of them, and then output the key and the changed value to create the new Map. The mapValues function makes this quite a bit easier. scala> foo.mapValues(2+) res6: scala.collection.immutable.Map[Int,Int] = Map(1 -> 4, 3 -> 6)Returning to the question about computing the length using mapValues or map — then it is just a question of which values you are transforming, as in the following examples. scala> val sentence = "here is a sentence with some words".split(" ").toList sentence: List = List(here, is, a, sentence, with, some, words)scala> sentence.map(_.length) res7: List[Int] = List(4, 2, 1, 8, 4, 4, 5)scala> val firstCharTokens = sentence.groupBy(x=>x(0)) firstCharTokens: scala.collection.immutable.Map[Char,List] = Map(s -> List(sentence, some), a -> List(a), i -> List(is), h -> List(here), w -> List(with, words))scala> firstCharTokens.mapValues(_.length) res9: scala.collection.immutable.Map[Char,Int] = Map(s -> 2, a -> 1, i -> 1, h -> 1, w -> 2)Q. Is there any function that splits a list into two lists with the elements in the alternating positions of the original list? For example, MainList =(1,2,3,4,5,6) List1 = (1,3,5) List2 = (2,4,6) A. Given the exact main list you provided, one can use the partition function and use the modulo operation to see whether the value is divisible evenly by 2 or not. scala> val mainList = List(1,2,3,4,5,6) mainList: List[Int] = List(1, 2, 3, 4, 5, 6)scala> mainList.partition(_ % 2 == 0) res0: (List[Int], List[Int]) = (List(2, 4, 6),List(1, 3, 5))So, partition returns a pair of Lists. The first has all the elements that match the condition and the second has all the ones that do not. Of course, this wouldn’t work in general for Lists that have Strings, or that don’t have Ints in order, etc. However, the indices of a List are always well-behaved in this way, so we just need to do a bit more work by zipping each element with its index and then partitioning based on indices. scala> val unordered = List("b","2","a","4","z","8") unordered: List = List(b, 2, a, 4, z, 8)scala> unordered.zipWithIndex res1: List[(java.lang.String, Int)] = List((b,0), (2,1), (a,2), (4,3), (z,4), (8,5))scala> val (evens, odds) = unordered.zipWithIndex.partition(_._2 % 2 == 0) evens: List[(java.lang.String, Int)] = List((b,0), (a,2), (z,4)) odds: List[(java.lang.String, Int)] = List((2,1), (4,3), (8,5))scala> evens.map(_._1) res2: List = List(b, a, z)scala> odds.map(_._1) res3: List = List(2, 4, 8)Based on this, you could of course write a function that does this for any arbitrary list. Q. How to convert a List to a Vector and vice-versa? A. Use toIndexSeq and toList. scala> val foo = List(1,2,3,4) foo: List[Int] = List(1, 2, 3, 4)scala> val bar = foo.toIndexedSeq bar: scala.collection.immutable.IndexedSeq[Int] = Vector(1, 2, 3, 4)scala> val baz = bar.toList baz: List[Int] = List(1, 2, 3, 4)scala> foo == baz res0: Boolean = trueQ. The advantage of a vector over a list is the constant time look-up. What is the advantage of using a list over a vector? A. A List is slightly faster for operations at the head (front) of the sequence, so if all you are doing is doing a traversal (accessing each element in order, e.g. when mapping), then Lists are perfectly adequate and may be more efficient. They also have some nice pattern matching behavior for case statements. However, common wisdom seems to be that you should default to using Vectors. See Daniel Spiewak’s nice answer on Stackoverflow: http://stackoverflow.com/questions/6928327/when-should-i-choose-vector-in-scala Q. With splitting strings, holmes.split(“\\s”) – \n and \t just requires a single ‘\’ to recognize its special functionality but why two ‘\’s are required for white space character? A. That’s because \n and \t actually mean something in a String. scala> println("Here is a line with a tab\tor\ttwo, followed by\na new line.") Here is a line with a tab or two, followed by a new line.scala> println("This will break\s.") <console>:1: error: invalid escape character println("This will break\s.")So, you are supplying a String argument to split, and it uses that to construct a regular expression. Given that \s is not a string character, but is a regex metacharacter, you need to escape it. You can of course use split(“””\s”””), though that isn’t exactly better in this case. Q. I have long been programming in C++ and Java. Therefore, I put semicolon at the end of the line unconsciously. It seems that the standard coding style of Scala doesn’t recommend to use semicolons. However, I saw that there are some cases that require semicolons as you showed last class. Is there any specific reason why semicolon loses its role in Scala? A. The main reason is to improve readability since the semicolon is rarely needed when writing standard code in editors (as opposed to one liners in the REPL). However, when you want to do something in a single line, like handling multiple cases, you need the semicolons. scala> val foo = List("a",1,"b",2) foo: List[Any] = List(a, 1, b, 2)scala> foo.map { case(x: String) => x; case(x: Int) => x.toString } res5: List[String] = List(a, 1, b, 2)But, in general, it’s best to just split these cases over multiple lines in any actual code. Q. Is there no way to use _ in map like methods for collections that consist of pairs? For example, List((1,1),(2,2)).map(e => e._1 + e._2) works, but List((1,1),(2,2)).map(_._1 + _._2) does not work. A. The scope in which the _ remains unanambigious runs out past its first invocation, so you only get to use it once. It is better anyway to use a case statement that makes it clear what the members of the pairs are. scala> List((1,1),(2,2)).map { case(num1, num2) => num1+num2 } res6: List[Int] = List(2, 4)Q. I am unsure about the exact meaning of and the difference between “=>” and “->”. They both seem to mean something like “apply X to Y” and I see that each is used in a particular context, but what is the logic behind that? A. The use of -> simply constructs a Tuple2, as is pretty clear in the following snippet. scala> val foo = (1,2) foo: (Int, Int) = (1,2)scala> val bar = 1->2 bar: (Int, Int) = (1,2)scala> foo == bar res11: Boolean = truePrimarily, it is syntactic sugar that provides an intuitive symbol for creating elements of a a Map. Compare the following two ways of declaring the same Map. scala> Map(("a",1),("b",2)) res9: scala.collection.immutable.Map = Map(a -> 1, b -> 2)scala> Map("a"->1,"b"->2) res10: scala.collection.immutable.Map = Map(a -> 1, b -> 2)The second seems more readable to me. The use of => indicates that you are defining a function. The basic form is ARGUMENTS => RESULT. scala> val addOne = (x: Int) => x+1 addOne: Int => Int = <function1>scala> addOne(2) res7: Int = 3scala> val addTwoNumbers = (num1: Int, num2: Int) => num1+num2 addTwoNumbers: (Int, Int) => Int = <function2>scala> addTwoNumbers(3,5) res8: Int = 8Normally, you use it in defining anonymous functions as arguments to functions like map, filter, and such. Q. Is there a more convenient way of expressing vowels as [AEIOUaeiou] and consonants as [BCDFGHJKLMNPQRSTVWXYZbcdfghjklmnpqrstvwxyz] in RegExes? A. You can use Strings when defining regexes, so you can have a variable for vowels and one for consonants. scala> val vowel = "[AEIOUaeiou]" vowel: java.lang.String = [AEIOUaeiou]scala> val consonant = "[BCDFGHJKLMNPQRSTVWXYZbcdfghjklmnpqrstvwxyz]" consonant: java.lang.String = [BCDFGHJKLMNPQRSTVWXYZbcdfghjklmnpqrstvwxyz]scala> val MyRE = ("("+vowel+")("+consonant+")("+vowel+")").r MyRE: scala.util.matching.Regex = ([AEIOUaeiou])([BCDFGHJKLMNPQRSTVWXYZbcdfghjklmnpqrstvwxyz])([AEIOUaeiou])scala> val MyRE(x,y,z) = "aJE" x: String = a y: String = J z: String = EQ. The “\b” in RegExes marks a boundary, right? So, it also captures the “-”. But if I have a single string “sdnfeorgn”, it does NOT capture the boundaries of that, is that correct? And if so, why doesn’t it? A. Because there are no boundaries in that string! Intermediate questions Q. The flatMap function takes lists of lists and merges them to single list. But in the example: scala> (1 to 10).toList.map(x=>squareOddNumber(x)) res16: List[Option[Int]] = List(Some(1), None, Some(9), None, Some(25), None, Some(49), None, Some(81), None)scala> (1 to 10).toList.flatMap(x=>squareOddNumber(x)) res17: List[Int] = List(1, 9, 25, 49, 81)Here it is not list of list but just a list. In this case it expects the list to be Option list. I tried running the code with function returning just number or None. It showed error. So is there any way to use flatmap without Option lists and just list. For example, List(1, None, 9, None, 25) should be returned as List(1, 9, 25). A. No, this won’t work because List(1, None, 9, None, 25) mixes Options with Ints. scala> val mixedup = List(1, None, 9, None, 25) mixedup: List[Any] = List(1, None, 9, None, 25)So, you should have your function return an Option which means returning Somes or Nones. Then flatMap will work happily. One way of think of Options is that they are like Lists with zero or one element, as can be noted by the parallels in the following snippet. scala> val foo = List(List(1),Nil,List(3),List(6),Nil) foo: List[List[Int]] = List(List(1), List(), List(3), List(6), List())scala> foo.flatten res12: List[Int] = List(1, 3, 6)scala> val bar = List(Option(1),None,Option(3),Option(6),None) bar: List[Option[Int]] = List(Some(1), None, Some(3), Some(6), None)scala> bar.flatten res13: List[Int] = List(1, 3, 6)Q. Does scala have generic templates (like C++, Java)? eg. in C++, we can use vector<int>, vector<string> etc. Is that possible in scala? If so, how? A. Yes, every collection type is parameterized. Notice that each of the following variables is parameterized by the type of the elements they are initialized with. scala> val foo = List(1,2,3) foo: List[Int] = List(1, 2, 3)scala> val bar = List("a","b","c") bar: List = List(a, b, c)scala> val baz = List(true, false, true) baz: List[Boolean] = List(true, false, true)You can create your own parameterized classes straightforwardly. scala> class Flexible[T] (val data: T) defined class Flexiblescala> val foo = new Flexible(1) foo: Flexible[Int] = Flexible@7cd0570escala> val bar = new Flexible("a") bar: Flexible = Flexible@31b6956fscala> val baz = new Flexible(true) baz: Flexible[Boolean] = Flexible@5b58539fscala> foo.data res0: Int = 1scala> bar.data res1: java.lang.String = ascala> baz.data res2: Boolean = trueQ. How can we easily create, initialize and work with multi-dimensional arrays (and dictionaries)? A. Use the fill function of the Array object to create them. scala> Array.fill(2)(1.0) res8: Array[Double] = Array(1.0, 1.0)scala> Array.fill(2,3)(1.0) res9: Array[Array[Double]] = Array(Array(1.0, 1.0, 1.0), Array(1.0, 1.0, 1.0))scala> Array.fill(2,3,2)(1.0) res10: Array[Array[Array[Double]]] = Array(Array(Array(1.0, 1.0), Array(1.0, 1.0), Array(1.0, 1.0)), Array(Array(1.0, 1.0), Array(1.0, 1.0), Array(1.0, 1.0)))Once you have these in hand, you can iterate over them as usual. scala> val my2d = Array.fill(2,3)(1.0) my2d: Array[Array[Double]] = Array(Array(1.0, 1.0, 1.0), Array(1.0, 1.0, 1.0))scala> my2d.map(row => row.map(x=>x+1)) res11: Array[Array[Double]] = Array(Array(2.0, 2.0, 2.0), Array(2.0, 2.0, 2.0))For dictionaries (Maps), you can use mutable HashMaps to create an empty Map and then add elements to it. For that, see this blog post: http://bcomposes.wordpress.com/2011/09/19/first-steps-in-scala-for-beginning-programmers-part-8/ Q. Is the apply function similar to constructor in C++, Java? Where will the apply function be practically used? Is it for intialising values of attributes? A. No, the apply function is like any other function except that it allows you to call it without writing out “apply”. Consider the following class. class AddX (x: Int) { def apply(y: Int) = x+y override def toString = "My number is " + x }Here’s how we can use it. scala> val add1 = new AddX(1) add1: AddX = My number is 1scala> add1(4) res0: Int = 5scala> add1.apply(4) res1: Int = 5scala> add1.toString res2: java.lang.String = My number is 1So, the apply method is just (very handy) syntactic sugar that allows you to specify one function as fundamental to a class you have designed (actually, you can have multiple apply methods as long as each one has a unique parameter list). For example, with Lists, the apply method returns the value at the index provided, and for Maps it returns the value associated with the given key. scala> val foo = List(1,2,3) foo: List[Int] = List(1, 2, 3)scala> foo(2) res3: Int = 3scala> foo.apply(2) res4: Int = 3scala> val bar = Map(1->2,3->4) bar: scala.collection.immutable.Map[Int,Int] = Map(1 -> 2, 3 -> 4)scala> bar(1) res5: Int = 2scala> bar.apply(1) res6: Int = 2Q. In the SBT tutorial you discuss “Node” and “Value” as being case classes. What is the alternative to a case class? A. A normal class. Case classes are the special case. They do two things (and more) for you. The first is that you don’t have to use “new” to create a new object. Consider the following otherwise identical classes. scala> class NotACaseClass (val data: Int) defined class NotACaseClassscala> case class IsACaseClass (val data: Int) defined class IsACaseClassscala> val foo = new NotACaseClass(4) foo: NotACaseClass = NotACaseClass@a5c0f8fscala> val bar = IsACaseClass(4) bar: IsACaseClass = IsACaseClass(4)That may seem like a little thing, but it can significantly improve code readability. Consider creating Lists within Lists within Lists if you had to use “new” all the time, for example. This is definitely true for Node and Value, which are used to build trees. Case classes also support matching, as in the following. scala> val IsACaseClass(x) = bar x: Int = 4A normal class cannot do this. scala> val NotACaseClass(x) = foo <console>:13: error: not found: value NotACaseClass val NotACaseClass(x) = foo ^ <console>:13: error: recursive value x needs type val NotACaseClass(x) = foo ^If you mix the case class into a List and map over it, you can match it like you can with other classes, like Lists and Ints. Consider the following heterogeneous List. scala> val stuff = List(IsACaseClass(3), List(2,3), IsACaseClass(5), 4) stuff: List[Any] = List(IsACaseClass(3), List(2, 3), IsACaseClass(5), 4)We can convert this to a List of Ints by processing each element according to its type by matching. scala> stuff.map { case List(x,y) => x; case IsACaseClass(x) => x; case x: Int => x } <console>:13: warning: match is not exhaustive! missing combination * Nil * *stuff.map { case List(x,y) => x; case IsACaseClass(x) => x; case x: Int => x } ^warning: there were 1 unchecked warnings; re-run with -unchecked for details res10: List[Any] = List(3, 2, 5, 4)If you don’t want to see the warning in the REPL, add a case for things that don’t match that throws a MatchError. scala> stuff.map { case List(x,y) => x; case IsACaseClass(x) => x; case x: Int => x; case _ => throw new MatchError } warning: there were 1 unchecked warnings; re-run with -unchecked for details res13: List[Any] = List(3, 2, 5, 4)Better yet, return Options (using None for the unmatched case) and flatMapping instead. scala> stuff.flatMap { case List(x,y) => Some(x); case IsACaseClass(x) => Some(x); case x: Int => Some(x); case _ => None } warning: there were 1 unchecked warnings; re-run with -unchecked for details res14: List[Any] = List(3, 2, 5, 4)Q. In C++ the default access specifier is private; in Java one needs to specify private or public for each class member where as in Scala the default access specifier for a class is public. What could be the design motivation behind this when one of the purpose of the class is data hiding? A. The reason is that Scala has a much more refined access specification scheme than Java that makes public the rational choice. See the discussion here: http://stackoverflow.com/questions/4656698/default-public-access-in-scala Another key aspecte of this is that the general emphasis in Scala is on using immutable data structures, so there isn’t any danger of someone changing the internal state of your objects if you have designed them in this way. This in turn gets rid of the ridiculous getter and setter methods that breed and multiply in Java programs. See “Why getters and setters are evil” for more discussion: http://www.javaworld.com/javaworld/jw-09-2003/jw-0905-toolbox.html After you get used to programming in Scala, the whole getter/setter thing that is so common in Java code is pretty much gag worthy. In general, it is still a good idea to use private[this] as a modifier to methods and variables whenever they are only needed by an object itself. Q. How do we define overloaded constructors in Scala? Q. The way a class is defined in Scala introduced in the tutorial, seems to have only one constructor. Is there any way to provide multiple constructors like Java? A. You can add additional constructors with this declarations. class SimpleTriple (x: Int, y: Int, z: String) { def this (x: Int, z: String) = this(x,0,z) def this (x: Int, y: Int) = this(x,y,"a") override def toString = x + ":" + y + ":" + z }scala> val foo = new SimpleTriple(1,2,"hello") foo: SimpleTriple = 1:2:helloscala> val bar = new SimpleTriple(1,"goodbye") bar: SimpleTriple = 1:0:goodbyescala> val baz = new SimpleTriple(1,3) baz: SimpleTriple = 1:3:aNotice that you must supply an initial value for every one of the parameters of the class. This contrasts with Java, which allows you to leave some fields uninitialized (and which tends to lead to nasty bugs and bad design). Note that you can also provide defaults to parameters. class SimpleTripleWithDefaults (x: Int, y: Int = 0, z: String = "a") { override def toString = x + ":" + y + ":" + z }scala> val foo = new SimpleTripleWithDefaults(1) foo: SimpleTripleWithDefaults = 1:0:ascala> val bar = new SimpleTripleWithDefaults(1,2) bar: SimpleTripleWithDefaults = 1:2:aHowever, you can’t omit a middle parameter while specifying the last one. scala> val foo = new SimpleTripleWithDefaults(1,"xyz") <console>:12: error: type mismatch; found : java.lang.String("xyz") required: Int Error occurred in an application involving default arguments. val foo = new SimpleTripleWithDefaults(1,"xyz") ^But, you can name the parameters in the initialization if you want to be able to do this. scala> val foo = new SimpleTripleWithDefaults(1,z="xyz") foo: SimpleTripleWithDefaults = 1:0:xyzYou then have complete freedom to change the parameters around. scala> val foo = new SimpleTripleWithDefaults(z="xyz",x=42,y=3) foo: SimpleTripleWithDefaults = 42:3:xyzQ. I’m still not clear on the difference between classes and traits. I guess I see a conceptual difference but I don’t really understand what the functional difference is — how is creating a “trait” different from creating a class with maybe fewer methods associated with it? A. Yes, they are different. First off, traits are abstract, which means you cannot create any members. Consider the following contrast. scala> class FooClass defined class FooClassscala> trait FooTrait defined trait FooTraitscala> val fclass = new FooClass fclass: FooClass = FooClass@1b499616scala> val ftrait = new FooTrait <console>:8: error: trait FooTrait is abstract; cannot be instantiated val ftrait = new FooTrait ^You can extend a trait to make a concrete class, however. scala> class FooTraitExtender extends FooTrait defined class FooTraitExtenderscala> val ftraitExtender = new FooTraitExtender ftraitExtender: FooTraitExtender = FooTraitExtender@53d26552This gets more interesting if the trait has some methods, of course. Here’s a trait, Animal, that declares two abstract methods, makeNoise and doBehavior. trait Animal { def makeNoise: String def doBehavior (other: Animal): String }We can extend this trait with new class definitions; each extending class must implement both of these methods (or else be declared abstract). case class Bear (name: String, defaultBehavior: String = "Regard warily...") extends Animal { def makeNoise = "ROAR!" def doBehavior (other: Animal) = other match { case b: Bear => makeNoise + " I'm " + name + "." case m: Mouse => "Eat it!" case _ => defaultBehavior } override def toString = name }case class Mouse (name: String) extends Animal { def makeNoise = "Squeak?" def doBehavior (other: Animal) = other match { case b: Bear => "Run!!!" case m: Mouse => makeNoise + " I'm " + name + "." case _ => "Hide!" } override def toString = name }Notice that Bear and Mouse have different parameter lists, but both can be Animals because they fully implement the Animal trait. We can now start creating objects of the Bear and Mouse classes and have them interact. We don’t need to use “new” because they are case classes (and this also allowed them to be used in the match statements of the doBehavior methods). val yogi = Bear("Yogi", "Hello!") val baloo = Bear("Baloo", "Yawn...") val grizzly = Bear("Grizzly") val stuart = Mouse("Stuart")println(yogi + ": " + yogi.makeNoise) println(stuart + ": " + stuart.makeNoise) println("Grizzly to Stuart: " + grizzly.doBehavior(stuart))We can also create a singleton object that is of the Animal type by using the following declaration. object John extends Animal { def makeNoise = "Hullo!" def doBehavior (other: Animal) = other match { case b: Bear => "Nice bear... nice bear..." case _ => makeNoise } override def toString = "John" }Here, John is an object, not a class. Because this object implements the Animal trait, it successfully extends it and can act as an Animal. This means that a Bear like baloo can interact with John. println("Baloo to John: " + baloo.doBehavior(John))The output of the above code when run as a script is the following. Yogi: ROAR! Stuart: Squeak? Grizzly to Stuart: Eat it! Baloo to John: Yawn… The closer distinction is between traits and abstract classes. In fact, everything shown above could have been done with Animal as an abstract class rather than as a trait. One difference is that an abstract class can have a constructor while traits cannot. Another key difference between them is that traits can be used to support limited multiple inheritance, as shown in the next question/answer. Q. Does Scala support multiple inheritance? A. Yes, via traits with implementations of some methods. Here’s an example, with a trait Clickable that has an abstract (unimplemented) method getMessage, an implemented method click, and a private, reassignable variable numTimesClicked (the latter two show clearly that traits are different from Java interfaces). trait Clickable { private var numTimesClicked = 0 def getMessage: String def click = { val output = numTimesClicked + ": " + getMessage numTimesClicked += 1 output } }Now let’s say we have a MessageBearer class (that we may have wanted for entirely different reasons having nothing to do with clicking). class MessageBearer (val message: String) { override def toString = message }A new class can be now created by extending MessageBearer and “mixing in” the Clickable trait. class ClickableMessageBearer(message: String) extends MessageBearer(message) with Clickable { def getMessage = message }ClickableMessageBearer now has the abilities of both MessageBearers (which is to be able to retrieve its message) and Clickables. scala> val cmb1 = new ClickableMessageBearer("I'm number one!") cmb1: ClickableMessageBearer = I'm number one!scala> val cmb2 = new ClickableMessageBearer("I'm number two!") cmb2: ClickableMessageBearer = I'm number two!scala> cmb1.click res3: java.lang.String = 0: I'm number one!scala> cmb1.message res4: String = I'm number one!scala> cmb1.click res5: java.lang.String = 1: I'm number one!scala> cmb2.click res6: java.lang.String = 0: I'm number two!scala> cmb1.click res7: java.lang.String = 2: I'm number one!scala> cmb2.click res8: java.lang.String = 1: I'm number two!Q. Why are there toString, toInt, and toList functions, but there isn’t a toTuple function? A. This is a basic question that leads directly to the more advanced topic of implicits. There are a number of reasons behind this. To start with, it is important to realize that there are many types of Tuples, starting with a Tuple with a single element (a Tuple1) up to 22 elements (a Tuple22). Note that when you use (,) to create a tuple, it is implicitly invoking a constructor for the corresponding TupleN of the correct arity. scala> val b = (1,2,3) b: (Int, Int, Int) = (1,2,3)scala> val c = Tuple3(1,2,3) c: (Int, Int, Int) = (1,2,3)scala> b==c res4: Boolean = trueGiven this, it is obviously not meaningful to have a function toTuple on Seqs (sequences) that are longer than 22. This means there is no generic way to have, say a List or Array, and then call toTuple on it and expect reliable behavior to happen. However, if you want this functionality (even though limited by the above constraint of 22 elements max), Scala allows you to “add” methods to existing classes by using implicit definitions. You can find lots of discussions about implicits by search for “scala implicits”. But, here’s an example that shows how it works for this particular case. val foo = List(1,2) val bar = List(3,4,5) val baz = List(6,7,8,9)foo.toTupleclass TupleAble[X] (elements: Seq[X]) { def toTuple = elements match { case Seq(a) => Tuple1(a) case Seq(a,b) => (a,b) case Seq(a,b,c) => (a,b,c) case _ => throw new RuntimeException("Sequence too long to be handled by toTuple: " + elements) } }foo.toTupleimplicit def seqToTuple[X](x: Seq[X]) = new TupleAble(x)foo.toTuple bar.toTuple baz.toTupleIf you put this into the Scala REPL, you’ll see that the first invocation of foo.toTuple gets an error: scala> foo.toTuple <console>:9: error: value toTuple is not a member of List[Int] foo.toTuple ^Note that class TupleAble takes a Seq in its constructor and then provides the method toTuple, using that Seq. It is able to do so for Seqs with 1, 2 or 3 elements, and above that it throws an exception. (We could of course keeping listing more cases out and go up to 22 element tuples, but this shows the point.) The second invocation of foo.toTuple still doesn’t work — and that is because foo is a List (a kind of Seq) and there isn’t a toTuple method for Lists. That’s where the implicit function seqToTuple comes in — once it is declared, Scala notes that you are trying to call toTuple on a Seq, notes that there is no such function for Seqs, but sees that there is an implicit conversion from Seqs to TupleAbles via seqToTuple, and then it sees that TupleAble has a toTuple method. Based on that, it compiles and the produces the desired behavior. This is a very handy ability of Scala that can really simplify your code if you use it well and with care. Reference: Student Questions about Scala, Part 1 from our JCG partner Jason Baldridge at the Bcomposes blog....

Flavor of the month – Neo4j and Heroku

Neo4j launched a challenge earlier this year called “Seed the Cloud” to get folks to create templates or demo applications on Heroku using the Neo4j add-on. After much internal debate, I decided to enter, only to be thrown into despair for lack of an idea. The idea came to me while I was doing nothing in particular- to build a simple app that would help one find ingredients whose flavors complement one another. Basically, you have these ingredients that pair really well together- knowing which ingredients have flavor affinities can produce some amazing new dishes. The app allows you to add pairings with an ‘affinity’- how well they pair together- and search for an ingredient to find others that pair with it. The app is built using Grails 2.0 for the front end, visualizations are the result of the very neat d3.js library, and interaction with Neo4j is done using the Neo4j REST Api. The entire thing is deployed on Heroku (with the Neo4j add-on), while the source is available on github To get started with Grails 2.0, Neo4j add-on and Heroku, read Aldrins post on the topic: http://thought-bytes.blogspot.in/2012/02/grails-20-heroku-and-neo4j-addon.html How to deploy Flavorwocky locally as well as on Heroku is documented in the Readme. The model lends itself very nicely to a graph- as you can see, it is very simple:Every category is color coded- for convenience, it is stored as a property on the category node, but of course, it doesn’t have to be. Flavorwocky was also picked as the basis for this challenge because it is a real world use case for a graph, and I wanted this entry to really focus on Neo4j primarily rather than supporting bells and whistles. Set up When the application starts up, we check if the database is empty. To do this, we perform a traversal from the reference node looking for any categories connected to it. If there are none, then we go ahead and create the set of categories by first creating their nodes, and then a relationship to the reference node. We also create the index “ingredients”, which is used to index ingredient names. Source code: https://github.com/luanne/flavorwocky/blob/master/grails-app/conf/BootStrap.groovy Add pairing: Adding a pairing involvesChecking whether either node already exists to avoid recreating it Creating both nodes and linking them to their categories in a single transaction Creating a relationship between them in the same transaction aboveTo accomplish this, we used the Batch operation (note, this is still experimental). Source: https://github.com/luanne/flavorwocky/blob/master/grails-app/controllers/com/herokuapp/flavorwocky/FlavorwockyController.groovy (fetchOrCreateNodes()) Auto complete: This was just an index lookup matching the partially entered ingredient by name. Source: https://github.com/luanne/flavorwocky/blob/master/grails-app/controllers/com/herokuapp/flavorwocky/FlavorwockyController.groovy (autosearch()) Visualization: We used d3.js to provide two visualizations for the search results. The “Explore” visualization is based on the Node-Link tree; we used a Cypher query to find all ingredients that pair with the searched ingredient up to 3 levels deep, transform it into the appropriate data structure, and render it as JSON. Note, although the visualization interactive, the fetching of data is not. The entire set of data for 3 levels is grabbed at once- a future enhancement would be fetching children only when you expand a node. Source: https://github.com/luanne/flavorwocky/blob/master/grails-app/controllers/com/herokuapp/flavorwocky/FlavorwockyController.groovy (getSearchVisualizationAsTreeJson()) Although the tree is pretty slick, one shortcoming is that it is a tree- so if children are linked to each other, then you see multiple instances of that ingredient in the tree. Hence we tried another visualization to capture the interconnections between ingredients and also surface interesting facts such as flavor trios- there’s a pretty high chance that if you see a triangle in the network visualization, those three ingredients can be combined together well. We used the Force directed graph for this. Again, a Cypher query came to the rescue (this time 5 levels deep to produce a richer model). Source: https://github.com/luanne/flavorwocky/blob/master/grails-app/controllers/com/herokuapp/flavorwocky/FlavorwockyController.groovy (getSearchVisualizationAsNetworkJson()) In both examples, the affinity of the pairing is used to calculate the length of the connector between ingredients, indicating shorter connections have ingredients that pair much better than those with longer connections. That’s about it! There’s so much more that can be done with this application, but it’s going to have to wait for a bit. Please tweet about Flavorwocky if you like it or just want me to win. Voting also helps! Reference: Flavor of the month- Neo4j add-on for Heroku & Flavor of the month- Neo4j and Heroku, part 2 from our JCG partners Aldrin & Luanne Misquitta at the Thought Bytes blog....

Investigating the HashDoS issue

Nearly one month ago I have written some thoughts on how the HashDoS problem presented at the 28C3 or other code defects could perhaps be fixed temporarily without interaction of vendors. Now it’s time to deeper investigate the complexity attack and have at look at the sources. I quitely assume that java.util.HashMap and java.util.Hashtable are the most common used data structures of Java which are affected by this attack, so this article will only focus the code behind these types. A brief kick start to hash functions and indexed data structures Hash indexed data structures are very popular because of their simple usage and benefits:no bothering with index tables to find the right position of desired data access to data by using keywords instead of index numbers nearly constant time for add or delete operationsTo achieve these benefits hash indexed data structures follow a clever idea on how to index data. The index is computed by hashing a keyword which is associated with data behind it. Consider the following example for an easy code-like example: myHashIndexedDataStructure[hash(keyword)] = particular data Sounds perfect, but it has one major drawback: the used hash functions are in most cases not cryptograhic ones. According to Wikipedia the only mandatory characteristic for a function to call itself hash function is to “map[s] large data sets of variable length, called keys, to smaller data sets of a fixed length” In contrast to call itself a cryptographic hash function (once again, a definition taken from Wikipedia) it has to fulfill further, even much stronger, requirements: ”it is easy (but not necessarily quick) to compute the hash value for any given message it is infeasible to generate a message that has a given hash it is infeasible to modify a message without changing the hash it is infeasible to find two different messages with the same hash” To make a long story short let’s summarize what we learned and what’s to conclude with this knowledge:hash indexed data structures utilize hash functions hash functions are not necessarily collision resistant as long as they are not cryptographic ones lack of collision resistance implies means to easily compute multiple values with the same hash valueIn case of colliding keywords a hash indexed data structure needs some kind of a plan b) – a fall-back algorithm – on how to handle multiple data sets with equal hash values of the keywords . In practice are several approaches feasible:probing (shift to fixed or computable intervals) multiple hashing entry chaining (building lists of colliding entries) overriding existing entriesWhich of these strategies takes Java? At first we will inspect the code of java.util. Hashtable (only the interesting parts are presented, the rest of the code is left out for clarity reasons: public synchronized V put(K key, V value) { ... // Makes sure the key is not already in the hashtable. Entry tab[] = table; int hash = key.hashCode(); int index = (hash & 0x7FFFFFFF) %tab.length; for (Entry<K,V> e = tab[index] ; e != null ; e = e.next) { if((e.hash == hash) && e.key.equals(key)) { V old = e.value; e.value = value; return old; } } ... // Creates the new entry. Entry<K,V> e = tab[index]; tab[index] = new Entry<>(hash, key, value, e); count++; return null; }As can be seen this class uses the hashCode() function of the key object (the keyword) to compute the hash value. It follows an ANDing (& operator), for proper representation as Integer, MODULO (% operator) the table size (building a cyclic ring structure: (table.length + 1) mod table.length ? 1, division with remainders) to always address an entry in tab[]. After this all Entry(-ies) are considered and checked if the hash values are identical and the object itself is identical. The if-clause prevents storing of multiple instances of an identical object – the older one is simply replaced by the newer. In case no identical object (concerning hash value and equals() method) was found at the current position identified by key.hashCode() a new Entry will be created, placed at the current position and handled the old Entry object at this position. So far it looks like java.util.Hashtable is using some kind of list as data structure behind each tab[]. This assumption is confirmed when looking at the code for java.util.Hashtable.Entry<K,V> which is a private inner class. private static class Entry<K,V> implements Map.Entry<K,V> { int hash; K key; V value; Entry<K,V> next;The next Entry-object simply points to the following Entry. This represents a custom built linked list. The code of java.util.HashMap is more complex and behaves partly different (null values are allowed, !unsynchronized!), but is based on the same idea. Investigating the code here will not reveal anything new, except the fact that Entry is reimplemented once again…). Both implementations rely on linked lists behind each entry of the hash indexed array. Attack idea Now that we know the implementation details behind java.util.Hashtable and java.util.HashMap we could come back to the attack referred as HashDoS. The attack implements the idea of Crosby, S.A., Wallach, D.S.: Denial of Service via Algorithmic Complexity Attacks. In: Proceedings of the 12th conference on USENIX Security Symposium – Volume 12, USENIX Association (2003) To summarize: a hash indexed data structure can be extremely slowed down by provoking an unfavorable state. The ideal hash indexed data structure looks like this: ... table[hash(keyA)] = dataA table[hash(keyB)] = dataB table[hash(keyC)] = dataC table[hash(keyD)] = dataD ...In this case time for altering, deleting or adding data with a keyword with different hash value is nearly constant. The position can easily be found by using the hash value of the keyword as index and the data is instantly present without iterating a list. Let’s have a look at a different, unfavorable state of a hash indexed data structure: ... hash(keyA) == hash(keyB) == hash(keyC) == hash(keyD) = k table[k] = dataA -> dataB -> dataC -> dataD ...With a structure like this times of constant time for CRUD operations are over… It is necessary tocompute the hash value for the keyword iterate through the linked list compare each entry’s keyword if it matches the one the application is looking forThis significantly slows down the processing thread. A very fast data structure has turned into a linked list with additional overhead (computing a hash value). All benefits of a hash indexed data structure are wiped. As if it isn’t bad enough most hash indexed data structures enable a feature called rehashing. When the data structure exceeds a defined load (in Java for example 75%) the table is rehashed for optimization reasons. Most times this feature is absolutely desirable, but in this special case it even more slows down the whole process. Exploiting the problem To exploit this behavior it is necessary to compute a whole bunch of colliding keywords. If we assume the keyword to be of type java.lang.String for example we may have a look at its hashCode() function: public int hashCode() { int h = hash; if (h == 0) { int off = offset; char val[] = value; int len = count; for (int i = 0; i < len; i++) { h = 31*h + val[off++]; } hash = h; } return h; }This seems to be a customized version of a function called DJBX33A, designed by D.J. Bernstein where collisions can easily be found. The function has an interesting property which will be demonstrated on the following example: "0w".hashCode() = 1607 "1X".hashCode() = 1607 "29".hashCode() = 1607 "0w1X".hashCode() = 1545934 "0w29".hashCode() = 1545934 "1X0w".hashCode() = 1545934 "1X29".hashCode() = 1545934 "290w".hashCode() = 1545934 "291X".hashCode() = 1545934 ...We see that concatenation of colliding values results again in colliding values. We can do this further on and get a huge collection of colliding keywords. This makes finding collisions even more easy than simply bruteforcing. We tested this against a local web service and could significantly slow down the running web application server by using the colliding keywords as tag attributes. I’m uncertain if one could actually crash the machine or if there is some kind of non-obvious mechanism to prevent the server from killing itself (we have not investigated the processing code at server side), but one can definitely prevent the server from acting properly in an acceptable period of time. Requests to the web service can so easily be delayed. Maybe I will put some effort in collection measurement data (#colliding keys – system response time) in the near future. If I do, you will find the data on this blog… Corner points to take with younever rely on hashCode() alone – it can be error prone avoid things like if(password.hashCode() == referencePassword.hashCode()) {    return godMode; } else {    return simpleMode; }  spend a few seconds on implementation details when making your decision for/against a data type/structure filter incoming data – crop its size, reject overlong parameters etc. be careful and always look out for coding best practices!Further interesting points to look at In this example we used java.lang.String as keyword objects. It would be interesting what else could be used and where at the JRE code or at heavy used projects colliding hash values are used for data structuring or even worse purposes. One could have a look at how Object.hashCode() is implemented (it’s native code) – this would be a great target since all other objects extend this base class. If an extending class does not override the hashCode() function and relies on proper, collision-free output this could be useful for more sophisticated attacks. Consider what happens if serialization relies on corresponding code…. If anybody already knows some vulnerable places please let us know! We are very interested, but couldn’t look as deep as we would like to, due to lack of time. Thanks Once again I would like to thank Juraj Somorovsky for rich joint research work! Further on, our thanks go to Andrea Barisani from the oCERT team and Vincent Danen from the Red Hat Security Response Team who discussed this problem with us! Reference: Investigating the HashDoS issue from our JCG partner Christopher Meyer at the Java security and related topics blog....

State Does Not Belong In The Code

What is “state” in your web application? It’s the data that gets stored (regardless of the destination – memory, database, file-system). The application itself must not store any state in the code. This means your classes should only have fields with objects that are also stateless. In other words – you should not store anything in your services, DAOs or controllers during the program flow. This is a complete “must” for your service layer. Why? Your application needs to be scalable. This means it needs to be run in a cluster, and state is the hardest thing to distribute. If you minimize the places where state is stored, you minimize the complexity of clustering. But state should exist, and here is where it is fine to have it:the database – be it SQL, NoSQL or even a search engine, it’s the main thing that stores state. It is the thing that is supposed to support clustering, or a huge dedicated machine that handles requests from multiple other “code” servers. The code communicates with the database, but the code itself does not store anything for more than one client request; cache – caching is relatively easy to distribute (it’s basically key-value). There are many ready-to-use solutions like EhCache and memcached. So instead of computing a result, or getting it from the DB on each request, you can configure caching and store the result in memory. But again – code does not store anything – it just populates and queries the cache; HTTP session – in web components (controllers, managed beans, whatever you call it). It is very similar to caching, though it has a different purpose – to allow identifying subsequent actions by the same user (http itself is stateless). But as your code runs on multiple machines, the load-balancer may not always send subsequent requests to the same server. So the session should also be replicated across all servers. Fortunately, most containers have that option built-in, so you just add one configuration line. Alternatively you can instruct the load-balancer to use a “sticky session” (identify which server to send the request depending on the session cookie), but it moves some state management to the load-balancer as well. Regardless of the option you choose, do not put too much data in the session the file system – when you store files, you need them to be accessible to all machines. There are multiple options here, including SAN or using a cloud storage service like Amazon S3, which are accessible through an APIAll these are managed outside the code. Your code simply consumes them through an API (the Session API, the cache API, JDBC, S3/file system API). If the code contained any of that state (as instance-variables of your objects) the application would be hard to support (you’d have to manage state yourself) and will be less scalable. Of course, there are some rare cases, where you can’t go without storing state in the code. Document these and make sure they do not rely on working in a cluster. But what can go wrong if you store state in the objects that perform the business logic? You have two options then:synchronize access to fields – this will kill performance, because all users that make requests will have to wait in queue for the service to manage its fields; make new instance of your class for each HTTP request, and manage the instances somehow. Managing these instances is the hard part. People may be inclined to choose the session to do it, which means the session grows very large and gets harder to replicate (sharing a lot of data across multiple machines is slower, and session replication must be fast). Not to mention the unnecessarily increased memory footprint.Here’s a trivial example of what not to do. You should pass these kinds of values as method arguments, rather than storing them in the instance: class OrderService { double orderPrice;void processOrder(OrderDto order) { for (Entry entry : order.getEntries() { orderPrice += entry.getPrice(); } boolean discounts = hasDiscounts(order); } boolean hasDiscounts(OrderDto order) { return order.getEntries().length > 5 && orderPrice > 200; } }So, make all your code stateless – this will ensure at least some level of scalability. Reference: State Does Not Belong In The Code from our JCG partner Bozhidar Bozhanov at the Bozho’s tech blog....

Spring Thread Pool Services

Thread Pools are very important to execute synchronous & asynchronous processes. This article shows how to develop and monitor Thread Pool Services by using Spring. Creating Thread Pool has been explained via two alternative methods. Used Technologies : JDK 1.6.0_21 Spring 3.0.5 Maven 3.0.2 STEP 1 : CREATE MAVEN PROJECT A maven project is created as below. (It can be created by using Maven or IDE Plug-in).STEP 2 : LIBRARIES Spring dependencies are added to Maven’ s pom.xml. <!-- Spring 3 dependencies --> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-core</artifactId> <version>${spring.version}</version> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-context</artifactId> <version>${spring.version}</version> </dependency>For creating runnable-jar, below plugin can be used. <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-shade-plugin</artifactId> <version>1.3.1</version> <executions> <execution> <phase>package</phase> <goals> <goal>shade</goal> </goals> <configuration> <transformers> <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"> <mainClass>com.otv.exe.Application</mainClass> </transformer> <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer"> <resource>META-INF/spring.handlers</resource> </transformer> <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer"> <resource>META-INF/spring.schemas</resource> </transformer> </transformers> </configuration> </execution> </executions> </plugin>STEP 3 : CREATE TASK CLASS A new TestTask Class is created by implementing Runnable Interface. This class shows to be executed tasks. package com.otv.task; import org.apache.log4j.Logger; /** * @author onlinetechvision.com * @since 17 Oct 2011 * @version 1.0.0 * */ public class TestTask implements Runnable { private static Logger log = Logger.getLogger(TestTask.class); String taskName; public TestTask() { } public TestTask(String taskName) { this.taskName = taskName; } public void run() { try { log.debug(this.taskName + " : is started."); Thread.sleep(10000); log.debug(this.taskName + " : is completed."); } catch (InterruptedException e) { log.error(this.taskName + " : is not completed!"); e.printStackTrace(); } } @Override public String toString() { return (getTaskName()); } public String getTaskName() { return taskName; } public void setTaskName(String taskName) { this.taskName = taskName; } }STEP 4 : CREATE TestRejectedExecutionHandler CLASS TestRejectedExecutionHandler Class is created by implementing RejectedExecutionHandler Interface. If there is no idle thread and queue overflows, tasks will be rejected. This class handles rejected tasks. package com.otv.handler; import java.util.concurrent.RejectedExecutionHandler; import java.util.concurrent.ThreadPoolExecutor; import org.apache.log4j.Logger; /** * @author onlinetechvision.com * @since 17 Oct 2011 * @version 1.0.0 * */ public class TestRejectedExecutionHandler implements RejectedExecutionHandler { private static Logger log = Logger.getLogger(TestRejectedExecutionHandler.class); public void rejectedExecution(Runnable runnable, ThreadPoolExecutor executor) { log.debug(runnable.toString() + " : has been rejected"); } }STEP 5 : CREATE ITestThreadPoolExecutorService INTERFACE ITestThreadPoolExecutorService Interface is created. package com.otv.srv; import java.util.concurrent.ThreadPoolExecutor; import com.otv.handler.TestRejectedExecutionHandler; /** * @author onlinetechvision.com * @since 17 Oct 2011 * @version 1.0.0 * */ public interface ITestThreadPoolExecutorService { public ThreadPoolExecutor createNewThreadPool(); public int getCorePoolSize(); public void setCorePoolSize(int corePoolSize); public int getMaxPoolSize(); public void setMaxPoolSize(int maximumPoolSize); public long getKeepAliveTime(); public void setKeepAliveTime(long keepAliveTime); public int getQueueCapacity(); public void setQueueCapacity(int queueCapacity); public TestRejectedExecutionHandler getTestRejectedExecutionHandler(); public void setTestRejectedExecutionHandler(TestRejectedExecutionHandler testRejectedExecutionHandler); }STEP 6 : CREATE TestThreadPoolExecutorService CLASS TestThreadPoolExecutorService Class is created by implementing ITestThreadPoolExecutorService Interface. This class creates a new Thread Pool. package com.otv.srv; import java.util.concurrent.ArrayBlockingQueue; import java.util.concurrent.ThreadPoolExecutor; import java.util.concurrent.TimeUnit; import com.otv.handler.TestRejectedExecutionHandler; /** * @author onlinetechvision.com * @since 17 Oct 2011 * @version 1.0.0 * */ public class TestThreadPoolExecutorService implements ITestThreadPoolExecutorService { private int corePoolSize; private int maxPoolSize; private long keepAliveTime; private int queueCapacity; TestRejectedExecutionHandler testRejectedExecutionHandler; public ThreadPoolExecutor createNewThreadPool() { ThreadPoolExecutor executor = new ThreadPoolExecutor(getCorePoolSize(), getMaxPoolSize(), getKeepAliveTime(), TimeUnit.SECONDS, new ArrayBlockingQueue(getQueueCapacity()), getTestRejectedExecutionHandler()); return executor; } public int getCorePoolSize() { return corePoolSize; } public void setCorePoolSize(int corePoolSize) { this.corePoolSize = corePoolSize; } public int getMaxPoolSize() { return maxPoolSize; } public void setMaxPoolSize(int maxPoolSize) { this.maxPoolSize = maxPoolSize; } public long getKeepAliveTime() { return keepAliveTime; } public void setKeepAliveTime(long keepAliveTime) { this.keepAliveTime = keepAliveTime; } public int getQueueCapacity() { return queueCapacity; } public void setQueueCapacity(int queueCapacity) { this.queueCapacity = queueCapacity; } public TestRejectedExecutionHandler getTestRejectedExecutionHandler() { return testRejectedExecutionHandler; } public void setTestRejectedExecutionHandler(TestRejectedExecutionHandler testRejectedExecutionHandler) { this.testRejectedExecutionHandler = testRejectedExecutionHandler; } }STEP 7 : CREATE IThreadPoolMonitorService INTERFACE IThreadPoolMonitorService Interface is created. package com.otv.monitor.srv; import java.util.concurrent.ThreadPoolExecutor; public interface IThreadPoolMonitorService extends Runnable { public void monitorThreadPool(); public ThreadPoolExecutor getExecutor(); public void setExecutor(ThreadPoolExecutor executor); }STEP 8 : CREATE ThreadPoolMonitorService CLASS ThreadPoolMonitorService Class is created by implementing IThreadPoolMonitorService Interface. This class monitors created thread pool. package com.otv.monitor.srv; import java.util.concurrent.ThreadPoolExecutor; import org.apache.log4j.Logger; /** * @author onlinetechvision.com * @since 17 Oct 2011 * @version 1.0.0 * */ public class ThreadPoolMonitorService implements IThreadPoolMonitorService { private static Logger log = Logger.getLogger(ThreadPoolMonitorService.class); ThreadPoolExecutor executor; private long monitoringPeriod; public void run() { try { while (true){ monitorThreadPool(); Thread.sleep(monitoringPeriod*1000); } } catch (Exception e) { log.error(e.getMessage()); } } public void monitorThreadPool() { StringBuffer strBuff = new StringBuffer(); strBuff.append("CurrentPoolSize : ").append(executor.getPoolSize()); strBuff.append(" - CorePoolSize : ").append(executor.getCorePoolSize()); strBuff.append(" - MaximumPoolSize : ").append(executor.getMaximumPoolSize()); strBuff.append(" - ActiveTaskCount : ").append(executor.getActiveCount()); strBuff.append(" - CompletedTaskCount : ").append(executor.getCompletedTaskCount()); strBuff.append(" - TotalTaskCount : ").append(executor.getTaskCount()); strBuff.append(" - isTerminated : ").append(executor.isTerminated()); log.debug(strBuff.toString()); } public ThreadPoolExecutor getExecutor() { return executor; } public void setExecutor(ThreadPoolExecutor executor) { this.executor = executor; } public long getMonitoringPeriod() { return monitoringPeriod; } public void setMonitoringPeriod(long monitoringPeriod) { this.monitoringPeriod = monitoringPeriod; } }STEP 9 : CREATE Starter CLASS Starter Class is created. package com.otv.start; import java.util.concurrent.ThreadPoolExecutor; import org.apache.log4j.Logger; import com.otv.handler.TestRejectedExecutionHandler; import com.otv.monitor.srv.IThreadPoolMonitorService; import com.otv.monitor.srv.ThreadPoolMonitorService; import com.otv.srv.ITestThreadPoolExecutorService; import com.otv.srv.TestThreadPoolExecutorService; import com.otv.task.TestTask; /** * @author onlinetechvision.com * @since 17 Oct 2011 * @version 1.0.0 * */ public class Starter { private static Logger log = Logger.getLogger(TestRejectedExecutionHandler.class); IThreadPoolMonitorService threadPoolMonitorService; ITestThreadPoolExecutorService testThreadPoolExecutorService; public void start() { // A new thread pool is created... ThreadPoolExecutor executor = testThreadPoolExecutorService.createNewThreadPool(); executor.allowCoreThreadTimeOut(true); // Created executor is set to ThreadPoolMonitorService... threadPoolMonitorService.setExecutor(executor); // ThreadPoolMonitorService is started... Thread monitor = new Thread(threadPoolMonitorService); monitor.start(); // New tasks are executed... for(int i=1;i<10;i++) { executor.execute(new TestTask("Task"+i)); } try { Thread.sleep(40000); } catch (Exception e) { log.error(e.getMessage()); } for(int i=10;i<19;i++) { executor.execute(new TestTask("Task"+i)); } // executor is shutdown... executor.shutdown(); } public IThreadPoolMonitorService getThreadPoolMonitorService() { return threadPoolMonitorService; } public void setThreadPoolMonitorService(IThreadPoolMonitorService threadPoolMonitorService) { this.threadPoolMonitorService = threadPoolMonitorService; } public ITestThreadPoolExecutorService getTestThreadPoolExecutorService() { return testThreadPoolExecutorService; } public void setTestThreadPoolExecutorService(ITestThreadPoolExecutorService testThreadPoolExecutorService) { this.testThreadPoolExecutorService = testThreadPoolExecutorService; } }STEP 10 : CREATE Application CLASS Application Class is created. This class runs the application. package com.otv.start; import org.springframework.context.ApplicationContext; import org.springframework.context.support.ClassPathXmlApplicationContext; /** * @author onlinetechvision.com * @since 17 Oct 2011 * @version 1.0.0 * */ public class Application { public static void main(String[] args) { ApplicationContext context = new ClassPathXmlApplicationContext("applicationContext.xml"); Starter starter = (Starter) context.getBean("Starter"); starter.start(); } }STEP 11 : CREATE applicationContext.xml applicationContext.xml is created. <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd"> <!-- Beans Declaration --> <bean id="TestTask" class="com.otv.task.TestTask"></bean> <bean id="ThreadPoolMonitorService" class="com.otv.monitor.srv.ThreadPoolMonitorService"> <property name="monitoringPeriod" value="5" /> </bean> <bean id="TestRejectedExecutionHandler" class="com.otv.handler.TestRejectedExecutionHandler"></bean> <bean id="TestThreadPoolExecutorService" class="com.otv.srv.TestThreadPoolExecutorService"> <property name="corePoolSize" value="1" /> <property name="maxPoolSize" value="3" /> <property name="keepAliveTime" value="10" /> <property name="queueCapacity" value="3" /> <property name="testRejectedExecutionHandler" ref="TestRejectedExecutionHandler" /> </bean> <bean id="Starter" class="com.otv.start.Starter"> <property name="threadPoolMonitorService" ref="ThreadPoolMonitorService" /> <property name="testThreadPoolExecutorService" ref="TestThreadPoolExecutorService" /> </bean> </beans>STEP 12 : ALTERNATIVE METHOD TO CREATE THREAD POOL ThreadPoolTaskExecutor Class provided by Spring can also be used to create Thread Pool. <bean id="threadPoolTaskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor"> <property name="corePoolSize" value="1" /> <property name="maxPoolSize" value="3" /> <property name="queueCapacity" value="3" /> </bean> <bean id="testTaskExecutor" class="TestTaskExecutor"> <constructor-arg ref="threadPoolTaskExecutor" /> </bean>STEP 13 : BUILD PROJECT After OTV_Spring_ThreadPool Project is build, OTV_Spring_ThreadPool-0.0.1-SNAPSHOT.jar will be created. STEP 14 : RUN PROJECT After created OTV_Spring_ThreadPool-0.0.1-SNAPSHOT.jar file is run, below output logs will be shown : 18.10.2011 20:08:48 DEBUG (TestRejectedExecutionHandler.java:19) - Task7 : has been rejected 18.10.2011 20:08:48 DEBUG (TestRejectedExecutionHandler.java:19) - Task8 : has been rejected 18.10.2011 20:08:48 DEBUG (TestRejectedExecutionHandler.java:19) - Task9 : has been rejected 18.10.2011 20:08:48 DEBUG (TestTask.java:25) - Task1 : is started. 18.10.2011 20:08:48 DEBUG (TestTask.java:25) - Task6 : is started. 18.10.2011 20:08:48 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 3 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 2 - CompletedTaskCount : 0 - TotalTaskCount : 5 - isTerminated : false 18.10.2011 20:08:48 DEBUG (TestTask.java:25) - Task5 : is started. 18.10.2011 20:08:53 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 3 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 3 - CompletedTaskCount : 0 - TotalTaskCount : 6 - isTerminated : false 18.10.2011 20:08:58 DEBUG (TestTask.java:27) - Task6 : is completed. 18.10.2011 20:08:58 DEBUG (TestTask.java:27) - Task1 : is completed. 18.10.2011 20:08:58 DEBUG (TestTask.java:25) - Task3 : is started. 18.10.2011 20:08:58 DEBUG (TestTask.java:25) - Task2 : is started. 18.10.2011 20:08:58 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 3 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 3 - CompletedTaskCount : 2 - TotalTaskCount : 6 - isTerminated : false 18.10.2011 20:08:58 DEBUG (TestTask.java:27) - Task5 : is completed. 18.10.2011 20:08:58 DEBUG (TestTask.java:25) - Task4 : is started. 18.10.2011 20:09:03 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 3 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 3 - CompletedTaskCount : 3 - TotalTaskCount : 6 - isTerminated : false 18.10.2011 20:09:08 DEBUG (TestTask.java:27) - Task2 : is completed. 18.10.2011 20:09:08 DEBUG (TestTask.java:27) - Task3 : is completed. 18.10.2011 20:09:08 DEBUG (TestTask.java:27) - Task4 : is completed. 18.10.2011 20:09:08 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 3 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 0 - CompletedTaskCount : 6 - TotalTaskCount : 6 - isTerminated : false 18.10.2011 20:09:13 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 3 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 0 - CompletedTaskCount : 6 - TotalTaskCount : 6 - isTerminated : false 18.10.2011 20:09:18 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 0 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 0 - CompletedTaskCount : 6 - TotalTaskCount : 6 - isTerminated : false 18.10.2011 20:09:23 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 0 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 0 - CompletedTaskCount : 6 - TotalTaskCount : 6 - isTerminated : false 18.10.2011 20:09:28 DEBUG (TestTask.java:25) - Task10 : is started. 18.10.2011 20:09:28 DEBUG (TestRejectedExecutionHandler.java:19) - Task16 : has been rejected 18.10.2011 20:09:28 DEBUG (TestRejectedExecutionHandler.java:19) - Task17 : has been rejected 18.10.2011 20:09:28 DEBUG (TestRejectedExecutionHandler.java:19) - Task18 : has been rejected 18.10.2011 20:09:28 DEBUG (TestTask.java:25) - Task14 : is started. 18.10.2011 20:09:28 DEBUG (TestTask.java:25) - Task15 : is started. 18.10.2011 20:09:28 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 3 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 3 - CompletedTaskCount : 6 - TotalTaskCount : 12 - isTerminated : false 18.10.2011 20:09:33 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 3 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 3 - CompletedTaskCount : 6 - TotalTaskCount : 12 - isTerminated : false 18.10.2011 20:09:38 DEBUG (TestTask.java:27) - Task10 : is completed. 18.10.2011 20:09:38 DEBUG (TestTask.java:25) - Task11 : is started. 18.10.2011 20:09:38 DEBUG (TestTask.java:27) - Task14 : is completed. 18.10.2011 20:09:38 DEBUG (TestTask.java:27) - Task15 : is completed. 18.10.2011 20:09:38 DEBUG (TestTask.java:25) - Task12 : is started. 18.10.2011 20:09:38 DEBUG (TestTask.java:25) - Task13 : is started. 18.10.2011 20:09:38 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 3 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 3 - CompletedTaskCount : 9 - TotalTaskCount : 12 - isTerminated : false 18.10.2011 20:09:43 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 3 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 3 - CompletedTaskCount : 9 - TotalTaskCount : 12 - isTerminated : false 18.10.2011 20:09:48 DEBUG (TestTask.java:27) - Task11 : is completed. 18.10.2011 20:09:48 DEBUG (TestTask.java:27) - Task13 : is completed. 18.10.2011 20:09:48 DEBUG (TestTask.java:27) - Task12 : is completed. 18.10.2011 20:09:48 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 0 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 0 - CompletedTaskCount : 12 - TotalTaskCount : 12 - isTerminated : true 18.10.2011 20:09:53 DEBUG (ThreadPoolMonitorService.java:39) - CurrentPoolSize : 0 - CorePoolSize : 1 - MaximumPoolSize : 3 - ActiveTaskCount : 0 - CompletedTaskCount : 12 - TotalTaskCount : 12 - isTerminated : trueSTEP 15 : DOWNLOAD OTV_Spring_ThreadPool Reference: How to develop and monitor Thread Pool Services by using Spring from our JCG partner Eren Avsarogullari at the Online Technology Vision blog....

Apache Camel – A little Scala DSL example

So we have a Scala DSL in Apache Camel for many years now, and I guess its about time I wrote a little blog entry about this (has been on my todo list for a while). So the Scala DSL is of course using the Scala programming language which has many bells and whistles over plain Java. However the uptake of the Scala DSL is not very high, as the Java and XML DSL’s is good enough for most people. Anyway I guess one of the nice thing about the Scala DSL would be using closures as expressions and predicates etc. So let’s do a little example using the Filter EIP pattern and use a closure as the filter predicate. When using the Scala DSL you should use the org.apache.camel.scala.dsl.builder.RouteBuilder, which is the Scala empowered DSL. So in the following we have a FilterRoute class where we use the Scala DSL, when we define the createMyFilterRoute function. class FilterRoute { def createMyFilterRoute = new RouteBuilder { from("direct:start") .filter(_.in("gold") == "true") .to("mock:gold") } }As you can see from the code above, inside the scope of RouteBuilder we have the Scala DSL at our disposal. Then we use the Filter EIP which accepts a function with the Exchange as parameter, which gets defaulted into the _ symbol. The result of that function is the evaluated as a predicate, using the Scala powerful (but a bit scary) type system with implicit type converter and case matching etc. The in(“gold”) is a function on a Exchange wrapped we have in the Scala DSL which adds additional methods on the Camel Exchange (RichExchange), the in is a function that look up a header. To unit test this route I wanted to use the existing and powerful camel-test module. This module offers the CamelTestSupport class you can extend for your unit tests. So the unit tests can be almost like in Java, but you would need to add the trait org.apache.camel.scala.dsl.RouteBuilderSupport which helps bridge the Scala RouteBuilder with the Java RouteBuilder, that the CamelTestSupport expects and uses. The code below shows an unit test example. class FilterRouteTest extends CamelTestSupport with RouteBuilderSupport {override def createRouteBuilder() = new FilterRoute().createMyFilterRoute@Test def testFilterRouteGold() { getMockEndpoint("mock:gold").expectedMessageCount(1) template.sendBodyAndHeader("direct:start", "Hello World", "gold", "true") assertMockEndpointsSatisfied() }@Test def testFilterRouteNotGold() { getMockEndpoint("mock:gold").expectedMessageCount(0) template.sendBodyAndHeader("direct:start", "Hello World", "gold", "false") assertMockEndpointsSatisfied() }}As you can see we use the RouteBuilderSupport trait, and then override the createRouteBuilder function to return the Scala DSL empowered RouteBuilder we created previously. The rest of the code is standard and plain Java code with JUnit @Test annotations. This example is provided in the source code of the camel-scala module, as part of a unit test. If you are a Scala fan and interested in Camel as well, then the Camel community could use people who are dedicated to Scala and help with the Scala DSL. The Camel team is often busy with other issues in our lives, so we only have a bit time to have fun and play with Scala. Reference: Apache Camel – A little Scala DSL example from our JCG partner Claus Ibsen at the Claus Ibsen riding the Apache Camel blog....

Tomcat & IntelliJ – Deploy war files outside webapps folder

At present I am working on developing an Android application that needs to be supported by a slew of REST services hosted in the cloud. I chose Google App Engine based on its support for Java, Groovy and most importantly Spring. I developed a Spring MVC based REST application and used ContentNegotiatingViewResolver to negotiate content based on request URL extensions. For example, an XML response will be returned if the request URL ends with .xml, a JSON response for .json and an HTML response if he URL doesn’t have any extension. Don’t get me started on Accept Header versus URL extension based content negotiation. That is a rant for another day. I was attempting to Serialize a Map<Enum, List<Model>>. All was well and I was able to retrieve both HTML and JSON representations, but when I tested retrieving the XML representation, JAXB complained that it cannot handle a Map instance in the root although Jackson was totally cool about it. As usual, Googling revealed that JAXB expected a Container class at its root which I didn’t want to create. I didn’t want to give up either. So, I tried my luck using XStreamMarshaller. This time GAE complained that XStream used a restricted API. WTH? Just out of curiosity, I wanted to check if XStreamMarshaller would work as expected when used outside of GAE. So, I created a Tomcat context file “myapp.xml” with the following definition and carefully placed it inside TOMCAT_HOME/conf/Catalina/localhost. I could have just started Tomcat from TOMCAT_HOME/bin/startup.bat to check if it works, but being an IDEA addict, I created a Run Configuration for the IDEA Tomcat plugin and started the server from inside IDEA. But the app refused to even be discovered, let alone be deployed. After a few frustrated attempts, I tried starting Tomcat directly outside IDEA. Thankfully the app got deployed successfully and to my surprise, the XStreamMarshaller skillfully streamed out the serialized XML. Problem Solved! <?xml version='1.0' encoding='utf-8'?> <Context docBase="PATH_TO_MY_APP" reloadable="true" path="/myapp"> </Context>But, why didn’t the app get deployed when I started Tomcat from inside IDEA? After all, I have linked IDEA to my local Tomcat installation and the script it executes is clearly in my TOMCAT_HOME/bin folder. Then, why why why in the world does the app refuse to be discovered? The solution came in the form of CATALINA_BASE. It seems that IDEA copies the contents of TOMCAT_HOME/conf folder into its HOME folder with some name like Unnamed_MyApp and sets this folder to be the CATALINA_BASE. That explains why “myapp.xml” is so totally ignored by Tomcat. Then, I navigated to “Tomcat Run Configuration -> Startup/Connection -> Environment Variables” and added CATALINA_BASE as an environment variable and pointed it to your local TOMCAT_HOME folder. After this configuration change IDEA started Tomcat as expected and my app was both discovered and deployed. Another Problem Solved! But the real problem – JAXB complaining about Map and GAE rejecting XStreamMarshaller as restricted – is yet to be solved. Maybe I should try one of the CastorMarshaller, XmlBeansMarshaller or JibxMarshaller. Any ideas? Reference: Tomcat & IntelliJ – Deploy war files outside webapps folder from our JCG partner Ganeshji Marwaha at the Ganesh blog....

Google Appengine Staging Server Howto

Out of the box, Google’s App Engine supports versioned deployments. You can switch back and forth between revisions very easily, which is a great feature for properly testing an application before going live. There is one major problem: All versions of the application share the same datastore. So if you’re migrating your data you run a serious risk of influencing your current production application. Hence the need for a proper staging environment. It’s no secret, I am a fan of Google’s App Engine. Once you get used to its peculiarities, it has a number of major advantageous. Since I started incorporating some of the continuous integration/lean startup ideas in my own project. I’ve run into the shared datastore issue and the need for a properly isolated staging environment has become apparent. Here’s how I did it. Setting up the Staging Application It’s possible to use namespaces to create an isolated datastore, however I didn’t want to create additional code for testing. So I took another approach, which I believe is a lot easier and less error-prone:In the appengine control panel, create a second application. You have 10 free ones so that shouldn’t be a problem. I added the “-staging” suffix to the name of the application under test, so I won’t mistake one for the other. If you want to start from a copy of the existing datastore, you can export the entire datastore using the Python development kit. Even if you’re using the Java development kit, it’s worth setting this up. It allows you to make backups of your datastore, which might come in handy when something is really messed up. Next, import the database into your staging application using the same tool. And finally, deploy your application to the staging application. If you’re using Eclipse, just change the application id, if not, you can find the property in the appengine-web.xml.A small note on using production data in your tests: Be very careful about it. You may want to anonymize some of the data and remove anything that could be remotely confidential. That should be it. There really wasn’t much to it, but you now should have a fully functioning copy of your production application. Surf around a little to make sure everything is working swiftly. When you’re happy, lets automate it. Automating Deployments I was about to throw out Maven, but I’ve now created a setup that I’m pretty happy with. So Maven is here to stay for now. As are the Maven Eclipse plugin and the GAE plugin for Maven. It’s thanks to the maven-gae-plugin that I could automate the staging and production deployments. Which has given me a very reproducible build and deployment set up. To seamlessly create a build for both the staging and production server, I’m using Maven profiles and its ability to filter resources while copying them. In the appengine-web.xml I added a gae.application variable: <?xml version="1.0" encoding="utf-8"?> <appengine-web-app xmlns="http://appengine.google.com/ns/1.0"> <application>${gae.application}</application> ...Next up I enabled filtering of the appengine-web.xml (all of the next few bits go into the pom.xml): <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-war-plugin</artifactId> <configuration> <webResources> <resource> <directory>src/main/webapp</directory> <filtering>true</filtering> <includes> <include>**/appengine-web.xml</include> </includes> </resource> </webResources> </configuration> </plugin>In the properties section, I added the default application, which is the staging one. This gives me the assurance that I’ll always be deploying to the staging environment, unless I really want to go to production: <properties> <gae.application>myapp-staging</gae.application> </properties>And for the production deployment I created a profile: <profiles> <profile> <id>production</id> <properties> <gae.application>myapp</gae.application> </properties> </profile> </profiles>With this configuration, I can easily run the local development server: > mvn gae:run Deploy to the staging server: > mvn gae:deploy And when I’m happy, deploy it to the production server: > mvn gae:deploy -Pproduction In addition to the name of the application, you can also configure other properties that differ between a test setup and a production one. For instance, I use the PayPal development servers locally and on the staging server, but the real PayPal site in production. Conclusion With a pretty simple Maven configuration, it’s possible to create a very reproducible build and deployment environment. Add a continuous integration server and you’re on your way to the perfect lean setup. Reference: A Google Appengine Staging Server Howto from our JCG partner Peter Backx at the Streamhead blog....

Spring MVC – Flash Attributes

Latest Spring Framework incarnation (3.1) brought interesting feature called Flash Attributes. It is remedy for the problem mentioned a long time ago, in one of my posts: Spring MVC – Session Attributes handling. This problem can be described in few words: if we want to pass the attributes via redirect between two controllers, we cannot use request attributes (they will not survive the redirect), and we cannot use Spring’s @SessionAttributes (because of the way Spring handles it), only ordinary HttpSession can be used, which is not very convenient. Below you will find an example of Flash Attributes usage, before you start reviewing it, read Using flash attributes section of Spring documentation. Suppose that we have two controllers: AController and BController, first one will prepare some data and pass to the second using Flash Attributes after the form submission. On the AController we will have something like this: @RequestMapping(method = RequestMethod.POST) public String handleFormSubmission(..., final RedirectAttributes redirectAttrs) { ... redirectAttrs.addFlashAttribute("AttributeName", value); return "redirect:to_some_url_handled_by_BController"; }When the form will be submitted, attribute value will be stored as Flash Attribute named “AttributeName”, and thanks to the Spring, will be passed to BController, where it can be used for example in following way: @Controller ... @SessionAttributes("AttributeName") public class SearchCriteriaHandler { ... @RequestMapping(method = RequestMethod.GET) public void handleGetRequest(@ModelAttribute("AttributeName") final SomeType value) { ... } ... }Before your handler method will be called, Spring Framework will populate the Model with the available Flash Attributes – at this point value passed from AController will become a model attribute for the BController. Note, that because we also defined this attribute as the Session Attribute, it will be automatically stored for future usage within this controller, after the GET request handling. Let me say that I was waiting for this feature for the long time, ;) Related Posts: Spring MVC – Session Attributes handling Reference: Spring MVC – Flash Attributes from our JCG partner Micha? Ja?tak at the Warlock’s Thoughts blog....

Set up a Java EE 6 development environment

This tutorial explains in short how to set-up a typical environment to develop Java EE 6 based applications. There is no prerequesite to this tutorial other then a working Windows XP client with sufficient CPU power and memory. During the tutorial we will need to install the following components:Java 6 JDK Update 26 Eclipse IDE for Java EE Developers 3.6.2 (Helios) Maven 3.0.3 Glassfish 3.1I suggest that you create a folder where you drop all the stuff we need to set-up the environment. My Folder is C:\jee6. I will call that <dev_home> when I refer to it. Note: In one of my recent posts I explained how to set-up a Spring environment. This process is very simmilar, therefore I will only explain deviations in this Java EE set-up.In this tutorial we will create a little demo project. The sample application uses JSF 2.0, CDI 1.0, EJB 3.1 and JPA 2.0 (persistence unit included). The demo can be used to start development of your own enterprise project.Installing Java 6 JDKSee this blog post.Installing Eclipse IDE for Java EE DevelopersSee this blog post.Creating the Eclipse Projects using Maven 3See my Spring blog where I have explained how to set-up Maven and create a Spring project. The process to create a Java EE 6 project is almost equivalent. The only difference in setting up a Java EE environment is the archetype to use to create the Eclipse Project. When you create the Eclipse project use the following Maven command: mvn archetype:generate -DarchetypeGroupId=org.jboss.weld.archetypes -DarchetypeArtifactId=weld-jsf-jee -DarchetypeVersion=1.0.0-BETA1 -DgroupId=com.mycompany.jeeapp -DartifactId=jee-app-weld -Dversion=1.0-SNAPSHOT -DinteractiveMode=falseNote: I have also marked the differences in my Spring post to make it obvious where the Spring specifics are during Maven set-up and project creation.Note: At the time this blog post was created the weld-jsf-jee archetype was in beta state. You may use the 1.0 version when it is available. See here to find out if there is a newer version of this archetype.Installing Glassfish 3.1In a Java EE project you need a Java EE server. At the time this post was created Glassfish 3.1 was the only server available that provided the full Java EE 6 stack. Therefore I opt to use Glassfish. To Intall Glassfish to your local machine do the following: Download Glassfish 3.1Go to Glassfish Download page.Select Download GlassFish Server Open Source Edition 3.1. Then choose the Zip-Archive glassfish-3.1.zip (GlassFish Server 3.1 Open Source Edition Full Platform) In your <dev_home> create a folder called glassfish. Unzip the Zip-Archive hereConfigure Eclipse After you successfully downloaded a Glassfish copy you need to configure your Eclipse IDE to use Glassfish 3.1 as the target runtime environment. To do this follow these steps:Open Eclipse.In the Eclipse menu bar select “Window > Show View > Servers”. In the “Servers” view right click on the white space and select “New > Server”.Select the Link “Download additional server adapters”. Eclipse now downloads available server adapters.Select “Oracle Glassfish Server Tools” from the list.Accept the license and select “Finish”. The download begins.Restart Eclipse to activate the new adapter. When you re-enter Eclipse you see the Glassfish server tools front page.Close that view and go to the Workbench.In Eclipse go to the “Servers” view (remember: window available at the bottom) and right click in the white space, then select “New > Server”.Select GlassFish Server Open Source Edition 3 (Java EE 6) and select “Next”Select your home directory for the GlassFish instance you downloaded, that should be <dev_home>/glassfish/glassfish. Select “Finish” if possible. Otherwise, you may have to specify a domain. In that case select “Next”.The domain should be preselected, then click “Finish”.You should see in the “Servers” view that Glassfish was added. To start the server right click on Glassfish server and select “Debug”.  Go to the “Console” view (another window available at the bottom in the Eclipse workbench) and see how GlassFish talks to you :-). You may need to change into the server.log to see the log entries displayed. I have higlighted the toolbar button where you can change to the server log. To maximize and minimize the “Console” view you need to double click the “Console” tab. To verify installation go to your web browser and enter http://localhost:8080.Select the Link “goto the Administration Console” to enter the Admin Console.You can manage GlassFish in the Admin Console. Take a look around.Your environment is complete now. It’s time to get the demo application going.Starting and using the demo application In one of the previous steps we have created a Java EE based application with Maven. We have to import it into Eclipse. You can do this as follows: Import your Java EE applicationOpen Eclipse.In the menu bar select “File > Import”.Enter “Existing” into “Select an import source”.Select “Existing Projects into Workspace” and click “Next”.Select the root directory of the application you have just created with Maven. This should be <dev_home>\workspace\jee-app-weld. Then select “Finish”.The project will be imported. You will notice when you go to the “Markers” view that there are several errors. We have to fix those. Right click the Project “jee-app-weld” and select “Properties” from the context menu.Goto “Targeted Runtimes”.Select GlassFish and then press “OK”. This should clear all compile errors in your “Markers” viewBugfix the demo application Unfortunately, when this post was written the demo application archetype was in beta state. So you have to do some stuff to make it functioning properly. It’s a good exercise to get used to Eclipse.In Eclipse, press Ctrl+Shift+R. The “Open Resource” dialog shows up. In “Select an Item” enter “persistence.xml”, select it in the list below and press “Open”.Repeat the last two steps for the file “home.xhtml”.  You should have opened two files now: persistence.xml and home.xhtml Look at my fixed persistence.xml file and make yours look the same.I have changed the jta-data-source to the GlassFish setting (default was JBoss) and I have added the property eclipselink.ddl-generation.output-mode to the “Properties for EclipseLink” section. The property makes sure that the database is created on application start-up.  Save your changes with Ctrl+S (or in the menu bar File > Save). The second file to adopt is home.xhtml. Here I have changed <h:dataTable var="_widget" value="#widgets"> to <h:dataTable var="_widget" value="#{widgets}">.Save your changes with Ctrl+S (or in the menu bar File > Save).Note: see this blog entry about persistence.xml and this blog entry about home.xhtml for additional info for the solutions explained above.Configure Eclipse for deployment We are almost done. Now, you need to explain Eclipse that the GlassFish server is the deployment container. You do this as follows:In the “Servers” view right click on the Glassfish server and select “Add and Remove”.Select your application and press “Add” to move it in the right section.Press “Finish”. The last thing to do is to make sure Glassfish starts the integrated Derby database, so you can use the default JDBC datasource. In the menu bar go to “Window > Preferences”.Select “GlassFish Preferences” and select the checkbox to enable the JavaDB when the server starts. Then press OK.That’s it already in terms of preparing Eclipse. Now start the demo application. Starting and verifying the demo applicationSelect the Glassfish server in the “Servers” view and richt click into the context menu, then select “Debug” (make sure the Server is not running already, if so stop the Server first). See the “Console” view to verify that the application started properly. Again: double click the “Console” tab to maximize and minimize the view in the workbench. Make sure you see the server.log.Go to your web browser and enter http://localhost:8080/jee-app-weldYou’re done. You have created your own Java EE development environment. From here look around a litlle and build your own project.Reference: “Setting up a Java EE 6 development environment” from our JCG partner Niklas....
Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.
Do you want to know how to develop your skillset and become a ...
Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you two of our best selling eBooks for FREE!

Get ready to Rock!
You can download the complementary eBooks using the links below: