Featured FREE Whitepapers

What's New Here?

enterprise-java-logo

Ultimate JPA Queries and Tips List – Part 1

There are several JPAs “how to” that we can find on the internet, here in this blog, that teaches how to do several tasks with JPA. Usually I see some people asking questions about Queries with JPA; usually to answer this kind of questions several links are provided trying to find a solution to the question. Until today I could not find a blog post that puts together a good subject about queries with JPA, tips about performance/utilization, source code to download… Today we will see:Model classes and a class that will generate database data Find method; Use the getReference method to get a better performance, displaying query parameters in the console with the log4j JPQL: Queries with simple parameters or objects, Joins, Order By, Navigating through relationships JPQL: Functions: AVG, COUNT, MAX, MIN, TRIM, SUM, UPPER, LOWER, MOD, LENGHT, SQRT; Using HAVING, GROUP BY JPQL: Filtering Conditions: LIKE, IN, DISTINCT, EMPTY, BETWEEN, NULL, MEMBER OF, EXISTS (Subqueries), ANY, ALL, SOME, CONCAT, CURRENT_DATE, CURRENT_TIME, CURRENT_TIMESTAMP, LOCATE, SIZE, SUBSTRING JPA: NamedQuery, querying with dates, warnings about the getSingleResult method JPA: NativeQuery, NamedNativeQuery JPA: Complex Native Queries JPA: Optimizing queries with EJB JPA: Pagination JPA: Database Hints JPA: Creating a object from a query JPQL: Bulk Update and Delete JPA: CriteriaYou will see that in every main class we will invoke the method “CodeGenerator.generateData()”. This class method only creates data in the database; with this data our queries will find the correct results. In the last page of this post you will find the link to download the source code of this post. In this post we will use JPA 2.0 with Hibernate as the provider. The database will be the HSQLDB and will be attached to this project. You can download the source code and run the project without the need of any extra configuration. We will not talk about how to set up the HSQLDB because the focus of this post is how to query data of the database. This post will not use best practices of development in some points. The focus of this post will be to display how the JPA queries works. Model classes and a class that will generate database data package com.model;import java.util.ArrayList; import java.util.List;import javax.persistence.CascadeType; import javax.persistence.Entity; import javax.persistence.GeneratedValue; import javax.persistence.GenerationType; import javax.persistence.Id; import javax.persistence.JoinColumn; import javax.persistence.OneToMany; import javax.persistence.OneToOne;@Entity public class Person {@Id @GeneratedValue(strategy = GenerationType.AUTO) private int id;private String name; private int age;public Person() {}public Person(String name, int age) { this.name = name; this.age = age; }@OneToMany(mappedBy = 'person', cascade = CascadeType.ALL) private List<Dog> dogs;@OneToOne(cascade = CascadeType.ALL) @JoinColumn(name='address_id') private Address address;public int getId() { return id; }public void setId(int id) { this.id = id; }public String getName() { return name; }public void setName(String name) { this.name = name; }public int getAge() { return age; }public void setAge(int age) { this.age = age; }public List<Dog> getDogs() { if (dogs == null) { dogs = new ArrayList<Dog>(); }return dogs; }public void setDogs(List<Dog> dogs) { this.dogs = dogs; }public Address getAddress() { return address; }public void setAddress(Address address) { this.address = address; }@Override public int hashCode() { return getId(); }@Override public boolean equals(Object obj) { if (obj instanceof Person) { Person person = (Person) obj; return person.getId() == getId(); }return false; } } package com.model;import java.util.Date;import javax.persistence.Entity; import javax.persistence.GeneratedValue; import javax.persistence.GenerationType; import javax.persistence.Id; import javax.persistence.ManyToOne; import javax.persistence.Temporal; import javax.persistence.TemporalType;@Entity public class Dog {@Id @GeneratedValue(strategy = GenerationType.AUTO) private int id;private String name; private double weight;@Temporal(TemporalType.TIMESTAMP) private Date dateOfBirth;public Dog() {}public Dog(String name, double weight, Date dateOfBirth) { this.name = name; this.weight = weight; this.dateOfBirth = dateOfBirth; }public static void main(String[] args) {System.out.println(new Date()); }@ManyToOne private Person person;public int getId() { return id; }public void setId(int id) { this.id = id; }public String getName() { return name; }public void setName(String name) { this.name = name; }public double getWeight() { return weight; }public void setWeight(double weight) { this.weight = weight; }public Date getDateOfBirth() { return dateOfBirth; }public void setDateOfBirth(Date dateOfBirth) { this.dateOfBirth = dateOfBirth; }public Person getPerson() { return person; }public void setPerson(Person person) { this.person = person; }@Override public int hashCode() { return getId(); }@Override public boolean equals(Object obj) { if (obj instanceof Dog) { Dog dog = (Dog) obj; return dog.getId() == getId(); }return false; } } package com.model;import javax.persistence.Entity; import javax.persistence.GeneratedValue; import javax.persistence.GenerationType; import javax.persistence.Id;@Entity public class Address {@Id @GeneratedValue(strategy = GenerationType.AUTO) private int id;private String streetName; private int houseNumber;public Address() {}public Address(String streetName, int houseNumber) { this.streetName = streetName; this.houseNumber = houseNumber; }public int getId() { return id; }public void setId(int id) { this.id = id; }public String getStreetName() { return streetName; }public void setStreetName(String streetName) { this.streetName = streetName; }public int getHouseNumber() { return houseNumber; }public void setHouseNumber(int houseNumber) { this.houseNumber = houseNumber; }@Override public int hashCode() { return getId(); }@Override public boolean equals(Object obj) { if (obj instanceof Address) { Address address = (Address) obj; return address.getId() == getId(); }return false; } } We got some basic classes with relationships unidirectional and bidirectional. These relationships will help us to manipulate all type of queries that we will be executing. To generate the database data we have the class bellow: package com.main;import java.text.ParseException; import java.text.SimpleDateFormat; import java.util.Date;import javax.persistence.EntityManager; import javax.persistence.EntityManagerFactory; import javax.persistence.Persistence;import com.model.Address; import com.model.Dog; import com.model.Person;public class CodeGenerator { private static EntityManagerFactory emf; private static EntityManager em;public static final String PERSON01_NAME = 'John'; public static final String PERSON02_NAME = 'Mary'; public static final String PERSON03_NAME = 'Anna'; public static final String PERSON04_NAME = 'Joseph'; public static final String PERSON05_NAME = 'Mark'; public static final String PERSON06_NAME = 'I will not have any relationship';public static void startConnection() { emf = Persistence.createEntityManagerFactory('JpaQuery'); em = emf.createEntityManager(); em.getTransaction().begin(); }public static void closeConnection() { em.getTransaction().commit(); emf.close(); }public static void generateData() { int year = 1995; int month = 1; int day = 10;Dog dog01 = new Dog('Yellow', 3.5d, createNewDate(day, month, year)); Dog dog02 = new Dog('Brown', 8.5d, createNewDate(++day, ++month, ++year)); Dog dog03 = new Dog('Dark', 15.5d, createNewDate(++day, ++month, ++year)); Dog dog04 = new Dog('Kaka', 4.3d, createNewDate(++day, ++month, ++year)); Dog dog05 = new Dog('Pepe', 8.2d, createNewDate(++day, ++month, ++year)); Dog dog06 = new Dog('Casillas', 6.1d, createNewDate(++day, ++month, ++year)); Dog dog07 = new Dog('Fish', 6.7d, createNewDate(++day, ++month, ++year)); Dog dog08 = new Dog('Lion', 3.1d, createNewDate(++day, ++month, ++year)); Dog dog09 = new Dog('Cat', 5.5d, createNewDate(++day, ++month, ++year)); Dog dog10 = new Dog('Java', 21.7d, createNewDate(++day, ++month, ++year)); Dog dog11 = new Dog('JSF', 23.65d, createNewDate(++day, ++month, ++year)); Dog dog12 = new Dog('VRaptor', 24.0d, createNewDate(++day, ++month, ++year)); Dog dog13 = new Dog('Ferrari', 3.7d, createNewDate(++day, ++month, ++year)); Dog dog14 = new Dog('Porshe', 1.33d, createNewDate(++day, ++month, ++year)); Dog dog15 = new Dog('Bike', 4.44d, createNewDate(++day, ++month, ++year)); Dog dog16 = new Dog('Rambo', 5.44d, createNewDate(++day, ++month, 2015)); Dog dog17 = new Dog('Terminator', 3.88d, createNewDate(++day, ++month, 2016)); Dog dog18 = new Dog('John McClan', 3.88d, createNewDate(++day, ++month, 2016));Person person01 = new Person(PERSON01_NAME, 33); person01.getDogs().add(dog01); person01.getDogs().add(dog02); person01.getDogs().add(dog03); person01.setAddress(new Address('Street A', 30)); dog01.setPerson(person01); dog02.setPerson(person01); dog03.setPerson(person01);Person person02 = new Person(PERSON02_NAME, 27); person02.getDogs().add(dog04); person02.getDogs().add(dog05); person02.getDogs().add(dog06); person02.setAddress(new Address('Street B', 60)); dog04.setPerson(person02); dog05.setPerson(person02); dog06.setPerson(person02);Person person03 = new Person(PERSON03_NAME, 7); person03.getDogs().add(dog07); person03.getDogs().add(dog08); person03.getDogs().add(dog09); person03.setAddress(new Address('Street B', 90)); dog07.setPerson(person03); dog08.setPerson(person03); dog09.setPerson(person03);Person person04 = new Person(PERSON04_NAME, 43); person04.getDogs().add(dog10); person04.getDogs().add(dog11); person04.getDogs().add(dog12); person04.setAddress(new Address('Street C', 120)); dog10.setPerson(person04); dog11.setPerson(person04); dog12.setPerson(person04);Person person05 = new Person(PERSON05_NAME, 70); person05.getDogs().add(dog13); person05.getDogs().add(dog14); person05.getDogs().add(dog15); person05.getDogs().add(dog16); person05.setAddress(new Address('Street D', 150)); dog13.setPerson(person05); dog14.setPerson(person05); dog15.setPerson(person05); dog16.setPerson(person05);Person person06 = new Person(PERSON06_NAME, 45);em.persist(person01); em.persist(person02); em.persist(person03); em.persist(person04); em.persist(person05); em.persist(person06);em.persist(dog17); em.persist(dog18);em.flush(); }private static Date createNewDate(int day, int month, int year) { SimpleDateFormat formatter = new SimpleDateFormat('dd/MM/yyyy'); try { return formatter.parse('' + day + '/' + month + '/' + year); } catch (ParseException e) { e.printStackTrace(); return null; } }public static EntityManager getEntityManager() { return em; } } Find method; Use the getReference method to get a better performance, displaying query parameters in the console with the log4j The find method usually is invoked before we execute some change in the database like update some object attribute, relationship or to delete it. Bellow you will find codes that uses the find method: package com.main;import javax.persistence.EntityManager;import com.model.Address; import com.model.Person;public class Page03 { public static void main(String[] args) { CodeGenerator.startConnection();CodeGenerator.generateData();EntityManager em = CodeGenerator.getEntityManager();Person person = em.find(Person.class, 1);int addressId = 2;// usually we send an id or a detached object from the view setAddressToOtherPerson(em, person, addressId);int personId = 4;// usually we send an id or a detached object from the view deletePerson(em, personId);CodeGenerator.closeConnection(); }private static void setAddressToOtherPerson(EntityManager em, Person person, int addressId) { Address address = em.find(Address.class, addressId); person.setAddress(address); em.merge(person); em.flush(); }private static void deletePerson(EntityManager em, int personId) { Person savedPerson = em.find(Person.class, personId); em.remove(savedPerson); em.flush(); } } Notice that the methods “setAddressToOtherPerson” and “deletePerson” uses the find method only to update a reference or to delete an object. The find() method has an optimized query functionality that will search for an object in the Persistence Context, if it does not find the object it will query the database to bring the data. If you got a relationship annotated with EAGER (e.g.: “@OneToMany(fetch=FetchType.EAGER)”) the find method will bring these objects from the database. Notice that there is no need to bring all this data from the database for simple tasks like a delete of reference update. The EntityManager has a specific method that helps with these simples tasks. The EntityManager will do a simple query like “select id from Person p where p.id = :personId“. We will have a faster and smaller query. Bellow you can see how we will use the getReference: package com.main;import javax.persistence.EntityManager;import com.model.Address; import com.model.Person;public class Page03 { public static void main(String[] args) { CodeGenerator.startConnection();CodeGenerator.generateData();EntityManager em = CodeGenerator.getEntityManager();Person person = em.find(Person.class, 1);int addressId = 2;// usually we send an id or a detached object from the view setAddressToOtherPerson(em, person, addressId);int personId = 4;// usually we send an id or a detached object from the view deletePerson(em, personId);CodeGenerator.closeConnection(); }private static void setAddressToOtherPerson(EntityManager em, Person person, int addressId) { Address address = em.getReference(Address.class, addressId); person.setAddress(address); em.merge(person); em.flush(); System.out.println('Merged'); }private static void deletePerson(EntityManager em, int personId) { // usually is find or merge Person savedPerson = em.getReference(Person.class, personId); em.remove(savedPerson); em.flush(); System.out.println('Deleted'); } } With the method “getReference” you will query only for the object ID, you will save some database traffic. Bellow you will find the lo4j.properties configuration needed to display the JPA queries parameters in the console. Usually when we invoke a query using the Hibernate, the Hibernate will format the query with “?” instead of using the real value. With the code bellow you will be able to see the query parameters: # Direct log messages to stdout log4j.appender.stdout=org.apache.log4j.ConsoleAppender log4j.appender.stdout.Target=System.out log4j.appender.stdout.layout=org.apache.log4j.PatternLayout log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %c{1}:%L - %m%n# Root logger option log4j.rootLogger=ERROR, stdout# Hibernate logging options (INFO only shows startup messages) log4j.logger.org.hibernate=ERROR# Log JDBC bind parameter runtime arguments log4j.logger.org.hibernate.type=TRACEIf you want to deactivate the log you just need to comment the lo4j.properties last line with the # symbol, and to set the show_log configuration in the “persistence.xml” to false. JPQL: Queries with simple parameters or objects, Joins, Order By, Navigating through relationships To do a basic query you just need to run a command like this: “select d from Dog d”. One thing that you always need to keep in your mind is that: to do this kind of query we use JPQL and not the regular SQL. The advantage of using JPQL is that it is very similar to SQL and it is portable. You may use the same query in every database without a problem. Never concatenate your query with a string. If you do a query like this: “select p from Person p where p.name” + person.getName(), you can be sure that hackers will love it. They use this kind of code to do an attack named “SQL Injection” (or JPQL Injection). The way to avoid this kind of attack is adding parameters to your query like we will see bellow. You will see bellow several ways to do a query: package com.main;import java.util.List;import javax.persistence.EntityManager; import javax.persistence.Query;import com.model.Dog; import com.model.Person;public class Page04 { public static void main(String[] args) { CodeGenerator.startConnection();CodeGenerator.generateData();EntityManager em = CodeGenerator.getEntityManager();List<Dog> dogs = listAllDogs(em);for (Dog dog : dogs) { System.out.println(dog.getName()); }Person person03 = findPersonByName(em, CodeGenerator.PERSON03_NAME); System.out.println(person03.getName());Person person01 = new Person(); person01.setId(1);Person savedPerson = findPersonByPersonObject(em, person01); System.out.println(savedPerson.getName());List<Dog> dogsByWeight = listAllDogsOrderingByWeight(em);for (Dog dog : dogsByWeight) { System.out.println(dog.getWeight()); }String addressName = findAddressNameOfPerson(em, CodeGenerator.PERSON04_NAME); System.out.println('Person 04 address is: ' + addressName);Person person02 = findPersonByNameWithAllDogs(em, CodeGenerator.PERSON02_NAME);for (Dog dog : person02.getDogs()) { System.out.println('Person 02 Dog: ' + dog.getName()); }Person person05 = findPersonByNameThatMayNotHaveDogs(em, CodeGenerator.PERSON06_NAME); System.out.println('Is the list of the Dogs from the Person 05 empty? ' + person05.getDogs().size());CodeGenerator.closeConnection(); }/** * Easiest way to do a query */ @SuppressWarnings('unchecked') private static List<Dog> listAllDogs(EntityManager em) { Query query = em.createQuery('select d from Dog d', Dog.class);return query.getResultList(); }/** * Easiest way to do a query with parameters */ private static Person findPersonByName(EntityManager em, String name) { Query query = em.createQuery('select p from Person p where name = :name', Person.class); query.setParameter('name', name); return (Person) query.getSingleResult(); }/** * Executes a query that has as parameter an object */ private static Person findPersonByPersonObject(EntityManager em, Person person) { Query query = em.createQuery('select p from Person p where p = :person'); query.setParameter('person', person); return (Person) query.getSingleResult(); }/** * Query that will list all dogs with an order */ @SuppressWarnings('unchecked') private static List<Dog> listAllDogsOrderingByWeight(EntityManager em) { Query query = em.createQuery('select d from Dog d order by d.weight desc', Dog.class);return query.getResultList(); }/** * Query that get only a field instead a complete class object */ private static String findAddressNameOfPerson(EntityManager em, String name) { Query query = em.createQuery('select p.address.streetName from Person p where p.name = :name'); query.setParameter('name', name); return (String) query.getSingleResult(); }/** * Query that will fetch a lazy relationship Be carefull, with this kind of * query only those who have the relationship will come in the result */ private static Person findPersonByNameWithAllDogs(EntityManager em, String name) { Query query = em.createQuery('select p from Person p join fetch p.dogs where p.name = :name', Person.class); query.setParameter('name', name); return (Person) query.getSingleResult(); }/** * With this query will will bring results that may not have arelationship */ private static Person findPersonByNameThatMayNotHaveDogs(EntityManager em, String name) { Query query = em.createQuery('select p from Person p left join fetch p.dogs where p.name = :name', Person.class); query.setParameter('name', name); return (Person) query.getSingleResult(); } } About the code above:Each query is invoked like “em.createQuery(“HHH”, HHH.class)” with the specific query text and return class. You can define a return class, e.g. Person.class. The Person.class parameter will indicate to the JPA which is the return object. We can use basic attributes as query parameters like “p.name = :name” or an object “p = :person“. If you use an object the JPA will compare by its @ID. If you want to order your query you just need to do: “order by d.weight desc“. The default order value is asc and you do not need to write it. About the join you must pay attention to the two kinds of joins that we used. In the “findPersonByNameWithAllDogs” method we just used “… Person p join fetch p.dogs …” to bring the dogs list. We needed to use the join fetch because the dog list is annotated with the “lazy” attribute; if we did not included the join fetch and executed a command like “person.getDogs()“, other “trip” to the database would be required. If you use this query to find a Person that has no dogs, the JPA will found no data in the database no matter if your database has a Person without dogs. If you want to do a query that brings a fetch dog collection and Persons that has or not dogs you will need to use “…Person p left join fetch p.dogs…” like we did in the method: “findPersonByNameThatMayNotHaveDogs“. The “left join fetch” will bring Persons that has an empty dog list.JPQL: Functions: AVG, COUNT, MAX, MIN, TRIM, SUM, UPPER, LOWER, MOD, LENGHT, SQRT; Using HAVING, GROUP BY JPQL also has a lot of functions that help us with the queries. Bellow you can see their description:AVG – Does a number average COUNT – Counts the records amount found by the query MAX – Gets the higher value of a column MIN – Gets the lower value of a column TRIM – Removes the white space at the begging/end of the text SUM – Sums all the values of a column UPPER – Modifies all the column text to UPPER CASE LOWER – Modifies all the column text to lower case MOD – Returns the modulus of a column LENGTH – Returns the size of a String SQRT – Returns the square root of a numberBellow you will see how to use these functions: package com.main;import java.util.List;import javax.persistence.EntityManager; import javax.persistence.Query;import com.model.Person;public class Page05 { public static void main(String[] args) { CodeGenerator.startConnection();CodeGenerator.generateData();EntityManager em = CodeGenerator.getEntityManager();Number average = getPersonsAgeAverage(em); System.out.println(average);List<Object[]> personsFilteredByDogsWeight = getPersonsWithDogsWeightHigherThan(em, 4d);for (Object[] objects : personsFilteredByDogsWeight) { Person person = (Person) objects[0]; Long count = (Long) objects[1]; System.out.println('The person : ' + person.getName() + ' has ' + count + ' dogs with the weight > 4'); }List<Object[]> dogsMinAndMaxWeightList = getDogMinAndMaxWeight(em); Object[] dogMinAndMaxWeightResult = dogsMinAndMaxWeightList.get(0); System.out.println('Min: ' + dogMinAndMaxWeightResult[0] + ' Max: ' + dogMinAndMaxWeightResult[1]);Number sumOfAllAges = getTheSumOfAllAges(em); System.out.println('All summed ages are: ' + sumOfAllAges);String loweredCaseName = getLoweredCaseNameFromUpperCase(em, CodeGenerator.PERSON03_NAME); System.out.println(loweredCaseName);Number personAgeMod = getPersonAgeMode(em, CodeGenerator.PERSON05_NAME, 6); System.out.println('Person modulus age: ' + personAgeMod);Number personAgeSqrt = getPersonAgeSqrtUsingTrim(em, ' ' + CodeGenerator.PERSON04_NAME + ' '); System.out.println('Person modulus age: ' + personAgeSqrt);List<Object[]> personsByDogsAmount = getPersonByHavingDogAmountHigherThan(em, 3);for (Object[] objects : personsByDogsAmount) { Person person = (Person) objects[0]; Long count = (Long) objects[1]; System.out.println(person.getName() + ' has ' + count + ' dogs'); }CodeGenerator.closeConnection(); }/** * Uses the AVG sql database function */ private static Number getPersonsAgeAverage(EntityManager em) { Query query = em.createQuery('select avg(p.age) from Person p'); return (Number) query.getSingleResult(); }/** * This query will use the count database function * * @return List<Object[]> where object[0] is a person, object [2] is a Long */ @SuppressWarnings('unchecked') private static List<Object[]> getPersonsWithDogsWeightHigherThan(EntityManager em, double weight) { Query query = em.createQuery('select p, count(p) from Person p join p.dogs d where d.weight > :weight group by p'); query.setParameter('weight', weight); return query.getResultList(); }/** * This query will use the min and max sql database function * * @return List<Object[]> where object[0] is the min, object [2] is the max */ @SuppressWarnings('unchecked') private static List<Object[]> getDogMinAndMaxWeight(EntityManager em) { Query query = em.createQuery('select min(weight), max(weight) from Dog'); return query.getResultList(); }/** * This query will use the sum sql database function */ private static Number getTheSumOfAllAges(EntityManager em) { Query query = em.createQuery('select sum(p.age) from Person p'); return (Number) query.getSingleResult(); }/** * Method that uses the UPPER and LOWER database functions */ private static String getLoweredCaseNameFromUpperCase(EntityManager em, String name) { Query query = em.createQuery('select lower(p.name) from Person p where UPPER(p.name) = :name'); query.setParameter('name', name.toUpperCase()); return (String) query.getSingleResult(); }/** * Method that uses the mod database function */ private static Number getPersonAgeMode(EntityManager em, String personName, int modBy) { Query query = em.createQuery('select mod(p.age, :modBy) from Person p where p.name = :name'); query.setParameter('modBy', modBy); query.setParameter('name', personName); return (Number) query.getSingleResult(); }/** * Method that uses the square root of a person age using the trim function in the name */ private static Number getPersonAgeSqrtUsingTrim(EntityManager em, String name) { Query query = em.createQuery('select sqrt(p.age) from Person p where p.name = trim(:name)'); query.setParameter('name', name); return (Number) query.getSingleResult(); }/** * Method that uses the having comparator with count */ @SuppressWarnings('unchecked') private static List<Object[]> getPersonByHavingDogAmountHigherThan(EntityManager em, long dogAmount) { Query query = em.createQuery('select p, count(p) from Person p join p.dogs group by p.id having count(p) > :dogAmount'); query.setParameter('dogAmount', dogAmount); return query.getResultList(); } } About the code above:In the method “getPersonsAgeAverage” we use the “avg” function to do an average calculation of the age column value. In the method “getPersonsWithDogsWeightHigherThan” we use the count function to bring the amount of dog with a person object. Notice that we have two distinct results, a number and a person object. These values will come inside an array Object[]. The LOWER and UPPER functions will change your string case and you can use it to change the result of your query (after the select) or in the where condition. The “getLoweredCaseNameFromUpperCase” method uses the LOWER and UPPER functions in both ways. The “getPersonAgeMode” uses a parameter after the word select. With JPA we can use a parameter in any place of the query, you just need to add the “:” with a variable. You can have the same parameter several times and pass the value with the query.setParameter method. In the method “getPersonByHavingDogAmountHigherThan” the “having” function is invoked with the “count” function. We can use the “having” function to help us to filter the query data result.JPQL: Filtering Conditions: LIKE, IN, DISTINCT, EMPTY, BETWEEN, NULL, MEMBER OF, EXISTS (Subqueries), ANY, ALL, SOME, CONCAT, CURRENT_DATE, CURRENT_TIME, CURRENT_TIMESTAMP, LOCATE, SIZE, SUBSTRING Some of these functions have the same purpose but handled differently. Bellow you can see how to use these functions: package com.main;import java.text.SimpleDateFormat; import java.util.Date; import java.util.List;import javax.persistence.EntityManager; import javax.persistence.Query;import com.model.Dog; import com.model.Person;public class Page06 {public static void main(String[] args) { CodeGenerator.startConnection();CodeGenerator.generateData();EntityManager em = CodeGenerator.getEntityManager();List<Person> personByLike = getPersonByNameUsingLike(em, 'oh');for (Person person : personByLike) { System.out.println(person.getName()); }List<Person> personsByAdressNumber = getPersonsByAddressNumberHigherThan(em, 90);for (Person person : personsByAdressNumber) { System.out.println(person.getName()); }List<Person> personsWithoutDogs = getPersonsWithoutDogs(em);System.out.println('Total of persons without dogs: ' + personsWithoutDogs.size());List<Person> personsWithoutAddress = getPersonsWithoutAddress(em);System.out.println('Total of persons without address: ' + personsWithoutAddress.size());try { SimpleDateFormat formatter = new SimpleDateFormat('dd/MM/yyyy');Date startDate = formatter.parse('01/01/1996'); Date endDate = formatter.parse('01/01/1999');List<Dog> dogsByBirth = getDogByBirthDate(em, startDate, endDate);for (Dog dog : dogsByBirth) { System.out.println(dog.getName() + ': ' + formatter.format(dog.getDateOfBirth())); } } catch (Exception e) { e.printStackTrace(); }Dog dog = (Dog) em.createQuery('select d from Dog d where d.id = 1', Dog.class).getSingleResult();boolean belongsTo = isThisDogBelongingToAperson(em, dog, CodeGenerator.PERSON01_NAME);System.out.println('Is this Dog member of Perons01? ' + belongsTo);Person personByConcatedName = getPersonConcatingName(em, 'Ma', 'ry');System.out.println('Found the person? ' + personByConcatedName.getName());List<Person> personByLocate = getPersonByLocatingStringInTheName(em, 'Mary');System.out.println('Amount of persons found by locate: ' + personByLocate.size());String personNameBySubstring = getPersonNameBySubstring(em, CodeGenerator.PERSON06_NAME, 12, 18);System.out.println('Name substring is: ' + personNameBySubstring);List<Person> personsDogWeight = getPersonByDogWeightOnlyHigherThan(em, 20);for (Person person : personsDogWeight) { System.out.println(person.getName()); }List<Person> distinctPersons = getDistinctPersonsByDogsWeight(em, 2d); System.out.println('With the distinct, the result size is: ' + distinctPersons.size());List<Person> personsWithDogsAmount = getPersonsWithDougsAmountOf(em, 4); System.out.println('Number of persons with 4 dogs: ' + personsWithDogsAmount.size());Number numberOfDogsByPerson = getDogAmountByPerson(em, CodeGenerator.PERSON04_NAME); System.out.println('The dog amount is to ' + CodeGenerator.PERSON04_NAME + ': ' + numberOfDogsByPerson);List<Dog> dogsBornedAfterToday = getDogsBornAfterToday(em); System.out.println('The amount of dogs borned after today is: ' + dogsBornedAfterToday.size());CodeGenerator.closeConnection(); }/** * This methods compares a value with LIKE */ @SuppressWarnings('unchecked') private static List<Person> getPersonByNameUsingLike(EntityManager em, String name) { Query query = em.createQuery('select p from Person p where p.name like :name'); query.setParameter('name', '%' + name + '%'); return query.getResultList(); }/** * This methods show several ways to do a query that checks if a part of a collection is inside another */ @SuppressWarnings('unchecked') private static List<Person> getPersonsByAddressNumberHigherThan(EntityManager em, int houseNumber) { Query query = em.createQuery('select p from Person p where p.address in (select a from Address a where a.houseNumber > :houseNumber)'); // Query query = em.createQuery('select p from Person p where (select a from Address a where a.houseNumber > :houseNumber and p.address = a) > 0'); // Query query = em.createQuery('select p from Person p where p.address = any (select a from Address a where a.houseNumber > :houseNumber)'); // Query query = em.createQuery('select p from Person p where p.address = some (select a from Address a where a.houseNumber > :houseNumber)'); // Query query = em.createQuery('select p from Person p where exists (select a from p.address a where a.houseNumber > :houseNumber)'); query.setParameter('houseNumber', houseNumber); return query.getResultList(); }/** * This methods show how to check if a collection is empty */ @SuppressWarnings('unchecked') private static List<Person> getPersonsWithoutDogs(EntityManager em) { Query query = em.createQuery('select p from Person p where p.dogs is empty'); return query.getResultList(); }/** * This method shows two ways to check if a relationship @OneToOne is empty */ @SuppressWarnings('unchecked') private static List<Person> getPersonsWithoutAddress(EntityManager em) { Query query = em.createQuery('select p from Person p where p.address is null'); // Query query = em.createQuery('select p from Person p where p.address is empty'); return query.getResultList(); }/** * Method that uses the between comparation */ @SuppressWarnings('unchecked') private static List<Dog> getDogByBirthDate(EntityManager em, Date startDate, Date endDate) { Query query = em.createQuery('select d from Dog d where d.dateOfBirth between :startDate and :endDate'); query.setParameter('startDate', startDate); query.setParameter('endDate', endDate); return query.getResultList(); }/** * Method that uses the member of comparation to check if an object belogs to a collection */ private static boolean isThisDogBelongingToAperson(EntityManager em, Dog dog, String name) { Query query = em.createQuery('select p from Person p where :dog member of p.dogs and p.name = :name'); query.setParameter('dog', dog); query.setParameter('name', name);try { return query.getSingleResult() != null; } catch (Exception e) { return false; } }/** * Methods that concats Strings */ private static Person getPersonConcatingName(EntityManager em, String firstWord, String secondWord) { Query query = em.createQuery('select p from Person p where p.name = concat(:firstWord, :secondWord)', Person.class); query.setParameter('firstWord', firstWord); query.setParameter('secondWord', secondWord); return (Person) query.getSingleResult(); }/** * Method that locates a string inside another */ @SuppressWarnings('unchecked') private static List<Person> getPersonByLocatingStringInTheName(EntityManager em, String valueToBeLocated) { Query query = em.createQuery('select p from Person p where locate(p.name, :value) > 0', Person.class); query.setParameter('value', valueToBeLocated); return query.getResultList(); }/** * Methods that uses the ALL comparator */ @SuppressWarnings('unchecked') private static List<Person> getPersonByDogWeightOnlyHigherThan(EntityManager em, double weight) { Query query = em.createQuery('select p from Person p where p.dogs is not empty and :weight < all (select d.weight from p.dogs d)'); query.setParameter('weight', weight);return query.getResultList(); }/** * Method that uses the distinct to remove any repetetition */ @SuppressWarnings('unchecked') private static List<Person> getDistinctPersonsByDogsWeight(EntityManager em, double weight) { Query query = em.createQuery('select distinct p from Person p join p.dogs d where d.weight > :weight'); query.setParameter('weight', weight); return query.getResultList(); }/** * Method that uses the substring to get just a position of chars inside the string */ private static String getPersonNameBySubstring(EntityManager em, String personName, int startPosition, int endPosition) { Query query = em.createQuery('select substring(p.name, :startPosition, :endPosition) from Person p where p.name = :personName'); query.setParameter('personName', personName); query.setParameter('startPosition', startPosition); query.setParameter('endPosition', endPosition); return (String) query.getSingleResult(); }/** * Method that checks the size of a collection */ @SuppressWarnings('unchecked') private static List<Person> getPersonsWithDougsAmountOf(EntityManager em, int dogAmount) { Query query = em.createQuery('select p from Person p where size(p.dogs) = :dogAmount'); query.setParameter('dogAmount', dogAmount); return query.getResultList(); }/** * Method that gets the size of a collection */ private static Number getDogAmountByPerson(EntityManager em, String personName) { Query query = em.createQuery('select size(p.dogs) from Person p where p.name = :personName'); query.setParameter('personName', personName); return (Number) query.getSingleResult(); }/** * Methods that uses the current database server date/time */ @SuppressWarnings('unchecked') private static List<Dog> getDogsBornAfterToday(EntityManager em) { Query query = em.createQuery('select d from Dog d where d.dateOfBirth > CURRENT_DATE'); return query.getResultList(); } } About the code above:You can add the “NOT” word in your queries. If you use the “IS EMPTY” you will search for a collection without values; if you use the “IS NOT EMPTY” you will search for a populated collection. The “getPersonsByAddressNumberHigherThan” shows how to do the same query with different functions. All commented command lines will bring the same result. In/Any/Some/Exists has a close syntax. According to the Pro EJB3 book “Some” is an alias to “Any”. The comparator “IS EMPTY” can be used to check a collection (e.g. @OneToMany) or a relationship class (e.g. @OneToOne). The “IS NULL” comparator cannot check a collection, but you can use it to check a non collection attribute (e.g. @OneToOne). The “MEMBER OF” comparator will check if a given parameter belongs to a collection. The “CONCAT” function can be used as condition comparator or as a query result. In the code above it was used just as comparator but you could use it like this: “select concat(firstName, lastName) from Person p” In the “getPersonByDogWeightOnlyHigherThan” method we use the ALL operator. This operator will return true only if all items of the condition (“:weight > ALL”) return true. In this method it will return true only if all the person dogs’ weight were higher than “:weight”, if only one of the dogs got the weight with a smaller value the comparator would return false. You must be aware of, if the list is empty the comparator will return true. To avoid this behavior you would need to check if the list is empty like was done in the method: “p.dogs is not empty”. The “distinct” function will remove the duplicated objects. In the method “getDistinctPersonsByDogsWeight” the “distinct” function removes the duplicated persons. The “SUBSTRING” function extracts a value from a given string. You will set the beginning and the end of the value that will be extracted from the original value. You can use this function as a comparator also. The “SIZE” function will return the number of elements inside the collection. You can use as comparator or to get the value. In the code above we use the “CURRENTE_DATE” function to compare the date, you could use also “CURRENT_TIME, CURRENT_TIMESTAMP”. The JPA specification says that the current date functions can only be used as comparator. JPA does not support any function to retrieve the database current date yet, because this kind of function is not database portable (4.6.16 Functional Expressions – JSR-000220 Enterprise JavaBeans 3.0 Final Release (persistence) ). If you want to query the database date you can use a NativeQuery to get this value. I must always remember that you cannot navigate inside a collection. You cannot do the following command: “person.dogs.name”. You can access a dogs name using a command like: select p from Person p fetch join p.dogs d where d.name = ‘’.Continue to the second part of the series. Reference: JPA Queries and Tips from our JCG partner Hebert Coelho at the uaiHebert blog....
enterprise-java-logo

Ultimate JPA Queries and Tips List – Part 2

This part continues from the first part of the series. JPA: NamedQuery, querying with dates, warnings about the getSingleResult method To avoid the repetition of queries codes, upgrade the performance and make easier to maintain the queries we can use the NamedQueries. A NamedQuery uses JPQL as syntax and it is declared in the entity class. It is easier to edit the query after an update in the class code. If you want to do a query using a date as parameter you can send only the date object or you can pass an enum that will describe the date type (recommended). Bellow you will see how to create and use a @NamedQuery and how to query with date: package com.model;import java.util.Date;import javax.persistence.*;@Entity @NamedQuery(name='Dog.FindByDateOfBirth', query='select d from Dog d where d.dateOfBirth = :dateOfBirth') public class Dog {public static final String FIND_BY_DATE_OF_BIRTH = 'Dog.FindByDateOfBirth';@Id @GeneratedValue(strategy = GenerationType.AUTO) private int id;// get and set } package com.model;import java.util.*;import javax.persistence.*;@Entity @NamedQueries({ @NamedQuery(name='Person.findByName', query='select p from Person p where p.name = :name'), @NamedQuery(name='Person.findByAge', query='select p from Person p where p.age = :age')}) }) public class Person {public static final String FIND_BY_NAME = 'Person.findByName'; public static final String FIND_BY_AGE = 'Person.findByAge';@Id @GeneratedValue(strategy = GenerationType.AUTO) private int id;// get and set} package com.main;import java.text.ParseException; import java.text.SimpleDateFormat; import java.util.Date; import java.util.List;import javax.persistence.EntityManager; import javax.persistence.Query; import javax.persistence.TemporalType;import com.model.Dog; import com.model.Person;public class Page07 { public static void main(String[] args) { CodeGenerator.startConnection();CodeGenerator.generateData();EntityManager em = CodeGenerator.getEntityManager();int age = 70; List<Person> personByAge = getPersonByAge(em, 70); System.out.println('Found ' + personByAge.size() + ' person(s) with the age of: ' + age);SimpleDateFormat formatter = new SimpleDateFormat('dd/MM/yyyy'); Date dateOfBirth = null; try { dateOfBirth = formatter.parse('10/1/1995'); } catch (ParseException e) { e.printStackTrace(); }List<Dog> dogsByDayOfBirth = getDogsByDayOfBirth(em, dateOfBirth); System.out.println('Found ' + dogsByDayOfBirth.size() + ' dog with birth date of ' + formatter.format(dateOfBirth));/* * This queries will raise Runtime Exceptions * * em.createQuery('select p from Person p').getSingleResult(); // NonUniqueResultException * * em.createQuery('select p from Person p where p.name = 'JJJJ'').getSingleResult(); //NoResultException */CodeGenerator.closeConnection(); }@SuppressWarnings('unchecked') private static List<Dog> getDogsByDayOfBirth(EntityManager em, Date dateOfBirth) { Query query = em.createNamedQuery(Dog.FIND_BY_DATE_OF_BIRTH); query.setParameter('dateOfBirth', dateOfBirth, TemporalType.DATE);return query.getResultList(); }@SuppressWarnings('unchecked') private static List<Person> getPersonByAge(EntityManager em, int age) { Query query = em.createNamedQuery(Person.FIND_BY_AGE); query.setParameter('age', age);return query.getResultList(); } } About the code above:If you have only one query you can use the @NamedQuery annotation; if you have more than one query you can use the @NamedQueries annotations. When you do a query using a date object you can also use the TemporalType enum to detail the type of the date. For date queries you can use “java.util.Date” or “java.util.GregorianCalendar”.getSingleResult() Be careful when you use this method. It has a special way to handle two behaviors that it is easy to happen and both behaviors will raise an exception:Find more than one object from the query result: NonUniqueResultException Find no result: NoResultExceptionYou always need to use a try/catch to avoid these exceptions to be thrown in the production environment. If you want to see this exceptions in real time, in the code above you can find two commented queries; it will raise the exceptions bellow: Exception in thread 'main' <span style='text-decoration: underline;'>javax.persistence.NonUniqueResultException</span>: result returns more than one elements at org.hibernate.ejb.QueryImpl.getSingleResult(<span style='text-decoration: underline;'>QueryImpl.java:287</span>) Exception in thread 'main' <span style='text-decoration: underline;'>javax.persistence.NoResultException</span>: No entity found for query at org.hibernate.ejb.QueryImpl.getSingleResult(<span style='text-decoration: underline;'>QueryImpl.java:280</span>) JPA: NativeQuery, NamedNativeQuery The JPA uses the JPQL that does not have any database specific feature. How could you do a query invoking a specific database function when the JPQL just provides the commons functionalities between the databases? “select p from Person p where p.name ~* :name” This query syntax is a valid query for Postgres database; if you try to run this query using a NamedQuery (with JPQL) you will see the exception bellow: <strong><em>ERROR SessionFactoryImpl:422 - Error in named query: Person.FindByName</em></strong> <strong><em>org.hibernate.QueryException: unexpected char: '~' [select p from com.model.Person p where p.name ~* :name]</em></strong> <strong><em>at org.hibernate.hql.ast.QueryTranslatorImpl.doCompile(QueryTranslatorImpl.java:229)</em></strong> <strong><em>at org.hibernate.hql.ast.QueryTranslatorImpl.compile(QueryTranslatorImpl.java:136)</em></strong> <strong><em>at org.hibernate.engine.query.HQLQueryPlan.<init>(HQLQueryPlan.java:101)</em></strong> <strong><em>at org.hibernate.engine.query.HQLQueryPlan.<init>(HQLQueryPlan.java:80)</em></strong> <strong><em>at org.hibernate.engine.query.QueryPlanCache.getHQLQueryPlan(QueryPlanCache.java:124)</em></strong> <strong><em>at org.hibernate.impl.SessionFactoryImpl.checkNamedQueries(SessionFactoryImpl.java:547)</em></strong> <strong><em>at org.hibernate.impl.SessionFactoryImpl.<init>(SessionFactoryImpl.java:411)</em></strong> <strong><em>at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1842)</em></strong> <strong><em>at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:902)</em></strong> <strong><em>at org.hibernate.ejb.HibernatePersistence.createEntityManagerFactory(HibernatePersistence.java:57)</em></strong> <strong><em>at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:48)</em></strong> <strong><em>at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:32)</em></strong> <strong><em>at com.main.CodeGenerator.startConnection(CodeGenerator.java:27)</em></strong> <strong><em>at com.main.Page05.main(Page05.java:12)</em></strong> <strong><em>Exception in thread “main” javax.persistence.PersistenceException: [PersistenceUnit: JpaQuery] Unable to build EntityManagerFactory</em></strong> <strong><em>at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:911)</em></strong> <strong><em>at org.hibernate.ejb.HibernatePersistence.createEntityManagerFactory(HibernatePersistence.java:57)</em></strong> <strong><em>at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:48)</em></strong> <strong><em>at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:32)</em></strong> <strong><em>at com.main.CodeGenerator.startConnection(CodeGenerator.java:27)</em></strong> <strong><em>at com.main.Page05.main(Page05.java:12)</em></strong> <strong><em>Caused by: org.hibernate.HibernateException: Errors in named queries: Person.FindByName</em></strong> <strong><em>at org.hibernate.impl.SessionFactoryImpl.<init>(SessionFactoryImpl.java:424)</em></strong> <strong><em>at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1842)</em></strong><strong><em> </em></strong> <strong><em>at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:902)</em></strong> <strong><em> ... 5 more</em></strong> The solution to this kind of situation is to use a query that does not use JPQL as its syntax. With NativeQuery you will be able to do queries using the database syntax. You can see in the code bellow how to use NativeQueries: package com.main;import javax.persistence.EntityManager; import javax.persistence.Query;import com.model.Dog;public class Page08 { public static void main(String[] args) { CodeGenerator.startConnection();CodeGenerator.generateData();EntityManager em = CodeGenerator.getEntityManager();String nameOfFirstPerson = getFirstPersonName(em); System.out.println(nameOfFirstPerson);Dog dog = getTopDogDescending(em); System.out.println(dog.getName());CodeGenerator.closeConnection(); }/** * Returns the name of the first person using a native sql */ private static String getFirstPersonName(EntityManager em) { Query query = em.createNativeQuery('select top 1 name from person'); return (String) query.getSingleResult(); }/** * Return an object using a native sql */ private static Dog getTopDogDescending(EntityManager em) { Query query = em.createNativeQuery('select top 1 id, name, weight from dog order by id desc', Dog.class); return (Dog) query.getSingleResult(); } } About the code above:Notice that we use a native query instead JPQL. We can have an entity as result of a native query. The method “getTopDogDescending” returns a Dog object after a native query invocation.You also can create your native query as @NamedNativeQuery. The difference between NamedNativeQuery and NativeQuery is that the NamedNativeQuery is defined on its entity class, and you can have only one with that name. The advantages of using a @NamedNativeQuery are:Easy to maintain the code: every query is on the class, if a class update an attribute it will be easier to update the query. Helps on a better performance: Once your query is already declared the JPA will map it and keep its syntax on the memory. The JPA will not need to “parse” you query every time that your project use it. Raises your code reutilization: Once you declare a @NamedNativeQuery you must provide the “name” parameter with a value and this name must be unique to the Persistence Unit scope (you set this value in the “persistence.xml” and it is used by the EntityManager).You will see bellow how to declare a @NamedNativeQuery: package com.model;import java.util.*;import javax.persistence.*;@Entity @NamedQueries({ @NamedQuery(name='Person.findByName', query='select p from Person p where p.name = :name'), @NamedQuery(name='Person.findByAge', query='select p from Person p where p.age = :age') })@NamedNativeQuery(name='Person.findByNameNative', query='select id, name, age from person where name = :name') public class Person {public static final String FIND_BY_NAME = 'Person.findByName'; public static final String FIND_BY_AGE = 'Person.findByAge';@Id @GeneratedValue(strategy = GenerationType.AUTO) private int id;// get and set } You will be able to use the @NamedNativeQuery just like the @NativeQuery: “em.createNamedQuery(“Person.findByNameNative”);”. Here it comes some bad news. Unfortunately the Hibernate does not have the @NamedNativeQuery implemented yet. If you try to run the code with this annotation you will see the exception bellow: <em>Caused by: <span style='text-decoration: underline;'>org.hibernate.cfg.NotYetImplementedException</span>: Pure native scalar queries are not yet supported</em> <em>at org.hibernate.cfg.annotations.QueryBinder.bindNativeQuery(<span style='text-decoration: underline;'>QueryBinder.java:140</span>)</em> <em>at org.hibernate.cfg.AnnotationBinder.bindQueries(<span style='text-decoration: underline;'>AnnotationBinder.java:339</span>)</em> <em>at org.hibernate.cfg.AnnotationBinder.bindClass(<span style='text-decoration: underline;'>AnnotationBinder.java:548</span>)</em> <em>at org.hibernate.cfg.Configuration$MetadataSourceQueue.processAnnotatedClassesQueue(<span style='text-decoration: underline;'>Configuration.java:3977</span>)</em> <em>at org.hibernate.cfg.Configuration$MetadataSourceQueue.processMetadata(<span style='text-decoration: underline;'>Configuration.java:3931</span>)</em> <em>at org.hibernate.cfg.Configuration.secondPassCompile(<span style='text-decoration: underline;'>Configuration.java:1368</span>)</em> <em>at org.hibernate.cfg.Configuration.buildMappings(<span style='text-decoration: underline;'>Configuration.java:1345</span>)</em> <em>at org.hibernate.ejb.Ejb3Configuration.buildMappings(<span style='text-decoration: underline;'>Ejb3Configuration.java:1477</span>)</em> <em>at org.hibernate.ejb.EventListenerConfigurator.configure(<span style='text-decoration: underline;'>EventListenerConfigurator.java:193</span>)</em> <em>at org.hibernate.ejb.Ejb3Configuration.configure(<span style='text-decoration: underline;'>Ejb3Configuration.java:1096</span>)</em> <em>at org.hibernate.ejb.Ejb3Configuration.configure(<span style='text-decoration: underline;'>Ejb3Configuration.java:278</span>)</em> <em>at org.hibernate.ejb.Ejb3Configuration.configure(<span style='text-decoration: underline;'>Ejb3Configuration.java:362</span>)</em> JPA: Complex Native Queries You will be able to create complexes mapping to a NativeQuery; this mappings will return more than one class or value. You can see bellow how our class map this complex results and how to execute a query with it: package com.model;import java.util.*;import javax.persistence.*;@Entity @NamedQueries({ @NamedQuery(name='Person.findByName', query='select p from Person p where p.name = :name'), @NamedQuery(name='Person.findByAge', query='select p from Person p where p.age = :age')}) }) @SqlResultSetMappings({ @SqlResultSetMapping(name='personAndAdress', entities={ @EntityResult(entityClass=Person.class), @EntityResult(entityClass=Address.class, fields={ @FieldResult(name='id', column='ADDRESS_ID') } ) }), @SqlResultSetMapping(name='personWithDogAmount', entities={@EntityResult(entityClass=Person.class)}, columns={@ColumnResult(name='dogAmount')} ) }) public class Person {public static final String FIND_BY_NAME = 'Person.findByName'; public static final String FIND_BY_AGE = 'Person.findByAge'; public static final String MAPPING_PERSON_AND_ADDRESS = 'personAndAdress'; public static final String MAPPING_DOG_AMOUNT = 'personWithDogAmount';@Id @GeneratedValue(strategy = GenerationType.AUTO) private int id;// get and set } package com.main;import java.math.BigInteger;import javax.persistence.EntityManager; import javax.persistence.Query;import com.model.Address; import com.model.Person;public class Page09 { public static void main(String[] args) { CodeGenerator.startConnection();CodeGenerator.generateData();EntityManager em = CodeGenerator.getEntityManager();Query query = em.createNativeQuery('select id, name, age, a.id as ADDRESS_ID, houseNumber, streetName ' + 'from person p join address a on a.id = p.address_id where p.id = 1', Person.MAPPING_PERSON_AND_ADDRESS);Object[] result = (Object[]) query.getSingleResult();Person personWithAdress = (Person) result[0]; Address address = (Address) result[1];System.out.println(personWithAdress.getName() + ' lives at ' + address.getStreetName());query = em.createNativeQuery('select p.id, p.name, count(0) as dogAmount ' + 'from person p join dog d on p.id = d.person_id where name = 'Mark' ' + 'group by p.id, p.name', Person.MAPPING_DOG_AMOUNT);result = (Object[]) query.getSingleResult();Person person = (Person) result[0]; BigInteger total = (BigInteger) result[1];System.out.println(person.getName() + ' has ' + total + ' dogs');CodeGenerator.closeConnection(); } } About the code above:With the “@SqlResultSetMapping” you will notify the JPA which entities we want as result. Notice that in the mapping “personAndAdress” we wrote the classes that will be returned. We also used an attribute named “@FieldResult”. This attribute will map query fields with the same name, in our query we got the person id and the address id. That is why we used the “@FieldResult” to notify the JPA to map the column ADDRESS_ID to the ID attribute of the Address class. In the “dogAmount” mapping we set the “@ColumnResult” attribute that notifies to the JPA that we will have an “extra column” in the query result and this “extra column” does not belong to any class.JPA: Optimizing queries with EJB Every time you do a query inside a transaction scope, the Persistence Context will keep the result “attached”; the Persistence Context will “watch” this objects just in case if any of this “attached” objects receive any update. All “attached” objects updates will be persisted in the database. @PersistenceContext(unitName = 'myPU') private EntityManager em;@TransactionAttribute(TransactionAttributeType.REQUIRED) public void editPersonName(Integer personId, String personNewName){ Person person = em.find(Person.class, personId); person.setName(personNewName); } You can see in the code above that we did not need to invoke the “em.merge()” to update the person name in the database. When we bring a collection from the database, usually to be displayed on a datatable or reports, all these objects will be attached to the Persistence Context. The process to make this objects attached will fire several processes, data validations and synchronizations. When higher is the number of objects, higher will be the allocated memory to the query result and higher will be the work of the Persistence Context to keep all this objects “attached”. What is the point of having all these objects “attached” when their final destination will be to be sent to the view? When the objects leave the EJB project and go to the view project they will be considered “detached”. With this scenario we have an unnecessary work grabbing all the data from the database, make than “attached” and send them to the view to make them “detached”. There is an easy way to do this objects to come from the database already detached. The advantage of this approach is that the Persistence Context will never waste time and container processor trying to synchronize the query result. You can see in the code bellow how this solution works. package com.main;import java.util.List;import javax.ejb.Stateless; import javax.ejb.TransactionAttribute; import javax.ejb.TransactionAttributeType; import javax.persistence.EntityManager; import javax.persistence.PersistenceContext; import javax.persistence.Query;import com.model.Person;@Stateless @TransactionAttribute(TransactionAttributeType.NOT_SUPPORTED) public class PersonDAO { @PersistenceContext(unitName = 'myPU') private EntityManager em;@TransactionAttribute(TransactionAttributeType.REQUIRED) public void editPersonName(Integer personId, String personNewName){ Person person = em.find(Person.class, personId); person.setName(personNewName); }@SuppressWarnings('unchecked') public List<Person> listAll(){ Query query = em.createQuery('select p from Person p'); return query.getResultList(); }@SuppressWarnings('unchecked') public List<Person> listAllWithoutDogs(){ Query query = em.createQuery('select p from Person p where p.dogs is empty'); return query.getResultList(); } } In the code above we got a DAO class that is an EJB. Our EJB has by default the lack of transaction (“@TransactionAttribute(TransactionAttributeType.NOT_SUPPORTED)”), with this kind of transaction the Persistence Context will not “attach” the result of the queries. The objects returned by the queries will be considered “detached”. Notice that the annotation of the method “editPersonName” stills the same: “@TransactionAttribute(TransactionAttributeType.REQUIRED)”. This kind of transaction indicates to the EJB that a new transaction should be started if none has been started yet. You can set a transaction attribute for the class, but a method can override this attribute just like we did in the “editPersonName”. The method transaction definition will have priority over the class transaction definition. JPA: Pagination If you want to do the JPA pagination just do like the method bellow: package com.main;import java.util.List;import javax.persistence.EntityManager; import javax.persistence.Query;import com.model.Dog;public class Page11 {@SuppressWarnings('unchecked') public static void main(String[] args) { CodeGenerator.startConnection();CodeGenerator.generateData();EntityManager em = CodeGenerator.getEntityManager();Query query = em.createQuery('select d from Dog d');List<Dog> dogs = query.getResultList();System.out.println('Total of dogs found: ' + dogs.size());query.setMaxResults(5); query.setFirstResult(0);List<Dog> fiveFirstDogs = query.getResultList();System.out.print('Total of dogs found: ' + fiveFirstDogs.size() + ' '); for (Dog dog : fiveFirstDogs) { System.out.print(dog.getName() + ' '); }System.out.println();query.setMaxResults(5); query.setFirstResult(5);List<Dog> fiveSecondDogs = query.getResultList();System.out.print('Total of dogs found: ' + fiveSecondDogs.size() + ' '); for (Dog dog : fiveSecondDogs) { System.out.print(dog.getName() + ' '); }CodeGenerator.closeConnection(); } } About the code above:The method “setMaxResults” will set the amount of result that the query will return. The method “setFirstResult” will set the first line that will be brought.In the first query we searched for all data in the database. In the second query we got five results starting from the position 0. In the last query we got the five results again but starting of the position 5. Remember that the first position is always zero and not one. JPA: Database Hints The database vendors provided to us specifics functions named Hints. These Hints are very useful because they optimize queries and help us with other tasks. Each database got its own Hints and these values are not portable. Bellow you can see some hints:SQLServer: OPTION ( OPTIMIZE FOR (@name = ‘Mark’, @age UNKNOWN) ); Oracle: select /*+ first_rows (100) */ name MySQL: select * from person ignore index (col3_index)Each database vendor will set the rules to its hints, rules like syntax and executions commands. There are two ways to define hints: package com.main;import java.util.List;import javax.persistence.EntityManager; import javax.persistence.Query;import com.model.Dog;public class Page12 {@SuppressWarnings('unchecked') public static void main(String[] args) { CodeGenerator.startConnection();CodeGenerator.generateData();EntityManager em = CodeGenerator.getEntityManager();Query query = em.createQuery('select d from Dog d'); query.setHint('org.hibernate.timeout', 1000);List<Dog> dogs = query.getResultList(); System.out.println('Found ' + dogs.size() + ' dogs');CodeGenerator.closeConnection(); } } package com.model;import java.util.*;import javax.persistence.*;@Entity @NamedQueries({ @NamedQuery(name='Person.findByName', query='select p from Person p where p.name = :name'), @NamedQuery(name='Person.findByAge', query='select p from Person p where p.age = :age', hints={@QueryHint(name='org.hibernate.timeout', value='1000')}) }) @SqlResultSetMappings({ @SqlResultSetMapping(name='personAndAdress', entities={ @EntityResult(entityClass=Person.class), @EntityResult(entityClass=Address.class, fields={ @FieldResult(name='id', column='ADDRESS_ID') } ) }), @SqlResultSetMapping(name='personWithDogAmount', entities={@EntityResult(entityClass=Person.class)}, columns={@ColumnResult(name='dogAmount')} ) }) public class Person {public static final String FIND_BY_NAME = 'Person.findByName'; public static final String FIND_BY_AGE = 'Person.findByAge'; public static final String MAPPING_PERSON_AND_ADDRESS = 'personAndAdress'; public static final String MAPPING_DOG_AMOUNT = 'personWithDogAmount';@Id @GeneratedValue(strategy = GenerationType.AUTO) private int id;// get and set } You can set a hint in the @NamedQuery and directly in the Query. You must always remember the disadvantage of the Hints. Once you set the Hint on your NamedQuery your code will not be portable to others database vendors. One solution would be using only Query instead NamedQuery. Before you add the hint to the Query you could check if the current database supports that hint; you could use a properties file to define the current database of the application. Continue to the third part of the series Reference: JPA Queries and Tips from our JCG partner Hebert Coelho at the uaiHebert blog....
enterprise-java-logo

Ultimate JPA Queries and Tips List – Part 3

Before you read the third part , remember the first and second part of the series JPA: Creating a object from a query The JPA allow us to create objects inside a query, just with the values that we need:               package com.model;public class PersonDogAmountReport { private int dogAmount; private Person person;public PersonDogAmountReport(Person person, int dogAmount) { this.person = person; this.dogAmount = dogAmount; }public int getDogAmount() { return dogAmount; }public void setDogAmount(int dogAmount) { this.dogAmount = dogAmount; }public Person getPerson() { return person; }public void setPerson(Person person) { this.person = person; } } package com.main;import java.util.List;import javax.persistence.EntityManager; import javax.persistence.Query;import com.model.PersonDogAmountReport;public class Page13 {@SuppressWarnings('unchecked') public static void main(String[] args) { CodeGenerator.startConnection();CodeGenerator.generateData();EntityManager em = CodeGenerator.getEntityManager();Query query = em.createQuery('select new com.model.PersonDogAmountReport(p, size(p.dogs)) from Person p group by p.id');List<PersonDogAmountReport> persons = query.getResultList(); for (PersonDogAmountReport personReport : persons) { System.out.println(personReport.getPerson().getName() + ' has: ' + personReport.getDogAmount() + ' dogs.'); }CodeGenerator.closeConnection(); } } Notice that inside our query we create a new object. The good news is that you can create any object, it does not need to be an entity. You just need to pass the full path of the class and the JPA will handle new class instantiation. This is a very useful functionality to use with reports that you need of specifics fields but those fields do not exists in the entity. JPQL: Bulk Update and Delete Sometimes we need do execute an operation to update several rows in a table database. E.g. update all persons with the age higher than 70 and define than as elderly. You can run a Bulk Update/Delete like this: package com.main;import javax.persistence.EntityManager; import javax.persistence.Query;import com.model.Person;public class Page14 {public static void main(String[] args) { CodeGenerator.startConnection();CodeGenerator.generateData();EntityManager em = CodeGenerator.getEntityManager();em.clear(); Query query = em.createQuery('update Person p set p.name = 'Fluffy, the destroyer of worlds!''); query.executeUpdate();query = em.createQuery('select p from Person p where p.id = 4');Person person = (Person) query.getSingleResult();System.out.println('My new name is: ' + person.getName());query = em.createQuery('delete from Person p where p.dogs is empty'); query.executeUpdate();query = em.createQuery('select p from Person p');System.out.println('We had 6, but was found ' + query.getResultList().size() + ' persons in the database');CodeGenerator.closeConnection(); } } The cascade option will not be triggered in this scenario; you will not be able to delete an object and hope that the JPA will delete the cascaded objects in the relationship. The database data integrity belongs to the Developer once we talk about bulk operations. If you want to remove an object from the database and its relationships you need to update the object setting the relationship to null before executing the delete. We can define this kind of operation as a very dangerous operation. If we comment the line 17 (“em.clear(); “), we will see that the name of the person stills the same after the update. The “problem” is that the Persistence Context keeps all the data in memory “attached”, but these kinds of bulk operations do not update the Persistence Context. We will have an operation done in our database but was not reflected at our Persistence Context yet. This kind of situation could give us synchronization problems. Picture the following scenario:A transaction is started. Person A is persisted in the database through the method em.persist(). Person B has its name updated to “Louanne” through the method em.merge(). Person A will be removed by a bulk delete. Person B has its name updated to “Fernanda” by a bulk update. Transaction finishes.What would happen in this scenario? The Person A was removed by the bulk operation but the Persistence Context will try to persist it the database. The Person B had its name updated to Fernanda but the Persistence Context will try to update to Louanne. There is no default behavior for kind of situation, but there are solutions that we can use to avoid these problems:Start a new transaction before the bulk operation: With a new transaction started just to the bulk operation, when this operations finishes the updates/deletes will be executed in the database. You will not have an entity manager trying to use data that has not been written to the database yet. Call the “entityManager.clear()” method before the bulk operation: if you invoke this method you will force the Persistence Context to release all cached data. After the bulk operation if you use the find method, the Persistence Context will get the data from the database because you cleaned the cached data before the bulk operation.Invoke the clear() method is not a silver bullet, if you use it too many times it can give you performance problems. If your Persistence Context has a lot of cached objects and you invoke the clear() method, your Persistence Context will have to do a lot of “trips” to get the needed data again. The Persistence Context has a wonderful data cache control and should take advantage of it. The bulk operations are an option that will help us in several situations, but you must use it carefully. JPA: Criteria JPA is a good framework do run your queries, but Criteria it is not a good way of doing queries with JPA. JPA Criteria is too much verbose, complicated and it need too much code to do some basic queries. Unfortunately it is not easy as Hibernate Criteria. The code bellow will show an easy Criteria code, but we will not see more of this subject. I already read 3 books about EJB/JPA and not a single one book talked about it. The code bellow has a criteria code: package com.main;import java.util.List;import javax.persistence.EntityManager; import javax.persistence.criteria.CriteriaQuery;import com.model.Person;public class Page15 {@SuppressWarnings({ 'unchecked', 'rawtypes' }) public static void main(String[] args) { CodeGenerator.startConnection();CodeGenerator.generateData();EntityManager em = CodeGenerator.getEntityManager();CriteriaQuery criteriaQuery = em.getCriteriaBuilder().createQuery(); criteriaQuery.select(criteriaQuery.from(Person.class));List<Person> result = em.createQuery(criteriaQuery).getResultList();System.out.println('Found ' + result.size() + ' persons.');CodeGenerator.closeConnection(); } } I am sorry to say what I think about Criteria like this here, but at this point I do not see simplicity to use Criteria in your code unless for A ListALL. The code above code easily applied to a Generic DAO, it would be easier to list all objects by it. The following link shows a Generic DAO in an application: Full WebApplication JSF EJB JPA JAAS. The end! I hope this post might help you. Click here do download the source code. You will not need to edit any configuration to run the code of this post, just import it to the Eclipse. If you got any doubt/comment just post it bellow. See you soon. Useful Links:http://www.mkyong.com/hibernate/how-to-display-hibernate-sql-parameter-values-log4j/ http://stackoverflow.com/questions/1659030/how-to-get-the-database-time-with-jpql Pro EJB 3: Java Persistence API, Mike Keith, Merrick Schincariol Enterprise JavaBeans 3.0 – Richard Monson-Haefel, Bill BurkeReference: JPA Queries and Tips from our JCG partner Hebert Coelho at the uaiHebert blog....
spring-logo

Spring Custom Namespaces

Spring Custom Namespaces provides a good way to simplify the xml files used to describe the bean definitions of a Spring Application Context. It is a fairly old concept, first introduced with Spring 2.0, but deserves being reviewed once in a while. Consider a case of having to configure a part of the beans for a Spring MVC application without custom namespaces – this would typically look like this: <bean name="handlerAdapter" class="org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter"> <property name="webBindingInitializer"> <bean class="org.springframework.web.bind.support.ConfigurableWebBindingInitializer"> <property name="conversionService" ref="conversionService"></property> <property name="validator"> <bean class="org.springframework.validation.beanvalidation.LocalValidatorFactoryBean"/> </property> </bean> </property> <property name="messageConverters"> <list> <bean class="org.springframework.http.converter.ByteArrayHttpMessageConverter"></bean> <bean class="org.springframework.http.converter.StringHttpMessageConverter"></bean> <bean class="org.springframework.http.converter.ResourceHttpMessageConverter"></bean> <bean class="org.springframework.http.converter.xml.SourceHttpMessageConverter"></bean> <bean class="org.springframework.http.converter.xml.XmlAwareFormHttpMessageConverter"></bean> <bean class="org.springframework.http.converter.xml.Jaxb2RootElementHttpMessageConverter"></bean> <bean class="org.springframework.http.converter.json.MappingJacksonHttpMessageConverter"></bean> </list> </property> </bean><bean name="handlerMapping" class="org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping"> <property name="useSuffixPatternMatch" value="false"></property> </bean>Here it is configuring two beans – a handlerAdapter to handle the MVC controller flow and a handlerMapping to keep the mapping between request URI’s and the Controller methods to handle the requests. The same configuration becomes very concise with a custom namespace, “http://www.springframework.org/schema/mvc” typically given a namspace prefix of “mvc”: <mvc:annotation-driven conversion-service="conversionService"> </mvc:annotation-driven>This is in essence the advantage of using a Custom namespace – a very concise way to describe the Spring bean definitions So how does a custom namespace work: This section in the Spring Reference document describes it much better than I can – http://static.springsource.org/spring/docs/3.1.x/spring-framework-reference/html/extensible-xml.html#extensible-xml-schema . To summarize it, a custom namespace has 4 parts to it:the schema - which describes the structure of the custom namespace – the tag names, attributes, child tags etc. a NamespaceHandler – which creates the bean definition for the xml elements. However typically a better mechanism suggested by Spring document is to extend NameSpaceHandlerSupport and to register a series of BeanDefinitionParser(s) for the different xml elements supported by the Custom namespace(say annotation-driven, interceptors elements of mvc namespace). BeanDefinitionParser – create the bean definition for the specific element – here is where a line like <mvc:annotation-driven/> will be expanded to the broader bean definitions with actual bean class names. Registering the schema, NamespaceHandler – This is for Spring to find the schema for the custom namespaces and to find the NamespaceHandler that will handle the custom namespace. The registration of the schema is done by a file called META-INF/spring.schemas, this is neat way for Spring to find the schema in the classpath rather than downloading the schema over the web. The NamespaceHandler is further specified using a META-INF/spring.handlers file and contains the NamespaceHandler name that will handle the custom namespace, for eg. from the Spring documentation –http\://www.mycompany.com/schema/myns=org.springframework.samples.xml.MyNamespaceHandlerTying it together  This information of how the Custom namespace works internally can be put to good use to understand some of behavior of a few custom namepsace tags. Consider a tag to load up the properties file: <context:property-placeholder location="classpath*:META-INF/spring/database.properties"/>So to find how a property-placeholder element works internally, first find the spring.handlers file. Since property-placeholder is in the context namespace, the spring.handlers file will be present in the spring-context.jar file and the file indicates that the NamespaceHandler is org.springframework.context.config.ContextNamespaceHandler The ContextNameSpaceHandler registers a BeanDefinition parser called org.springframework.context.config.PropertyPlaceholderBeanDefinitionParser. This BeanDefinitionParser creates a bean definition with a class name of ” PropertyPlaceholderConfigurer” and so essentially we could have replaced: <context:property-placeholder location="classpath*:META-INF/spring/database.properties" local-override="true" properties-ref="localProperties"/>with but losing conciseness in the process – <bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"> <property name="location" value="classpath*:META-INF/spring/database.properties"></property> <property name="localOverride" value="true"></property> <property name="properties"> <ref bean="localProperties"/> </property> </bean>This however provides a good way to understand some of the nuances in how Spring handles things underlying a custom namespace. Reference: Spring Custom Namespaces from our JCG partner Biju Kunjummen at the all and sundry blog....
devops-logo

Does Devops have a Culture Problem?

At the Devopsdays conference in Mountain View, Spike Morelli led an Open Space discussion on the importance of culture. He was concerned that when people think and talk about devops they think and talk too much about tools and practices, and not enough about culture and collaboration and communication, not enough about getting people to work closely together and caring about working closely together – and that this is getting worse as more vendors tie themselves to devops.At the Open Space we talked about the problem of defining culture and communicating it and other “soft and squishy” things. What was important and why. What got people excited about devops and how to communicate this and get it out to more people, and how to try to hold onto this in companies that aren’t agile and transparent. Culture isn’t something that you build by talking about itCulture is like quality. You don’t build culture, or quality, by talking about it. You build it by doing things, by acting, by making things happen and making things change, and reinforcing these actions patiently and continually over time.It’s important to talk – but to talk about the right things. To tell good stories, stories about people and organizations making things better and how they did it. What they did that worked, and what they did that didn’t work – how they failed, and what they learned by failing and how they got through it, so that others can learn from them. This transparency and honesty is one of the things that makes the devops community so compelling and so convincing – organizations that compete against each other for customers and talent and funding openly share their operational failures and lessons learned, as well as sharing some of the technology that they used to build their success. You need tools to make Devops workDevops needs good tools to succeed. So does anything that involves working with software. Take Agile development. Agile is about “people over process and tools”. You can’t get agile by putting in an Agile tool of some kind. Developers can, and often prefer to, use informal and low-tech methods, organize their work on a card wall for example. But I don’t know any successful agile development teams that don’t rely on a good continuous integration platform and automated testing tools at least. Without these tools, Agile development wouldn’t scale and Agile teams couldn’t deliver software successfully.The same is even more true for devops. Tools like Puppet and Chef and cfengine, and statsd and Graphite and the different log management toolsets are all a necessary part of devops. Without tools like these devops can’t work. Tools can’t make change, but people can’t change the way that they work without the right tools.  Devops doesn’t have a culture problem – not yetFrom what I can see, devops doesn’t have a culture problem – at least not yet. Everyone who has talked to me about devops (except for maybe a handful of vendors who are just looking for an angle), at this conference or at Velocity or at other conferences or meetups or in forums, all seem to share the same commitment to making operations better, and are all working earnestly and honestly with other people to make this happen.I hear people talking about deployment and configuration and monitoring and metrics and self-service APIs and about automated operations testing frameworks, and about creating catalogs of operations architecture patterns. About tools and practices. But everyone is doing this in a serious and open and transparent and collaborative way. They are living the culture.As Devops ideas catch on and eventually go mainstream – and I think it will happen – devops will change, just like Agile development has changed as it has gone mainstream. Agile in a big, bureaucratic enterprise with thousands of developers is heavier and more formal and not as much fun. But it is still agile. Developers are still able to deliver working software faster. They talk more with each other and with the customer to understand what they are working on and to solve problems and to find new ways to work better and faster. They focus more on getting things done, and less on paper pushing and process. This is a good thing.As bigger companies learn more about devops and adopt it and adapt, it will become something different. As devops is taken up in different industries outside of online web services, with different tolerances for risk different constraints, it will change. And as time goes on and the devops community gets bigger and the people who are involved change and more people find more ways to make more money from devops, devops will become more about training and certification and coaching and consulting and more about commercial tools or about making money from open source tools.Devops will change – not always for the better. That will suck. But at its core I think devops will still be about getting dev and ops together to solve operational problems, about getting feedback and responding to change, about making operations simpler and more transparent. And that’s going to be a good thing.Reference: Does Devops have a Culture Problem? from our JCG partner Jim Bird at the Building Real Software blog....
couchbase-logo

Couchbase 101 : install, store and query data

Introduction In this post I just want to show how easily is to get started with Couchbase, and also explain how to “query” the data. The basic steps of this tutorial are:Install Couchbase Create Data Query DataI will try to post more articles, if I have time to show how to use Couchbase from your applications (starting with Java). Prerequisites :Could not be simpler : Couchbase 2.0 available here. (Currently in Developer Preview)Couchbase 101 : Insert and Query data Installation I am using Couchbase on Mac OS X, so let me describe the installation in this environment. If you are using other operating system just take a look to the Couchbase documentation. Couchbase installation is very (very!) fast:Download the Mac OS X Zip file. Double-click the downloaded Zip installation file to extract the contents. This will create a single file, the Couchbase.app application. Drag and Drop the Couchbase.app to your chosen installation folder, such as the system Applications folder.Start and Configure Couchbase Server   To start Couchbase Server, just double click on the Couchbase Server. Once the server is started, a new icon is added in the OS X Menu to indicate that the Server is up and running. You can now configure your Couchbase instance, for this you just need to access the Admin Console, available at the following location http://127.0.0.1:8091 (change the IP address if needed) or simply by going in the Couchbase menu and click on Open Admin Console entry.Welcome Screen : Click Setup Set the disk and cluster configuration. On my instance I keep the default location for the on disk storage. Just configure the size of the memory usage for your instance, for example 800Mb. So far, we have a single instance, so no need to join a cluster. Choose to generate sample data. This will be interesting to learn more about data and views. Create the default bucket (use for testing only). A bucket is used by Couchbase to store data. It could be compared to a “database” in RDBMS world. Configure update notifications to be alerted when new version of Couchbase is released Configure the server with a final step with the administrator username and password When this is done you are automatically redirected to the Admin Console.This is it! You are ready to use your Couchbase server. Couchbase has many interesting features, especially around scalability and elasticity but for not in this article let’s focus on the basics :Insert some data and query themInsert Data Couchbase has many ways to manipulate data from you favorite programming language using the different client libraries : Java, Python, PHP, Ruby, .Net, C. For now let’s use the Admin Console to create and query data. Couchbase can store any type of data, but when you need to manipulate some data with a structure the best way is to use JSON Documents. So let’s use the console and create documents. To create new documents in your database, click on the “Data Buckets” tab. If you have installed the sample you see 2 buckets: default and gamesim-sample. Let’s create a new documents in the default bucket:Click on Documents button Click on Create Document Since each document must have an id for example 100. Couchbase save the document and add some metadata such as _rev, $flags, expiration Add new attributes to the document that describe an employee : Name, Departement and Salary, then save it. You just need to update the JSON object with values { “_id”: “100″, “name”: “Thomas”, “dept”: “Sales”, “salary”: 5000 }Repeat the operation with some other employees : 200,Jason,Technology,5500 300,Mayla,Technology,7000 400,Nisha,Marketing,9500 500,Randy,Technology,6000 501,Ritu,Accounting,5400 You have now a list of employees in your database. That was easy isn’t? Let’s now query them. Query Data Access document directly from its ID First of all you can quickly access a document using a simple HTTP request using its id. For example to access the Mayla with the id 300 just enter the following URL:http://127.0.0.1:8092/default/300In this URL you have :8092 is the Couch API REST port used to access data (where 8091 is the port for the Admin console) default is the bucket in which the document is stored 300 is the id of the documentSearch your data with queries So we have seen how you can access one document. But what if my need is :“Give me all the employee of the Technology department”To achieve such query it is necessary to create views. The views are used by Couchbase server to index and search your data. A view is a Map function written in JavaScript, that will generate a key value list that is compliant with logic you put in the Map function. Note that this key,value is now indexed and sorted by key. This is important when you query your data. So let’s create a new view from the Admin Console:Click on the Views tab (be sure you are on the default bucket) Click on the “Create Development View” Enter the Document and View name:Document Name : _design/dev_dept View Name : deptCick Save Click on your View to edit itSince we need to provide the list of employees that are part of a the Technology department, we need to create a view that use the department as key, so the map function looks like : function (doc) { emit(doc.dept, null); }Save the view This function takes the document and create a list that contains the “dept” as key and null as value. The value itself is not that important in our case. A simple rule will be : do not put too much data in the value since at the end Couchbase server creates an index with this map. Will see that Couchbase allows developer to easily get the document information when accessing a view. Click on the “Show Results” button, the result will look like: {"total_rows":6,"rows":[ {"id":"501","key":"Accounting","value":null}, {"id":"400","key":"Marketing","value":null}, {"id":"100","key":"Sales","value":null}, {"id":"200","key":"Technology","value":null}, {"id":"300","key":"Technology","value":null}, {"id":"500","key":"Technology","value":null} ] }As we have seen in earlier it is possible to access the document using a single URL, it is the same for views. You can for example access the view we have just created using the following URL:http://127.0.0.1:8092/default/_design/dev_dept/_view/deptNow it is possible to use query parameter to filter the results using the key parameter with the value entered using a JSON String :http://127.0.0.1:8092/default/_design/dev_dept/_view/dept?key=”Technology”The result of this query is now : {"total_rows":6,"rows":[ {"id":"200","key":"Technology","value":null}, {"id":"300","key":"Technology","value":null}, {"id":"500","key":"Technology","value":null} ] }You have many other parameters you can use when accessing a view to control the size, the time out, …. One of them is quite interesting is include_docs that ask Couchbase to include the full content of the document in the result. So if you call :http://127.0.0.1:8092/default/_design/dev_dept/_view/dept?key=”Technology”&include_docs=trueThe result is : {"total_rows":6,"rows":[ {"id":"200","key":"Technology","value":null,"doc": {"_id":"200","_rev":"1-1de6e6751608eada0000003200000000","$flags":0,"$expiration":0,"name":"Jason","dept":"Technology","salary":5500}}, {"id":"300","key":"Technology","value":null,"doc":{"_id":"300","_rev":"1-f3e44cee742bfae10000003200000000","$flags":0,"$expiration":0,"name":"Mayla","dept":"Technology","salary":7000}}, {"id":"500","key":"Technology","value":null,"doc": {"_id":"500","_rev":"1-05780359aac8f3790000003200000000","$flags":0,"$expiration":0,"name":"Randy","dept":"Technology","salary":6000}} ] }Let’s now create a little more complicated view to answer the following business requirement: “Give me all the employee with a salary between 5000 and 6000″ So now you know that you need to create a new view with the salary as key let’s with the following Map function: function (doc) { emit(doc.salary, null); }Couchbase is automatically sorting the key when creating/updating the index so, let’s use the startkey and endkey parameter when calling the view. So let’s call the view with from the following URL:http://127.0.0.1:8092/default/_design/dev_salary/_view/salary?startkey=5000&endkey=6000&include_docs=trueThe result is : {"total_rows":6,"rows":[ {"id":"100","key":5000,"value":null,"doc":{"_id":"100","_rev":"1-0f33580d780014060000002e00000000","$flags":0,"$expiration":0,"name":"Thomas","dept":"Sales","salary":5000}}, {"id":"501","key":5400,"value":null,"doc":{"_id":"501","_rev":"1-b1fe5bc79637720e0000003100000000","$flags":0,"$expiration":0,"name":"Ritu","dept":"Accounting","salary":5400}}, {"id":"200","key":5500,"value":null,"doc":{"_id":"200","_rev":"1-1de6e6751608eada0000003200000000","$flags":0,"$expiration":0,"name":"Jason","dept":"Technology","salary":5500}}, {"id":"500","key":6000,"value":null,"doc":{"_id":"500","_rev":"1-05780359aac8f3790000003200000000","$flags":0,"$expiration":0,"name":"Randy","dept":"Technology","salary":6000}} ] }Conclusion In this short article you have learn how to:Install Couchbase Create data using the Admin Console Query data with viewsWhen I get more time I will write another article that do the same from Java, and other languages. Reference: Couchbase 101 : install, store and query data from our JCG partner Tugdual Grall at the Tug’s Blog blog....
spring-security-logo

Spring Security Part 1 – Simple Login application with database

What is Spring Security? Spring security is a framework that provides security solution, handling authentication and authorization at both the web request level and the method level. Spring security handle security in two ways. One is secure web request and other one is restrict access at the URL level. Spring security uses servlet filters. In this post I’m going to create a simple web application that handle the login authentication and authorization. Download project : http://www.mediafire.com/?bb9x88uxvkb0uuv or http://dl.dropbox.com/u/7215751/JavaCodeGeeks/SpringSecurityTutorialPart1/spring-security-login-example.rar Before create the project need to execute some queries to mysql to create a new database , tables and add some sample data. create-table.sql CREATE DATABASE IF NOT EXISTS `spring-test`; -- create user CREATE USER 'user'@'localhost' IDENTIFIED BY 'test'; GRANT ALL ON spring-test.* TO 'user'@'localhost'; USE `spring-test`; CREATE TABLE USER_DETAILS ( USERNAME VARCHAR(10) NOT NULL, PASSWORD VARCHAR(32) NOT NULL, PRIMARY KEY (USERNAME) ); CREATE TABLE USER_AUTH ( USERNAME VARCHAR(10) NOT NULL, AUTHORITY VARCHAR(10) NOT NULL, FOREIGN KEY (USERNAME) REFERENCES USER_DETAILS(USERNAME) ); test-data.sql insert into USER_DETAILS values ('user','123'); insert into USER_DETAILS values ('admin','admin'); insert into USER_AUTH values ('user', 'ROLE_USER'); insert into USER_AUTH values ('admin', 'ROLE_ADMIN'); After that I create a web project using maven and add below dependencies to pom.xml <properties> <spring.version>3.0.5.RELEASE</spring.version> </properties> <dependencies> <dependency> <groupId>javax.validation</groupId> <artifactId>validation-api</artifactId> <version>1.0.0.GA</version> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-core</artifactId> <version>${spring.version}</version> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-web</artifactId> <version>${spring.version}</version> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-webmvc</artifactId> <version>${spring.version}</version> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-jdbc</artifactId> <version>${spring.version}</version> </dependency> <!-- Spring Security --> <dependency> <groupId>org.springframework.security</groupId> <artifactId>spring-security-core</artifactId> <version>${spring.version}</version> </dependency> <dependency> <groupId>org.springframework.security</groupId> <artifactId>spring-security-web</artifactId> <version>${spring.version}</version> </dependency> <dependency> <groupId>org.springframework.security</groupId> <artifactId>spring-security-config</artifactId> <version>${spring.version}</version> </dependency> <dependency> <groupId>org.springframework.security</groupId> <artifactId>spring-security-taglibs</artifactId> <version>${spring.version}</version> </dependency> <dependency> <groupId>org.springframework.security</groupId> <artifactId>spring-security-acl</artifactId> <version>${spring.version}</version> </dependency> <!-- jstl --> <dependency> <groupId>javax.servlet</groupId> <artifactId>jstl</artifactId> <version>1.2</version> </dependency> <!-- MySQL database driver --> <dependency> <groupId>mysql</groupId> <artifactId>mysql-connector-java</artifactId> <version>5.1.9</version> </dependency> <dependency> <groupId>c3p0</groupId> <artifactId>c3p0</artifactId> <version>0.9.1</version> </dependency> </dependencies> After that change the web.xml like this <!DOCTYPE web-app PUBLIC '-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN' 'http://java.sun.com/dtd/web-app_2_3.dtd' > <web-app> <display-name>spring-security-login</display-name> <servlet> <servlet-name>login</servlet-name> <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class> <load-on-startup>1</load-on-startup> </servlet> <servlet-mapping> <servlet-name>login</servlet-name> <url-pattern>/</url-pattern> </servlet-mapping> <listener> <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class> </listener> <context-param> <param-name>contextConfigLocation</param-name> <param-value> /WEB-INF/login-servlet.xml, /WEB-INF/login-security.xml, /WEB-INF/login-service.xml </param-value> </context-param> <!-- Spring Security --> <filter> <filter-name>springSecurityFilterChain</filter-name> <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class> </filter> <filter-mapping> <filter-name>springSecurityFilterChain</filter-name> <url-pattern>/*</url-pattern> </filter-mapping> <welcome-file-list> <welcome-file>login.jsp</welcome-file> </welcome-file-list> </web-app> Now I need to create login-servlet.xml, login-security.xml and login-service.xml spring configuration files. In this example we are using c3p0 connection pool with Mysql database. Here is the login-servlet.xml file <?xml version='1.0' encoding='UTF-8'?> <beans xmlns='http://www.springframework.org/schema/beans' xmlns:context='http://www.springframework.org/schema/context' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xsi:schemaLocation='http://www.springframework.org/schema/beanshttp://www.springframework.org/schema/beans/spring-beans-3.0.xsdhttp://www.springframework.org/schema/contexthttp://www.springframework.org/schema/context/spring-context-3.0.xsd'><context:component-scan base-package='rd.controller'/> <bean id='internalResourceResolver' class='org.springframework.web.servlet.view.InternalResourceViewResolver'> <property name='prefix' value='/WEB-INF/views/'/> <property name='suffix' value='.jsp'/> </bean> <bean class='org.springframework.web.servlet.mvc.annotation.DefaultAnnotationHandlerMapping'></bean> <bean class='org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter'/> <bean id='placeholderConfig' class='org.springframework.beans.factory.config.PropertyPlaceholderConfigurer'> <property name='locations'> <list> <value>classpath:login.properties</value> </list> </property> </bean> </beans> Here is the login-security.xml <?xml version='1.0' encoding='UTF-8'?> <beans:beans xmlns='http://www.springframework.org/schema/security' xmlns:beans='http://www.springframework.org/schema/beans' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xsi:schemaLocation='http://www.springframework.org/schema/beanshttp://www.springframework.org/schema/beans/spring-beans-3.0.xsdhttp://www.springframework.org/schema/securityhttp://www.springframework.org/schema/security/spring-security-3.0.xsd'><beans:import resource='login-service.xml'/> <http> <intercept-url pattern='/home*' access='ROLE_USER,ROLE_ADMIN' /> <intercept-url pattern='/admin*' access='ROLE_ADMIN' /> <form-login login-page='/login.jsp' default-target-url='/home' authentication-failure-url='/login.jsp?error=true'/> <logout logout-success-url='/login.jsp' /> <anonymous username='guest' granted-authority='ROLE_GUEST'/> <remember-me/> </http> <authentication-manager> <authentication-provider> <!--<user-service>--> <!--<user name='admin' password='secret' authorities='ROLE_ADMIN,ROLE_USER' />--> <!--<user name='user1' password='1111' authorities='ROLE_USER' />--> <!--</user-service>--> <jdbc-user-service data-source-ref='dataSource' users-by-username-query='select username,password, 'true' as enabled from USER_DETAILS where username=?' authorities-by-username-query='select USER_DETAILS.username , USER_AUTH.AUTHORITY as authorities from USER_DETAILS,USER_AUTH where USER_DETAILS.username = ? AND USER_DETAILS.username=USER_AUTH.USERNAME '/> </authentication-provider> </authentication-manager> </beans:beans> Here is the login-service.xml <beans xmlns='http://www.springframework.org/schema/beans' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xsi:schemaLocation='http://www.springframework.org/schema/beanshttp://www.springframework.org/schema/beans/spring-beans-3.0.xsd'><bean id='dataSource' class='com.mchange.v2.c3p0.ComboPooledDataSource'> <!--Driver name to connect to the database--> <property name='driverClass'> <value>${login.jdbc.driver}</value> </property> <!--DB URL--> <property name='jdbcUrl'> <value>${login.url}</value> </property> <!--DB User used to connect to the schema--> <property name='user'> <value>${login.username}</value> </property> <!--Password required to access for the above user--> <property name='password'> <value>${login.password}</value> </property> <!-- configuration pool via c3p0--> <property name='acquireIncrement'> <value>${login.c3p0.acquireIncrement}</value> </property> <property name='idleConnectionTestPeriod'> <value>${login.c3p0.idleConnectionTestPeriod}</value> <!-- seconds --> </property> <property name='maxPoolSize'> <value>${login.c3p0.maxPoolSize}</value> </property> <property name='maxStatements'> <value>${login.c3p0.maxStatements}</value> </property> <property name='minPoolSize'> <value>${login.c3p0.minPoolSize}</value> </property> <property name='initialPoolSize'> <value>${login.c3p0.initialPoolSize}</value> </property> <property name='maxIdleTime'> <value>${login.c3p0.maxIdleTime}</value> </property> <property name='acquireRetryAttempts'> <value>${login.c3p0.acquireRetryAttempts}</value> </property> <property name='acquireRetryDelay'> <value>${login.c3p0.acquireRetryDelay}</value> </property> <property name='breakAfterAcquireFailure'> <value>${login.c3p0.breakAfterAcquireFailure}</value> </property> </bean> </beans> The login.jsp page looks like this. (Need to place is under webapp directory. But not in under WEB_INF directory) <%@ taglib prefix='c' uri='http://java.sun.com/jsp/jstl/core' %> <html> <head> <title>Login</title> </head> <body> <c:if test='${not empty param.error}'> <font color='red'> Login error. <br /> Reason : ${sessionScope['SPRING_SECURITY_LAST_EXCEPTION'].message} </font> </c:if> <form method='POST' action='<c:url value='/j_spring_security_check' />'> <table> <tr> <td align='right'>Username</td> <td><input type='text' name='j_username' /></td> </tr> <tr> <td align='right'>Password</td> <td><input type='password' name='j_password' /></td> </tr> <tr> <td colspan='2' align='right'> <input type='submit' value='Login' /> </td> </tr> </table> </form> </body> </html> home.jsp page <%@ taglib prefix='c' uri='http://java.sun.com/jsp/jstl/core' %> <%@ taglib prefix='sec' uri='http://www.springframework.org/security/tags' %> <html> <head> <title>Home</title> </head> <body> <a href=<c:url value='/j_spring_security_logout'/>>Logout</a><br/> <sec:authorize ifAnyGranted='ROLE_ADMIN'> <h1>Only admin can see this</h1><br/> <a href='admin'> Admin Home </a> </sec:authorize> <h1>Welcome</h1> </body> </html> admin-home.jsp page <%@ taglib prefix='c' uri='http://java.sun.com/jsp/jstl/core' %> <%@ page contentType='text/html;charset=UTF-8' language='java' %> <html> <head> <title>Admin</title> </head> <body> <a href=<c:url value='/j_spring_security_logout'/>>Logout</a><br/> <h1>Only Admin allowed here</h1> </body> </html> After that you need to write two controller to retrieve home page and admin-home page. Here is the HomeController.java package rd.controller; import org.springframework.stereotype.Controller; import org.springframework.ui.Model; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMethod; @Controller public class HomeController { @RequestMapping(value = '/home' , method = RequestMethod.GET) public String setUp(Model model){ return 'home'; } } Here is the AdminController.java package rd.controller; import org.springframework.stereotype.Controller; import org.springframework.ui.Model; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMethod; @Controller public class AdminController { @RequestMapping(value = '/admin' , method = RequestMethod.GET) public String setUp(Model model){ return 'admin-home'; } } That’s it. Run mvn clean install command to create war file. Copy the war file under tomcat/webapps directory and access the web app in your favorite browser. url : localhost:<port>/spring-login/login.jsp Test case 1 : Try to log with user as username 123 as password. You will get users home page. Test case 2 : Try to log with admin as username admin as password. You will get users home page with visible admin page link. In spring security part 2 I will modify this project and add remember me feature and md5 password encryption feature. In near future Ill try to post some interesting article about spring security with CAS integration and LDAP integration. Stay tuned :) Reference: Spring Security Part 1 – Simple Login application with database from our JCG partner Rajith Delantha at the Looping around with Rajith… blog....
logback-logo

Logback: Application errors notification

Some months ago when I was doing big application refactoring I found really annoying pieces of log4j based code used for logging repeated hundreds of times:                   if (LOG.isDebugEnabled()) { LOG.debug("Logging some stuff = " + stuff); }I wanted to get rid of isXXXEnabled an thats how I found Logback and SLF4J.   Logback and SLF4J Logback in conjunction with SLF4J provides great API and fast and powerful implementation of logging framework. Reasons why to switch to Logback from log4j are not topic of this post and they are already described in details on logback website. Just in short notice what do you get with Logback and SL4FJ:simple and fast way to skip isXXXEnabled with: LOG.debug("Logging some stuff = {}", stuff);automatic reloading of configuration files powerful logging filters loading properties file into configuration xml conditional configurationThanks to SLF4J migration tool migration from log4j api to SLF4J is super fast. I’ve decided to switch all projects to Logback and after several months I have to say that I am really satisfied with that decision. In order to use Logback in your project add following dependencies: <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-api</artifactId> <version>1.6.4</version> </dependency><dependency> <groupId>ch.qos.logback</groupId> <artifactId>logback-classic</artifactId> <version>1.0.1</version> </dependency><dependency> <groupId>org.slf4j</groupId> <artifactId>jcl-over-slf4j</artifactId> <version>1.6.4</version> </dependency>Setting up error notifications with Logback In application with proper logging every entry logged with level ERROR should be considered as a problem that should be fixed. Most common way of logging errors is of course logging exceptions with stacktraces to logfile but if you have plenty of applications on several hosts checking for errors can be time consuming. Smarter way is to log them into database – that can be achieved with DBAppender. I found the most helpful to send all exceptions on my email to be able to fix them immediately. In order to setup Logback to send mails with exceptions we need to add Java Mail dependency: <dependency> <groupId>javax.mail</groupId> <artifactId>mail</artifactId> <version>1.4</version> </dependency>And define configuration for SMTPAppender in logback.xml: <?xml version="1.0" encoding="UTF-8"?> <configuration> <!-- some application specific configuration here --><appender name="sheriff" class="ch.qos.logback.classic.net.SMTPAppender"> <smtpHost>localhost</SMTPHost> <from>sheriff@mycompany.com</From> <to>john@mycompany.com</To> <subject>Something went wrong</Subject> <layout class="ch.qos.logback.classic.html.HTMLLayout"/><filter class="ch.qos.logback.classic.filter.ThresholdFilter"> <level>ERROR</level> </filter> </appender><root level="ERROR"> <appender-ref ref="sheriff" /> </root> </configuration>Thanks to ThresholdFilter with level set to ERROR we are sure that even if root level will be changed – only error mails will be sent. Now you should get emails that look similar to:Skipping error notifications in dev environment Probably you will not need to receive those mails coming from development environment and thanks to conditionals in logback configuration you can easily skip them. In order to use conditionals you need to add dependency to Janino: <dependency> <groupId>janino</groupId> <artifactId>janino</artifactId> <version>2.5.10</version> </dependency>Next thing is to know whats your environment. In Logback configuration you can access system properties, logback properties or special variables like HOSTNAME and CONTEXT_NAME and them to determine your environment. In my application I use Spring profiles for that so my ROOT logger configuration is: <root level="ERROR"> <if condition='"${spring.profiles.active}" == "production"'> <appender-ref ref="sheriff" /> </if> </root>Few words for log4j users If you decide to stick to log4j you can use it’s SMTPAppender. Log4j can be used together with SLF4J thanks to log4j-over-slf4j library.   Conclusion Sending mails with Logback is not the only way to notify about errors. Its possible to use Jabber appender or even write appender that would send errors through SMS gateway. Its up to you. What solutions for error notifications do you use in your projects? Reference: Application errors notification with Logback from our JCG partner Maciej Walkowiak at the Software Development Journey blog....
junit-logo

Tomcat Context JUnit @Rule

A first draft of a JUnit @Rule that create the test context. This can be used with the Spring context rule for this post to create a complete Spring context for integration tests. import org.apache.commons.dbcp.BasicDataSource; import org.apache.log4j.Logger; import org.junit.rules.TestRule; import org.junit.runner.Description; import org.junit.runners.model.Statement; import org.w3c.dom.Document; import org.w3c.dom.Element; import org.w3c.dom.NodeList; import javax.naming.Context; import javax.naming.InitialContext; import javax.naming.NamingException; import javax.sql.DataSource; import javax.xml.parsers.DocumentBuilder; import javax.xml.parsers.DocumentBuilderFactory; import java.io.*; import java.lang.reflect.Method; import java.sql.Driver; import java.sql.DriverManager; /** * Creates an context for tests using an Apache Tomcat server.xml. * * https://blogs.oracle.com/randystuph/entry/injecting_jndi_datasources_for_junit * * @author alex.collins */ public class TomcatContextRule implements TestRule { public static final Logger LOGGER = Logger.getLogger(CatalinaContextRule.class); /** * Creates all the sub-contexts for a name. */ public static void createSubContexts(Context ctx, String name) { String subContext = ''; for (String x : name.substring(0, name.lastIndexOf('/')).split('/')) { subContext += x; try { ctx.createSubcontext(subContext); } catch (NamingException e) { // nop } subContext += '/'; } } private final File serverXml; public TomcatContextRule(File serverXml, Object target) { if (serverXml == null || !serverXml.isFile()) {throw new IllegalArgumentException();} if (target == null) {throw new IllegalArgumentException();} this.serverXml = serverXml; } public Statement apply(final Statement statement, Description description) { return new Statement() { @Override public void evaluate() throws Throwable { createInitialContext(); try { statement.evaluate(); } finally { destroyInitialContext(); } } }; } private void createInitialContext() throws Exception { LOGGER.info('creating context'); System.setProperty(Context.INITIAL_CONTEXT_FACTORY, org.apache.naming.java.javaURLContextFactory.class.getName()); System.setProperty(Context.URL_PKG_PREFIXES, 'org.apache.naming'); final InitialContext ic = new InitialContext(); createSubContexts(ic, 'java:/comp/env'); final DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance(); final DocumentBuilder builder = factory.newDocumentBuilder(); final Document document = builder.parse(serverXml); // create Environment { final NodeList envs = document.getElementsByTagName('Environment'); for (int i = 0; i < envs.getLength(); i++) { final Element env = (Element)envs.item(i); // must be Element final String name = 'java:comp/env/' + env.getAttribute('name'); final Object instance = Class.forName(env.getAttribute('type')).getConstructor(String.class) .newInstance(env.getAttribute('value')); LOGGER.info('binding ' + name + ' <' + instance + '>'); createSubContexts(ic, name); ic.bind(name, instance); } } // Resource { final NodeList resources = document.getElementsByTagName('Resource'); for (int i = 0; i < resources.getLength(); i++) { final Element resource = (Element)resources.item(i); // must be Element final String name = 'java:comp/env/' + resource.getAttribute('name'); final Class<?> type = Class.forName(resource.getAttribute('type')); final Object instance; if (type.equals(DataSource.class)) { { @SuppressWarnings('unchecked') // this mus be driver? final Class<? extends Driver> driverClass = (Class<? extends Driver>) Class.forName(resource.getAttribute('driverClassName')); DriverManager.registerDriver(driverClass.newInstance()); } final BasicDataSource dataSource = new BasicDataSource(); // find all the bean attributes and set them use some reflection for (Method method : dataSource.getClass().getMethods()) { if (!method.getName().matches('^set.*')) {continue;} final String x = method.getName().substring(3, 4).toLowerCase() + method.getName().substring(4); if (!resource.hasAttribute(x)) {continue;} Class<?> y = method.getParameterTypes()[0]; // might be primitive if (y.isPrimitive()) { if (y.getName().equals('boolean')) y = Boolean.class; if (y.getName().equals('byte')) y = Byte.class; if (y.getName().equals('char')) y = Character.class; if (y.getName().equals('double')) y = Double.class; if (y.getName().equals('float')) y = Float.class; if (y.getName().equals('int')) y = Integer.class; if (y.getName().equals('long')) y = Long.class; if (y.getName().equals('short')) y = Short.class; if (y.getName().equals('void')) y = Void.class; } method.invoke(dataSource, y.getConstructor(String.class).newInstance(resource.getAttribute(x))); } instance = dataSource; } else { // not supported, yet... throw new AssertionError('type ' + type + ' not supported'); } LOGGER.info('binding ' + name + ' <' + instance + '>'); createSubContexts(ic, name); ic.bind(name, instance); } } } private void destroyInitialContext() { System.clearProperty(Context.INITIAL_CONTEXT_FACTORY); System.clearProperty(Context.URL_PKG_PREFIXES); LOGGER.info('context destroyed'); } } For example: @Rule public TestRule rules = RuleChain.outerRule(new CatalinaContextRule(new File(getClass().getResource('/server.xml').getFile()), this)) .around(new ContextRule(new String[] {'/applicationContext.xml'}, this)); This code is on Github. Reference: Tomcat Context JUnit @Rule from our JCG partner Alex Collins at the Alex Collins ‘s blog blog....
oasis-saml-logo

Implementing SAML to XACML

Before Implementing SAMLThis is how a XACML request will looks like when it is arriving at PDP(Policy Decision Point) to be evaluated. <Request xmlns='urn:oasis:names:tc:xacml:2.0:context:schema:os'> <Subject> <Attribute AttributeId='urn:oasis:names:tc:xacml:1.0:subject:subject-id' DataType='http://www.w3.org/2001/XMLSchema#string'> <AttributeValue>admin</AttributeValue> </Attribute> </Subject> <Resource> <Attribute AttributeId='urn:oasis:names:tc:xacml:1.0:resource:resource-id' DataType='http://www.w3.org/2001/XMLSchema#string'> <AttributeValue>http://localhost:8280/services/echo/echoString</AttributeValue> </Attribute> </Resource> <Action> <Attribute AttributeId='urn:oasis:names:tc:xacml:1.0:action:action-id' DataType='http://www.w3.org/2001/XMLSchema#string'><AttributeValue>read</AttributeValue> </Attribute> </Action> <Environment/> </Request> Basically it states who is(Subject) wanting to access which resource and what action it wants to perform on the resource. PDP trusts that request made is not altered while being sent and received, evaluates the request against existing enabled policies and reply with the decision which will be as follows. <Response> <Result ResourceId='http://localhost:8280/services/echo/echoString'> <Decision>Permit</Decision> <Status> <StatusCode Value='urn:oasis:names:tc:xacml:1.0:status:ok'/> </Status> </Result> </Response> Again there is no guarantee for the party who is using this response that this decision is not altered since sent from PDP until been received. In order achieve the security of XACML requests and responses in server to server communication SAML profile for XACML is defined by OASIS.This take the system security to a higher level by allowing the usage of fine-grained authorization provided by XACML, to be signed.After Implementing SAMLFollowing is how the previous XACML request looks like after wrapped into a XACMLAuthzDecisionQueryType, which is generated using OpenSAML 2.0.0 library which is supporting SAML profile of XACML as declared in 2004 . The diagram shows the basic structure of a XACMLAuthzDecisionQueryType.Following is a sample XACMLAuthzDecisionQuery. <xacml-samlp:XACMLAuthzDecisionQueryType InputContextOnly='true' IssueInstant='2011-10-31T06:44:57.766Z' ReturnContext='false' Version='2.0' xmlns:xacml-samlp='urn:oasis:names:tc:xacml:2.0:profile:saml2.0:v2:schema:protocol'> <saml:Issuer SPProvidedID='SPPProvierId' xmlns:saml='urn:oasis:names:tc:SAML:2.0:assertion'> https://identity.carbon.wso2.org</saml:Issuer> <ds:Signature xmlns:ds='http://www.w3.org/2000/09/xmldsig#'> <ds:SignedInfo> <ds:CanonicalizationMethod Algorithm='http://www.w3.org/2001/10/xml-exc-c14n#'/> <ds:SignatureMethod Algorithm='http://www.w3.org/2000/09/xmldsig#rsa-sha1'/> <ds:Reference URI=''> <ds:Transforms> <ds:Transform Algorithm='http://www.w3.org/2000/09/xmldsig#enveloped-signature'/> <ds:Transform Algorithm='http://www.w3.org/2001/10/xml-exc-c14n#'> <ec:InclusiveNamespaces PrefixList='ds saml xacml-context xacml-samlp' xmlns:ec='http://www.w3.org/2001/10/xml-exc-c14n#'/> </ds:Transform> </ds:Transforms> <ds:DigestMethod Algorithm='http://www.w3.org/2000/09/xmldsig#sha1'/> <ds:DigestValue>7T1ScatC2Xg7pSpjB2X9HB3EH8M=</ds:DigestValue> </ds:Reference> </ds:SignedInfo> <ds:SignatureValue>XQBUVH3j16HVm3aTFSFh5EYFyiYjn0IU4PJfXelzK6BfXp GGTBGouVJEe2Kk26sa3Yj0nEgh51pKsNWxk8xQFWdXg6/UlMkq+CaKrYj7laYlM9yGuIlEBT6t yzjIQBa8wskHeITL6tHE+G0aMa5YnTqtb+9IaJKGPIrl/K5Zn2A=</ds:SignatureValue> <ds:KeyInfo> <ds:X509Data> <ds:X509Certificate>MIICNTCCAZ6gAwIBAgIES343gjANBgkqhkiG9w0BA QUFADBVMQswCQYDVQQGEwJVUzELMAkGA1UECAwCQ0ExFjAUBgNVBAcMDU1vdW50YWluIFZ pZXcxDTALBgNVBAoMBFdTTzIxEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xMDAyMTkwNzAyM jZaFw0zNTAyMTMwNzAyMjZaMFUxCzAJBgNVBAYTAlVTMQswCQYDVQQIDAJDQTEWMBQGA1U EBwwNTW91bnRhaW4gVmlldzENMAsGA1UECgwEV1NPMjESMBAGA1UEAwwJbG9jYWxob3N0M IGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCUp/oV1vWc8/TkQSiAvTousMzOM4asB2i ltr2QKozni5aVFu818MpOLZIr8LMnTzWllJvvaA5RAAdpbECb+48FjbBe0hseUdN5Hpwvn H/DW8ZccGvk53I6Orq7hLCv1ZHtuOCokghz/ATrhyPq+QktMfXnRS4HrKGJTzxaCcU7OQI DAQABoxIwEDAOBgNVHQ8BAf8EBAMCBPAwDQYJKoZIhvcNAQEFBQADgYEAW5wPR7cr1LAdq +IrR44iQlRG5ITCZXY9hI0PygLP2rHANh+PYfTmxbuOnykNGyhM6FjFLbW2uZHQTY1jMrP prjOrmyK5sjJRO4d1DeGHT/YnIjs9JogRKv4XHECwLtIVdAbIdWHEtVZJyMSktcyysFcvu hPQK8Qc/E/Wq8uHSCo=</ds:X509Certificate> </ds:X509Data> </ds:KeyInfo> </ds:Signature> <xacml-context:Request xmlns:xacml-context='urn:oasis:names:tc:xacml:2.0:context:schema:os'><xacml-context:Subject SubjectCategory='urn:oasis:names:tc:xacml:1.0:subject-category:access-subject' xmlns:xacml-context='urn:oasis:names:tc:xacml:2.0:context:schema:os'><xacml-context:Attribute AttributeId='urn:oasis:names:tc:xacml:1.0:subject:subject-id' DataType='http://www.w3.org/2001/XMLSchema#string'><xacml-context:AttributeValue>admin</xacml-context:AttributeValue></xacml-context:Attribute></xacml-context:Subject><xacml-context:Resource xmlns:xacml-context='urn:oasis:names:tc:xacml:2.0:context:schema:os'><xacml-context:Attribute AttributeId='urn:oasis:names:tc:xacml:1.0:resource:resource-id' DataType='http://www.w3.org/2001/XMLSchema#string'><xacml-context:AttributeValue>http://localhost:8280/services/echo/echoString</xacml-context:AttributeValue></xacml-context:Attribute></xacml-context:Resource><xacml-context:Action xmlns:xacml-context='urn:oasis:names:tc:xacml:2.0:context:schema:os'><xacml-context:Attribute AttributeId='urn:oasis:names:tc:xacml:1.0:action:action-id' DataType='http://www.w3.org/2001/XMLSchema#string'><xacml-context:AttributeValue>read</xacml-context:AttributeValue></xacml-context:Attribute></xacml-context:Action><xacml-context:Environment xmlns:xacml-context='urn:oasis:names:tc:xacml:2.0:context:schema:os'/> </xacml-context:Request> </xacml-samlp:XACMLAuthzDecisionQueryType> As you can see it carries lot of information related to the content of the request like who issued it , when, signature with the X509Certificate and the XACML request. Data integrity can be preserved in this way. After executing the request and gaining the response from PDP, it is also sent secured with a signature. The diagram shows the structure of a basic SAML Response.Following is a sample SAML response that carries XACML response. <samlp:Response IssueInstant='2011-10-31T06:49:51.013Z' Version='2.0' xmlns:samlp='urn:oasis:names:tc:SAML:2.0:protocol'> <saml:Issuer SPProvidedID='SPPProvierId' xmlns:saml='urn:oasis:names:tc:SAML:2.0:assertion'>https://identity.carbon.wso2.org</saml:Issuer><ds:Signature xmlns:ds='http://www.w3.org/2000/09/xmldsig#'> <ds:SignedInfo> <ds:CanonicalizationMethod Algorithm='http://www.w3.org/2001/10/xml-exc-c14n#'/> <ds:SignatureMethod Algorithm='http://www.w3.org/2000/09/xmldsig#rsa-sha1'/> <ds:Reference URI=''> <ds:Transforms> <ds:Transform Algorithm='http://www.w3.org/2000/09/xmldsig#enveloped-signature'/> <ds:Transform Algorithm='http://www.w3.org/2001/10/xml-exc-c14n#'> <ec:InclusiveNamespaces PrefixList='ds saml samlp xacml-context xacml-saml' xmlns:ec='http://www.w3.org/2001/10/xml-exc-c14n#'/> </ds:Transform> </ds:Transforms> <ds:DigestMethod Algorithm='http://www.w3.org/2000/09/xmldsig#sha1'/> <ds:DigestValue>uct4nBcdqAV4FIO50WMmFjSy9sE=</ds:DigestValue> </ds:Reference> </ds:SignedInfo> <ds:SignatureValue>dLaXFl6+HHqtaQoE8l22bCCM8byxblyBOYUTdUdG/LeYIR+ NUTn6nTRe9MJqWqrXT4qLtQ2Jvb3Cjrw66YZTdVrBXNjD1t6oWAg3YFXtZcO4s1+z5y4BeN6Mq spLLKIUnovCADNbHvhhVDwtMkCOcUs0x35R0zENiU1PYVMLQMM=</ds:SignatureValue> <ds:KeyInfo> <ds:X509Data> <ds:X509Certificate> MIICNTCCAZ6gAwIBAgIES343gjANBgkqhkiG9w0BAQUFADBVMQswCQYDVQQGEwJVUzELMAkGA1 UECAwCQ0ExFjAUBgNVBAcMDU1vdW50YWluIFZpZXcxDTALBgNVBAoMBFdTTzIxEjAQBgNVBAMM CWxvY2FsaG9zdDAeFw0xMDAyMTkwNzAyMjZaFw0zNTAyMTMwNzAyMjZaMFUxCzAJBgNVBAYTAl VTMQswCQYDVQQIDAJDQTEWMBQGA1UEBwwNTW91bnRhaW4gVmlldzENMAsGA1UECgwEV1NPMjES MBAGA1UEAwwJbG9jYWxob3N0MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCUp/oV1vWc8/ TkQSiAvTousMzOM4asB2iltr2QKozni5aVFu818MpOLZIr8LMnTzWllJvvaA5RAAdpbECb+48F jbBe0hseUdN5HpwvnH/DW8ZccGvk53I6Orq7hLCv1ZHtuOCokghz/ATrhyPq+QktMfXnRS4HrK GJTzxaCcU7OQIDAQABoxIwEDAOBgNVHQ8BAf8EBAMCBPAwDQYJKoZIhvcNAQEFBQADgYEAW5wP R7cr1LAdq+IrR44iQlRG5ITCZXY9hI0PygLP2rHANh+PYfTmxbuOnykNGyhM6FjFLbW2uZHQTY 1jMrPprjOrmyK5sjJRO4d1DeGHT/YnIjs9JogRKv4XHECwLtIVdAbIdWHEtVZJyMSktcyysFcv uhPQK8Qc/E/Wq8uHSCo= </ds:X509Certificate> </ds:X509Data> </ds:KeyInfo> </ds:Signature> <saml:Assertion IssueInstant='2011-10-31T06:49:51.008Z' Version='2.0' xmlns:saml='urn:oasis:names:tc:SAML:2.0:assertion'> <saml:Issuer SPProvidedID='SPPProvierId'>https://identity.carbon.wso2.org</saml:Issuer> <saml:Statement xmlns:xacml-saml='urn:oasis:names:tc:xacml:2.0:profile:saml2.0:v2:schema:assertion' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xsi:type='xacml-saml:XACMLAuthzDecisionStatementType'> <xacml-context:Response xmlns:xacml-context='urn:oasis:names:tc:xacml:2.0:context:schema:os'> <xacml-context:Result ResourceId='http://localhost:8280/services/echo/echoString' xmlns:xacml-context='urn:oasis:names:tc:xacml:2.0:context:schema:os'> <xacml-context:Decision>Permit</xacml-context:Decision> <xacml-context:Status><xacml-context:StatusCode Value='urn:oasis:names:tc:xacml:1.0:status:ok'/> </xacml-context:Status> </xacml-context:Result> </xacml-context:Response> </saml:Statement> </saml:Assertion> </samlp:Response> The XACML response is wrapped into a SAML statement which is included in a SAML assertion that is again wrapped by a SAML response.I have only signed the response according to the context and included only one assertion. We can separately sign both the assertion and response according to the spec and include more assertions in one response. Also it is possible to send the relevant XACML request inside the response and lot more options are there according to the spec. With OpenSAML we can get most of them into action. Reference: Implementing SAML to XACML from our JCG partner Pushpalanka at the Pushpalanka’s Blog blog....
Java Code Geeks and all content copyright © 2010-2014, Exelixis Media Ltd | Terms of Use | Privacy Policy
All trademarks and registered trademarks appearing on Java Code Geeks are the property of their respective owners.
Java is a trademark or registered trademark of Oracle Corporation in the United States and other countries.
Java Code Geeks is not connected to Oracle Corporation and is not sponsored by Oracle Corporation.

Sign up for our Newsletter

20,709 insiders are already enjoying weekly updates and complimentary whitepapers! Join them now to gain exclusive access to the latest news in the Java world, as well as insights about Android, Scala, Groovy and other related technologies.

As an extra bonus, by joining you will get our brand new e-books, published by Java Code Geeks and their JCG partners for your reading pleasure! Enter your info and stay on top of things,

  • Fresh trends
  • Cases and examples
  • Research and insights
  • Two complimentary e-books