Enterprise Java

Using MongoDB with Morphia

In the past few years, NoSQL databases like CouchDB, Cassandra and MongoDB have gained some popularity for applications that don’t require the semantics and overhead of running a traditional RDBMS. I won’t get into the design decisions to go into choosing a NoSQL database as others have done a good enough job already, but I will relate my experience with MongoDB and some tricks on using it effectively in Java.

I recently have had a chance to work with MongoDB (as in humongoous), which is a document-oriented database written in C++. It is ideal for storing documents which may vary in structure, and it uses a format similar to JSON, which means it supports similar data types and structures as JSON. It provides a rich yet simple query language and still allows us to index key fields for fast retrieval. Documents are stored in collections which effectively limit the scope of a query, but there is really no limitation on the types of heterogeneous data that you can store in a collection. The MongoDB site has decent docs if you need to learn the basics of MongoDB.

MongoDB in Java

The Mongo Java driver basically exposes all documents as key-value pairs exposed as map, and lists of values. This means that if we have to store or retrieve documents in Java, we will have to do some mapping of our POJOs to that map interface. Below is an example of the type of code we would normally have to write to save a document to MongoDB from Java:

BasicDBObject doc = new BasicDBObject();

doc.put("user", "carfey");

BasicDBObject post1 = new BasicDBObject();
post1.put("subject", "spam & eggs");
post1.put("message", "first!");

BasicDBObject post2 = new BasicDBObject();
post2.put("subject", "sorry about the spam");

doc.put("posts", Arrays.asList(post1, post2));

coll.insert(doc);

This is fine for some use cases, but for others, it would be better to have a library to do the grunt work for us.

Enter Morphia

Morphia is a Java library which acts sort of like an ORM for MongoDB – it allows us to seamlessly map Java objects to the MongoDB datastore. It uses annotations to indicate which collection a class is stored in, and even supports polymorphic collections. One of the nicest features is that it can be used to automatically index your collections based on your collection- or property-level annotations. This greatly simplifies deployment and rolling out changes.

I mentioned polymorphic storage of multiple types in the same collection. This can help us map varying document structures and acts somewhat like a discriminator in something like Hibernate.

Here’s an example of how to define entities which will support polymorphic storage and querying. The Return class is a child of Order and references the same collection-name. Morphia will automatically handle the polymorphism when querying or storing data. You would pretty much do the same thing for annotating collections that aren’t polymorphic, but you wouldn’t have multiple classes using the same collection name.

Note: This isn’t really an example of the type of data I would recommend storing in MongoDB since it is more suited to a traditional RDBMS, but it demonstrates the principles nicely.

@Entity("orders") // store in the orders collection
@Indexes({ @Index("-createdDate, cancelled") }) // multi-column index
public class Order {
    @Id private ObjectId id; // always required

    @Indexed
    private String orderId;

    @Embedded // let's us embed a complex object
    private Person person;
    @Embedded
    private List<Item> items;

    private Date createdDate;
    private boolean cancelled;

    // .. getters and setters aren't strictly required
    // for mapping, but they would be here
}

@Entity("orders") // note the same collection name
public class Return extends Order {
    // maintain legacy name but name it nicely in mongodb
    @Indexed
    @Property("rmaNumber") private String rma;
    private Date approvedDate;
    private Date returnDate;
}

Now, below I will demonstrate how to query those polymorphic instances. Note that we don’t have to do anything special when storing the data. MongoDB stores a className attribute along with the document so it can support polymorphic fetches and queries. Following the example above, I can query for all order types by doing the following:

// ds is a Datastore instance
Query<Order> q = ds.createQuery(Order.class).filter("createdDate >=", date);
List<Order> ordersAndReturns = q.asList();

// and returns only
Query<Return> rq = ds.createQuery(Return.class).filter("createdDate >=", date);
List<Return> returnsOnly = rq.asList();

If I only want to query plain orders, I would have to use a className filter as follows. This allows us to effectively disable the polymorphic behaviour and limit results to a single target type.

Query<Order> q = ds.createQuery(Order.class)
    .filter("createdDate >=", cutOffDate)
    .filter("className", Order.class.getName());

List<Order> ordersOnly = q.asList();

Morphia currently uses the className attribute to filter results, but at some point in the future is likely to use a discriminator column, in which case you may have to filter on that value instead.

Note: At some point during startup of your application, you need to register your mapped classes so they can be used by Morphia. See here for full details. A quick example is below.

Morphia m = ...
Datastore ds = ...

m.map(MyEntity.class);
ds.ensureIndexes(); //creates all defined with @Indexed
ds.ensureCaps(); //creates all collections for @Entity(cap=@CappedAt(...))

Problems with Varying Structure in Documents

One of the nice features of document-oriented storage in MongoDB is that it allows you to store documents with different structure in the same collection, but still perform structured queries and index values to get good performance.

Morphia unfortunately doesn’t really like this as it is meant to map all stored attributes to known POJO fields. There are currently two ways I’ve found that let us deal with this.

The first is disabling validation on queries. This will mean that values which exist in the datastore but can’t be mapped to our POJOs will be dropped rather than blowing up:

// drop unmapped fields quietly
Query<Order> q = ds.createQuery(Order.class).disableValidation();

The other option is to store all unstructured content under a single bucket element using a Map. This could contain any basic types supported by the MongoDB driver including Lists and Maps, but no complex objects unless you have registered converters with Morphia (e.g. morphia.getMapper().getConverters().addConverter(new MyCustomTypeConverter()) .

@Entity("orders")
public class Order {
    // .. our base attributes here
    private Map<String, Object> attributes; // bucket for everything else (
}

Note that Morphia may complain on startup that it can’t validate the field (since the generics declaration is not strict), but as of the current release version (0.99), it will work with no problem and store any attributes normally and retrieve them as maps and lists of values.

Note: When it populates a loosely-typed map from a retrieved document, it will use the basic MongoDB Java driver types BasicDBObject and BasicDBList. These implement Map and List respectively, so they will work pretty much as you expect, except that they will not be equals() to any input maps or lists you may have stored, even if the structure and content appear to be equal. If you want to avoid this, you can use the @PostLoad annotation to annotate a method which can perform normalization to JDK maps and lists after the document is loaded. I personally did this to ensure we always see a consistent view of MongoDB documents whether they are pulled from a collection or not yet persisted.

Reference: Using MongoDB with Morphia from our JCG partners at the Carfey Software blog.

Related Articles :

Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Inline Feedbacks
View all comments
Back to top button