This article is part of our Academy Course titled MongoDB – A Scalable NoSQL DB.
In this course, you will get introduced to MongoDB. You will learn how to install it and how to operate it via its shell. Moreover, you will learn how to programmatically access it via Java and how to leverage Map Reduce with it. Finally, more advanced concepts like sharding and replication will be explained. Check it out here!
1. Introduction
In this final part of the tutorial we are going to take a look on MongoDB security model, different types of indexes, query plans and profiling, server-side cursors and bulk operations.
2. Security
MongoDB security features include authentication, authorization and auditing. Its foundation is a role-based access control with flexible set of privileges. It is worth mentioning that MongoDB provides a basis for user-defined roles by defining a set of build-in roles (for more details please refer to official documentation). As always, MongoDB shell provides rich set of command helpers to deal with security configuration.
Command |
createUser |
Parameters |
{
createUser: <username>,
pwd: <password>,
customData: { <any information> },
roles: [
{ role: <role>, db: <database> } | <role>,
...
],
writeConcern: { <write concern> }
}
|
Wrapper |
db.createUser(user, writeConcern) |
Description |
The command creates a new user with name username on the current database. If the user with such a username already exists, an error will be raised. |
Example |
In MongoDB shell, let us issue the command:
db.createUser(
{
user: "testuser",
pwd: "testpassword",
customData: {
"firstName": "John",
"lastName": "Smith"
},
roles: [
{ role: "readWrite", db: "test" }
]
},
{
w: "majority" ,
wtimeout: 5000
}
)
|
Reference |
http://docs.mongodb.org/manual/reference/command/createUser/
http://docs.mongodb.org/manual/reference/method/db.createUser/ |
createUser
Command |
updateUser |
Parameters |
{
updateUser: <username>,
pwd: <password>,
customData: { <any information> },
roles: [
{ role: <role>, db: <database> } | <role>,
...
],
writeConcern: { <write concern> }
}
|
Wrapper |
db.updateUser(username, update, writeConcern) |
Description |
The command updates user’s data with name username on the current database. Please notice that an update to a property completely replaces the previous property values (including updates to the user’s roles). |
Example |
In MongoDB shell, let us issue the command (to update the user created previously by createUser command):
createUser command):
db.updateUser(
"testuser",
{
customData: {
"firstName": "John",
"lastName": "Smith",
"country": "US"
},
roles: [
"read"
]
},
{
w: "majority" ,
wtimeout: 5000
}
)
|
Reference |
http://docs.mongodb.org/manual/reference/command/updateUser/
http://docs.mongodb.org/manual/reference/method/db.updateUser/ |
updateUser
Command |
createRole |
Parameters |
{
createRole: <rolename>,
privileges: [
{ resource: { <resource> }, actions: [<action>, ... ] },
...
],
roles: [
{ role: <role>, db: <database> } | <role>,
...
],
writeConcern: { <write concern> }
}
|
Wrapper |
db.createRole(role, writeConcern) |
Description |
The command creates a user-defined role with a name rolename and particular set of privileges. The role will be applicable to the current database. |
Example |
In MongoDB shell, let us issue the command:
db.createRole(
{
role: "testrole",
privileges: [
{ resource: {
db: "test",
collection: "test"
}, actions: [ "find" ] }
],
roles: [
{ role: "read", db: "test" }
],
},
{ w: "majority" , wtimeout: 5000 }
)
|
Reference |
http://docs.mongodb.org/manual/reference/command/createRole/
http://docs.mongodb.org/manual/reference/method/db.createRole/ |
createRole
Command |
updateRole |
Parameters |
{
updateRole: <rolename>,
privileges: [
{ resource: { <resource> }, actions: [<action>, ... ] },
...
],
roles: [
{ role: <role>, db: <database> } | <role>,
...
],
writeConcern: { <write concern> }
}
|
Wrapper |
db.updateRole(rolename, update, writeConcern) |
Description |
The command updates user-defined role with name rolename on the current database. Please notice that an update to a property completely replaces the previous property values (including updates to the roles and privileges). |
Example |
In MongoDB shell, let us issue the command (to update the role created previously by createRole command):
db.updateRole(
"testrole",
{
privileges: [
{ resource: {
db: "test",
collection: "test"
}, actions: [ "find" ] }
],
roles: [
{ role: "read", db: "test" },
{ role: "readWrite", db: "test" }
]
},
{ w: "majority" , wtimeout: 5000 }
)
|
Reference |
http://docs.mongodb.org/manual/reference/command/updateRole/
http://docs.mongodb.org/manual/reference/method/db.updateRole/ |
updateRole
Command |
grantPrivilegesToRole |
Parameters |
{
grantPrivilegesToRole: <rolename>,
privileges: [
{ resource: { <resource> }, actions: [<action>, ... ] },
...
],
writeConcern: { <write concern> }
}
|
Wrapper |
db.grantPrivilegesToRole(rolename, privileges, writeConcern) |
Description |
The command assigns additional privileges to a user-defined role with name rolename defined on the current database. |
Example |
In MongoDB shell, let us issue the command (to grant more privileges to the role created previously by createRole command):
db.grantPrivilegesToRole(
"testrole",
[
{ resource: {
db: "test",
collection: "test"
}, actions: [ "insert" ] }
],
{ w: "majority" , wtimeout: 5000 }
)
|
Reference |
http://docs.mongodb.org/manual/reference/command/grantPrivilegesToRole/
http://docs.mongodb.org/manual/reference/method/db.grantPrivilegesToRole/ |
grantPrivilegesToRole
Command |
revokePrivilegesFromRole |
Parameters |
{
revokePrivilegesFromRole: <rolename>,
privileges: [
{ resource: { <resource> }, actions: [<action>, ... ] },
...
],
writeConcern: { <write concern> }
}
|
Wrapper |
db.revokePrivilegesFromRole(rolename, privileges, writeConcern) |
Description |
The command removes the specified privileges from a user-defined role with name rolename defined on the current database. |
Example |
In MongoDB shell, let us issue the command (to revoke privileges from the role created previously by createRole command):
db.revokePrivilegesFromRole(
"testrole",
[
{ resource: {
db: "test",
collection: "test"
}, actions: [ "find" ] }
],
{ w: "majority" , wtimeout: 5000 }
)
|
Reference |
http://docs.mongodb.org/manual/reference/command/revokePrivilegesFromRole/
http://docs.mongodb.org/manual/reference/method/db.revokePrivilegesFromRole/ |
revokePrivilegesFromRole
Command |
grantRolesToRole |
Parameters |
{
grantRolesToRole: <rolename>,
roles: [
{ role: <role>, db: <database> } | <role>,
...
],
writeConcern: { <write concern> }
}
|
Wrapper |
db.grantRolesToRole(rolename, roles, writeConcern) |
Description |
The command assigns additional roles to a user-defined role with name rolename defined on the current database. |
Example |
In MongoDB shell, let us issue the command (to assign more roles to the role created previously by createRole command):
db.grantRolesToRole(
"testrole",
[
"readWrite"
],
{ w: "majority" , wtimeout: 5000 }
)
|
Reference |
http://docs.mongodb.org/manual/reference/command/grantRolesToRole/
http://docs.mongodb.org/manual/reference/method/db.grantRolesToRole/ |
grantRolesToRole
Command |
revokeRolesFromRole |
Parameters |
{
revokeRolesFromRole: <rolename>,
roles: [
{ role: <role>, db: <database> } | <role>,
...
],
writeConcern: { <write concern> }
}
|
Wrapper |
db.revokeRolesFromRole(rolename, roles, writeConcern) |
Description |
The command removes specified roles from a user-defined role with name rolename defined on the current database. |
Example |
In MongoDB shell, let us issue the command (to remove some roles from the role created previously by createRole command):
db.revokeRolesFromRole(
"testrole",
[
{ role: "read", db: "test" }
],
{ w: "majority" , wtimeout: 5000 }
)
|
Reference |
http://docs.mongodb.org/manual/reference/command/revokeRolesFromRole/
http://docs.mongodb.org/manual/reference/method/db.revokeRolesFromRole/ |
revokeRolesFromRole
Command |
invalidateUserCache |
Description |
The command immediately flushes user information from in-memory cache, including removal of each user’s credentials and roles. |
Example |
In MongoDB shell, let us issue the command: db.runCommand( { invalidateUserCache: 1 } )
|
Reference |
http://docs.mongodb.org/manual/reference/command/invalidateUserCache/ |
invalidateUserCache
2.1. Additional Resources
3. Indexing
Choosing the right indexes may boost your query performance (and consequently applications performance) in most times. MongoDB supports different types of indexes to pick from:
- _id: all collections have an index on the _id field that exists by default
- single field: indexes on a single field of a document, f.e. { “title”: 1 }
- compound index: indexes on multiple fields, f.e. { “title”: 1, “price”: 1 }
- multikey index: indexes on content stored in arrays, f.e. { “categories”: 1 }
- geospatial index: 2d/2sphere indexes to support efficient geospatial queries, f.e. { “location”: “2d” }
- text indexes: indexes on string content to support full-text search, f.e. { “title”: “text” }
- hashed indexes: indexes to support hash-based sharding (please refer to Part 4. MongoDB Sharding Guide of the tutorial for more details)
Additionally, each index may be defined as:
- unique: duplicate values for the indexed field will be rejected
- sparse: index only contain entries for documents that have the indexed field
For indexed collections, the values for the indexed fields have a maximum index key length limit: the total size of an index entry must be less than 1024 bytes (please refer to limits and thresholds section of the official documentation).
For more details about indexing, please refer to official documentation.
4. Profiling
MongoDB provides a very useful tool to collect server performance data: the database profiler. It collects fine grained data about queries, write operations, cursors, and other database commands on a running server instance. The profiling can be enabled on a per-database or per-server instance level.
To get more insights about analyzing the performance of database operations please refer to official documentation.
5. Query Cache
Among many other new features, MongoDB 2.6 supports a new set of commands to view and manipulate the query cache:
- list all known query shapes
- display cached plans for a query shape
- remove a query shape from the cache
- clear the whole cache
The query optimizer executes queries and picks the most efficient query plan for a query given the defined indexes. Later on, this query plan is used each time the query (with such a shape) runs. The query optimizer only caches the plans for those query shapes that can have more than one viable plan and occasionally reevaluates query plans as the content of the collection changes.
To experiment with query plans, we need a small dataset and the example from Part 3. MongoDB and Java Tutorial is going to be handy again. Let us switch to books collection in bookstore database, create couple of indexes and insert a few documents into it using MongoDB shell.
use bookstore
db.books.ensureIndex( { "publisher.name": 1 } )
db.books.ensureIndex( { "publisher.name": 1, "price": 1 } )
db.books.insert( {
"title" : "MongoDB: The Definitive Guide",
"publisher" : { "name" : "O'Reilly" },
"price" : 32.99
} );
db.books.insert( {
"title" : "MongoDB Applied Design Patterns",
"publisher" : { "name" : "O'Reilly" },
"price" : 32.99
} );
db.books.insert( {
"title" : "MongoDB in Action, 2nd Edition",
"publisher" : { "name" : "Manning" },
"price" : 26.66
} );
Once the preparation is done, let us run simple query which will trigger the query plans evaluation: db.books.find( { "publisher.name": "O'Reilly" }, { "title": 1 } )
db.<collection>.getPlanCache
Command |
planCacheListFilters |
Parameters |
{
planCacheListFilters: <collection>
}
|
Description |
The command lists the index filters associated with query shapes for a collection collection. |
Example |
In MongoDB shell, let us issue the command (to list the filters set by planCacheSetFilter before): db.runCommand( { planCacheListFilters: "books" } )
|
Reference |
http://docs.mongodb.org/manual/reference/command/planCacheListFilters/ |
planCacheListFilters
Command |
planCacheSetFilter |
Parameters |
{
planCacheSetFilter: <collection>,
query: <query>,
sort: <sort>,
projection: <projection>,
indexes: [ <index1>, <index2>, ...]
}
|
Description |
The command set an index filter for a collection collection. If an index filter already exists for the query shape, it will be overridden. |
Example |
In MongoDB shell, let us issue the command:
db.runCommand({
planCacheSetFilter: "books" ,
query: { "publisher.name": "O'Reilly" },
sort: {},
projection: { "title": 1 },
indexes: [ { "publisher.name": 1 } ]
})
|
Reference |
http://docs.mongodb.org/manual/reference/command/planCacheSetFilter/ |
planCacheSetFilter
Command |
planCacheClearFilters |
Parameters |
{
planCacheClearFilters: <collection>,
query: <query>,
sort: <sort>,
projection: <projection>
}
|
Description |
The command clears index filter(s) for a collection collection. |
Example |
In MongoDB shell, let us issue the command:
db.runCommand({
planCacheClearFilters: "books" ,
query: { "publisher.name": "O'Reilly" },
sort: {},
projection: { "title": 1 }
})
|
Reference |
http://docs.mongodb.org/manual/reference/command/planCacheClearFilters/ |
planCacheClearFilters
With log level set to 1 or greater MongoDB will log plan cache changes. The log level could be set using the following command (please notice that it should be run in context of admin database): db.adminCommand( { setParameter: 1, logLevel: 1 } )
6. Cursors
Cursors are the basic way to access documents returned by read operations, f.e. db.<collection>.find()
. In MongoDB shell, if the returned cursor is not assigned to a variable, then only first 20 documents are taken from the cursor and shown as the result. However, cursors are very powerful and provide a lot of useful methods.
cursor.maxTimeMS(<milliseconds>)
cursor.readPref(mode, tagSet)
Method |
cursor.snapshot() |
Description |
The method ensures that the query will not return a document multiple times, even if intervening write operations result in a move of the document due to the growth in document size. It should be called before retrieving any documents from the database and works with unsharded collections only. |
Reference |
http://docs.mongodb.org/manual/reference/method/cursor.snapshot/ |
cursor.snapshot()
Command |
parallelCollectionScan |
Parameters |
{
parallelCollectionScan: <collection>,
numCursors: <integer>
}
|
Description |
The command allows applications to use multiple parallel cursors when reading all the documents from a collection. It returns a document that contains an array of cursor information. |
Example |
In MongoDB shell, let us issue the command: db.runCommand( { parallelCollectionScan: “books”, numCursors: 1 } )
|
Reference |
http://docs.mongodb.org/manual/reference/command/parallelCollectionScan/ |
parallelCollectionScan
Armored with deeper knowledge about cursors, let us take a look on the example of different cursor methods in action using the books collection from section Query Cache.
[
{
"title" : "MongoDB in Action, 2nd Edition",
"publisher" : {
"name" : "Manning"
},
"price" : 26.66,
"$diskLoc" : {
"file" : 0,
"offset" : 17072
}
},
{
"title" : "MongoDB: The Definitive Guide",
"publisher" : {
"name" : "O'Reilly"
},
"price" : 32.99,
"$diskLoc" : {
"file" : 0,
"offset" : 16560
}
}
]
7. Bulk Operations
One of the coolest features of latest MongoDB 2.6 release is the introduction of bulk API. In a nutshell, this new API supports ordered and unordered bulk operations. In an ordered bulk operation, the execution of every operation follows the order it was added to the bulk operation. Consequently, in an unordered bulk operation the order of every operation is not guaranteed.
initializeUnorderedBulkOp
Those two commands are the starting point to begin using the bulk API. They return bulk builder object (of type Bulk) which provides the fluent API to construct bulk operation.
Method |
Bulk.find(<query>) |
Description |
The method specifies a query condition for an update or a remove operation. It could be used with following bulk operations:
– Bulk.find(<query>).removeOne()
– Bulk.find(<query>).remove()
– Bulk.find(<query>).replaceOne()
– Bulk.find(<query>).updateOne()
– Bulk.find(<query>).update() |
Reference |
http://docs.mongodb.org/manual/reference/method/Bulk.find/ |
Bulk.find(<query>)
Bulk.find(<query>).removeOne
Bulk.find(<query>).remove
Bulk.find(<query>).replaceOne
Bulk.find(<query>).updateOne
Bulk.find(<query>).update
Bulk.execute(writeConcern)
To finish with the bulk API, let us create and execute an example bulk operation using MongoDB shell and initialization commands. In the example we are going to bulk the following actions:
- insert 3 books into books collection
- update all books by setting the categories field
- update book’s (with title “MongoDB: The Definitive Guide”) categories with additional category
- remove all books published by Manning
var bulk = db.books.initializeOrderedBulkOp();
bulk.insert( {
"title" : "MongoDB: The Definitive Guide",
"publisher" : { "name" : "O'Reilly" },
"price" : 32.99
} );
bulk.insert( {
"title" : "MongoDB Applied Design Patterns",
"publisher" : { "name" : "O'Reilly" },
"price" : 32.99
} );
bulk.insert( {
"title" : "MongoDB in Action, 2nd Edition",
"publisher" : { "name" : "Manning" },
"price" : 26.66
} );
bulk.find( { "publisher.name": "O'Reilly" } )
.update( { $set: { "categories" : [ "Databases", "NoSQL" ] } } );
bulk.find( { "title": "MongoDB: The Definitive Guide" } )
.updateOne( { $addToSet: { "categories" : "Programming" } } );
bulk.find( { "publisher.name": "Manning" } )
.remove();
bulk.execute( { w: "majority" , wtimeout: 5000 } );
As you can see, each action within ordered bulk operation may rely on previous actions to succeed (for example, find/update depends on insert). This is not the case for unordered bulk operation. The execution of this bulk operation in MongoDB shell yields the following result document:
BulkWriteResult({
"writeErrors" : [ ],
"writeConcernErrors" : [ ],
"nInserted" : 3,
"nUpserted" : 0,
"nMatched" : 3,
"nModified" : 3,
"nRemoved" : 1,
"upserted" : [ ]
})
7.1. Additional Resources
8. What’s next
This section concludes the MongoDB tutorial. Hopefully, you have found this NoSQL document database fitting your current or future application demands and this tutorial has helped you to make the right decisions.