Software Development

MongoDB Security, Profiling, Indexing, Cursors and Bulk Operations Guide

This article is part of our Academy Course titled MongoDB – A Scalable NoSQL DB.

In this course, you will get introduced to MongoDB. You will learn how to install it and how to operate it via its shell. Moreover, you will learn how to programmatically access it via Java and how to leverage Map Reduce with it. Finally, more advanced concepts like sharding and replication will be explained. Check it out here!

1. Introduction

In this final part of the tutorial we are going to take a look on MongoDB security model, different types of indexes, query plans and profiling, server-side cursors and bulk operations.

2. Security

MongoDB security features include authentication, authorization and auditing. Its foundation is a role-based access control with flexible set of privileges. It is worth mentioning that MongoDB provides a basis for user-defined roles by defining a set of build-in roles (for more details please refer to official documentation). As always, MongoDB shell provides rich set of command helpers to deal with security configuration.

CommandcreateUser
Parameters
{
    createUser: <username>,
    pwd: <password>,
    customData: { <any information> },
    roles: [
        { role: <role>, db: <database> } | <role>,
        ...
    ],
    writeConcern: { <write concern> }
}
Wrapperdb.createUser(user, writeConcern)
DescriptionThe command creates a new user with name username on the current database. If the user with such a username already exists, an error will be raised.
ExampleIn MongoDB shell, let us issue the command:

db.createUser(  
    {
        user: "testuser",
        pwd: "testpassword",
        customData: { 
            "firstName": "John",
            "lastName": "Smith"
        },
        roles: [
            { role: "readWrite", db: "test" }
        ]
    },	
    { 
        w: "majority" , 
        wtimeout: 5000 
    }
)

07.DB.CREATEUSER

Referencehttp://docs.mongodb.org/manual/reference/command/createUser/

http://docs.mongodb.org/manual/reference/method/db.createUser/

createUser

 

CommandupdateUser
Parameters
{
    updateUser: <username>,
    pwd: <password>,
    customData: { <any information> },
    roles: [
        { role: <role>, db: <database> } | <role>,
        ...
    ],
    writeConcern: { <write concern> }
}
Wrapperdb.updateUser(username, update, writeConcern)
DescriptionThe command updates user’s data with name username on the current database. Please notice that an update to a property completely replaces the previous property values (including updates to the user’s roles).
ExampleIn MongoDB shell, let us issue the command (to update the user created previously by createUser command):

createUser command):

db.updateUser(
    "testuser",  
    {
        customData: { 
            "firstName": "John",
            "lastName": "Smith",
            "country": "US"
        },
        roles: [
            "read"
        ]
    },	
    { 
        w: "majority" , 
        wtimeout: 5000 
    }
)

07.DB.UPDATEUSER

Referencehttp://docs.mongodb.org/manual/reference/command/updateUser/

http://docs.mongodb.org/manual/reference/method/db.updateUser/

updateUser

 

CommanddropUser
Parameters
{
    dropUser: <username>,
    writeConcern: { <write concern> }
}
Wrapperdb.dropUser(username, writeConcern)
DescriptionThe command removes the user with name username from the current database.
ExampleIn MongoDB shell, let us issue the command (to drop the user created previously by createUser command):

db.dropUser(
    "testuser",  
    { 
        w: "majority" , 
        wtimeout: 5000 
    }
)

07.DB.DROPUSER

Referencehttp://docs.mongodb.org/manual/reference/command/dropUser/

http://docs.mongodb.org/manual/reference/method/db.dropUser/

dropUser

 

Commanddb.changeUserPassword(username, password)
DescriptionThe command updates the password for the existing user with name username.
ExampleIn MongoDB shell, let us issue the command (to drop the user created previously by createUser command): db.changeUserPassword("testuser", "newpassword")

07.DB.CHANGEUSERPASSWORD

Referencehttp://docs.mongodb.org/manual/reference/method/db.changeUserPassword/

db.changeUserPassword

 

CommanddropAllUsersFromDatabase
Parameters
{ 
    dropAllUsersFromDatabase: 1,
    writeConcern: { <write concern> }
}
Wrapperdb.dropAllUsers(writeConcern)
DescriptionThe command removes all users from the current database. It returns a number of users removed.
ExampleIn MongoDB shell, let us issue the command: db.dropAllUsers()

07.DB.DROPALLUSERS

Referencehttp://docs.mongodb.org/manual/reference/command/dropAllUsersFromDatabase/

http://docs.mongodb.org/manual/reference/method/db.dropAllUsers/

dropAllUsersFromDatabase

 

CommandgrantRolesToUser
Parameters
{ 
    grantRolesToUser: <username>,
    roles: [
        { role: <role>, db: <database> } | <role>,
        ...
    ],
    writeConcern: { <write concern> }
}
Wrapperdb.grantRolesToUser(username, roles, writeConcern)
DescriptionThe command grants additional roles to a user with name username.
ExampleIn MongoDB shell, let us issue the command (to grant roles to the user created previously by createUser command):

db.grantRolesToUser(
    "testuser",
    [ "read" ],
    { w: "majority" }
)

07.DB.GRANTROLESTOUSER

Referencehttp://docs.mongodb.org/manual/reference/command/grantRolesToUser/

http://docs.mongodb.org/manual/reference/method/db.grantRolesToUser/

grantRolesToUser

 

CommandrevokeRolesFromUser
Parameters
{ 
    revokeRolesFromUser: <username>,
    roles: [
        { role: <role>, db: <database> } | <role>,
        ...
    ],
    writeConcern: { <write concern> }
}
Wrapperdb.revokeRolesFromUser(username, roles, writeConcern)
DescriptionThe command removes a one or more roles from a user with name username on the current database.
ExampleIn MongoDB shell, let us issue the command (to revoke roles to the user created previously by createUser command):

db.revokeRolesFromUser(
    "testuser",
    [ "readWrite" ],
    { w: "majority" }
)

07.DB.REVOKEROLESFROMUSER

Referencehttp://docs.mongodb.org/manual/reference/command/revokeRolesFromUser/

http://docs.mongodb.org/manual/reference/method/db.revokeRolesFromUser/

revokeRolesFromUser

 

CommandusersInfo
Parameters
{ 
    usersInfo: { user: <username>, db: <db> } | <name> | 1,
    showCredentials: <true|false>,
    showPrivileges: <true|false>
}
Wrapperdb.getUser(username)

db.getUsers()

DescriptionThe command returns information about one or more users on the database.
ExampleIn MongoDB shell, let us issue the command (to get the information about the user created previously by createUser command): db.getUser("testuser") (or alternatively, db.getUsers())

07.DB.GETUSER

Referencehttp://docs.mongodb.org/manual/reference/command/usersInfo/

http://docs.mongodb.org/manual/reference/method/db.getUser/

http://docs.mongodb.org/manual/reference/method/db.getUsers/

usersInfo

 

CommandcreateRole
Parameters
{ 
    createRole: <rolename>,
    privileges: [
        { resource: { <resource> }, actions: [<action>, ... ] },
        ...
    ],
    roles: [
        { role: <role>, db: <database> } | <role>,
        ...
    ],
    writeConcern: { <write concern> }
}
Wrapperdb.createRole(role, writeConcern)
DescriptionThe command creates a user-defined role with a name rolename and particular set of privileges. The role will be applicable to the current database.
ExampleIn MongoDB shell, let us issue the command:

db.createRole(
    { 
        role: "testrole",
        privileges: [
            { resource: { 
                db: "test", 
                collection: "test" 
            }, actions: [ "find" ] }
        ],
        roles: [
            { role: "read", db: "test" }
        ],
    },
    { w: "majority" , wtimeout: 5000 }
)

07.DB.CREATEROLE

Referencehttp://docs.mongodb.org/manual/reference/command/createRole/

http://docs.mongodb.org/manual/reference/method/db.createRole/

createRole

 

CommandupdateRole
Parameters
{ 
    updateRole: <rolename>,
    privileges: [
        { resource: { <resource> }, actions: [<action>, ... ] },
        ...
    ],
    roles: [
        { role: <role>, db: <database> } | <role>,
        ...
    ],
    writeConcern: { <write concern> }
}
Wrapperdb.updateRole(rolename, update, writeConcern)
DescriptionThe command updates user-defined role with name rolename on the current database. Please notice that an update to a property completely replaces the previous property values (including updates to the roles and privileges).
ExampleIn MongoDB shell, let us issue the command (to update the role created previously by createRole command):

db.updateRole(
    "testrole",
    { 
        privileges: [
            { resource: { 
                db: "test", 
                collection: "test" 
            }, actions: [ "find" ] }
        ],
        roles: [
            { role: "read", db: "test" },
            { role: "readWrite", db: "test" }
        ]
    },
    { w: "majority" , wtimeout: 5000 }
)

07.DB.UPDATEROLE

Referencehttp://docs.mongodb.org/manual/reference/command/updateRole/

http://docs.mongodb.org/manual/reference/method/db.updateRole/

updateRole

 

CommanddropRole
Parameters
{ 
    dropRole: <rolename>,
    writeConcern: { <write concern> }
}
Wrapperdb.dropRole(rolename, writeConcern)
DescriptionThe command deletes a user-defined role with name rolename from the current database.
ExampleIn MongoDB shell, let us issue the command (to drop the role created previously by createRole command): db.dropRole( "testrole" )

07.DB.DROPROLE

Referencehttp://docs.mongodb.org/manual/reference/command/dropRole/

http://docs.mongodb.org/manual/reference/method/db.dropRole/

dropRole

 

CommanddropAllRolesFromDatabase
Parameters
{
    dropAllRolesFromDatabase: 1,
    writeConcern: { <write concern> }
}
Wrapperdb.dropAllRoles(writeConcern)
DescriptionThe command deletes all user-defined roles from the current database. It returns a number of roles removed.
ExampleIn MongoDB shell, let us issue the command: db.dropAllRoles()

07.DB.DROPALLROLES

Referencehttp://docs.mongodb.org/manual/reference/command/dropAllRolesFromDatabase/

http://docs.mongodb.org/manual/reference/method/db.dropAllRoles/

dropAllRolesFromDatabase

 

CommandgrantPrivilegesToRole
Parameters
{ 
    grantPrivilegesToRole: <rolename>,
    privileges: [
        { resource: { <resource> }, actions: [<action>, ... ] },
        ...
    ],
    writeConcern: { <write concern> }
}
Wrapperdb.grantPrivilegesToRole(rolename, privileges, writeConcern)
DescriptionThe command assigns additional privileges to a user-defined role with name rolename defined on the current database.
ExampleIn MongoDB shell, let us issue the command (to grant more privileges to the role created previously by createRole command):

db.grantPrivilegesToRole(
    "testrole",
    [
        { resource: { 
            db: "test", 
            collection: "test" 
        }, actions: [ "insert" ] }
    ],
    { w: "majority" , wtimeout: 5000 }
)

07.DB.GRANTPRIVILEGESTOROLE

Referencehttp://docs.mongodb.org/manual/reference/command/grantPrivilegesToRole/

http://docs.mongodb.org/manual/reference/method/db.grantPrivilegesToRole/

grantPrivilegesToRole

 

CommandrevokePrivilegesFromRole
Parameters
{ 
    revokePrivilegesFromRole: <rolename>,
    privileges: [
        { resource: { <resource> }, actions: [<action>, ... ] },
        ...
    ],
    writeConcern: { <write concern> }
}
Wrapperdb.revokePrivilegesFromRole(rolename, privileges, writeConcern)
DescriptionThe command removes the specified privileges from a user-defined role with name rolename defined on the current database.
ExampleIn MongoDB shell, let us issue the command (to revoke privileges from the role created previously by createRole command):

db.revokePrivilegesFromRole(
    "testrole",
    [
        { resource: { 
            db: "test", 
            collection: "test" 
        }, actions: [ "find" ] }
    ],
    { w: "majority" , wtimeout: 5000 }
)

07.DB.REVOKEPRIVILEGESFROMROLE

Referencehttp://docs.mongodb.org/manual/reference/command/revokePrivilegesFromRole/

http://docs.mongodb.org/manual/reference/method/db.revokePrivilegesFromRole/

revokePrivilegesFromRole

 

CommandgrantRolesToRole
Parameters
{ 
    grantRolesToRole: <rolename>,
    roles: [
        { role: <role>, db: <database> } | <role>,
        ...
    ],
    writeConcern: { <write concern> }
}
Wrapperdb.grantRolesToRole(rolename, roles, writeConcern)
DescriptionThe command assigns additional roles to a user-defined role with name rolename defined on the current database.
ExampleIn MongoDB shell, let us issue the command (to assign more roles to the role created previously by createRole command):

db.grantRolesToRole(
    "testrole",
    [
        "readWrite"
    ],
    { w: "majority" , wtimeout: 5000 }
)

07.DB.GRANTROLESTOROLE

Referencehttp://docs.mongodb.org/manual/reference/command/grantRolesToRole/

http://docs.mongodb.org/manual/reference/method/db.grantRolesToRole/

grantRolesToRole

 

CommandrevokeRolesFromRole
Parameters
{ 
    revokeRolesFromRole: <rolename>,
    roles: [
        { role: <role>, db: <database> } | <role>,
        ...
    ],
    writeConcern: { <write concern> }
}
Wrapperdb.revokeRolesFromRole(rolename, roles, writeConcern)
DescriptionThe command removes specified roles from a user-defined role with name rolename defined on the current database.
ExampleIn MongoDB shell, let us issue the command (to remove some roles from the role created previously by createRole command):

db.revokeRolesFromRole(
    "testrole",
    [
        { role: "read", db: "test" }
    ],
    { w: "majority" , wtimeout: 5000 }
)

07.DB.REVOKEROLESFROMROLE

Referencehttp://docs.mongodb.org/manual/reference/command/revokeRolesFromRole/

http://docs.mongodb.org/manual/reference/method/db.revokeRolesFromRole/

revokeRolesFromRole

 

CommandrolesInfo
Parameters
{
    rolesInfo: <rolename> | { role: <rolename>, db: <database> },
    showPrivileges: <true|false>,
    showBuiltinRoles: <true|false>
}
Wrapperdb.getRole(rolename, showPrivileges)

db.getRoles()

DescriptionThe command returns inheritance and privilege information for specified roles (including both user-defined roles and built-in roles).
ExampleIn MongoDB shell, let us issue the command (to get the information for the role created previously by createRole command): db.getRole( "testrole" )

07.DB.GETROLE

Referencehttp://docs.mongodb.org/manual/reference/command/rolesInfo/

http://docs.mongodb.org/manual/reference/method/db.getRole/

http://docs.mongodb.org/manual/reference/method/db.getRoles/

rolesInfo

 

CommandinvalidateUserCache
DescriptionThe command immediately flushes user information from in-memory cache, including removal of each user’s credentials and roles.
ExampleIn MongoDB shell, let us issue the command: db.runCommand( { invalidateUserCache: 1 } )

07.INVALIDATEUSERCACHE

Referencehttp://docs.mongodb.org/manual/reference/command/invalidateUserCache/

invalidateUserCache

 

Commandauthenticate
Parameters
{ 
    authenticate : 1, 
    user : <username>, 
    nonce : <nonce>, 
    key : <digest> 
}
Wrapperdb.auth(username, password)
DescriptionThe command allows a user to authenticate to the database from within the MongoDB shell or client session.
ExampleIn MongoDB shell, let us issue the command (to authenticate session using the user created previously by createUser command): db.auth( "testuser", "testpassword" )

07.DB.AUTH

Referencehttp://docs.mongodb.org/manual/reference/command/authenticate/

http://docs.mongodb.org/manual/reference/method/db.auth/

authenticate

 

Commandlogout
Wrapperdb.logout()
DescriptionThe command terminates the current authenticated session (if current session is not using authentication, the command has no effect).
ExampleIn MongoDB shell, let us issue the command: db.logout()
07.DB.LOGOUT
Referencehttp://docs.mongodb.org/manual/reference/command/logout/

http://docs.mongodb.org/manual/reference/method/db.logout/

logout

2.1. Additional Resources

3. Indexing

Choosing the right indexes may boost your query performance (and consequently applications performance) in most times. MongoDB supports different types of indexes to pick from:

  • _id: all collections have an index on the _id field that exists by default
  • single field: indexes on a single field of a document, f.e. { “title”: 1 }
  • compound index: indexes on multiple fields, f.e. { “title”: 1, “price”: 1 }
  • multikey index: indexes on content stored in arrays, f.e. { “categories”: 1 }
  • geospatial index: 2d/2sphere indexes to support efficient geospatial queries, f.e. { “location”: “2d” }
  • text indexes: indexes on string content to support full-text search, f.e. { “title”: “text” }
  • hashed indexes: indexes to support hash-based sharding (please refer to Part 4. MongoDB Sharding Guide of the tutorial for more details)

Additionally, each index may be defined as:

  • unique: duplicate values for the indexed field will be rejected
  • sparse: index only contain entries for documents that have the indexed field

For indexed collections, the values for the indexed fields have a maximum index key length limit: the total size of an index entry must be less than 1024 bytes (please refer to limits and thresholds section of the official documentation).

For more details about indexing, please refer to official documentation.

4. Profiling

MongoDB provides a very useful tool to collect server performance data: the database profiler. It collects fine grained data about queries, write operations, cursors, and other database commands on a running server instance. The profiling can be enabled on a per-database or per-server instance level.

CommandProfile
Parameters
{
    profile: <level>
}
Wrapperdb.setProfilingLevel(level, slowms)
DescriptionThe command modifies the current profiler level used by the database profiling system to capture data about performance. The level parameter has the following meaning:

-1 – No change. Returns the current profile level.

0 – Off. No profiling.

1 – On. Only includes slow operations.

2 – On. Includes all operations.

ExampleIn MongoDB shell, let us issue the command: db.setProfilingLevel(2)
07.SHOW.PROFILE
Referencehttp://docs.mongodb.org/manual/reference/command/profile/

http://docs.mongodb.org/manual/reference/method/db.setProfilingLevel/

Profile

 

Commanddb.getProfilingLevel()
DescriptionThe command returns the current profiling level for database operations.
ExampleIn MongoDB shell, let us issue the command: db.getProfilingLevel()

07.DB.GETPROFILINGLEVEL

Referencehttp://docs.mongodb.org/manual/reference/method/db.getProfilingLevel/
Commanddb.getProfilingStatus()
DescriptionThe command returns a document that reflects the current profiling level and the profiling threshold.
ExampleIn MongoDB shell, let us issue the command: db.getProfilingStatus()
07.DB.GETPROFILINGSTATUS
Referencehttp://docs.mongodb.org/manual/reference/method/db.getProfilingStatus/

db.getProfilingLevel

 

Commandshow profile
DescriptionOutputs the five most recent operations that took 1 millisecond or more.
ExampleIn MongoDB shell, let us issue the command: show profile

07.SHOW.PROFILE

Note: Only a fragment of the output is shown.

Referencehttp://docs.mongodb.org/manual/reference/mongo-shell/#command-helpers

show profile

 

To get more insights about analyzing the performance of database operations please refer to official documentation.

5. Query Cache

Among many other new features, MongoDB 2.6 supports a new set of commands to view and manipulate the query cache:

  • list all known query shapes
  • display cached plans for a query shape
  • remove a query shape from the cache
  • clear the whole cache

The query optimizer executes queries and picks the most efficient query plan for a query given the defined indexes. Later on, this query plan is used each time the query (with such a shape) runs. The query optimizer only caches the plans for those query shapes that can have more than one viable plan and occasionally reevaluates query plans as the content of the collection changes.

To experiment with query plans, we need a small dataset and the example from Part 3. MongoDB and Java Tutorial is going to be handy again. Let us switch to books collection in bookstore database, create couple of indexes and insert a few documents into it using MongoDB shell.

use bookstore

db.books.ensureIndex( { "publisher.name": 1 } )
db.books.ensureIndex( { "publisher.name": 1, "price": 1 } )

db.books.insert( { 
    "title" : "MongoDB: The Definitive Guide", 
    "publisher" : { "name" : "O'Reilly" },
    "price" : 32.99
} );

db.books.insert( { 
    "title" : "MongoDB Applied Design Patterns", 
    "publisher" : { "name" : "O'Reilly" },
    "price" : 32.99 
} );

db.books.insert( { 
    "title" : "MongoDB in Action, 2nd Edition", 
    "publisher" : { "name" : "Manning" },
    "price" : 26.66  
} );

Once the preparation is done, let us run simple query which will trigger the query plans evaluation: db.books.find( { "publisher.name": "O'Reilly" }, { "title": 1 } )

Commanddb.<collection>.getPlanCache().help()
DescriptionThe command displays the methods available for a collection’s query plan cache with brief description.
ExampleIn MongoDB shell, let us issue the command: db.books.getPlanCache().help()

07.DB.GETPLANCACHE.HELP

Referencehttp://docs.mongodb.org/manual/reference/method/PlanCache.help/

db.<collection>.getPlanCache

 

CommandplanCacheListFilters
Parameters
{ 
    planCacheListFilters: <collection> 
}
DescriptionThe command lists the index filters associated with query shapes for a collection collection.
ExampleIn MongoDB shell, let us issue the command (to list the filters set by planCacheSetFilter before): db.runCommand( { planCacheListFilters: "books" } )

07.PLANCACHEGETFILTERS

Referencehttp://docs.mongodb.org/manual/reference/command/planCacheListFilters/

planCacheListFilters

 

CommandplanCacheSetFilter
Parameters
{
    planCacheSetFilter: <collection>,
    query: <query>,
    sort: <sort>,
    projection: <projection>,
    indexes: [ <index1>, <index2>, ...]
}
DescriptionThe command set an index filter for a collection collection. If an index filter already exists for the query shape, it will be overridden.
ExampleIn MongoDB shell, let us issue the command:

db.runCommand({ 
    planCacheSetFilter: "books" ,
    query: { "publisher.name": "O'Reilly" }, 
    sort: {},
    projection: { "title": 1 },
    indexes: [ { "publisher.name": 1 } ]
})

07.PLANCACHESETFILTER

Referencehttp://docs.mongodb.org/manual/reference/command/planCacheSetFilter/

planCacheSetFilter

 

CommandplanCacheClearFilters
Parameters
{
    planCacheClearFilters: <collection>,
    query: <query>,
    sort: <sort>,
    projection: <projection>
}
DescriptionThe command clears index filter(s) for a collection collection.
ExampleIn MongoDB shell, let us issue the command:

db.runCommand({ 
    planCacheClearFilters: "books" ,
    query: { "publisher.name": "O'Reilly" }, 
    sort: {},
    projection: { "title": 1 }
})

07.PLANCACHECLEARFILTERS

Referencehttp://docs.mongodb.org/manual/reference/command/planCacheClearFilters/

planCacheClearFilters

 

CommandplanCacheListQueryShapes
Parameters
{
    planCacheListQueryShapes: <collection>
}
Wrapperdb.<collection>.getPlanCache().listQueryShapes()
DescriptionThe command displays the query shapes for which cached query plans exist for collection collection.
ExampleIn MongoDB shell, let us issue the command: db.books.getPlanCache().listQueryShapes()

07.DB.GETPLANCACHE.LISTQUERYSHAPES

Referencehttp://docs.mongodb.org/manual/reference/command/planCacheListQueryShapes/

http://docs.mongodb.org/manual/reference/method/PlanCache.listQueryShapes/

planCacheListQueryShapes

 

CommandplanCacheListPlans
Parameters
{
    planCacheListPlans: <collection>,
    query: <query>,
    sort: <sort>,
    projection: <projection>
}
Wrapperdb.<collection>.getPlanCache().getPlansByQuery( <query>, <projection>, <sort> )
DescriptionThe command displays the cached query plans for the specified query shape for collection collection.
ExampleIn MongoDB shell, let us issue the command: db.books.getPlanCache().getPlansByQuery( { "publisher.name": "O'Reilly" }, { "title": 1 }, {} )

07.DB.GETPLANSBYQUERY

Note: Only a fragment of the output is shown.

Referencehttp://docs.mongodb.org/manual/reference/command/planCacheListPlans/

http://docs.mongodb.org/manual/reference/method/PlanCache.getPlansByQuery/

planCacheListPlans

 

CommandplanCacheClear
Parameters
{
    planCacheClear: <collection>,
    query: <query>,
    sort: <sort>,
    projection: <projection>
}
Wrapperdb.<collection>.getPlanCache().clearPlansByQuery( <query>, <projection>, <sort> )

db.<collection>.getPlanCache().clear()

DescriptionThe command clears the cached query plans for the specified query shape for collection collection. If the query shape is omitted, all cached query plans will be cleared.
ExampleIn MongoDB shell, let us issue the command: db.books.getPlanCache().clear()

07.PLANCACHECLEARFILTERS

Referencehttp://docs.mongodb.org/manual/reference/command/planCacheClear/

http://docs.mongodb.org/manual/reference/method/PlanCache.clearPlansByQuery/

planCacheClear

With log level set to 1 or greater MongoDB will log plan cache changes. The log level could be set using the following command (please notice that it should be run in context of admin database): db.adminCommand( { setParameter: 1, logLevel: 1 } )

6. Cursors

Cursors are the basic way to access documents returned by read operations, f.e. db.<collection>.find(). In MongoDB shell, if the returned cursor is not assigned to a variable, then only first 20 documents are taken from the cursor and shown as the result. However, cursors are very powerful and provide a lot of useful methods.

Methodcursor.addOption(flag)
DescriptionThe method adds special wire protocol flags that modify the behavior of the query.
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.addOption/

cursor.addOption(flag)

 

Methodcursor.batchSize(size)
DescriptionThe method specifies the number of documents to return in each batch of the response from the server instance.
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.batchSize/

cursor.batchSize(size)

 

Methodcursor.count()
DescriptionThe method counts the number of documents referenced by a cursor.
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.count/

cursor.count()

 

Methodcursor.explain(verbose)
DescriptionThe method reports on the query execution plan (including indexes usages) for a cursor.
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.explain/

cursor.explain(verbose)

 

Methodcursor.forEach(function)
DescriptionThe method iterates over the cursor and applies a JavaScript function to each document.
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.forEach/

cursor.forEach(function)

 

Methodcursor.hasNext()
DescriptionThe method returns true if the cursor has more documents and can be iterated.
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.hasNext/

cursor.hasNext()

 

Methodcursor.hint(index)
DescriptionThe method forces server instance to use a specific index for a query.
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.hint/

cursor.hint(index)

 

Methodcursor.limit(limit)
DescriptionThe method limits the size of a cursor’s result set (returned documents).
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.limit/

cursor.limit(limit)

 

Methodcursor.map(function)
DescriptionThe method applies a JavaScript function to each document in a cursor and collects the return values in an array.
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.map/

cursor.map(function)

 

Methodcursor.maxTimeMS(<milliseconds>)
DescriptionThe method specifies a cumulative time limit in milliseconds for processing operations on a cursor.
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.maxTimeMS/

cursor.maxTimeMS(<milliseconds>)

 

Methodcursor.max({ field1: <max value1>, field2: <max value2> … fieldN:<max valueN> })
DescriptionThe method specifies an exclusive upper index bound for a cursor.
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.max/

cursor.max

 

Methodcursor.min({ field1: <min value1>, field2: <min value2> … fieldN:<min valueN> })
DescriptionThe method specifies an inclusive lower index bound for a cursor.
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.min/

cursor.min

 

Methodcursor.next()
DescriptionThe method returns the next document in a cursor.
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.next/

cursor.next()

 

Methodcursor.objsLeftInBatch()
DescriptionThe method returns the number of documents left in the current cursor batch.
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.objsLeftInBatch/

cursor.objsLeftInBatch()

 

Methodcursor.readPref(mode, tagSet)
DescriptionThe method specifies a read preference for the cursor to control how the clients direct queries to a replica set.
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.readPref/

cursor.readPref(mode, tagSet)

 

Methodcursor.showDiskLoc()
DescriptionThe method returns a cursor with modified documents that include the on-disk location of the document in the special $diskLoc property.
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.showDiskLoc/

cursor.showDiskLoc()

 

Methodcursor.size()
DescriptionThe method returns a count of the documents in the cursor after applying cursor.skip() and cursor.limit() methods.
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.size/

cursor.size()

 

Methodcursor.skip(number)
DescriptionThe method returns a cursor that begins returning results only after passing or skipping a number of documents. It should be called before retrieving any documents from the database.
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.skip/

cursor.skip(number)

 

Methodcursor.snapshot()
DescriptionThe method ensures that the query will not return a document multiple times, even if intervening write operations result in a move of the document due to the growth in document size. It should be called before retrieving any documents from the database and works with unsharded collections only.
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.snapshot/

cursor.snapshot()

 

Methodcursor.sort(sort)
DescriptionThe method returns the results ordered according to a sort specification.
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.sort/

cursor.sort(sort)

 

Methodcursor.toArray()
DescriptionThe method returns an array that contains all documents returned by the cursor
Referencehttp://docs.mongodb.org/manual/reference/method/cursor.toArray/

cursor.toArray()

 

CommandparallelCollectionScan
Parameters
{
    parallelCollectionScan: <collection>,
    numCursors: <integer>
}
DescriptionThe command allows applications to use multiple parallel cursors when reading all the documents from a collection. It returns a document that contains an array of cursor information.
ExampleIn MongoDB shell, let us issue the command: db.runCommand( { parallelCollectionScan: “books”, numCursors: 1 } )

07.PARALLELCOLLECTIONSCAN

Referencehttp://docs.mongodb.org/manual/reference/command/parallelCollectionScan/

parallelCollectionScan

 

Armored with deeper knowledge about cursors, let us take a look on the example of different cursor methods in action using the books collection from section Query Cache.

[
    {
        "title" : "MongoDB in Action, 2nd Edition",
        "publisher" : {
            "name" : "Manning"
        },
        "price" : 26.66,
        "$diskLoc" : {
            "file" : 0,
            "offset" : 17072
        }
    },
    {
        "title" : "MongoDB: The Definitive Guide",
        "publisher" : {
            "name" : "O'Reilly"
        },
        "price" : 32.99,
        "$diskLoc" : {
            "file" : 0,
            "offset" : 16560
        }
    }
]

7. Bulk Operations

One of the coolest features of latest MongoDB 2.6 release is the introduction of bulk API. In a nutshell, this new API supports ordered and unordered bulk operations. In an ordered bulk operation, the execution of every operation follows the order it was added to the bulk operation. Consequently, in an unordered bulk operation the order of every operation is not guaranteed.

Commanddb.<collection>.initializeOrderedBulkOp()
DescriptionThe command Initializes and returns a new ordered bulk operation builder for a collection collection. The builder constructs an ordered list of the operations to be executed in bulk.
ExampleIn MongoDB shell, let us issue the command:
var bulk = db.testcollection.initializeOrderedBulkOp()
07.DB.INITIALIZEORDEREDBULKOP
Referencehttp://docs.mongodb.org/manual/reference/method/db.collection.initializeOrderedBulkOp/

initializeOrderedBulkOp

 

Commanddb.<collection>.initializeUnorderedBulkOp()
DescriptionThe command Initializes and returns a new unordered bulk operation builder for a collection collection. The builder constructs an unordered list of the operations to be executed in bulk.
ExampleIn MongoDB shell, let us issue the command:

var bulk = db.testcollection.initializeUnorderedBulkOp()

 

Referencehttp://docs.mongodb.org/manual/reference/method/db.collection.initializeUnorderedBulkOp/

initializeUnorderedBulkOp

 

Those two commands are the starting point to begin using the bulk API. They return bulk builder object (of type Bulk) which provides the fluent API to construct bulk operation.

MethodBulk.insert(<document>)
DescriptionThe method adds an insert operation to a bulk operations list.
Referencehttp://docs.mongodb.org/manual/reference/method/Bulk.insert/

Bulk.insert(<document>)

 

MethodBulk.find(<query>)
DescriptionThe method specifies a query condition for an update or a remove operation. It could be used with following bulk operations:

Bulk.find(<query>).removeOne()

Bulk.find(<query>).remove()

Bulk.find(<query>).replaceOne()

Bulk.find(<query>).updateOne()

Bulk.find(<query>).update()

Referencehttp://docs.mongodb.org/manual/reference/method/Bulk.find/

Bulk.find(<query>)

 

MethodBulk.find(<query>).removeOne()
DescriptionThe method adds a single document remove operation to a bulk operations list.
Referencehttp://docs.mongodb.org/manual/reference/method/Bulk.find.removeOne/

Bulk.find(<query>).removeOne

 

MethodBulk.find(<query>).remove()
DescriptionThe method adds a remove operation to a bulk operations list.
Referencehttp://docs.mongodb.org/manual/reference/method/Bulk.find.remove/

Bulk.find(<query>).remove

 

MethodBulk.find(<query>).replaceOne()
DescriptionThe method adds a single document replacement operation to a bulk operations list.
Referencehttp://docs.mongodb.org/manual/reference/method/Bulk.find.replaceOne/

Bulk.find(<query>).replaceOne

 

MethodBulk.find(<query>).updateOne()
DescriptionThe method adds a single document update operation to a bulk operations list.
Referencehttp://docs.mongodb.org/manual/reference/method/Bulk.find.updateOne/

Bulk.find(<query>).updateOne

 

MethodBulk.find(<query>).update()
DescriptionThe method adds a multi-update operation to a bulk operations list.
Referencehttp://docs.mongodb.org/manual/reference/method/Bulk.find.update/

Bulk.find(<query>).update

 

MethodBulk.execute(writeConcern)
DescriptionThe method executes the list of operations built by the bulk operations builder.
Referencehttp://docs.mongodb.org/manual/reference/method/Bulk.execute/

Bulk.execute(writeConcern)

 

To finish with the bulk API, let us create and execute an example bulk operation using MongoDB shell and initialization commands. In the example we are going to bulk the following actions:

  • insert 3 books into books collection
  • update all books by setting the categories field
  • update book’s (with title “MongoDB: The Definitive Guide”) categories with additional category
  • remove all books published by Manning
var bulk = db.books.initializeOrderedBulkOp();
bulk.insert( { 
    "title" : "MongoDB: The Definitive Guide", 
    "publisher" : { "name" : "O'Reilly" },
    "price" : 32.99
} );
bulk.insert( { 
    "title" : "MongoDB Applied Design Patterns", 
    "publisher" : { "name" : "O'Reilly" },
    "price" : 32.99 
} );
bulk.insert( { 
    "title" : "MongoDB in Action, 2nd Edition", 
    "publisher" : { "name" : "Manning" },
    "price" : 26.66  
} );
bulk.find( { "publisher.name": "O'Reilly" } )
    .update( { $set: { "categories" : [ "Databases", "NoSQL" ] } } );
bulk.find( { "title": "MongoDB: The Definitive Guide" } )
    .updateOne( { $addToSet: { "categories" : "Programming" } } );
bulk.find( { "publisher.name": "Manning" } )
    .remove();
bulk.execute( { w: "majority" ,  wtimeout: 5000  } );

As you can see, each action within ordered bulk operation may rely on previous actions to succeed (for example, find/update depends on insert). This is not the case for unordered bulk operation. The execution of this bulk operation in MongoDB shell yields the following result document:

BulkWriteResult({
    "writeErrors" : [ ],
    "writeConcernErrors" : [ ],
    "nInserted" : 3,
    "nUpserted" : 0,
    "nMatched" : 3,
    "nModified" : 3,
    "nRemoved" : 1,
    "upserted" : [ ]
})

7.1. Additional Resources

8. What’s next

This section concludes the MongoDB tutorial. Hopefully, you have found this NoSQL document database fitting your current or future application demands and this tutorial has helped you to make the right decisions.

Andrey Redko

Andriy is a well-grounded software developer with more then 12 years of practical experience using Java/EE, C#/.NET, C++, Groovy, Ruby, functional programming (Scala), databases (MySQL, PostgreSQL, Oracle) and NoSQL solutions (MongoDB, Redis).
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Inline Feedbacks
View all comments
Back to top button