Software Development

MongoDB Replication Guide

This article is part of our Academy Course titled MongoDB – A Scalable NoSQL DB.

In this course, you will get introduced to MongoDB. You will learn how to install it and how to operate it via its shell. Moreover, you will learn how to programmatically access it via Java and how to leverage Map Reduce with it. Finally, more advanced concepts like sharding and replication will be explained. Check it out here!

1. Introduction

Replication is a foundational technique to keep your data safe (by providing redundancy) and highly available all the time (by providing multiple instances serving the exact copy of the data). Replication helps a lot to recover from hardware failure and prevent service interruptions. Very often it is being used to off-load some work (for example, reporting or backup) from the primary servers to dedicated replicas.

There are server classes of replication: Master – Master (or Active – Active) and Master – Slave (or Active – Passive). At the moment, MongoDB implements Master – Slave (or Active – Passive) replication (only one node can accept write operations at a time) with automatic master (primary) election in case of failure.

MongoDB supports the replication in a form of replica sets: a group of MongoDB instances that maintain the same (synchronized) data across multiple instances (servers).

A replica set consist of a single primary MongoDB instance (which accepts all write operations) and one or more secondary instances which synchronize with primary so to have the same data set. To support replication, the primary logs all changes to its data sets in its oplog: a special capped collection that keeps a rolling record of all operations that modify the data stored in the databases. Consequently, the secondaries replicate the primary’s oplog and apply the operations to their data sets so the databases are kept in sync (please refer to official documentation for more details). Please notice that those operations are applied asynchronously so the secondary instances may not always return the most up-to-date data (please refer to official documentation for more details), the fact known as replication lag.

Optionally, each replica set could include one or more arbiters: MongoDB instances which do not maintain a data set but only exist to vote in elections by contributing to majority of votes. Interestingly, a primary instance may step down and become secondary, a secondary may be promoted to primary but arbiters never change their roles (please refer to official documentation for more details).

When a primary is not available to other members of the replica set for more than 10 seconds, the replica set will attempt to promote one of the secondary instances to become a new primary by starting the election process: the first secondary that receives a majority of the votes becomes primary (please refer to official documentation for more details).

Before introducing replica set (which is the recommended way to configure replication), MongoDB supported a bit different master / slave replication model, which at the moment is considered legacy (for more details please refer to official documentation). We are not going to cover this model in this part of the tutorial.

2. Configuring Replication

It is worth mentioning that each secondary member in the replica set might be configured to serve a particular purpose:

  • priority 0 member: never becomes a primary in an elections (please refer to official documentation for more details)
  • hidden member: invisible to client applications (please refer to official documentation for more details)
  • delayed member: reflects an earlier, or delayed, state of the dataset (please refer to official documentation for more details)

While configuring your replica set, it is very important to have an odd number of members so to ensure that the replica set is always able to elect a primary by reaching a majority of votes (please refer to official documentation for more thorough clarification). In the sample replica set configuration we are about to configure there is one primary instance, three secondary instances and one arbiter, totaling 5 members.

Picture 1. A sample replica set configuration.
Picture 1. A sample replica set configuration.

The first step in configuring a replica set is to start all MongoDB instances which are supposed to be its members. The process is very similar to the one we have covered in Part 1. MongoDB Installation – How to install MongoDB except a new command line argument --replSet which specifies the replica set name.

bin/mongod --replSet "rs-demo" --bind_ip 192.168.100.1 --dbpath data
bin/mongod --replSet "rs-demo" --bind_ip 192.168.100.2 --dbpath data
bin/mongod --replSet "rs-demo" --bind_ip 192.168.100.3 --dbpath data
bin/mongod --replSet "rs-demo" --bind_ip 192.168.100.4 --dbpath data
bin/mongod --replSet "rs-demo" --bind_ip 192.168.100.5 --dbpath data

From this point, all other configuration steps are going to be performed using MongoDB shell and a rich set of its command for replication configuration (please refer to Replication commands and command helpers for more details).

Let us connect to the first member of the replica set using MongoDB shell: bin/mongo --host 192.168.100.1. The initial command to initialize the replica set is rs.initiate(): it will initiate a new replica set that consists of the current member and uses the default configuration.

Picture 2. Initiating the replica set with default configuration.
Picture 2. Initiating the replica set with default configuration.

Let us immediately issue another helpful command rs.conf() to inspect current replica set members and configuration (only current instance should be listed):

Picture 3. Inspecting current replica set members and configuration.
Picture 3. Inspecting current replica set members and configuration.

Let us move on by firstly adding an arbiter to replica set using rs.addArb() command passing the arbiter:27017 instance as a parameter: rs.addArb( "arbiter:27017" ). The call to rs.conf() shows off a new member with arbiterOnly flag set to true.

Picture 4. Adding an arbiter and inspecting current replica set members and configuration.
Picture 4. Adding an arbiter and inspecting current replica set members and configuration.

Following the same procedure let us add all other, secondary members to the replica set but this time using regular rs.add() command and providing hostname and port, very similar to rs.addArb():

rs.add( "secondary1:27017" )
rs.add( "secondary2:27017" )
rs.add( "secondary3:27017" )

Picture 5. Adding secondary members and inspecting current replica set configuration.
Picture 5. Adding secondary members and inspecting current replica set configuration.

Great, the replica set is fully configured! Another very useful command rs.status() provides a verbose report about current replica set.

Picture 6. Retrieving current replica set status.
Picture 6. Retrieving current replica set status.

For a demonstration purposes, let us reuse the bookstore example from Part 3. MongoDB and Java Tutorial and insert couple of the documents into books collection. Please notice that those operations should be issued against primary member of the replica set (secondary members nor arbiters do not accept write operations).

db.books.insert( { 
    "title" : "MongoDB: The Definitive Guide", 
    "published" : "2013-05-23", 
    "categories" : [ "Databases", "NoSQL", "Programming" ], 
    "publisher" : { "name" : "O'Reilly" } 
} )

db.books.insert( { 
    "title" : "MongoDB Applied Design Patterns", 
    "published" : "2013-03-19", 
	"categories" : [ "Databases", "NoSQL", "Patterns", "Programming" ], 
	"publisher" : { "name" : "O'Reilly" } 
} )

Picture 7. Inserting new documents into books collection by issuing commands against primary member of replica set.
Picture 7. Inserting new documents into books collection by issuing commands against primary member of replica set.

To make sure the documents have been replicated, let us connect to any secondary member of the replica set and query for all documents in books collection: bin/mongo --host secondary3 bookstore.

Please notice that for any secondary member the error will be raised if rs.slaveOk() command has not been issues before running read operations:

rs.slaveOk()
db.books.find( {}, { title: 1 }).pretty()

Picture 8. Querying books collection on the secondary member of replica set.
Picture 8. Querying books collection on the secondary member of replica set.

3. Replication and Sharding (Partitioning)

Sharding and replication go side by side. In the Part 4. MongoDB Sharding Guide of the tutorial we have mentioned that it is strongly recommended to have each shard configured as a replica set. Such deployments allow having redundant copies of every partition of your data, plus high availability in case the primary member of the shard’s replica set fails.

Luckily, it is very easy to do just by following different member naming convention while calling sh.addShard() command: each hostname should be prefixed by replica set name. For example, the commands we have seen in Part 4. MongoDB Sharding Guide are:

sh.addShard( "ubuntu:27000" )
sh.addShard( "ubuntu:27001" )

In case each shard is a replica set, the commands are going to look like this:

sh.addShard( " rs1/ubuntu:27000" ) 
sh.addShard( " rs1/ubuntu:27001" )

4. Replication commands and command helpers

MongoDB shell provides a command helpers and rs context variable to simplify replication management and deployment.

Commandrs.help()
DescriptionOutputs the brief description for all replication-related shell functions.
ExampleIn MongoDB shell, let us issue the command:

rs.help()

05.RS.HELP

Referencehttp://docs.mongodb.org/manual/reference/method/rs.help/

rs.help()

 

CommandreplSetInitiate
Parameters
{ 
    replSetInitiate : <configuration> 
}
Wrapperrs.initiate(configuration)
DescriptionThe command initiates a new replica set. Optionally, it takes a configuration to initiate with.
ExampleSee please Configuring Replication section.
Referencehttp://docs.mongodb.org/manual/reference/command/replSetInitiate/

http://docs.mongodb.org/manual/reference/method/rs.initiate/

replSetInitiate

 

Commandrs.conf()
DescriptionThe command returns the current replica set configuration.
ExampleSee please Configuring Replication section.
Referencehttp://docs.mongodb.org/manual/reference/method/rs.conf/

rs.conf()

 

Commandrs.addArb(host)
DescriptionThe command adds a new arbiter to an existing replica set.
ExampleSee please Configuring Replication section.
Referencehttp://docs.mongodb.org/manual/reference/method/rs.addArb/

rs.addArb(host)

 

Commandrs.add(host, arbiterOnly)
DescriptionAdds a new member to an existing replica set. With arbiterOnly flag set to true, the new member will be added as an arbiter, similarly to rs.addArb() command.
ExampleSee please Configuring Replication section.
Referencehttp://docs.mongodb.org/manual/reference/method/rs.add/

rs.add(host, arbiterOnly)

 

Commandrs.remove(host)
DescriptionThe command removes existing member from the current replica set. Please notice that this function will disconnect the MongoDB shell briefly and forces a reconnection as the replica set renegotiates which member will be primary. As a result, the shell will display an error even if this command succeeds.
ExampleIn MongoDB shell, let us issue the command:

rs.remove( "secondary3:27017" )

05.RS.REMOVE

Referencehttp://docs.mongodb.org/manual/reference/method/rs.remove/

rs.remove(host)

 

CommandreplSetGetStatus
Wrapperrs.status()
DescriptionThe command returns the status of the replica set from the point of view of the current MongoDB instance. It should be run in context of admin database.
ExampleSee please Configuring Replication section.
Referencehttp://docs.mongodb.org/manual/reference/command/replSetGetStatus/

http://docs.mongodb.org/manual/reference/method/rs.status/

replSetGetStatus

 

CommandreplSetFreeze
Parameters
{ 
    replSetFreeze: <seconds> 
}
Wrapperrs.freeze(seconds)
DescriptionThe command prevents a replica set member from seeking election for the specified number of seconds. This command is often used in conjunction with the replSetStepDown command to make a different member in the replica set a new primary.
ExampleLet us reconnect MongoDB shell to one of the secondary instances bin/mongo –host secondary1 and issue the command:

rs.freeze( 10 )

05.RS.FREEZE

Referencehttp://docs.mongodb.org/manual/reference/command/replSetFreeze/

http://docs.mongodb.org/manual/reference/method/rs.freeze/

replSetFreeze

 

Commandrs.printSlaveReplicationInfo()

db.printSlaveReplicationInfo()

DescriptionThe command outputs a status report of a replica set from the perspective of the secondary member of the set.
ExampleIn MongoDB shell, let us issue the command:

rs.printSlaveReplicationInfo()

05.RS.PRINTSLAVEREPLICATIONINFO

Referencehttp://docs.mongodb.org/manual/reference/method/rs.printSlaveReplicationInfo/

http://docs.mongodb.org/manual/reference/method/db.printSlaveReplicationInfo/

rs.printSlaveReplicationInfo()
db.printSlaveReplicationInfo()

 

Commandrs.printReplicationInfo()
db.printReplicationInfo()
DescriptionThe command outputs a status report of a replica set from the perspective of the primary member of the set.
ExampleIn MongoDB shell, let us issue the command:

rs.printReplicationInfo()

05.RS.PRINTREPLICATIONINFO

Referencehttp://docs.mongodb.org/manual/reference/method/rs.printReplicationInfo/

http://docs.mongodb.org/manual/reference/method/db.printReplicationInfo/

rs.printReplicationInfo()
db.printReplicationInfo()

 

Commanddb.getReplicationInfo()
DescriptionThe command outputs the status of the replica set, using data polled from the oplog.
ExampleIn MongoDB shell, let us issue the command:

rs.getReplicationInfo()

05.DB.GETREPLICATIONINFO

Referencehttp://docs.mongodb.org/manual/reference/method/db.getReplicationInfo/

db.getReplicationInfo()

 

CommandisMaster
Wrapperdb.isMaster()
DescriptionThe command displays information about current member’s role in the replica set, including whether it is a primary (master), secondary or arbiter.
ExampleIn MongoDB shell, let us issue the command:

db.isMaster()

05.DB.ISMASTER

Referencehttp://docs.mongodb.org/manual/reference/command/isMaster/

http://docs.mongodb.org/manual/reference/method/db.isMaster/

isMaster

 

CommandreplSetMaintenance
Parameters{

replSetMaintenance: <true|false>

}

DescriptionThe command enables or disables the maintenance mode for a secondary member of a replica set. It should be run in context of admin database.
ExampleLet us reconnect MongoDB shell to one of the secondary instances bin/mongo –host secondary1 and issue the command: db.adminCommand( { replSetMaintenance: true } )

05.REPLSETMAINTENANCE

Referencehttp://docs.mongodb.org/manual/reference/command/replSetMaintenance/

replSetMaintenance

 

CommandreplSetSyncFrom
Parameters{

replSetSyncFrom: <host>

}

Wrapperrs.syncFrom(host)
DescriptionThis command sets the member that this replica set member will sync from (overriding the default sync target selection logic).
ExampleLet us reconnect MongoDB shell to one of the secondary instances bin/mongo –host secondary1 and issue the command: rs.syncFrom( “secondary2:27017” )

05.REPLSETSYNCFROM

Referencehttp://docs.mongodb.org/manual/reference/command/replSetSyncFrom/

http://docs.mongodb.org/manual/reference/method/rs.syncFrom/

replSetSyncFrom

 

CommandreplSetReconfig
Parameters
{

replSetReconfig: <configuration>,

force: <true|false>

}
Wrapperrs.reconfig(configuration, force)
DescriptionThe command modifies the configuration of an existing replica set. It may be used to add and remove members from the set and/or to change the configuration of existing members. Please notice that this function may disconnect the shell briefly and force a reconnection as the replica set renegotiates which member will be primary. As a result, the shell may display an error even if this command succeeds.
ExampleIn MongoDB shell, let us issue the command:

rs.reconfig( rs.conf() )

05.RS.RECONFIG

Referencehttp://docs.mongodb.org/manual/reference/command/replSetReconfig/

http://docs.mongodb.org/manual/reference/method/rs.reconfig/

replSetReconfig

 

CommandreplSetStepDown
Parameters
{

replSetStepDown: <seconds> ,

force: <true|false>

}
Wrapperrs.stepDown(seconds)
DescriptionThe command forces the current replica set member to step down as primary and then attempts to avoid election as primary for the designated number of seconds. The command raises an error if the current member is not the primary.
ExampleIn MongoDB shell, let us issue the command:

rs.stepDown( 10 )

05.RS.STEPDOWN

The consecutive call of rs.isMaster() command confirms that current member has been stepped down as a primary and became a secondary node.

Referencehttp://docs.mongodb.org/manual/reference/command/replSetStepDown/

http://docs.mongodb.org/manual/reference/method/rs.stepDown/

replSetStepDown

 

Commandrs.slaveOk()
DescriptionThe command allows the current connection to run read operations on secondary members of the replica set.
ExampleSee please Configuring Replication section.
Referencehttp://docs.mongodb.org/manual/reference/method/rs.slaveOk/

rs.slaveOk()

5. What´s next

In this section we have covered replication – a very important aspect of data management. We have seen how easy it is to configure replication in MongoDB using replica sets feature and how it relates to sharding. In the next part of the tutorial we are going to cover the Map/Reduce programming model which MongoDB supports out of the box.

Andrey Redko

Andriy is a well-grounded software developer with more then 12 years of practical experience using Java/EE, C#/.NET, C++, Groovy, Ruby, functional programming (Scala), databases (MySQL, PostgreSQL, Oracle) and NoSQL solutions (MongoDB, Redis).
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Back to top button