In this post you will learn how to use Google’s Cloud Launcher to set up instances for a MongoDB replica set in the Google Compute Engine.
Replication in MongoDB
A minimal MongoDB replica set consists of two data bearing nodes and one so-called arbiter that only takes part in the election of a new primary in case of failure. Unlike other distributed databases, MongoDB does not offer auto-discovery of cluster nodes. If all nodes are up, you have to initialize the cluster via an administrative command inside the Mongo CLI that takes list of all replica set members. This fact makes it hard (but not impossible) to script this with the cloud provisioning tool of your choice.
Cloud Launcher for MongoDB
With the help of Google’s Cloud Launcher for MongoDB, the provisioning of a replica set is done in just a few steps.
First, we define a deployment name (that will prefix the names of the instances), a zone and the name of the replica set.
Then, we set up the instances that will run in the Compute Engine. We define a minimal replica set with two server instances and one arbiter. All other parameters will use the defaults for now (the server instances will use a n1-highmem-4 machine type).
This will finally lead to three running instances.
Compute Engine Instances
The instances will show up in the Compute Engine dashboard where they can be managed:
If you prefer your CLI, you can list the instances with the gcloud tool:
$ gcloud compute instances list
NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS
mongo-0-arbiters-vm-0 europe-west3-b n1-standard-1 10.156.0.2 184.108.40.206 RUNNING
mongo-0-servers-vm-0 europe-west3-b n1-highmem-4 10.156.0.4 220.127.116.11 RUNNING
mongo-0-servers-vm-1 europe-west3-b n1-highmem-4 10.156.0.3 18.104.22.168 RUNNING
Replica Set Status
In order to check if everything is up and running, we open a SSH window to one of the instances from the dasboard and start the mongo CLI:
After connecting to the MongoDB, we run the rs.status() to get the status of the replica set:
"set" : "rs0",
"date" : ISODate("2018-02-12T12:52:47.562Z"),
"myState" : 7,
"members" : [
"_id" : 0,
"name" : "mongo-0-servers-vm-0:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 787,
"_id" : 1,
"name" : "mongo-0-servers-vm-1:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 787,
"_id" : 2,
"name" : "mongo-0-arbiters-vm-0:27017",
"health" : 1,
"state" : 7,
"stateStr" : "ARBITER",
"uptime" : 801,
"configVersion" : 3,
"self" : true
"ok" : 1
(I ommitted some of the output, so we can focus on the important things.)
We can see the name of the replica set rs0 and our three instances
Restore Data Dump from Google Storage
We’ll restore a data dump from a file located in the Google Storage. Since only the the primary node can do write operations, we connect to the instance mongo-0-servers-vm-0 that has to role PRIMARY right now.
Inside that instance, we download a BSON dump from the storage by using the gsutil tool, assuming your storage bucket is called [BUCKET_ID]:
mongo-0-servers-vm-1:/tmp$ cd /tmp
mongo-0-servers-vm-1:/tmp$ gsutil cp gs://[BUCKET_ID].appspot.com/pois.* ./
/ [2 files][ 5.6 KiB/ 5.6 KiB]
Operation completed over 2 objects/5.6 KiB.
Now the dump is on our disk and we restore its data to the MongoDB replica set:
mongo-0-servers-vm-1:/tmp$ mongorestore pois.bson
2018-02-12T13:12:33.402+0000 checking for collection data in pois.bson
2018-02-12T13:12:33.402+0000 reading metadata for test.pois from pois.metadata.json
2018-02-12T13:12:33.418+0000 restoring test.pois from pois.bson
2018-02-12T13:12:33.481+0000 restoring indexes for collection test.pois from metadata
2018-02-12T13:12:33.590+0000 finished restoring test.pois (7 documents)
Finally, we perfom a simple query from the OS command line:
mongo-0-servers-vm-0:/tmp$ mongo --quiet --eval "db.pois.count()" test
to check if our 7 imported documents are really there.
We created a MongoDB replica set and deployed it to the Compute Engine. Then we checked the replica set status and imported some data.
In one of my next posts I will show you how to build a REST-ful microservice in the AppEngine that will access our MongoDB backend in the Compute Engine.