//

Cloud Launcher for MongoDB in the Google Compute Engine

5.3.2018 | 3 minutes of reading time

In this post you will learn how to use Google’s Cloud Launcher to set up instances for a MongoDB replica set in the Google Compute Engine.

Replication in MongoDB

A minimal MongoDB replica set consists of two data bearing nodes and one so-called arbiter that only takes part in the election of a new primary in case of failure.

Unlike other distributed databases, MongoDB does not offer auto-discovery of cluster nodes. If all nodes are up, you have to initialize the cluster via an administrative command inside the Mongo CLI that takes list of all replica set members. This fact makes it hard (but not impossible) to script this with the cloud provisioning tool of your choice.

Cloud Launcher for MongoDB

With the help of Google’s Cloud Launcher for MongoDB , the provisioning of a replica set is done in just a few steps.

First, we define a deployment name (that will prefix the names of the instances), a zone and the name of the replica set.

Then, we set up the instances that will run in the Compute Engine. We define a minimal replica set with two server instances and one arbiter. All other parameters will use the defaults for now (the server instances will use a n1-highmem-4 machine type ).

This will finally lead to three running instances.

Compute Engine Instances

The instances will show up in the Compute Engine dashboard where they can be managed:

If you prefer your CLI, you can list the instances with the gcloud tool:

1$ gcloud compute instances list
2NAME                   ZONE            MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP     STATUS
3mongo-0-arbiters-vm-0  europe-west3-b  n1-standard-1               10.156.0.2   35.198.139.198  RUNNING
4mongo-0-servers-vm-0   europe-west3-b  n1-highmem-4                10.156.0.4   35.198.158.94   RUNNING
5mongo-0-servers-vm-1   europe-west3-b  n1-highmem-4                10.156.0.3   35.198.189.164  RUNNING
6

Replica Set Status

In order to check if everything is up and running, we open a SSH window to one of the instances from the dasboard and start the mongo CLI:

After connecting to the MongoDB, we run the rs.status() to get the status of the replica set:

1rs0:ARBITER>rs.status()
2{
3  "set" : "rs0",
4  "date" : ISODate("2018-02-12T12:52:47.562Z"),
5  "myState" : 7,
6  ...
7  },
8  "members" : [
9      {
10      "_id" : 0,
11      "name" : "mongo-0-servers-vm-0:27017",
12      "health" : 1,
13      "state" : 1,
14      "stateStr" : "PRIMARY",
15      "uptime" : 787,
16      ...
17      },
18      {
19      "_id" : 1,
20      "name" : "mongo-0-servers-vm-1:27017",
21      "health" : 1,
22      "state" : 2,
23      "stateStr" : "SECONDARY",
24      "uptime" : 787,
25      ...
26      },
27      {
28        "_id" : 2,
29        "name" : "mongo-0-arbiters-vm-0:27017",
30        "health" : 1,
31        "state" : 7,
32        "stateStr" : "ARBITER",
33        "uptime" : 801,
34        "configVersion" : 3,
35        "self" : true
36      }
37  ],
38  "ok" : 1
39}
40

(I ommitted some of the output, so we can focus on the important things.)

We can see the name of the replica set rs0 and our three instances

  • mongo-0-servers-vm-0
  • mongo-0-servers-vm-1
  • mongo-0-arbiters-vm-0

Restore Data Dump from Google Storage

We’ll restore a data dump from a file located in the Google Storage . Since only the the primary node can do write operations, we connect to the instance mongo-0-servers-vm-0 that has to role PRIMARY right now.

Inside that instance, we download a BSON dump from the storage by using the gsutil tool, assuming your storage bucket is called [BUCKET_ID]:

1mongo-0-servers-vm-1:/tmp$ cd /tmp
2mongo-0-servers-vm-1:/tmp$ gsutil cp gs://[BUCKET_ID].appspot.com/pois.* ./
3Copying gs://[BUCKET_ID].appspot.com/pois.bson...
4Copying gs://[BUCKET_ID].appspot.com/pois.metadata.json...             
5/ [2 files][  5.6 KiB/  5.6 KiB]                                                
6Operation completed over 2 objects/5.6 KiB.
7

Now the dump is on our disk and we restore its data to the MongoDB replica set:

1mongo-0-servers-vm-1:/tmp$ mongorestore pois.bson
22018-02-12T13:12:33.402+0000    checking for collection data in pois.bson
32018-02-12T13:12:33.402+0000    reading metadata for test.pois from pois.metadata.json
42018-02-12T13:12:33.418+0000    restoring test.pois from pois.bson
52018-02-12T13:12:33.481+0000    restoring indexes for collection test.pois from metadata
62018-02-12T13:12:33.590+0000    finished restoring test.pois (7 documents)
72018-02-12T13:12:33.590+0000    done
8

Finally, we perfom a simple query from the OS command line:

1mongo-0-servers-vm-0:/tmp$ mongo --quiet --eval "db.pois.count()" test
27
3

to check if our 7 imported documents are really there.

Summary

We created a MongoDB replica set and deployed it to the Compute Engine. Then we checked the replica set status and imported some data.

In one of my next posts I will show you how to build a REST-ful microservice in the AppEngine that will access our MongoDB backend in the Compute Engine.

share post

Likes

0

//

More articles in this subject area\n

Discover exciting further topics and let the codecentric world inspire you.

//

Gemeinsam bessere Projekte umsetzen

Wir helfen Deinem Unternehmen

Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.

Hilf uns, noch besser zu werden.

Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.