Go at 10gen

Sam Helman, Software Engineer

What is Go?

“Go is an open source programming environment that makes it easy to build simple, reliable, and efficient software”

- Google

What is Go?

- A programming language (Golang)

Language features

- Code organized into packages

- Distinction b/t pointers and values

- Structs instead of objects

Language features

- First-class functions

- Multiple return values

- Concurrency via goroutines

Development features

- Compiles quickly

- Easy to learn, small

- Strong standard libraries

- http

- subprocess

- time

- Nice plugins for editors

- sublime text, vim, emacs

What else is Go?

- A collection of development tools

mgo (mango) - Go Driver

- labix.org/v2/mgo

- Uses standard API for MongoDB drivers

- A little syntactic sugar (UpdateId, Upsert)

- Connection pooling, cluster discovery, read

preference, auth, GridFS

mgo's BSON Implementation

- labix.org/v2/mgo/bson

- Fully to spec

- Supports marshalling, unmarshalling, ordered,

unordered

Built-in ODM

- Uses struct tags to define db->struct mappings

- Tags aid mgo in unmarshalling documents

How we came to use Go

MMS Backup

- Hosted backups for MongoDB

- Lots of flexibility

o Periodic snapshots

o Point-in-time restore

o Support for sharded clusters

How it works (in a nutshell)

- Backup agent

o Runs on customer’s machine

o Syncs data, then tails oplog

- Backend storage ("Mothership")

o Collection of oplog operations (sent from agent)

o Deduped block storage of periodic snapshots

Backup Agent

- Originally written in Java

- However, needs to live on customer's

machines

- Needs to be easily deployable

- Java relies on customer's JVM, installed

libraries, etc.

Benefits of Go

- Go is compiled and statically linked

o Doesn't rely on anything on target machine

o Large binaries, but no dependencies

- Can be cross-compiled easily

o http://www.laher.net.nz/goxc/

o GOOS, GOARCH

Mongo Continuous Integration (MCI)

- Our home-brewed CI system

- Used to test MongoDB cross-platform

- Built for a number of considerations

o Parallelization

o Flexibility

o Ease of tuning

o Speed

- (Will be open-source soon)

How it works

- One master machine

o Stores config info

o Polls github for changes

o Divides steps (compile, test) among workers

o Sends agent process over to workers

- Numerous worker machines

o Run steps (via agent), report back to master

MCI Agent

- Independent of master process

- Runs on worker machines

- Needs to run on all supported platforms

o With minimal maintenance

- Want to minimize worker machine setup

o Many are dynamically spun up

o As few dependencies as possible

Benefits of Go- No dependencies

o Worker machines need only the agent binary

- Cross-platform

o Build on master machine, scp to workers

- Single binary to copy to workers

- Go dynamicallycompiles quickly

o Agent code changes a lot

o Rebuilt

MongoDB Management Service:Backup Overview

21

What is MongoDB Management Service (MMS)?

How It Works

23

Overview

24

• From the initial sync, we rebuild your data in our datacenters and take a snapshot

• We take snapshots every 6 hours

• Oplog is stored for 48 hours

Under the Hood

25

• Balancer paused every 6 hours

• A no-op token is inserted across all shards, mongoses and config servers

• Oplog applied to replica sets until point in which token was inserted

• Provides a consistent state of database across shards

Sharded Clusters

Key Benefits

27

• Oplog stored for 48 hours

• Restore your replica set to any point-in-time in the last 48 hours by creating a custom snapshot

Point-In-Time

28

• Pull from custom URL

• Push via SCP

Easy to Restore

29

• Confidence in your restore process

• Build development, QA, analytics environments without impacting production

Unlimited, Free Restores

30

• Created by the engineers that build MongoDB

• No need to write or maintain custom backup scripts

Fully Managed

Getting Started

32

1. Create an account at mms.10gen.com

2. Install MMS Monitoring Agent on your deployment

3. Register at mms.10gen.com/backup

4. Install MMS Backup Agent on your deployment

5. Start initial sync

6. Rest easy!

Getting Started

Free Month – Register UsingCoupon Code

“MUG_NYC”