Concurrency Control in MongoDB

Kaloian ManassievSenior Engineer at MongoDB

kaloianm@mongodb.com@kaloianm

2

Audience

• Operations engineers• Application developers• Third-party storage engine developers• Anybody who is curious

3

What is concurrency control?

4

Locking

5

Data consistency

$100 + $100 = $200 (not $100 )

6

Performance

7

Motivation

• Understand MongoDB’s concurrency control responsibilities

• Learn how MongoDB achieves document-level locking

8

Talk outline

• MongoDB concurrency control• Multiple-granularity locking• WiredTiger vs MMAP V1• Questions

9

Collection Collection

Storage Engine API

10

MongoDB 2.0 (and before)

THETOP LOCK

R WR OK NOW NO NO

11

MongoDB 2.2

Lock per database

db1.coll.insert({a:1}) db4.coll.find({c:5})

12

MongoDB 2.2

THETOP LOCK + intents

13

TOP

db1.coll.insert({a:1}) db4.coll.find({c:5})

IX IS

X S

14

TOP

db.coll.insert({a:1}) db.coll.find({c:5})

IX IS

X S

15

Why intents?

• Get rid of the global lock?

• Use read (shared) lock?

16

TOP

db1.coll.insert({a:1}) db4.coll.find({c:5})

S S

X S

17

TOP

db1.coll.insert({a:1}) db.fsyncLock({lock:1})

S S

X

18

TOP

db1.coll.insert({a:1}) db.fsyncLock({lock:1})

IX S

X

19

TOP

db1.coll.insert({a:1}) db.fsyncLock({lock:1})

IX S

XS S S S

20

TOP

db.fsyncLock({lock:1})

S

S S S S

db1.coll.find({a:1})

IS

S

21

Intents

• “Intention” to access one or more children of an item–Compatible with other intents–Need to acquire lock further down–No overhead

22

Locks

• “Lock” an item for particular access–Item being locked–All items below it

23

Intents compatibility

IS IX S XIS OK OK OK NOIX OK OK NO NOS OK NO OK NOX NO NO NO NO

24

MongoDB 3.0

TOP

25

Multiple-granularity locking

TOP

26

Lock Manager

• Ensures the locking protocol is obeyed

• Very low overhead

• Testable

27

Lock Manager

• Improved fairness

• Support for locking policies

• Lock statistics gathering

28

TOP

1

db.coll1.update({a:1}) db.coll1.update({a:4})

IX IX

IX IX

IX IX

X X

29

Storage Engine API

TOP

30

Storage Engine API

TOP

Storage Engine API

31

How does WiredTiger work?

• Supports document concurrency

• Takes intents for the entire hierarchy

32

TOP

db.coll1.update({a:1}) db.coll1.update({a:4})

IX IX

IX IX

IX IX

WiredTiger MVCC

33

TOP

db.coll1.update({a:1}) db.coll1.drop()

IX IX

IX X

IX

WiredTiger MVCC

34

How does MMAP V1 work?

• Supports collection concurrency

• Takes lock on the collection–Instead of intent

35

TOP

db.coll1.update({a:1}) db.coll2.update({a:4})

IX IX

IX IX

X X

MMAP V1

36

TOP

db.coll1.update({a:1}) db.coll1.find({a:4})

IX IS

IX IS

X S

MMAP V1

37

WiredTiger

• Intents: Global, Database, Collection

• Document-level concurrency left to the storage engine

38

MMAP V1

• Intents: Global, Database

• Locks: Collection

39

Conclusion

• Storage engines have direct control over concurrency–Completely decoupled from top-level locking

–Enabled document-level control

40

Conclusion

• Retrofitted MMAP V1 to use multi-granularity locks–Support for collection-level locking–Got rid of the global lock usage for journaling

QUESTIONS?

#MDBDaysmongodb.com

Get your technical questions answered

In the foyer, 10:00 - 5:00By appointment only – register in person

Tell me how I did today on Guidebook and enter for a chance to win one of these

How to do it: Download the Guidebook App

Search for MongoDB Silicon Valley Submit session feedback

EXTRA SLIDES

45

46

47

48

49

50

51

52