Building Real Time Systems on MongoDB Using the Oplog at Stripe
description
Transcript of Building Real Time Systems on MongoDB Using the Oplog at Stripe
![Page 1: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/1.jpg)
MongoDB and the OplogEVAN BRODER @ebroder
![Page 2: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/2.jpg)
AGENDAINTRO TO THE OPLOGEXAMPLE APPLICATIONS
![Page 3: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/3.jpg)
INTROTO THE OPLOG
![Page 4: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/4.jpg)
PRIMARY
SECONDARIES
APPLICATION
![Page 5: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/5.jpg)
APPLICATION
save{_id: 1, a: 2}
![Page 6: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/6.jpg)
THINGS I’VE DONE:
- save {_id: 1, a: 2}
![Page 7: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/7.jpg)
APPLICATION
update where{a: 2},{$set: {a: 3}}
![Page 8: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/8.jpg)
THINGS I’VE DONE:
- save {_id: 1, a: 2}- update {_id: 1}, {$set: {a: 3}}
![Page 9: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/9.jpg)
THINGS I’VE DONE:
- save {_id: 1, a: 2}- update {_id: 1}, {$set: {a: 3}}- insert…- delete…- delete…- save…- update…
![Page 10: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/10.jpg)
THINGS I’VE DONE:
save…
THINGS I’VE DONE:
![Page 11: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/11.jpg)
THINGS I’VE DONE:
save…
THINGS I’VE DONE:
save…
![Page 12: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/12.jpg)
![Page 13: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/13.jpg)
![Page 14: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/14.jpg)
![Page 15: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/15.jpg)
![Page 16: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/16.jpg)
![Page 17: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/17.jpg)
![Page 18: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/18.jpg)
![Page 19: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/19.jpg)
![Page 20: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/20.jpg)
TRIGGERS
![Page 21: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/21.jpg)
GOAL:EVENT PROCESSING
![Page 22: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/22.jpg)
![Page 23: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/23.jpg)
![Page 24: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/24.jpg)
![Page 25: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/25.jpg)
GOAL:DETECT INSERTIONS
![Page 26: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/26.jpg)
![Page 27: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/27.jpg)
WARNINGTHIS CODE IS NOT PRODUCTION-READY
![Page 28: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/28.jpg)
oplog = mongo_connection['local']['oplog.rs']
ns = 'eventdb.events'
oplog.find({'op' => 'i', 'ns' => ns}) do |cursor|
cursor.each do |op|
puts op['o']['_id']
end
end
![Page 29: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/29.jpg)
oplog = mongo_connection['local']['oplog.rs']
ns = 'eventdb.events'
oplog.find({'op' => 'i', 'ns' => ns}) do |cursor|
cursor.each do |op|
puts op['o']['_id']
end
end
![Page 30: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/30.jpg)
oplog = mongo_connection['local']['oplog.rs']
ns = 'eventdb.events'
oplog.find({'op' => 'i', 'ns' => ns}) do |cursor|
cursor.each do |op|
puts op['o']['_id']
end
end
![Page 31: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/31.jpg)
oplog = mongo_connection['local']['oplog.rs']
ns = 'eventdb.events'
oplog.find({'op' => 'i', 'ns' => ns}) do |cursor|
cursor.each do |op|
puts op['o']['_id']
end
end
![Page 32: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/32.jpg)
oplog.find({'op' => 'i', 'ns' => ns}) do |cursor|
cursor.add_option(Mongo::Constants::OP_QUERY_TAILABLE)
cursor.add_option(Mongo::Constants::OP_QUERY_AWAIT_DATA)
loop do
cursor.each do |op|
puts op['o']['_id']
end
end
end
![Page 33: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/33.jpg)
oplog.find({'op' => 'i', 'ns' => ns}) do |cursor|
cursor.add_option(Mongo::Constants::OP_QUERY_TAILABLE)
cursor.add_option(Mongo::Constants::OP_QUERY_AWAIT_DATA)
loop do
cursor.each do |op|
puts op['o']['_id']
end
end
end
![Page 34: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/34.jpg)
oplog.find({'op' => 'i', 'ns' => ns}) do |cursor|
cursor.add_option(Mongo::Constants::OP_QUERY_TAILABLE)
cursor.add_option(Mongo::Constants::OP_QUERY_AWAIT_DATA)
loop do
cursor.each do |op|
puts op['o']['_id']
end
end
end
![Page 35: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/35.jpg)
oplog.find({'op' => 'i', 'ns' => ns}) do |cursor|
cursor.add_option(Mongo::Constants::OP_QUERY_TAILABLE)
cursor.add_option(Mongo::Constants::OP_QUERY_AWAIT_DATA)
loop do
cursor.each do |op|
puts op['o']['_id']
end
end
end
![Page 36: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/36.jpg)
![Page 37: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/37.jpg)
![Page 38: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/38.jpg)
![Page 39: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/39.jpg)
![Page 40: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/40.jpg)
![Page 41: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/41.jpg)
![Page 42: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/42.jpg)
start_entry = oplog.find_one({},
{:sort => {'$natural' => -1}})
start = start_entry['ts']
oplog.find({'ts' => {'$gt' => start},
'op' => 'i',
'ns' => ns}) do |cursor|
cursor.add_option(Mongo::Constants::OP_QUERY_TAILABLE)
cursor.add_option(Mongo::Constants::OP_QUERY_AWAIT_DATA)
loop do
cursor.each do |op|
puts op['o']['_id']
end
end
end
![Page 43: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/43.jpg)
start_entry = oplog.find_one({},
{:sort => {'$natural' => -1}})
start = start_entry['ts']
oplog.find({'ts' => {'$gt' => start},
'op' => 'i',
'ns' => ns}) do |cursor|
cursor.add_option(Mongo::Constants::OP_QUERY_OPLOG_REPLAY)
cursor.add_option(Mongo::Constants::OP_QUERY_TAILABLE)
cursor.add_option(Mongo::Constants::OP_QUERY_AWAIT_DATA)
loop do
cursor.each do |op|
puts op['o']['_id']
end
end
end
![Page 44: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/44.jpg)
start_entry = oplog.find_one({},
{:sort => {'$natural' => -1}})
start = start_entry['ts']
oplog.find({'ts' => {'$gt' => start},
'op' => 'i',
'ns' => ns}) do |cursor|
cursor.add_option(Mongo::Constants::OP_QUERY_OPLOG_REPLAY)
cursor.add_option(Mongo::Constants::OP_QUERY_TAILABLE)
cursor.add_option(Mongo::Constants::OP_QUERY_AWAIT_DATA)
loop do
cursor.each do |op|
puts op['o']['_id']
end
end
end
![Page 45: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/45.jpg)
start_entry = oplog.find_one({},
{:sort => {'$natural' => -1}})
start = start_entry['ts']
oplog.find({'ts' => {'$gt' => start},
'op' => 'i',
'ns' => ns}) do |cursor|
cursor.add_option(Mongo::Constants::OP_QUERY_OPLOG_REPLAY)
cursor.add_option(Mongo::Constants::OP_QUERY_TAILABLE)
cursor.add_option(Mongo::Constants::OP_QUERY_AWAIT_DATA)
loop do
cursor.each do |op|
puts op['o']['_id']
end
end
end
![Page 46: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/46.jpg)
DATA TRANSFORMATIONS
![Page 47: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/47.jpg)
GOAL:MONGODB TO POSTGRESQL
![Page 48: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/48.jpg)
start_entry = oplog.find_one({},
{:sort => {'$natural' => -1}})
start = start_entry['ts']
oplog.find({'ts' => {'$gt' => start},
'op' => 'i',
'ns' => ns}) do |cursor|
cursor.add_option(Mongo::Constants::OP_QUERY_OPLOG_REPLAY)
cursor.add_option(Mongo::Constants::OP_QUERY_TAILABLE)
cursor.add_option(Mongo::Constants::OP_QUERY_AWAIT_DATA)
loop do
cursor.each do |op|
puts op['o']['_id']
end
end
end
![Page 49: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/49.jpg)
start_entry = oplog.find_one({},
{:sort => {'$natural' => -1}})
start = start_entry['ts']
oplog.find({'ts' => {'$gt' => start}}) do |cursor|
cursor.add_option(Mongo::Constants::OP_QUERY_OPLOG_REPLAY)
cursor.add_option(Mongo::Constants::OP_QUERY_TAILABLE)
cursor.add_option(Mongo::Constants::OP_QUERY_AWAIT_DATA)
loop do
cursor.each do |op|
puts op['o']['_id']
end
end
end
![Page 50: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/50.jpg)
cursor.each do |op|
puts op['o']['_id']
end
![Page 51: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/51.jpg)
cursor.each do |op|
case op['op']
when 'i'
puts op['o']['_id']
else
# ¯\_(ツ)_/¯
end
end
![Page 52: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/52.jpg)
cursor.each do |op|
case op['op']
when 'i'
query = "INSERT INTO #{op['ns']} (" +
op['o'].keys.join(', ') +
') VALUES (' +
op['o'].values.map(&:inspect).join(', ') + ')'
else
# ¯\_(ツ)_/¯
end
end
![Page 53: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/53.jpg)
![Page 54: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/54.jpg)
![Page 55: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/55.jpg)
![Page 56: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/56.jpg)
cursor.each do |op|
case op['op']
when 'i'
query = "INSERT INTO #{op['ns']} (" +
op['o'].keys.join(', ') +
') VALUES (' +
op['o'].values.map(&:inspect).join(', ') + ')'
when 'd'
query = "DELETE FROM #{op['ns']} WHERE _id=" +
op['o']['_id'].inspect
else
# ¯\_(ツ)_/¯
end
end
![Page 57: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/57.jpg)
![Page 58: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/58.jpg)
![Page 59: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/59.jpg)
![Page 60: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/60.jpg)
query = "DELETE FROM #{op['ns']} WHERE _id=" +
op['o']['_id'].inspect
when 'u'
query = "UPDATE #{op['ns']} SET"
updates = op['o']['$set'] ? op['o']['$set'] : op['o']
updates.each do |k, v|
query += " #{k}=#{v.inspect}"
end
query += " WHERE _id="
query += op['o2']['_id'].inspect
else
# ¯\_(ツ)_/¯
end
end
![Page 61: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/61.jpg)
cursor.each do |op|
case op['op']
when 'i'
query = "INSERT INTO #{op['ns']} (" +
op['o'].keys.join(', ') + ') VALUES (' +
op['o'].values.map(&:inspect).join(', ') + ')'
when 'd'
query = "DELETE FROM #{op['ns']} WHERE _id=" +
op['o']['_id'].inspect
when 'u'
query = "UPDATE #{op['ns']} SET"
updates = op['o']['$set'] ? op['o']['$set'] : op['o']
updates.each do |k, v|
query += " #{k}=#{v.inspect}"
end
query += " WHERE _id=" + op['o2']['_id'].inspect
else
# ¯\_(ツ)_/¯
end
end
![Page 62: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/62.jpg)
github.com/stripe/mosql
![Page 63: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/63.jpg)
github.com/stripe/zerowing
![Page 64: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/64.jpg)
cursor.each do |op|
case op['op']
when 'i'
query = "INSERT INTO #{op['ns']} (" +
op['o'].keys.join(', ') + ') VALUES (' +
op['o'].values.map(&:inspect).join(', ') + ')'
when 'd'
query = "DELETE FROM #{op['ns']} WHERE _id=" +
op['o']['_id'].inspect
when 'u'
query = "UPDATE #{op['ns']} SET"
updates = op['o']['$set'] ? op['o']['$set'] : op['o']
updates.each do |k, v|
query += " #{k}=#{v.inspect}"
end
query += " WHERE _id=" + op['o2']['_id'].inspect
else
# ¯\_(ツ)_/¯
end
end
![Page 65: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/65.jpg)
DISASTER RECOVERY
![Page 66: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/66.jpg)
task = collection.find_one({'finished' => nil}
# do something with task…
collection.update({'_id' => task.id},
{'$set' => {'finished' => Time.now.to_i}})
![Page 67: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/67.jpg)
loop do
collection.remove(
{'finished' => {'$lt' => Time.now.to_i - 30}})
sleep(10)
end
![Page 68: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/68.jpg)
evan@caron:~$ mongo
MongoDB shell version: 2.4.10
connecting to: test
normal:PRIMARY> null < (Date.now() / 1000) - 30
true
![Page 69: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/69.jpg)
![Page 70: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/70.jpg)
![Page 71: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/71.jpg)
THINGS I’VE DONE:
insertdelete…
THINGS I’VE DONE:
![Page 72: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/72.jpg)
> db.getReplicationInfo()
{
"logSizeMB" : 48964.3541015625,
"usedMB" : 46116.4,
"timeDiff" : 316550,
"timeDiffHours" : 87.93,
"tFirst" : "Thu Apr 11 2013 07:24:29 GMT+0000 (UTC)",
"tLast" : "Sun Apr 14 2013 23:20:19 GMT+0000 (UTC)",
"now" : "Sat May 24 2014 07:52:35 GMT+0000 (UTC)"
}
![Page 73: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/73.jpg)
> db.getReplicationInfo()
{
"logSizeMB" : 48964.3541015625,
"usedMB" : 46116.4,
"timeDiff" : 316550,
"timeDiffHours" : 87.93,
"tFirst" : "Thu Apr 11 2013 07:24:29 GMT+0000 (UTC)",
"tLast" : "Sun Apr 14 2013 23:20:19 GMT+0000 (UTC)",
"now" : "Sat May 24 2014 07:52:35 GMT+0000 (UTC)"
}
![Page 74: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/74.jpg)
new_oplog.find({'ts' => {'$gt' => start}}) do |cursor| cursor.add_option(Mongo::Constants::OP_QUERY_OPLOG_REPLAY) cursor.each do |op| if op['op'] == 'd' && op['ns'] == 'monsterdb.tasks' old_task = old_tasks.find_one({'_id' => op['o']['_id']}) if old_task['finished'] == nil # found one! # save old_task to a file, and we'll re-queue it later end end
old_connection['admin'].command({'applyOps' => [op]}) endend
![Page 75: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/75.jpg)
new_oplog.find({'ts' => {'$gt' => start}}) do |cursor| cursor.add_option(Mongo::Constants::OP_QUERY_OPLOG_REPLAY) cursor.each do |op| if op['op'] == 'd' && op['ns'] == 'monsterdb.tasks' old_task = old_tasks.find_one({'_id' => op['o']['_id']}) if old_task['finished'] == false # found one! # save old_task to a file, and we'll re-queue it later end end
old_connection['admin'].command({'applyOps' => [op]}) endend
![Page 76: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/76.jpg)
new_oplog.find({'ts' => {'$gt' => start}}) do |cursor| cursor.add_option(Mongo::Constants::OP_QUERY_OPLOG_REPLAY) cursor.each do |op| if op['op'] == 'd' && op['ns'] == 'monsterdb.tasks' old_task = old_tasks.find_one({'_id' => op['o']['_id']}) if old_task['finished'] == false # found one! # save old_task to a file, and we'll re-queue it later end end
old_connection['admin'].command({'applyOps' => [op]}) endend
![Page 77: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/77.jpg)
THINGS I’VE DONE:
save…
THINGS I’VE DONE:
save…
![Page 78: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/78.jpg)
![Page 79: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/79.jpg)
![Page 80: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/80.jpg)
![Page 81: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/81.jpg)
![Page 82: Building Real Time Systems on MongoDB Using the Oplog at Stripe](https://reader033.fdocuments.in/reader033/viewer/2022052623/559a3a881a28ab2e688b463f/html5/thumbnails/82.jpg)
QUESTIONS?