Hidden Gems in the 2.6 Release
-
Upload
mongodb -
Category
Technology
-
view
210 -
download
0
description
Transcript of Hidden Gems in the 2.6 Release
![Page 1: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/1.jpg)
Unearthing 4 Hidden Gems of MongoDB 2.6
Dan PasetteVP of Core Engineering
![Page 2: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/2.jpg)
MongoDB 2.6 was big.
![Page 3: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/3.jpg)
> Power of 2 as Default Allocation Strategy> Server-side Timeouts> Query Engine Introspection> Background Indexing on Secondaries
Here’s what you missed:
Allocation. Timeouts. Introspection. Indexing.
![Page 4: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/4.jpg)
Gem 1.Power of 2 Allocation is now on by default.
Allocation. Timeouts. Introspection. Indexing.
![Page 5: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/5.jpg)
What happened?Before 2.6, the default record allocation used an exact-fit strategy.
Allocation. Timeouts. Introspection. Indexing.
![Page 6: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/6.jpg)
Why does the allocation strategy matter?
Allocation. Timeouts. Introspection. Indexing.
![Page 7: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/7.jpg)
Allocation. Timeouts. Introspection. Indexing.
Take this section of disk with some records of various sizes.
![Page 8: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/8.jpg)
Allocation. Timeouts. Introspection. Indexing.
Documents are resized, and records are removed.
![Page 9: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/9.jpg)
Allocation. Timeouts. Introspection. Indexing.
Space is wasted because new records cannot fit.
![Page 10: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/10.jpg)
Allocation. Timeouts. Introspection. Indexing.
Fragmentation!
![Page 11: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/11.jpg)
Allocation. Timeouts. Introspection. Indexing.
When document needs to move, references must be updated. That costs I/O.
![Page 12: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/12.jpg)
The 2.6 FixBy default, round up the record size to the next power of two.
Allocation. Timeouts. Introspection. Indexing.
![Page 13: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/13.jpg)
Allocation. Timeouts. Introspection. Indexing.
5121024
![Page 14: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/14.jpg)
Allocation. Timeouts. Introspection. Indexing.
5121024
804 412Reduce disk frag!
![Page 15: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/15.jpg)
We’ve made capacity planning predictable.
Allocation. Timeouts. Introspection. Indexing.
![Page 16: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/16.jpg)
We’ve made capacity planning predictable.What about operations?
Allocation. Timeouts. Introspection. Indexing.
![Page 17: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/17.jpg)
Gem 2.Server-side Timeouts.
Allocation. Timeouts. Introspection. Indexing.
![Page 18: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/18.jpg)
Server
“Our collection was indexed in staging,but we forgot to index in production.”
Allocation. Timeouts. Introspection. Indexing.
![Page 19: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/19.jpg)
ServerC
Allocation. Timeouts. Introspection. Indexing.
“Our collection was indexed in staging,but we forgot to index in production.”
![Page 20: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/20.jpg)
ServerC
Cretry!
Allocation. Timeouts. Introspection. Indexing.
“Our collection was indexed in staging,but we forgot to index in production.”
![Page 21: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/21.jpg)
ServerC
C
Cretry
!
retry!
Allocation. Timeouts. Introspection. Indexing.
“Our collection was indexed in staging,but we forgot to index in production.”
![Page 22: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/22.jpg)
The 2.6 FixUse maxTimeMS to limit how long an op can run in the database.
Allocation. Timeouts. Introspection. Indexing.
![Page 23: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/23.jpg)
We’ve stopped the op.
Allocation. Timeouts. Introspection. Indexing.
![Page 24: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/24.jpg)
We’ve stopped the op.But why was it slow?
Allocation. Timeouts. Introspection. Indexing.
![Page 25: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/25.jpg)
Gem 3.Get inside the Query Optimizer.
Allocation. Timeouts. Introspection. Indexing.
![Page 26: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/26.jpg)
What happened?For 2.6, we rewrote the query execution framework — and most of the time, it just works.
Allocation. Timeouts. Introspection. Indexing.
![Page 27: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/27.jpg)
Allocation. Timeouts. Introspection. Indexing.
Query PlannerQuery Parser
![Page 28: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/28.jpg)
Allocation. Timeouts. Introspection. Indexing.
Query Planner Plan Cache
?
Query Parser
![Page 29: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/29.jpg)
Allocation. Timeouts. Introspection. Indexing.
Query Planner Plan Cache
?
Hit
ReturnPlan Runner
Miss
Query Parser
![Page 30: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/30.jpg)
Allocation. Timeouts. Introspection. Indexing.
Query Parser
Query Planner Plan Cache
?
Hit
ReturnPlan Runner
Miss
Plan Enumerator
Plan
Plan
Plan
![Page 31: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/31.jpg)
Allocation. Timeouts. Introspection. Indexing.
Query Parser
Query Planner Plan Cache
?
Hit
ReturnPlan Runner
Miss
Plan Enumerator
Plan
Plan
Plan
Multiplan Runner
![Page 32: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/32.jpg)
Allocation. Timeouts. Introspection. Indexing.
Query Parser
Query Planner Plan Cache
?
Hit
ReturnPlan Runner
Miss
Plan Enumerator
Plan
Plan
Plan
Multiplan Runner
Winner!
Cache
![Page 33: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/33.jpg)
Allocation. Timeouts. Introspection. Indexing.
Query Parser
Query Planner Plan Cache
?
Hit
ReturnPlan Runner
Miss
Plan Enumerator
Plan
Plan
Plan
Multiplan Runner
Winner!
Cache
![Page 34: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/34.jpg)
Allocation. Timeouts. Introspection. Indexing.
Query Planner Plan Cache
?
Hit
ReturnPlan Runner
Query Parser
The plan cache cuts out a lot of processing.
![Page 35: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/35.jpg)
But sometimes,queries are slow.
Allocation. Timeouts. Introspection. Indexing.
![Page 36: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/36.jpg)
Allocation. Timeouts. Introspection. Indexing.
Query Parser
Query Planner Plan Cache
?
Hit
ReturnPlan Runner
!!!
The plan cache is cachinga suboptimal plan.
![Page 37: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/37.jpg)
The 2.6 FixAdd logging and introspection tools to view and manipulate the cache, such as getPlanCache.
Allocation. Timeouts. Introspection. Indexing.
![Page 38: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/38.jpg)
Now we know what’s wrong with the query.
Allocation. Timeouts. Introspection. Indexing.
![Page 39: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/39.jpg)
Now we know what’s wrong with the query.Let’s fix it!
Allocation. Timeouts. Introspection. Indexing.
![Page 40: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/40.jpg)
Gem 4.Background Indexing on Secondaries.
Allocation. Timeouts. Introspection. Indexing.
![Page 41: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/41.jpg)
P
S S
Allocation. Timeouts. Introspection. Indexing.
![Page 42: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/42.jpg)
P
S S
> db.coll.ensureIndex({ … }, { background: true })
Allocation. Timeouts. Introspection. Indexing.
![Page 43: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/43.jpg)
CP
S S
> db.coll.ensureIndex({ … }, { background: true })
Allocation. Timeouts. Introspection. Indexing.
![Page 44: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/44.jpg)
What happened?Before 2.6, background index builds became foreground index builds when replicated to secondaries.
Allocation. Timeouts. Introspection. Indexing.
![Page 45: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/45.jpg)
The 2.6 FixNow, background index buildsstay in the background.
Allocation. Timeouts. Introspection. Indexing.
![Page 46: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/46.jpg)
CP
S S
> db.coll.ensureIndex({ … }, { background: true })
Allocation. Timeouts. Introspection. Indexing.
![Page 47: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/47.jpg)
> Power of 2 as Default Allocation Strategy> Server-side Timeouts> Query Engine Introspection> Background Indexing on Secondaries
Here’s what we covered:
Allocation. Timeouts. Introspection. Indexing.
![Page 48: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/48.jpg)
Small gems, big impact.
![Page 49: Hidden Gems in the 2.6 Release](https://reader033.fdocuments.in/reader033/viewer/2022061110/5453d0f3af7959aa678b7b97/html5/thumbnails/49.jpg)
Small gems, big impact.Thank you!