Best Practices for Upgrading to MongoDB 3.2
Richard Kreuter MongoDB
• Upgrading the database itself is pretty easy.• Upgrading all the bits around the database is…
less easy.• The effort is worth it.
• We can help you.
What’s cool in 3.2
More use cases. Pluggable storage engines enables you to use MongoDB in more
projects with the same core database.
Mission-critical apps. MongoDB delivers major advances in the critical areas of
governance, high availability, and disaster recovery.
New tools for new users. Now MongoDB is an integral part of the tooling and
workflows of Data Analysts, DBAs, and Operations teams.
3.2 Overview
Document Validation
What you get • Implement data governance • Enforce data quality• Use familiar MongoDB queries to specify validation rules
More power to the DBAs • The DBA can specify which documents in a collection should be validated
– Pre-existing documents can be left unvalidated, optionally.• Validation failure consequences are tunable
• Hard error• Just a warning
Document Validation Example
The example on the left adds a rule to the
contacts collection that validates:
• The year of birth is no later than 1994
• The document contains a phone number and / or
an email address
• When present, the phone number and email
addresses are strings
Partial Indexes
{"_id" : 1234,"archived" : "true"... }
A partial index canignore all documentswhere ignoredis true.
$lookup Join
What you get • Equivalent to left outer join• Built into aggregation framework
$lookup: { localField : "stock", foreignField : "stockQuote", from : "other_collection", as : "joined_stock" }
Distributed Database Enhancements
Faster Failover for Replica Sets
• Enhanced algorithm detects failure and isolation of primary.• Clusters more resilient to overloaded or unreliable networks
What you get • <2 seconds failover (tunable with the electionTimeoutMillis)
How to upgrade:• Conceptually:
– cfg=rs.conf()– cfg.protocolVersion=1– rs.reconfig(cfg)
Config Servers as Replica Sets
Overview• Config server replica sets can span more than 3 data centers with up to 50 replica set
members supported
What you get • Simplified sharded deployments
– Config servers are deployed as replica sets• Improved metadata consistency • Easily scale to many data centers How to upgrade• New clusters only, or a cluster reboot (for now)
Storage Engines
Storage Engine Architecture in 3.2
Content Repo
IoT Sensor Backend Ad Service Customer
Analytics Archive
MongoDB Query Language (MQL) + Native Drivers
MongoDB Document Data Model
WT MMAP
Available for MongoDB 3.2
Man
agem
ent
Sec
urity
In-memory (beta) Encrypted 3rd party
WiredTiger is the Default
What you get • Best general-purpose storage engine• 7-10x better throughput• Up to 80% compression
“Out of the box” • No special configuration needed
7x-10x Performance, 50%-80% Less Storage
How: WiredTiger Storage Engine• Same data model, same query
language, same ops• Write performance gains driven
by document-level concurrency control
• Storage savings driven by native compression
• 100% backwards compatible• Non-disruptive upgrade
MongoDB 3.0MongoDB 2.6
Performance
50%-80% Less Storage via Compression
• Better storage utilization• Higher I/O scalability• Multiple compression options
– Snappy– zlib– None
• Data and journal compressed on disk• Indexes compressed on disk and in memory
Compression in action
Encrypted Storage
What you get • Encryption of sensitive data for regulated industries• ~ 15% overhead
• Better than many 3rd party encryption tools
Note• Based on the WiredTiger Storage Engine • Available in MongoDB Enterprise Advanced
How to upgrade• Rolling upgrade
In-Memory Storage
• Predictable throughput and latency• In-memory computing without trading away guarantees of disk-based
databases like – Rich query flexibility – Real-time analytics– Scalable capacity – Durability
• Durability via replica set secondaries (using mmap/wt)
One Deployment Powering Multiple Apps
New Tools for New Users
Data Analysts• BI Connector• $lookup
• Aggregation Enhancements: Faster & simpler real-time analytics
DBAsMongoDB Compass
• Visualize schema• View Documents• Graphically build queries• Authenticated access
OperationsIntegration with standard operational workflow reduces overhead• Start from global view of infrastructure: APM tools
integration, e.g. New Relic, AppDynamics.• Then, drill down: Profiler visualization in Ops Manager • Then, deploy: Automated index builds• Next, refine: Partial Indexes improve resource utilization
Ops Manager & Cloud Manager
Single-click provisioning, scaling & upgrades, admin tasks
Monitoring, with charts, dashboards and alerts on 100+ metrics
Backup and restore, with point-in-time recovery, support for sharded clusters
MongoDB Ops Manager / Cloud Manager
The Best Way to Manage MongoDB In Your Data CenterUp to 95% Reduction in Operational Overhead
How Ops Manager / Cloud Manager Helps You
Scale EasilyMeet SLAs
Best Practices, Automated
Cut Management Overhead
Query Perf. Visualizations & Optimization
Fast and simple query optimization with the
new Visual Query Profiler
• Query and write latency are consolidated and
displayed visually; your ops teams can easily
identify slower queries and latency spikes
• Visual query profiler analyzes the data it
displays and provides recommendations for
new indexes that can be created to improve
query performance
• Ops Manager and Cloud Manager can automate
the rollout of new indexes, reducing risk and
your team’s operational overhead
Ops Manager Backup Enhancements3.2 includes Ops Manager enhancements to improve
the productivity of your ops teams and further
simplify installation and management
• MongoDB backup on standard network-mountable filesystems;
integrates with your existing storage infrastructure
• Automated database restores; Build clusters from backup in a few
clicks
• Faster time to first database snapshot
• Support for maintenance windows
• Centralized UI for installation and config of all application and
backup components
These new capabilities and enhancements are why it's worth
upgrading to MongoDB 3.2.
Upgrading MongoDB in Outline
• Manual upgrades – lots of steps, but doable• Automatic – Ops Manager / Cloud Manager
Manual Upgrade
Upgrading a cluster manually
• Follow the steps in the fine documentation:– http://docs.mongodb.org/manual/release-notes/3.2-upgrade/– First result if Googling for “MongoDB Upgrade”
• It's critical to note that upgrade order matters:– For replica sets, first upgrade the secondaries, then the primaries.– For a sharded cluster, follow the 7 steps in the documentation.
Automatic
Upgrading a cluster with Automation
• Ops Manager / Cloud Manager is the easiest, fastest, and best way to upgrade a MongoDB deployment– One-click upgrades.– Rolling upgrades, so zero downtime.
Upgrades
• Select Desired Version from Dropdown
As you can see, upgrading the database software itself is pretty easy.
The fine details
• Backward compatibility & your application• Backward compatibility & your operations• Database behavior & performance• Enabling 3.2 Features
Backward Compatibility
Backward Compatibility: App
• Comprehensive documentation about incompatible changes– https://docs.mongodb.org/manual/release-notes/3.2-compatibility
• Highlights– You may need to upgrade your applications' drivers before upgrading
MongoDB!– Some commands/methods deprecated, removed, changed.
• Audit your codebase for uses of deprecated functionality.
Backward Compatibility: Ops
• Comprehensive documentation about incompatible changes– https://docs.mongodb.org/manual/release-notes/3.2-compatibility
• Highlights– Some settings deprecated, others mmapv1-only.– Stricter replica set configuration validation– mongo* tools' options changed– Legacy (pre-2.6) user account model removed
• Audit scripts & other infrastructure you've got around.
Database Behavior & Performance
Database Behavior & Performance
• Always test your application with the new database version before upgrading production– Apps sometimes rely on unspecified behaviors (e.g. stability of results
from unsorted queries)– Query optimizer improvements may affect index selection (check
important queries with explain)
Good practice: Test against Staging
• Run all your performance correctness tests against a staging environment for some time before upgrading production.
• Best practice: run a real application workload against a 3.2 staging environment, too.– Replicate the workload manually– Use Facebook’s Flashback tool
In our experience, upgrading everything around the database software is where the work is.
Recap
Upgrading Revisited
• Plan• Practice• Productionize
Plan Your Upgrade
• Review the compatibility notes– Audit apps, ops for deprecated details
• Upgrade drivers, scripts, if necessary– Might require some recoding, in edge cases
• Make sure you've got good tests– App behavior, performance, etc.
• Write a checklist of upgrade steps for your environments
Practice Your Upgrade
• Upgrade apps & database in staging environment• Test, test, test.
– If no problems appear within (say) a week of continuous testing, probably a decent candidate.
Productionize Your Upgrade
• Upgrade apps & database in production environment– Ideally, during minimum load/traffic conditions, just for good measure
MongoDB's team can help you!
How we help users upgrade
• MongoDB's Professional Services team offers consulting engagements specifically for upgrading:– To ensure upward compatibility for your apps, tools– To assist with performance & behavior testing around database upgrades– To prepare & execute upgrade plans with your team in your environments.
Contact us for more info: http://mongodb.com/upgrade
Top Related