MongoDB Backups and Disaster Recovery - Austin MongoDB Meetup
I Am MongoDB – And So Can You!
-
Upload
mongodb -
Category
Technology
-
view
113 -
download
1
Transcript of I Am MongoDB – And So Can You!
![Page 1: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/1.jpg)
Follow [email protected]
AGENDAReally, really brief history of Rakuten Affiliate NetworkMoving faster with simpler processesMoving more efficiently with simpler technologySome things we learned along the way
![Page 2: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/2.jpg)
REALLY, REALLY BRIEF HISTORY OF RAKUTENAFFILIATE NETWORK
A dinosaur in internet years.LinkShare was founded in 1996Rakuten acquired LinkShare in 2005Re-branded to Rakuten Affiliate Network in 2014
![Page 3: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/3.jpg)
WHAT THAT GOT USTechnical DebtProcess Debt
![Page 4: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/4.jpg)
BUT WHAT WE REALLY WANTED...Tech to feel empoweredMinimize distractionsEliminate blockers
![Page 5: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/5.jpg)
THE CUSTOM REPORTING PROJECT
![Page 6: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/6.jpg)
PROJECT GOALSSoftware Development
Introduce more structure to our JavascriptMore automated testsCleaner code, clearer patternsAPIs first
Operations
Move fast, efficient and iterate quicklyUse MongoDBShip Faster
![Page 7: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/7.jpg)
THE SCHEMA { "_id" : ObjectId("5453ce5ee02222341b0000ea"), "user" : DBRef("User", "5453ce5de0945657b00000a"), "name" : "Sam Witwicky", "slug" : "sam-witwicky", "type" : "custom", "description" : "Cool human", "category" : "Humans", "updated" : ISODate("2015-02-24T03:00:57.802Z"), "created" : ISODate("2015-02-24T03:00:57.802Z"), "is_runnable" : true, "filter_groups" : [ { "_id" : ObjectId("5453ce5ee0909e9f1b0000eb"), "filters" : [ {
View on Github
![Page 8: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/8.jpg)
THE MODEL namespace RM\Document;
use RM\Document\ReportColumn; use RM\Document\FilterGroup; use RM\Document\User;
use Doctrine\Common\Collections\ArrayCollection; use Doctrine\ODM\MongoDB\Mapping\Annotations as MongoDB;
/** * @MongoDB\Document(repositoryClass="RM\Document\ReportRepository") */ class Report { /** * @MongoDB\Id
View on Github
![Page 9: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/9.jpg)
THE CONTROLLER namespace AppBundle\Controller;
use AppBundle\Document\Report; use AppBundle\Form\ReportType;
use Symfony\Bundle\FrameworkBundle\Controller\Controller; use Symfony\Component\HttpFoundation\Request; use Symfony\Component\HttpFoundation\Response;
use FOS\RestBundle\View\View, FOS\RestBundle\View\ViewHandler, FOS\RestBundle\View\RouteRedirectView;
use FOS\RestBundle\Controller\FOSRestController; use Symfony\Component\Routing\Route;
View on Github
![Page 10: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/10.jpg)
THE HTML <div class="report"> <div class="report-id">5453ce5ee02222341b0000ea</div> <div class="user"> <div class="username">SomeUser</div> </div> <div class="name">Sam Witwicky</div> <div class="slug">sam-witwicky</div> <div class="type">custom</div> <p class="description">Cool human</p> <div class="category">Humans</div> <div class="updated">2015-02-24T03:00:57.802Z</div> <div class="created">2015-02-24T03:00:57.802Z</div> <div class="is-runnable">true</div> <div class="is-deleted">false</div> <div class="filter-groups"> <div class="filters">
![Page 11: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/11.jpg)
THE JAVASCRIPT function Report() { var self = this;
this.name; this.description;
this.filterGroups = []; this.reportColumns = [];
this.addFilterGroup = function(filter){ self.filterGroups.push(filter); }
this.addReportColumn = function(reportColumn){ self.reportColumns.push(reportColumn); }
![Page 12: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/12.jpg)
THE CSS .report { .name {} .description {}
.filter-groups { .filter { .operator {} .default-value {} } }
.report-columns{ .report-column { .format {} .sort-direction {} }
![Page 13: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/13.jpg)
THE PLAYERS
![Page 14: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/14.jpg)
SHIP IT!
![Page 15: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/15.jpg)
Threat Level: LowJoeThe Product Manager
Automated testing helps him tooBeware of featuritis.Work with him to simplify the requirements
![Page 16: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/16.jpg)
AUTOMATED TESTINGCreate a culture of testingImprove communication between stakeholders
![Page 17: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/17.jpg)
BDD
![Page 18: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/18.jpg)
FEATURE FILES
Scenario: Valid CRUD requests Given body of request: """ { "name": "My Special Report", "description": "The description of my special report" } """ When I make a POST request to "/reports/" Then the status code should be 200
When I make a GET request to "/reports/5453ce5ee02222341b0000ea/" Then the status code should be 200 Then the response should be: """
![Page 19: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/19.jpg)
Threat Level: MediumDannyQA Guy
Long regressionsResistant to introducing code too close toreleasePush back on unrealistic test casesYou need to earn his trustGet him involved in the Feature File process
![Page 20: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/20.jpg)
MOVE FASTER AND MORE EFFICIENTLYRelease more oftenEliminate distractionsDon't get woken up
![Page 21: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/21.jpg)
Threat Level: HighSheilaThe System Administrator
Forced to limit her drinking for 1 week everymonth.Won't support your system withoutEVERYTHING being documented.She just wants to know how your systemworksYou are going to have to start sharingsupport duties
![Page 22: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/22.jpg)
TAKING ON SUPPORTOne SOP to start: call the application ownerIncrementally build on existing proceduresTransition SOPs to Operations
![Page 23: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/23.jpg)
USE MONGODBEasy to setupIntuitive query interfaceEasy to maintainJavascript friendly
![Page 24: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/24.jpg)
No shardingLimited audience = Limited riskMongoDB was involved the entire wayConfidence was high
![Page 25: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/25.jpg)
Threat Level: HighSamuelThe DBA
Fear introducing new thingsHates ORMsDoesn't like you designing your own schemaThinks NoSQL is a fadEnforcing data integrity at the app layerDe-normalization is okLow maintenanceAutomated backup and restore
![Page 26: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/26.jpg)
WE MADE IT TO PRODUCTION. NOW WHAT?
![Page 27: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/27.jpg)
THE PRIMARY NODE IS GETTING A BUNCH OF READS
![Page 28: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/28.jpg)
READ YOUR OWN WRITE
WHAT IS IT?
![Page 29: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/29.jpg)
IMPLEMENTATION WITH DOCTRINE ODM
namespace AppBundle\EventListener;
use Symfony\Component\DependencyInjection\Container; use Symfony\Component\HttpFoundation\Session\Session; use Symfony\Component\HttpKernel\Event\GetResponseEvent;
use Symfony\Component\EventDispatcher\EventSubscriberInterface;
class MongoReadPreferredSubscriber implements EventSubscriberInterface { /** * @var Session */ protected $session;
/**
![Page 30: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/30.jpg)
PERIODIC NETWORK PARTITIONS
![Page 31: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/31.jpg)
WHY WE MOVED AWAY FROM ARBITERS
5 Node Replica Set (normal)
![Page 32: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/32.jpg)
A & B cannot see each other. A1 wins an election.
![Page 33: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/33.jpg)
Writes go to A1
![Page 34: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/34.jpg)
A & C cannot see each other. B1 wins an election
![Page 35: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/35.jpg)
Writes go to B1
![Page 36: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/36.jpg)
Connectivity is restored
![Page 37: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/37.jpg)
USING A DATA NODE INSTEAD
5 Node Replica Set (normal)
![Page 38: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/38.jpg)
A & B cannot see each other. A1 wins an election.
![Page 39: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/39.jpg)
Writes go to A1
![Page 40: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/40.jpg)
A & C cannot see each other. C1 wins an election
![Page 41: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/41.jpg)
Connectivity is restored
![Page 42: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/42.jpg)
APP HANGS WHEN THE PRIMARY CHANGES(No candidate servers found)
![Page 43: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/43.jpg)
PAY ATTENTION TO YOUR TIMEOUT SETTINGS
![Page 44: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/44.jpg)
connectTimeoutMSThe timeout for establishing a connection.
![Page 45: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/45.jpg)
socketTimeoutMSThe timeout for read-write operations, isMaster and ping requests
![Page 46: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/46.jpg)
A PROPOSED FIXPHP-1223
Use connectTimeoutMS instead of socketTimeoutMS for isMaster and pingrequests
![Page 47: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/47.jpg)
CONSIDERATIONSTimeouts vary driver to driverKeep connectTimeoutMS as low as possible
![Page 48: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/48.jpg)
SINCE THEN...9 number of certified MongoDB DevelopersDBAs are currently handling production support issues for all but one cluster(no issues so far)We've built a couple sharded clustersLeverage the MongoDB support team as early as possible
![Page 49: I Am MongoDB – And So Can You!](https://reader033.fdocuments.in/reader033/viewer/2022051113/55b38bf0bb61eb58748b463e/html5/thumbnails/49.jpg)
WHAT'S NEXTGLOBAL EXPANSION
Tag Aware ShardingReplace homegrown backup solution with MMS Backup