Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler...
Transcript of Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler...
![Page 1: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/1.jpg)
©2017Percona1
VadimTkachenko
GainaMongoDBAdvantagewiththePerconaMemoryEngineWebinarDecember6th,2017
CTO,Percona
![Page 2: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/2.jpg)
©2017Percona2
Me• CTOatPercona
• LeadingSoftwareEngineeringatPercona
• Technology&(database,hardware,filesystems)PerformanceEnthusiast
• Currentareasofinterest:MySQL,MongoDBandClickHouse
• @VadimTk
![Page 3: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/3.jpg)
©2017Percona3
MemoryEngine• https://www.percona.com/software/mongo-database/percona-
memory-engine-for-mongodb
• StorageEngineinPerconaServerForMongoDB• BasedonwiredTiger• butwithoutwritestothestorage• HighWriteThroughput• HighReadThroughput• Predictable,lowlatencies
![Page 4: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/4.jpg)
©2017Percona4
MemoryEngineusecases
![Page 5: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/5.jpg)
©2017Percona5
ApplicationcacheReplaceservicessuchasmemcached andcustomapplication-leveldatastructureswiththefullpowerofMongoDBfeatures.
![Page 6: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/6.jpg)
©2017Percona6
Typicalarchitecture
Databaselayer(MongoDB,MySQL,etc)
Cachinglayer(memcached or
Redis)
Key-valuestoreLimitedcapabilities
![Page 7: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/7.jpg)
©2017Percona7
Betterway
Databaselayer(MongoDB,MySQL,etc)
CachinglayerPerconaMemory
Engine
• Documentstore• Flexiblequerylanguage• Replication/sharding• FamiliarMongoDBdeployment
![Page 8: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/8.jpg)
©2017Percona8
SessionmanagementDecreaseapplicationresponsetimesbykeepingactiveusersessionsinmemory.
![Page 9: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/9.jpg)
©2017Percona9
TransientruntimestateStoreapplicationstateful runtimedatathatdoesn'trequireon-diskstorage.
![Page 10: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/10.jpg)
©2017Percona10
Multi-tierobjectsharingFacilitatesharingofdatainmulti-tier/multi-languageapplications.
![Page 11: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/11.jpg)
©2017Percona11
ApplicationtestingReduceturnaroundtimeforautomatedapplicationtests.
![Page 12: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/12.jpg)
©2017Percona12
SophisticateddatamanipulationIncreaseperformancefordatamanipulationoperationssuchasaggregationandmapreduction.
![Page 13: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/13.jpg)
©2017Percona13
Real-timeAnalyticsUsesin-memorycomputinginsituationswhereresponsetimeismorecriticalthanpersistence.
![Page 14: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/14.jpg)
©2017Percona14
MemoryEnginedeployments
![Page 15: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/15.jpg)
©2017Percona15
• Fortestinganddevelopment• Usefulforsizingandquickresets• NoRedundancy• NoPersistence• Limitedtoserversize
Standalone
PSMDB Memory
![Page 16: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/16.jpg)
©2017Percona16
./mongod --storageEngine inMemory -–dbpath /data/db -–inMemorySizeGB 150
Standaloneconfig
PSMDB Memory
![Page 17: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/17.jpg)
©2017Percona17
• Forproduction• Failovercapability• Dataredundancy• NoPersistence• Readscaling
ReplicaSet- Memory
PSMDB Memory
primary secondary
secondary
PSMDB Memory
PSMDB Memory
![Page 18: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/18.jpg)
©2017Percona18
ReplicaSet- config
PSMDB Memory
primary secondary
secondary
PSMDB Memory
PSMDB Memory
./mongod --storageEngine inMemory -–dbpath /data/db -–inMemorySizeGB 150 --replSet repSet1
![Page 19: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/19.jpg)
©2017Percona19
• Forproduction• Failovercapability• Dataredundancy• DataPersistence• Readscaling• Configurepermanentmemberasahiddenmember(i.e.hidden:trueandpriority:0)
ReplicaSet– withPersistentOption
PSMDB Memory
primary secondary
secondary
PSMDB Memory
PSMDB wiredTiger or MongoRocks
Permanentstorage
![Page 20: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/20.jpg)
©2017Percona20
Replicatesetwithpermanentoption• Onlythemongod instancesrunningwiththeMemorystorage
enginecanbecometheprimary.• ClientsconnectonlytotheMemorystorageenginemongod
instances.• Evenifbothmongod instancesrunningMemorystorageengine
crashandrestart,theycansyncfromthememberrunningpermanentstorageengine.
• Thehiddenmongod instancerunningwithpermanentstoragepersiststhedatatodisk,includingtheuserdata,indexes,andreplicationconfigurationinformation.
![Page 21: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/21.jpg)
©2017Percona21
Sharding – scalingformorememory
PSMDB Memory PSMDB Memory
mongos
Shard by id
![Page 22: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/22.jpg)
©2017Percona22
Sharding – scalingmongos
Shard by id
PSMDB Memory
primary secondary
secondary PSMDB Memory
PSMDB wiredTiger or MongoRocks
PSMDB Memory
primary secondary
secondary PSMDB Memory
PSMDB wiredTiger or MongoRocks
• Forproduction• Failovercapability• Dataredundancy• DataPersistence• Write scaling
![Page 23: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/23.jpg)
©2017Percona23
PSMDBextras
![Page 24: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/24.jpg)
©2017Percona24
• Dataretrievalbydocumentfields
PSMDB– secondaryindexes{"_id“ : “Vadim”,“visits“ : 45,”created“ : ISODate("2016-01-07T15:46:32.085Z”)
}
> db.coll.ensureIndex({visits:1});> db.coll.find({visits : {$gte :40}});
![Page 25: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/25.jpg)
©2017Percona25
• Toremovedataautomatically
PSMDB– TTLindexes{"_id“ : “Vadim”,“visits“ : 45,”created“ : ISODate("2016-01-07T15:46:32.085Z”)
}
> db.coll.createIndex({"created": 1},{expireAtferSeconds: 7200 });
![Page 26: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/26.jpg)
©2017Percona26
Morefeatures• Authentication• IncludingExternalSASLAuthentication• Role-BasedAccessControl• Auditlogging• GeospatialIndexes• TextSearch
![Page 27: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/27.jpg)
©2017Percona27
PSMDBMemoryEngine- summary
StoreType DocumentStore
Failover/HA ReplicaSet
Scaling Sharding
Persistence ExtraReplicawithwiredTiger orMongoRocks
CRUD Insert,find,update,remove
Expiredata TTL
Concurrency Multi-threading
![Page 28: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/28.jpg)
©2017Percona28
Scalability– MultiThreading• Intel(R)Xeon(R)[email protected]/56
threads• 256GBRAM• Sysbench 1.0witholtp_mongo.lua• Availableathttps://github.com/Percona-Lab/sysbench-mongodb-lua• 100GBdataset• Userthreadsfrom1to128• Operation:OLTPevent
![Page 29: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/29.jpg)
©2017Percona29
![Page 30: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/30.jpg)
©2017Percona30
Stability– MultiThreading• Intel(R)Xeon(R)[email protected]/56
threads• 256GBRAM• Sysbench 1.0witholtp_mongo.lua• Availableatgithub.com/percona-labs• 100GBdataset• Userthreads256• Operation:OLTPevent
![Page 31: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/31.jpg)
©2017Percona31
![Page 32: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/32.jpg)
©2017Percona32
![Page 33: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/33.jpg)
©2017Percona33
Predictableresponsetime130ms(withpracticallynovariation)256userthreads
![Page 34: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/34.jpg)
©2017Percona34
MonitoringwithPerconaMonitoringandManagement(PMM)
![Page 35: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/35.jpg)
©2017Percona35
PMM• NewsoftwarefromPercona• Freeandopen-sourceplatformformanagingandmonitoring
MySQL® andMongoDB® performance.• YoucanrunPMMinyourownenvironment• Providesthoroughtime-basedanalysisforMySQL,MariaDB® and
MongoDBserverstoensurethatyourdataworksasefficientlyaspossible.
![Page 36: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/36.jpg)
©2017Percona36
PMMbenefits• Providesqueryandmetricinformationthatenables
administratorstooptimizedatabaseperformance• Displayscurrentqueriesandhighlightspotentialqueryissuesto
enablefasterissueresolution• Mapsqueriesagainstmetricstohelpmakeinformeddecisions
aboutcrucialdatabaseresources:platformneeds,systemgrowth,teamfocusandthemostimportantdatabaseactivities
![Page 37: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/37.jpg)
©2017Percona37
PMMfeatures• Point-in-timevisibilityandhistoricaltrendingofdatabase
performance• DatafromtheMongoDBqueryprofiler• SpecializedMongoDBdashboardsforgraphingMMAPv1,InMemory,
MongoRocks• Ahistoricalviewofmetricsthatarecriticaltoadatabaseserver• Querymetrics,includingbytessent,locktime,rowssent,andmore• Best-of-breedtools,includingGrafana,Prometheus,andConsul,as
wellasPercona-developedqueryanalytics,administration,API,agentandexportercomponents
• Asingle,easytomanagevirtualappliance
![Page 38: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/38.jpg)
©2017Percona38
![Page 39: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/39.jpg)
©2017Percona39
![Page 40: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/40.jpg)
©2017Percona40
PMM– QueryAnalytics
![Page 41: Gain a MongoDB Advantage with the Percona Memory Engine...• Data from the MongoDB query profiler • Specialized MongoDB dashboards for graphing MMAPv1, InMemory, MongoRocks •](https://reader034.fdocuments.in/reader034/viewer/2022050404/5f8209e08b95307d1440ad9d/html5/thumbnails/41.jpg)
©2017Percona41
Questions?