Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related...
Transcript of Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related...
![Page 1: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/1.jpg)
![Page 2: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/2.jpg)
![Page 3: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/3.jpg)
MonitoringDocker
![Page 4: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/4.jpg)
TableofContents
MonitoringDocker
Credits
AbouttheAuthor
AbouttheReviewer
www.PacktPub.com
Supportfiles,eBooks,discountoffers,andmore
Whysubscribe?
FreeaccessforPacktaccountholders
Preface
Whatthisbookcovers
Whatyouneedforthisbook
Whothisbookisfor
Conventions
Readerfeedback
Customersupport
Downloadingtheexamplecode
Downloadingthecolorimagesofthisbook
Errata
Piracy
Questions
1.IntroductiontoDockerMonitoring
Pets,Cattle,Chickens,andSnowflakes
Pets
Cattle
Chickens
Snowflakes
Sowhatdoesthisallmean?
Docker
Launchingalocalenvironment
![Page 5: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/5.jpg)
Cloningtheenvironment
Runningavirtualserver
Haltingthevirtualserver
Summary
2.UsingtheBuilt-inTools
Dockerstats
RunningDockerstats
Whatjusthappened?
Whataboutprocesses?
Dockertop
Dockerexec
Summary
3.AdvancedContainerResourceAnalysis
WhatiscAdvisor?
RunningcAdvisorusingacontainer
CompilingcAdvisorfromsource
Collectingmetrics
TheWebinterface
Overview
Processes
CPU
Memory
Network
Filesystem
Viewingcontainerstats
Subcontainers
Driverstatus
Images
Thisisallgreat,what’sthecatch?
Prometheus
LaunchingPrometheus
![Page 6: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/6.jpg)
QueryingPrometheus
Dashboard
Thenextsteps
Alternatives?
Summary
4.ATraditionalApproachtoMonitoringContainers
Zabbix
InstallingZabbix
Usingcontainers
Usingvagrant
Preparingourhostmachine
TheZabbixwebinterface
Dockermetrics
Createcustomgraphs
Comparecontainerstoyourhostmachine
Triggers
Summary
5.QueryingwithSysdig
WhatisSysdig?
InstallingSysdig
UsingSysdig
Thebasics
Capturingdata
Containers
Furtherreading
UsingCsysdig
Summary
6.ExploringThirdPartyOptions
Awordaboutexternallyhostedservices
DeployingDockerinthecloud
WhyuseaSaaSservice?
![Page 7: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/7.jpg)
SysdigCloud
Installingtheagent
Exploringyourcontainers
Summaryandfurtherreading
Datadog
Installingtheagent
Exploringthewebinterface
Summaryandfurtherreading
NewRelic
Installingtheagent
Exploringthewebinterface
Summaryandfurtherreading
Summary
7.CollectingApplicationLogsfromwithintheContainer
Viewingcontainerlogs
ELKStack
Startingthestack
Logspout
Reviewingthelogs
Whataboutproduction?
Lookingatthirdpartyoptions
Summary
8.WhatAretheNextSteps?
Somescenarios
Pets,Cattle,Chickens,andSnowflakes
Pets
Cattle
Chickens
Snowflakes
Scenarioone
Scenariotwo
![Page 8: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/8.jpg)
Scenariothree
Alittlemoreaboutalerting
Chickens
CattleandPets
Sendingalerts
Keepingup
Summary
Index
![Page 9: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/9.jpg)
![Page 10: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/10.jpg)
MonitoringDocker
![Page 11: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/11.jpg)
![Page 12: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/12.jpg)
MonitoringDockerCopyright©2015PacktPublishing
Allrightsreserved.Nopartofthisbookmaybereproduced,storedinaretrievalsystem,ortransmittedinanyformorbyanymeans,withoutthepriorwrittenpermissionofthepublisher,exceptinthecaseofbriefquotationsembeddedincriticalarticlesorreviews.
Everyefforthasbeenmadeinthepreparationofthisbooktoensuretheaccuracyoftheinformationpresented.However,theinformationcontainedinthisbookissoldwithoutwarranty,eitherexpressorimplied.Neithertheauthor,norPacktPublishing,anditsdealersanddistributorswillbeheldliableforanydamagescausedorallegedtobecauseddirectlyorindirectlybythisbook.
PacktPublishinghasendeavoredtoprovidetrademarkinformationaboutallofthecompaniesandproductsmentionedinthisbookbytheappropriateuseofcapitals.However,PacktPublishingcannotguaranteetheaccuracyofthisinformation.
Firstpublished:December2015
Productionreference:1041215
PublishedbyPacktPublishingLtd.
LiveryPlace
35LiveryStreet
BirminghamB32PB,UK.
ISBN978-1-78588-275-3
www.packtpub.com
![Page 13: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/13.jpg)
![Page 14: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/14.jpg)
CreditsAuthor
RussMcKendrick
Reviewer
MarceloCorreiaPinheiro
CommissioningEditor
VeenaPagare
AcquisitionEditor
RahulNair
ContentDevelopmentEditor
AnishSukumaran
TechnicalEditor
SaurabhMalhotra
CopyEditor
TrishyaHajare
ProjectCoordinator
IzzatContractor
Proofreader
SafisEditing
Indexers
MariammalChettiyar
PriyaSane
ProductionCoordinator
ShantanuN.Zagade
CoverWork
ShantanuN.Zagade
![Page 15: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/15.jpg)
![Page 16: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/16.jpg)
AbouttheAuthorRussMcKendrickisanexperiencedsolutionsarchitectwhohasbeenworkinginITandIT-relatedindustriesforthebetterpartof23years.Duringhiscareer,hehashadvariedresponsibilitiesinanumberofindustries,rangingfromlookingafterentireITinfrastructurestoprovidingfirstline,secondline,andseniorsupportinclientfacing,andinternalteamsforcorporateorganizations.
HeworksalmostexclusivelywithLinux,usingopensourcesystemsandtoolsonvariousplatformsrangingfromdedicatedhardwareandvirtualmachinestopublicclouds.
![Page 17: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/17.jpg)
![Page 18: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/18.jpg)
AbouttheReviewerMarceloCorreiaPinheiroisaBraziliansoftwareengineerfromPortoAlegre.Hestartedtoworkasawebdesignerandprogrammerin2000withASPandPHP,naturallygettingintouchwiththeMicrosoft.NETframeworkandJavarunningrespectivedatabasesofchoiceforwebapplications.Since2003,hehasusedLinuxandUNIX-relatedoperationalsystems,fromSlackwaretoGoboLinux,Archlinux,CentOS,Debian,andtodayOSX,havingsomecontactwithBSDdistributionstoo.HehaslostsomenightscompilingandapplyingpatchestotheLinuxkerneltomakeitsdesktopwork.Sincethebeginning,hehasbeenactingasaproblemsolver,nomatterwhattheprogramminglanguage,database,orplatformis—opensourceenthusiast.
Afterafewyears,hedecidedtoliveinSãoPaulotoworkwithnewertechnologiessuchasNoSQL,cloudcomputing,andRuby,wherehestartedtoconducttechtalkswiththislanguageinLocaweb.HecreatedsometoolstostandardizedevelopmentusingtoolssuchasvagrantandRubygems—someoftheseintheirGitHub—inLocawebtoensurefastapplicationpackagingandreduceddeploymentrollbacks.In2013,hechangedhiscareertobeafull-stackdeveloperfollowingtheDevOpsmovement.Since2012,hehasattended,asaspeaker,someofthebiggestsoftwareconferencesinBrazil—RSonRails,QConSP,TheDeveloper’sConference,andRubyConfBrazil—talkingnotonlyaboutRuby,butalsoaboutsomeofthewell-knownDevOpstoolssuchasTerraform,Packer,Ansible,andDocker.Today,heworksasaDevOpsconsultantintheircompany.
Inhisfreetime,helovesplayingtheguitar,havingsomefunwithcats,traveling,anddrinkingbeer.Hecanbefoundonhisblog(http://salizzar.net),Twitter(https://twitter.com/salizzar),GitHub(https://github.com/salizzar)andLinkedin(https://www.linkedin.com/in/salizzar).
HehasworkedasareviewerforVagrantVirtualDevelopmentEnvironmentCookbook,aPacktPublishingbookwithusefulrecipesusingvagrantwithconfigurationmanagementtoolssuchasPuppet,Chef,Ansible,andSaltStack.
Iwanttothankallmyfriends,whobelievedinmypotentialsincethebeginningandwhostillfollowmedespitethedistance.Iwouldalsoliketothankmymentors,GleiconMoraes,RobertoGaiser,andRodrigoCampos,whogavemetheincentiveandtipstobeabettersoftwareengineerandperson.
![Page 19: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/19.jpg)
![Page 20: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/20.jpg)
www.PacktPub.com
![Page 21: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/21.jpg)
Supportfiles,eBooks,discountoffers,andmoreForsupportfilesanddownloadsrelatedtoyourbook,pleasevisitwww.PacktPub.com.
DidyouknowthatPacktofferseBookversionsofeverybookpublished,withPDFandePubfilesavailable?YoucanupgradetotheeBookversionatwww.PacktPub.comandasaprintbookcustomer,youareentitledtoadiscountontheeBookcopy.Getintouchwithusat<[email protected]>formoredetails.
Atwww.PacktPub.com,youcanalsoreadacollectionoffreetechnicalarticles,signupforarangeoffreenewslettersandreceiveexclusivediscountsandoffersonPacktbooksandeBooks.
https://www2.packtpub.com/books/subscription/packtlib
DoyouneedinstantsolutionstoyourITquestions?PacktLibisPackt’sonlinedigitalbooklibrary.Here,youcansearch,access,andreadPackt’sentirelibraryofbooks.
![Page 22: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/22.jpg)
Whysubscribe?FullysearchableacrosseverybookpublishedbyPacktCopyandpaste,print,andbookmarkcontentOndemandandaccessibleviaawebbrowser
![Page 23: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/23.jpg)
FreeaccessforPacktaccountholdersIfyouhaveanaccountwithPacktatwww.PacktPub.com,youcanusethistoaccessPacktLibtodayandview9entirelyfreebooks.Simplyuseyourlogincredentialsforimmediateaccess.
![Page 24: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/24.jpg)
![Page 25: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/25.jpg)
PrefaceWiththeincreaseintheadoptionofDockercontainers,theneedtomonitorwhichcontainersarerunning,whatresourcestheyareconsuming,andhowitaffectstheoverallperformanceofthesystem,hasbecomeatime-relatedneed.MonitoringDockerwillteachyouhowmonitoringcontainersandkeepingakeeneyeontheworkingofapplicationshelptoimprovetheoverallperformanceoftheapplicationsthatrunonDocker.
ThisbookwillcovermonitoringcontainersusingDocker’snativemonitoringfunctions,variousplugins,andalsothird-partytoolsthathelpinmonitoring.Thebookwillfirstcoverhowtoobtaindetailedstatsfortheactivecontainers,resourcesconsumed,andcontainerbehavior.Thisbookwillalsoshowthereadershowtousethesestatstoimprovetheoverallperformanceofthesystem.
![Page 26: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/26.jpg)
WhatthisbookcoversChapter1,IntroductiontoDockerMonitoring,discusseshowdifferentitistomonitorcontainerscomparedtomoretraditionalserverssuchasvirtualmachines,baremetalmachines,andcloudinstances(PetsversusCattleandChickensversusSnowflakes).Thischapteralsodetailstheoperatingsystemscoveredintheexampleslaterinthisbookandalsogivesalittleinformationonhowtogetalocaltestenvironmentupandrunningusingvagrant,sothatinstallationinstructionsandpracticalexamplescanbeeasilyfollowed.
Chapter2,UsingtheBuilt-inTools,helpsyoulearnaboutthebasicmetricsyoucangetoutofthevanillaDockerinstallationandhowyoucanusethem.Also,wewillunderstandhowtogetreal-timestatisticsonourrunningcontainers,howtousecommandsthatarefamiliartous,andhowtogetinformationontheprocessesthatarelaunchedaspartofeachcontainer.
Chapter3,AdvancedContainerResourceAnalysis,introducescAdvisorfromGoogle,whichaddsalotmoreprecisiontothebasictoolsprovidedbyDocker.YouwillalsolearnhowtoinstallcAdvisorandstartcollectingmetrics.
Chapter4,ATraditionalApproachtoMonitoringContainers,looksatatraditionaltoolformonitoringservices.Bytheendofthischapter,youshouldknowyourwayaroundZabbixandthevariouswaysyoucanmonitoryourcontainers.
Chapter5,QueryingwithSysdig,describesSysdigas“anopensource,system-levelexplorationtooltocapturesystemstateandactivityfromarunningLinuxinstance,thensave,filter,andanalyzeit.”Inthischapter,youwilllearnhowtouseSysdigtobothviewyourcontainers’performancemetricsinrealtimeandalsorecordsessionstoquerylater.
Chapter6,ExploringThirdPartyOptions,walksyouthroughafewoftheSoftwareasaService(SaaS)optionsthatareavailable,whyyouwouldusethem,andhowtoinstalltheirclientsonthehostserver.
Chapter7,CollectingApplicationLogsfromwithintheContainer,looksathowwecangetthecontentofthelogfilesfortheapplicationsrunningwithinourcontainerstoacentrallocationsothattheyareavailableevenifyouhavetodestroyandreplaceacontainer.
Chapter8,WhatAretheNextSteps?,looksatthenextstepsyoucantakeinmonitoringyourcontainersbytalkingaboutthebenefitsofaddingalertingtoyourmonitoring.Also,wewillcoversomedifferentscenariosandlookatwhichtypeofmonitoringisappropriateforeachofthem.
![Page 27: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/27.jpg)
![Page 28: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/28.jpg)
WhatyouneedforthisbookToensuretheexperienceisasconsistentaspossible,wewillbeinstallingvagrantandVirtualBoxtorunthevirtualmachinethatwillactasahosttorunourcontainers.VagrantisavailableforLinux,OSX,andWindows;fordetailsonhowtoinstallthis,seethevagrantwebsiteathttps://www.vagrantup.com/.ThedetailsofhowtodownloadandinstallVirtualBoxcanbefoundathttps://www.virtualbox.org/;again,VirtualBoxcanbeinstalledonLinux,OSX,andWindows.
![Page 29: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/29.jpg)
![Page 30: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/30.jpg)
WhothisbookisforThisbookisforDevOpsengineersandsystemadministratorswhowanttomanageDockercontainers,bettermanagethesecontainersusingexperttechniquesandmethods,andbettermaintainapplicationsbuiltonDocker.
![Page 31: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/31.jpg)
![Page 32: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/32.jpg)
ConventionsInthisbook,youwillfindanumberoftextstylesthatdistinguishbetweendifferentkindsofinformation.Herearesomeexamplesofthesestylesandanexplanationoftheirmeaning.
Codewordsintext,databasetablenames,foldernames,filenames,fileextensions,pathnames,dummyURLs,userinput,andTwitterhandlesareshownasfollows:“Wecanincludeothercontextsthroughtheuseoftheincludedirective.”
Ablockofcodeissetasfollows:
{
"fields":{
"@timestamp":[
1444567706641
]
},
"sort":[
1444567706641
]
}
Whenwewishtodrawyourattentiontoaparticularpartofacodeblock,therelevantlinesoritemsaresetinbold:
{
"fields":{
"@timestamp":[
1444567706641
]
},
"sort":[
1444567706641
]
}
Anycommand-lineinputoroutputiswrittenasfollows:
cd~/Documents/Projects/monitoring-docker/vagrant-ubuntu
vagrantup
Newtermsandimportantwordsareshowninbold.Wordsthatyouseeonthescreen,forexample,inmenusordialogboxes,appearinthetextlikethis:“ClickingtheNextbuttonmovesyoutothenextscreen.”
NoteWarningsorimportantnotesappearinaboxlikethis.
TipTipsandtricksappearlikethis.
![Page 33: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/33.jpg)
![Page 34: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/34.jpg)
ReaderfeedbackFeedbackfromourreadersisalwayswelcome.Letusknowwhatyouthinkaboutthisbook—whatyoulikedordisliked.Readerfeedbackisimportantforusasithelpsusdeveloptitlesthatyouwillreallygetthemostoutof.
Tosendusgeneralfeedback,simplye-mail<[email protected]>,andmentionthebook’stitleinthesubjectofyourmessage.
Ifthereisatopicthatyouhaveexpertiseinandyouareinterestedineitherwritingorcontributingtoabook,seeourauthorguideatwww.packtpub.com/authors.
![Page 35: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/35.jpg)
![Page 36: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/36.jpg)
CustomersupportNowthatyouaretheproudownerofaPacktbook,wehaveanumberofthingstohelpyoutogetthemostfromyourpurchase.
![Page 37: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/37.jpg)
DownloadingtheexamplecodeYoucandownloadtheexamplecodefilesfromyouraccountathttp://www.packtpub.comforallthePacktPublishingbooksyouhavepurchased.Ifyoupurchasedthisbookelsewhere,youcanvisithttp://www.packtpub.com/supportandregistertohavethefilese-maileddirectlytoyou.
![Page 38: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/38.jpg)
DownloadingthecolorimagesofthisbookWealsoprovideyouwithaPDFfilethathascolorimagesofthescreenshots/diagramsusedinthisbook.Thecolorimageswillhelpyoubetterunderstandthechangesintheoutput.Youcandownloadthisfilefrom:http://www.packtpub.com/sites/default/files/downloads/Monitoring_Docker_ColorImages.pdf
![Page 39: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/39.jpg)
ErrataAlthoughwehavetakeneverycaretoensuretheaccuracyofourcontent,mistakesdohappen.Ifyoufindamistakeinoneofourbooks—maybeamistakeinthetextorthecode—wewouldbegratefulifyoucouldreportthistous.Bydoingso,youcansaveotherreadersfromfrustrationandhelpusimprovesubsequentversionsofthisbook.Ifyoufindanyerrata,pleasereportthembyvisitinghttp://www.packtpub.com/submit-errata,selectingyourbook,clickingontheErrataSubmissionFormlink,andenteringthedetailsofyourerrata.Onceyourerrataareverified,yoursubmissionwillbeacceptedandtheerratawillbeuploadedtoourwebsiteoraddedtoanylistofexistingerrataundertheErratasectionofthattitle.
Toviewthepreviouslysubmittederrata,gotohttps://www.packtpub.com/books/content/supportandenterthenameofthebookinthesearchfield.TherequiredinformationwillappearundertheErratasection.
![Page 40: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/40.jpg)
PiracyPiracyofcopyrightedmaterialontheInternetisanongoingproblemacrossallmedia.AtPackt,wetaketheprotectionofourcopyrightandlicensesveryseriously.IfyoucomeacrossanyillegalcopiesofourworksinanyformontheInternet,pleaseprovideuswiththelocationaddressorwebsitenameimmediatelysothatwecanpursuearemedy.
Pleasecontactusat<[email protected]>withalinktothesuspectedpiratedmaterial.
Weappreciateyourhelpinprotectingourauthorsandourabilitytobringyouvaluablecontent.
![Page 41: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/41.jpg)
QuestionsIfyouhaveaproblemwithanyaspectofthisbook,youcancontactusat<[email protected]>,andwewilldoourbesttoaddresstheproblem.
![Page 42: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/42.jpg)
![Page 43: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/43.jpg)
Chapter1.IntroductiontoDockerMonitoringDockerhasbeenarecentbutveryimportantadditiontoaSysAdminstoolbox.
Dockerdescribesitselfasanopenplatformforbuilding,shipping,andrunningdistributedapplications.Thismeansthatdeveloperscanbundletheircodeandpassittotheiroperationsteam.Fromhere,theycandeploysafeintheknowledgethatitwillbedonesoinawaythatintroducesconsistencywiththeenvironmentinwhichthecodeisrunning.
Whenthisprocessisfollowed,itshouldmaketheage-olddevelopersversusoperationsargumentof“itworkedonmylocaldevelopmentserver”—athingofthepast.Sincebeforeits“productionready”1.0releasebackinJune2014,therehadbeenover10,000Dockerizedapplicationsavailable.Bytheendof2014,thatnumberhadrisentoover71,000.YoucanseehowDockergrewin2014bylookingattheinfographicthatwaspublishedbyDockerinearly2015,whichcanbefoundathttps://blog.docker.com/2015/01/docker-project-2014-a-whirlwind-year-in-review/.
Whilethedebateisstillragingabouthowproductionreadythetechnologyis,Dockerhasgainedanimpressivelistoftechnologypartners,includingRedHat,Canonical,HP,andevenMicrosoft.
CompaniessuchasGoogle,Spotify,Soundcloud,andCenturyLink,haveallopensourcedtoolsthatsupportDockerinsomeway,shape,orformandtherehasalsobeennumerousindependentdeveloperswhohavereleasedappsthatprovideadditionalfunctionalitytothecoreDockerproductset.Also,allthecompanieshavesprunguparoundtheDockerecosystem.
Thisbookassumesthatyouhavehadsomelevelofexperiencebuilding,running,andmanagingDockercontainers,andthatyouwouldnowliketostarttometricsfromyourrunningapplicationstofurthertunethem,orthatyouwouldliketoknowwhenaproblemoccurswithacontainersothatyoucandebuganyongoingissues.
IfyouhaveneverusedDockerbefore,youmaywanttotryoneoftheexcellentbooksthatserveandintroduceyoutoallthethingsthatDockerprovides,bookssuchasLearningDocker,PacktPublishing,orDocker’sownintroductiontocontainers,whichcanbefoundattheirdocumentationpages,asfollows:
LearningDocker:https://www.packtpub.com/virtualization-and-cloud/learning-dockerOfficialDockerdocs:https://docs.docker.com/
Now,wehaveabroughtourselvesuptospeedwithwhatDockeris;therestofthischapterwillcoverthefollowingtopics:
Howdifferentisittomonitorcontainersversusmoretraditionalserverssuchasvirtualmachines,baremetalmachine,andcloudinstances(Pets,Cattle,Chickens,andSnowflakes).
![Page 44: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/44.jpg)
WhataretheminimumversionsofDockeryoushouldberunning?HowtofollowinstructionsonbringingupanenvironmentlocallyusingVagrantinordertofollowthepracticalexercisesinthisbook
![Page 45: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/45.jpg)
Pets,Cattle,Chickens,andSnowflakesBeforewestartdiscussingthevariouswaysinwhichyoucanmonitoryourcontainers,weshouldgetanunderstandingofwhataSysAdminsworldlookslikethesedaysandalsowherecontainersfitintoit.
AtypicalSysAdminwillprobablybelookingafteranestateofserversthatarehostedineitheranon-siteorthird-partydatacenter,somemayevenmanageinstanceshostedinapubliccloudsuchasAmazonWebServicesorMicrosoftAzure,andsomeSysAdminsmayjugglealltheirserverestatesacrossmultiplehostingenvironments.
Eachofthesedifferentenvironmentshasitsownwayofdoingthings,aswellasperformingbestpractices.BackinFebruary2012,RandyBiasgaveatalkatCloudscalingthatdiscussedarchitecturesforopenandscalableclouds.Towardstheendoftheslidedeck,RandyintroducedtheconceptofPetsversusCattle(whichheattributestoBillBaker,whowasthenanengineeratMicrosoft).
Youcanviewtheoriginalslidedeckathttp://www.slideshare.net/randybias/architectures-for-open-and-scalable-clouds.
PetsversusCattleisnowwidelyacceptedasagoodanalogytodescribemodernhostingpractices.
![Page 46: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/46.jpg)
PetsPetsareakintotraditionalphysicalserversorvirtualmachines,asfollows:
Eachpethasaname;forexample,myserver.domain.com.Whenthey’renotwell,youtakethemtothevettohelpthemgetbetter.YouemploySysAdminstolookafterthem.Youpaycloseattentiontothem,sometimesforyears.Youtakebackups,patchthem,andensurethattheyarefullydocumented.
![Page 47: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/47.jpg)
CattleCattle,ontheotherhand,representmoremoderncloudcomputinginstances,asfollows:
You’vegottoomanytoname,soyougivethemnumbers;forexample,theURLcouldlooksomethinglikeip123123123123.eu.public-cloud.com.Whentheygetsick,youshootthemandifyourherdrequiresit,youreplaceanythingyou’vekilled:Aservercrashesorshowssignsthatitishavingproblems,youterminateitandyourconfigurationautomaticallyreplacesitwithanexactreplica.Youputtheminafieldandwatchthemfromfarandyoudon’texpectthemtolivelong.Ratherthanmonitoringtheindividualinstances,youmonitorthecluster.Whenmoreresourcesareneeded,youaddmoreinstancesandoncetheresourceisnolongerrequired,youterminatetheinstancestogetyoubacktoyourbaseconfiguration.
![Page 48: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/48.jpg)
ChickensNextupisatermthatisagoodwayofdescribinghowcontainersfitintothePetsversusCattleworld;inablogposttitle“CloudComputing:Pets,Cattleand…Chickens?”onActiveState,BernardGoldendescribescontainersasChickens:
They’remoreefficientthancattlewhenitcomestoresourceuse.Acontainercanbootinsecondswhereainstanceorservercantakeminutes;italsouseslessCPUpowerthanatypicalvirtualmachineorcloudinstance.Therearemanymorechickensthancattle.Youcanquitedenselypackcontainersontoyourinstancesorservers.Chickenstendtohaveashorterlifespanthancattleandpets.Containerslendthemselvestorunningmicros-services;thesecontainersmayonlybeactiveforafewminutes.
Theoriginalblogpostcanbefoundathttp://www.activestate.com/blog/2015/02/cloud-computing-pets-cattle-and-chickens.
![Page 49: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/49.jpg)
SnowflakesThefinaltermisnotanimal-relatedanditdescribesatypeofserverthatyoudefiantlydon’twanttohaveinyourserverestate,aSnowflake.ThistermwaspennedbyMartinFowlerinablogposttitled“SnowflakeServer”.Snowflakesisatermappliedto“legacy”or“inherited”servers:
Snowflakesaredelicateandaretreatedwithkidgloves.Typically,theserverhasbeeninthedatacentersinceyoustarted.Nooneknowswhooriginallyconfigureditandthereisnodocumentationofit;allyouknowisthatitisimportant.Eachoneisuniqueandisimpossibletoexactlyreproduce.EventhemosthardenedSysAdminfearstorebootthemachineincaseitdoesn’tbootafterwards,asitisrunningend-of-lifesoftwarethatcannoteasilybereinstalled.
Martin’spostcanbefoundathttp://martinfowler.com/bliki/SnowflakeServer.html.
![Page 50: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/50.jpg)
Sowhatdoesthisallmean?Dependingonyourrequirementsandtheapplicationyouwanttodeploy,yourcontainerscanbelaunchedontoeitherpetorcattlestyleservers.Youcanalsocreateaclutchofchickensandhaveyourcontainersrunmicro-services.
Also,intheory,youcanreplaceyourfearedsnowflakeserverswithacontainer-basedapplicationthatmeetsalltheend-of-lifesoftwarerequirementswhileremainingdeployableonamodernsupportableplatform.
Eachofthedifferentstylesofserverhasdifferentmonitoringrequirements,inthefinalchapterwewilllookatPets,Cattle,Chickens,andSnowflakesagainanddiscussthetoolswehavecoveredinthecomingchapters.Wewillalsocoverbestpracticesyoushouldtakeintoconsiderationwhenplanningyourmonitoring.
![Page 51: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/51.jpg)
![Page 52: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/52.jpg)
DockerWhileDockerhititsversion1.0milestoneoverayearago,itisstillinit’sinfancy;witheachnewreleasecomesnewfeatures,bugfixes,andevensupportforsomeearlyfunctionalitythatisbeingdepreciated.
Dockeritselfisnowacollectionofseveralsmallerprojects;theseincludethefollowing:
DockerEngineDockerMachineDockerComposeDockerSwarmDockerHubDockerRegistryKitmatic
Inthisbook,wewillbeusingDockerEngine,DockerCompose,andtheDockerHub.
DockerEngineisthecorecomponentoftheDockerprojectanditprovidesthemainbulkoftheDockerfunctionality.WheneverDockerorthedockercommandismentionedinthisbook,IwillbereferringtoDockerEngine.
ThebookassumesyouhaveDockerEngineversion1.71orlaterinstalled;olderversionsofDockerEnginemaynotcontainthenecessaryfunctionalityrequiredtorunthecommandsandsoftwarecoveredintheupcomingchapters.
DockerComposestarteditslifeasathird-partyorchestrationtoolcalledFigbeforebeingpurchasedbyDockerin2014.Itisdescribedasawayofdefiningamulti-containerapplicationusingYAML(http://yaml.org).Simplyput,thismeansthatyouquicklydeploycomplexapplicationsusingasinglecommandthatcallsahumanreadableconfigurationfile.
WeassumethatyouhaveDockerCompose1.3.3orlaterinstalled;thedocker-compose.ymlfilesmentionedinthisbookhavebeenwrittenwiththisversioninmind.
Finally,themajorityoftheimageswewillbedeployingduringthisbookwillbesourcedfromtheDockerHub(https://hub.docker.com/),whichnotonlyhousesapublicregistrycontainingover40,000publicimagesbutalso100officialimages.ThefollowingscreenshotshowstheofficialrepositorieslistingontheDockerHubwebsite:
![Page 53: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/53.jpg)
YoucanalsosignupandusetheDockerHubtohostyourownpublicandprivateimages.
![Page 54: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/54.jpg)
![Page 55: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/55.jpg)
LaunchingalocalenvironmentWhereverpossible,Iwilltrytoensurethatthepracticalexercisesinthisbookwillbeabletoberunonalocalmachinesuchasyourdesktoporlaptop.Forthepurposesofthisbook,IwillassumethatyourlocalmachineisrunningeitherarecentversionOSXoranup-to-dateLinuxdistributionandhasahighenoughspecificationtorunthesoftwarementionedinthischapter.
ThetwotoolswewillbeusingtolaunchourDockerinstanceswillalsorunonWindows;therefore,itshouldbepossibletofollowtheinstructionswithinthis,althoughyoumayhavetorefertheusageguidesforanychangestothesyntax.
DuetothewayinwhichDockerisarchitected,alotofthecontentofthisbookwillhaveyourunningcommandsandinteractingwiththecommandlineonthevirtualserverthatisactingasthehostmachine,ratherthanthecontainersthemselves.Becauseofthis,wewillnotbeusingeitherDockerMachineorKitematic.
BothofthesearetoolsprovidedbyDockertoquicklybootstrapaDocker-enabledvirtualserveronyourlocalmachine,asunfortunatelythehostmachinesdeployedbythesetoolscontainastrippeddownoperatingsystemthatisoptimizedforrunningDockerwiththesmallestfootprintaspossible.
Aswewillbeinstallingadditionalpackagesonthehostmachines,astrippeddown“Dockeronly”operatingsystemmaynothavethecomponentsavailabletomeettheprerequisitesofthesoftwarethatwewillberunninginthelaterchapters;therefore,toensurethattherearenoproblemsfurtheron,weberunningafulloperatingsystem.
Personally,IpreferaRPM-basedoperatingsystemsuchasRedHatEnterpriseLinux,Fedora,orCentOS,asIhavebeenusingthemprettymuchsincethedayIfirstloggedintoaLinuxserver.
However,asalotofreaderswillbefamiliarwiththeDebian-basedUbuntu,Iwillbeprovidingpracticalexamplesforbothoperatingsystems.
Toensuretheexperienceisasconsistentaspossible,wewillbeinstallingVagrantandVirtualBoxtorunthevirtualmachinethatwillactasahosttorunourcontainers.
Vagrant,writtenbyMitchellHashimoto,isacommandlinetoolforcreatingandconfiguringreproducibleandportablevirtualmachineenvironments.TherehavebeennumerousblogpostsandarticlesthatactuallypitchDockeragainstVagrant;however,inourcase,thetwotechnologiesworkquitewelltogetherinprovidingarepeatableandconsistentenvironment.
VagrantisavailableforLinux,OSX,andWindows.Fordetailsonhowtoinstall,gototheVagrantwebsiteathttps://www.vagrantup.com/.
VirtualBoxisagreatallroundopensourcevirtualizationplatformoriginallydevelopedbySunandnowmaintainedbyOracle.Itallowsyoutorunboth32-bitand64-bitguestoperatingsystemsonyourlocalmachine.DetailsonhowtodownloadandinstallVirtualBoxcanbefoundathttps://www.virtualbox.org/;again,VirtualBoxcanbeinstalled
![Page 56: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/56.jpg)
onLinux,OSX,andWindows.
![Page 57: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/57.jpg)
![Page 58: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/58.jpg)
CloningtheenvironmentThesourcefortheenvironmentalongwiththepracticalexamplescanbefoundonGitHubintheMonitoringDockerrepositoryathttps://github.com/russmckendrick/monitoring-docker.
Toclonetherepositoryonaterminalonyourlocalmachine,runthefollowingcommands(replacingthefilepathasneeded):
mkdir~/Documents/Projects
cd~/Documents/Projects/
gitclonehttps://github.com/russmckendrick/monitoring-docker.git
Oncecloned,youshouldseeadirectorycalledmonitoring-dockerandthenenterthatdirectory,asfollows:
cd~/Documents/Projects/monitoring-docker
![Page 59: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/59.jpg)
![Page 60: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/60.jpg)
RunningavirtualserverIntherepository,youwillfindtwofolderscontainingthenecessaryVagrantfiletolauncheitheraCentOS7oraUbuntu14.04virtualserver.
IfyouwouldliketousetheCentOS7vagrantbox,changethedirectorytovagrant-centos:
cdvagrant-centos
Onceyouareinthevagrant-centosdirectory,youwillseethatthereisaVagrantfile;thisfileisallyouneedtolaunchaCentOS7virtualserver.Afterthevirtualserverhasbeenbooted,thelatestversionofdockeranddocker-composewillbeinstalledandthemonitoring-dockerdirectorywillalsobemountedinsidethevirtualmachineusingthemountpoint/monitoring-docker.
Tolaunchthevirtualserver,simplytypethefollowingcommand:
vagrantup
Thiswilldownloadthelatestversionofthevagrantboxfromhttps://atlas.hashicorp.com/russmckendrick/boxes/centos71andthenbootthevirtualserver;it’sa450MBdownloadsoitmaytakeseveralminutestodownload;itonlyhastodothisonce.
Ifallgoeswell,youshouldseesomethingsimilartothefollowingoutput:
![Page 61: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/61.jpg)
Nowthatyouhavebootedthevirtualserver,youcanconnecttoitusingthefollowingcommand:
vagrantssh
Onceloggedin,youshouldverifythatdockeranddocker-composearebothavailable:
Finally,youcantryrunningthehello-worldcontainerusingthefollowingcommand:
dockerrunhello-world
Ifeverythinggoesasexpected,youshouldseethefollowingoutput:
Totrysomethingmoreambitious,youcanrunanUbuntucontainerwiththefollowingcommand:
dockerrun-itubuntubash
BeforewelaunchandentertheUbuntucontainer,letsconfirmthatwearerunningtheCentOShostmachinebycheckingthereleasefilethatcanbefoundin/etc:
![Page 62: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/62.jpg)
Now,wecanlaunchtheUbuntucontainer.Usingthesamecommand,wecanconfirmthatweareinsidetheUbuntucontainerbyviewingitsreleasefile:
Toexitthecontainerjusttypeinexit.Thiswillstopthecontainerfromrunning,asithasterminatedtheonlyrunningprocesswithinthecontainer,whichwasbash,andreturnedyoutothehostCentOSmachine.
AsyoucanseeherefromourCentOS7host,wehavelaunchedandremovedanUbuntucontainer.
BoththeCentOS7andUbuntuVagrantfileswillconfigureastaticIPaddressonyourvirtualmachine.Itis192.168.33.10;also,thereisaDNSrecordforthisIPaddress
![Page 63: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/63.jpg)
availableatdocker.media-glass.es.Thesewillallowyoutoaccessanycontainersthatexposethemselvestoabrowserateitherhttp://192.168.33.10/orhttp://docker.media-glass.es/.
TipTheURLhttp://docker.media-glass.es/willonlyworkwhilethevagrantboxisup,andyouhaveacontainerrunningwhichservesWebpages.
Youcanseethisinactionbyrunningthefollowingcommand:
dockerrun-d-p80:80russmckendrick/nginx-php
TipDownloadingtheexamplecode
Youcandownloadtheexamplecodefilesfromyouraccountathttp://www.packtpub.comforallthePacktPublishingbooksyouhavepurchased.Ifyoupurchasedthisbookelsewhere,youcanvisithttp://www.packtpub.com/supportandregistertohavethefilese-maileddirectlytoyou.
ThiswilldownloadandlaunchacontainerrunningNGINX.Youcanthengotohttp://192.168.33.10/orhttp://docker.media-glass.es/inyourbrowser;youshouldseeaforbiddenpage.ThisisbecausewehavenotyetgivenNGINXanycontenttoserve(moreonthiswillbecoveredlaterinthebook):
Formoreexamplesandideas,gotothewebsiteathttp://docs.docker.com/userguide/.
![Page 64: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/64.jpg)
![Page 65: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/65.jpg)
HaltingthevirtualserverTologoutofthevirtualserverandreturntoyourlocalmachine,youtypeexit.
Youshouldnowseeyourlocalmachine’sterminalprompt;however,thevirtualserveryoubootedwillstillberunninginthebackgroundhappily,usingresources,untilyoueitherpoweritdownusingthefollowingcommand:
vagranthalt
Terminatethevirtualserveraltogetherusingvagrantdestroy:
vagrantdestroy
Tocheckthecurrentstatusofthevirtualserver,youcanrunthefollowingcommand:
vagrantstatus
Theresultoftheprecedingcommandisgiveninthefollowingoutput:
Eitherpoweringthevirtualserverbackonorcreatingitfromscratchagain,canbeachievedbyissuingthevagrantupcommandagain.
TheprecedingdetailsshowhowtousetheCentOS7vagrantbox.IfyouwouldprefertolaunchanUbuntu14.04virtualserver,youcandownloadandinstallthevagrantboxbygoingintothevagrant-ubuntudirectoryusingthefollowingcommand:
cd~/Documents/Projects/monitoring-docker/vagrant-ubuntu
vagrantup
Fromhere,youwillbeablerunvagrantupandfollowthesameinstructionsusedtobootandinteractwiththeCentOS7virtualserver.
![Page 66: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/66.jpg)
![Page 67: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/67.jpg)
SummaryInthischapter,wetalkedaboutdifferenttypesofserverandalsodiscussedhowyourcontainerizedapplicationscanfitintoeachofthecategories.WehavealsoinstalledVirtualBoxandusedVagranttolauncheitheraCentOS7orUbuntu14.04virtualserver,withdockeranddocker-composeinstalled.
Ournewvirtualserverenvironmentwillbeusedthroughouttheupcomingchapterstotestthevariousdifferenttypesofmonitoring.Inthenextchapter,wewillstartourjourneybyusingDocker’sin-builtfunctionalitytoexploremetricsaboutourrunningcontainers.
![Page 68: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/68.jpg)
![Page 69: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/69.jpg)
Chapter2.UsingtheBuilt-inToolsInthelaterchaptersofthisbook,wewillexplorethemonitoringpartsofthelargeeco-systemthathasstartedtoflourisharoundDockeroverthelast24months.However,beforewepressaheadwiththat,weshouldtakealookatwhatispossiblewithavanillainstallationofDocker.Inthischapter,wewillcoverthefollowingtopics:
UsingthetoolsbuiltintoDockertogetreal-timemetricsoncontainerperformanceUsingstandardoperatingsystemcommandstogetmetricsonwhatDockerisdoingGeneratingatestloadsoyoucanviewthemetricschanging
![Page 70: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/70.jpg)
DockerstatsSinceversion1.5,therehasbeenabasicstatisticcommandbuiltintoDocker:
dockerstats--help
Usage:dockerstats[OPTIONS]CONTAINER[CONTAINER…]
Displayalivestreamofoneormorecontainers'resourceusagestatistics
--help=falsePrintusage
--no-stream=falseDisablestreamingstatsandonlypullthefirst
result
Thiscommandwillstreamdetailsoftheresourceutilizationofyourcontainersinrealtime.Thebestwaytofindoutaboutthecommandistoseeitinaction.
![Page 71: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/71.jpg)
RunningDockerstatsLet’slaunchacontainerusingthevagrantenvironment,whichwecoveredinthelastchapter:
[russ@mac~]$cd~/Documents/Projects/monitoring-docker/vagrant-centos/
[russ@mac~]$vagrantup
Bringingmachine'default'upwith'virtualbox'provider…
==>default:Importingbasebox'russmckendrick/centos71'...
==>default:MatchingMACaddressforNATnetworking…
==>default:Checkingifbox'russmckendrick/centos71'isuptodate…
.....
==>default:=>Installingdocker-engine…
==>default:=>Configuringvagrantuser…
==>default:=>Startingdocker-engine…
==>default:=>Installingdocker-compose…
==>default:=>FinishedinstallationofDocker
[russ@mac~]$vagrantssh
Nowthatyouareconnectedtothevagrantserver,launchthecontainerusingtheDockercomposefilein/monitoring_docker/Chapter01/01-basic/:
[vagrant@centos7~]$cd/monitoring_docker/Chapter01/01-basic/
[vagrant@centos701-basic]$docker-composeup-d
Creating01basic_web_1…
Youhavenowpulleddownandlaunchedacontainerinthebackground.Thecontaineriscalled01basic_web_1anditrunsNGINXandPHPservingasinglePHPinformationpage(http://php.net/manual/en/function.phpinfo.php).
Tocheckwhethereverythinghasbeenlaunchedasexpected,rundocker-composeps.YoushouldseeyoursinglecontainerwithStateofUp:
[vagrant@centos701-basic]$docker-composeps
NameCommandStatePorts
---------------------------------------------------------------
01basic_web_1/usr/local/bin/runUp0.0.0.0:80->80/tcp
Finally,youshouldbeabletoseethepagecontainingtheoutputofthePHPinformationathttp://192.168.33.10/(thisIPaddressishardcodedintothevagrantconfiguration),ifyouputitinyourlocalbrowser:
![Page 72: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/72.jpg)
Now,youhaveacontainerupandrunning;let’slookatsomeofthebasicstats.Weknowfromtheoutputofdocker-composethatourcontaineriscalled01basic_web_1,soenterthefollowingcommandtostartstreamingstatisticsinyourterminal:
dockerstats01basic_web_1
Itwilltakeasecondtoinitiate;afterthisisdone,youshouldseeyourcontainerlistedalongwiththestatisticsforthefollowing:
CPU%:ThisshowsyouhowmuchoftheavailableCPUresourcethecontaineriscurrentlyusing.MEMUSEAGE/LIMIT:ThistellsyouhowmuchRAMthecontainerisutilizing;italsodisplayshowmuchallowancethecontainerhas.Ifyouhaven’texplicitlysetalimit,itwillshowthetotalamountofRAMonthehostmachine.MEM%:ThisshowsyouwhatpercentageoftheRAMallowancethecontainerisusing.
![Page 73: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/73.jpg)
NETI/O:Thisgivesarunningtotalofhowmuchbandwidthhasbeentransferredinandoutofthecontainer.
Ifyougobacktoyourbrowserwindowandstarttorefreshhttp://192.168.33.10/,youwillseethatthevaluesineachofthecolumnsstarttochange.Tostopstreamingthestatistics,pressCtrl+c.
Ratherthankeepingonhittingrefreshoverandoveragain,let’sgeneratealotoftrafficto01basic_web_1,whichshouldputthecontainerunderaheavyload.
Here,wewilllaunchacontainerthatwillsend10,000requeststo01basic_web_1usingApacheBench(https://httpd.apache.org/docs/2.2/programs/ab.html).Althoughitwilltakeaminuteortwotoexecute,weshouldrundockerstatsassoonaspossible:
dockerrun-d--name=01basic_load--link=01basic_web_1russmckendrick/abab
-k-n10000-c5http://01basic_web_1/&&dockerstats01basic_web_1
01basic_load
AftertheApacheBenchimagehasbeendownloadedandthecontainerthatwillbecalled01basic_loadstarts,youshouldseethestatisticsforboth01basic_web_1and01basic_loadbegintostreaminyourterminal:
CONTAINERCPU%MEMUSAGE/LIMITMEM%NETI/O
01basic_load18.11%12.71MB/1.905GB0.67%335.2MB/5.27MB
01basic_web_1139.62%96.49MB/1.905GB5.07%5.27MB/335.2MB
Afterawhile,youwillnoticethatmostofthestatisticsfor01basic_loadwilldropofftozero;thismeansthatthetesthascompletedandthatthecontainerrunningthetesthasexited.Thedockerstatscommandcanonlystreamstatisticsfortherunningcontainers;onesthathaveexitedarenolongerrunningand,therefore,donotproduceoutputwhenrunningdockerstats.
ExitfromdockerstatsusingCtrl+c;toseetheresultsoftheApacheBenchcommand,youcantypedockerlogs01basic_load;youshouldseesomethinglikethefollowingscreenshot:
![Page 74: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/74.jpg)
Youshouldn’tworryifyouseeanyfailureslikeintheprecedingoutput.ThisexercisewaspurelytodemonstratehowtoviewthestatisticsoftherunningcontainersandnottotuneawebservertohandletheamountoftrafficwesenttoitusingApacheBench.
Toremovethecontainersthatwelaunched,runthefollowingcommands:
[vagrant@centos701-basic]$docker-composestop
Stopping01basic_web_1…
[vagrant@centos701-basic]$docker-composerm
Goingtoremove01basic_web_1
Areyousure?[yN]y
Removing01basic_web_1…
[vagrant@centos701-basic]$dockerrm01basic_load
01basic_load
Tocheckwhethereverythinghasbeenremovedsuccessfully,rundockerps-aandyoushouldnotbeabletoseeanyrunningorexitedcontainersthathave01basic_intheirnames.
![Page 75: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/75.jpg)
![Page 76: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/76.jpg)
Whatjusthappened?WhilerunningtheApacheBenchtest,youmayhavenoticedthattheCPUutilizationonthecontainerrunningNGINXandPHPwashigh;intheexampleintheprevioussection,itwasusing139.62percentoftheavailableCPUresource.
Aswedidnotattachanyresourcelimitstothecontainerswelaunched,itwaseasyforourtesttousealloftheavailableresourcesonthehostVirtualMachine(VM).IfthisVMwasbeingusedbyseveralusers,allrunningtheirowncontainers,theymayhavestartedtonoticethattheirapplicationshadstartedtoslowdownor,evenworse,theapplicationshadstartedshowingerrors.
Ifyoueverfindyourselfinthissituation,youcanusedockerstatstohelptrackdowntheculprit.
Runningdockerstats$(dockerps-q)willstreamthestatisticsforallthecurrentlyrunningcontainers:
CONTAINERCPU%MEMUSAGE/LIMITMEM%NETI/O
361040b7b33e0.07%86.98MB/1.905GB4.57%2.514kB/738B
56b459ae9092120.06%87.05MB/1.905GB4.57%2.772kB/738B
a3de616f84ba0.04%87.03MB/1.905GB4.57%2.244kB/828B
abdbee7b52070.08%86.61MB/1.905GB4.55%3.69kB/738B
b85c49cf740c0.07%86.15MB/1.905GB4.52%2.952kB/738B
Asyoumayhavenoticed,thisdisplaysthecontainerIDratherthanthename;thisinformationshould,however,beenoughtospottheresourcehogsothatyoucanquicklystopit:
[vagrant@centos701-basic]$dockerstop56b459ae9092
56b459ae9092
Oncestopped,youcanthengetthenameoftheroguecontainerbyrunningthefollowingcommand:
[vagrant@centos701-basic]$dockerps-a|grep56b459ae9092
56b459ae9092russmckendrick/nginx-php"/usr/local/bin/run"9
minutesagoExited(0)26secondsagomy_bad_container
Alternatively,formoredetailedinformation,youcanrundockerinspect56b459ae9092,whichwillgiveyoualltheinformationyouneedonthecontainer.
![Page 77: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/77.jpg)
![Page 78: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/78.jpg)
Whataboutprocesses?OneofthegreatthingsaboutDockeristhatitisn’treallyvirtualization;asmentionedinthepreviouschapter,itisagreatwayofisolatingprocessesratherthanrunninganentireoperatingsystem.
Thiscangetconfusingwhenrunningtoolssuchastoporps.Togetanideajusthowconfusingthiscanget,letslaunchseveralcontainersusingdocker-composeandseeforourselves:
[vagrant@centos7~]$cd/monitoring_docker/Chapter01/02-multiple
[vagrant@centos702-multiple]$docker-composeup-d
Creating02multiple_web_1…
[vagrant@centos702-multiple]$docker-composescaleweb=5
Creating02multiple_web_2…
Creating02multiple_web_3…
Creating02multiple_web_4…
Creating02multiple_web_5…
Starting02multiple_web_2…
Starting02multiple_web_3…
Starting02multiple_web_4…
Starting02multiple_web_5…
Now,wehavefivewebserversthathaveallbeenlaunchedfromthesameimageusingthesameconfiguration.OneofthefirstthingsIdowhenloggingintoaservertotroubleshootaproblemisrunps-aux;thiswillshowalltherunningprocesses.Asyoucansee,whenrunningthecommand,therearealotprocesseslisted.
EvenjusttryingtolookattheprocessesforNGINXisconfusing,asthereisnothingtodifferentiatetheprocessesfromonecontainertoanother,asshowninthefollowingoutput:
![Page 79: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/79.jpg)
So,howcanyouknowwhichcontainerownswhichprocesses?
![Page 80: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/80.jpg)
DockertopThiscommandlistsalltheprocessesthatarerunningwithinacontainer;thinkofitasawayoffilteringtheoutputofthepsauxcommandweranonthehostmachine:
Asdockertopisanimplementationofthestandardpscommand,anyflagsyouwouldnormallypasstopsshouldworkasfollows:
[vagrant@centos702-multiple]$dockertop02multiple_web_3–aux
[vagrant@centos702-multiple]$dockertop02multiple_web_3-faux
![Page 81: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/81.jpg)
DockerexecAnotherwaytoviewwhatisgoingonwithinacontaineristoenterit.Toenableyoutodothis,Dockerintroducedthedockerexeccommand.Thisallowsyoutospawnanadditionalprocesswithinanalreadyrunningcontainerandthenattachtotheprocess;so,ifwewantedtolookatwhatiscurrentlyrunningon02multiple_web_3,weshouldusethefollowingcommandspawnabashshellwithinanalreadyrunningcontainer:
dockerexec-t-i02multiple_web_3bash
Onceyouhaveanactiveshellonthecontainer,youwillnoticethatyourprompthaschangedtothecontainer’sID.Yoursessionisnowisolatedtothecontainer’senvironment,meaningthatyouwillonlybeabletointeractwiththeprocessesbelongingtothecontaineryouentered.
Fromhere,youcanrunthepsauxortopcommandasyouwoulddoonthehostmachine,andonlyseetheprocessesassociatedwiththecontaineryouareinterestedin:
Toleavethecontainer,typeinexit,youshouldseeyourpromptchangebackinyourhostmachine.
Finally,youcanstopandremovethecontainersbyrunningdocker-composestopanddocker-composekill.
![Page 82: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/82.jpg)
![Page 83: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/83.jpg)
SummaryInthischapter,wesawhowwecangetreal-timestatisticsonourrunningcontainersandhowwecanusecommandsthatarefamiliartous,togetinformationontheprocessesthatarelaunchedaspartofeachcontainer.
Onthefaceofit,dockerstatsseemslikeareallybasicpieceoffunctionalitythatisn’treallyanythingmorethanatooltohelpyouidentifywhichcontainerisusingalltheresourceswhileaproblemisoccurring.However,theDockercommandisactuallypullingtheinformationfromaquitepowerfulAPI.
ThisAPIformsthebasisforalotofthemonitoringtoolswewillbelookingatinthenextfewchapters.
![Page 84: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/84.jpg)
![Page 85: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/85.jpg)
Chapter3.AdvancedContainerResourceAnalysisInthelastchapter,welookedathowyoucanusetheAPIbuiltintoDockertogainaninsighttowhatresourcesyourcontainersarerunning.Now,wearetoseehowwecantakeittothenextlevelbyusingcAdvisorfromGoogle.Inthischapter,youwillcoverthefollowingtopics:
HowtoinstallcAdvisorandstartcollectingmetricsLearnallaboutthewebinterfaceandreal-timemonitoringWhatyouroptionsareforshippingmetricstoaremotePrometheusdatabaseforlong-termstorageandtrendanalysis
![Page 86: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/86.jpg)
WhatiscAdvisor?GoogledescribescAdvisorasfollows:
“cAdvisor(ContainerAdvisor)providescontainerusersanunderstandingoftheresourceusageandperformancecharacteristicsoftheirrunningcontainers.Itisarunningdaemonthatcollects,aggregates,processes,andexportsinformationaboutrunningcontainers.Specifically,foreachcontainer,itkeepsresourceisolationparameters,historicalresourceusage,histogramsofcompletehistoricalresourceusage,andnetworkstatistics.Thisdataisexportedbyacontainerandismachine-wide.”
TheprojectstartedofflifeasaninternaltoolatGoogleforgaininganinsightintocontainersthathadbeenlaunchedusingtheirowncontainerstack.
NoteGoogle’sowncontainerstackwascalled“LetMeContainThatForYou”orlmctfyforshort.TheworkonlmctfyhasbeeninstalledasaGoogleportfunctionalityovertolibcontainerthatispartoftheOpenContainerInitiative.Furtherdetailsonlmctfycanbefoundathttps://github.com/google/lmctfy/.
cAdvisoriswritteninGo(https://golang.org);youcaneithercompileyourownbinaryoryoucanusethepre-compiledbinarythataresuppliedviaacontainer,whichisavailablefromGoogle’sownDockerHubaccount.Youcanfindthisathttp://hub.docker.com/u/google/.
Onceinstalled,cAdvisorwillsitinthebackgroundandcapturemetricsthataresimilartothatofthedockerstatscommand.Wewillgothroughthesestatsandunderstandwhattheymeanlaterinthischapter.
cAdvisortakesthesemetricsalongwiththoseforthehostmachineandexposesthemviaasimpleandeasy-to-usebuilt-inwebinterface.
![Page 87: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/87.jpg)
![Page 88: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/88.jpg)
RunningcAdvisorusingacontainerThereareanumberofwaystoinstallcAdvisor;theeasiestwaytogetstartedistodownloadandrunthecontainerimagethatcontainsacopyofaprecompiledcAdvisorbinary.
BeforerunningcAdvisor,let’slaunchafreshvagranthost:
[russ@mac~]$cd~/Documents/Projects/monitoring-docker/vagrant-centos/
[russ@mac~]$vagrantup
Bringingmachine'default'upwith'virtualbox'provider…
==>default:Importingbasebox'russmckendrick/centos71'...
==>default:MatchingMACaddressforNATnetworking…
==>default:Checkingifbox'russmckendrick/centos71'isuptodate…
.....
==>default:=>Installingdocker-engine…
==>default:=>Configuringvagrantuser…
==>default:=>Startingdocker-engine…
==>default:=>Installingdocker-compose…
==>default:=>FinishedinstallationofDocker
[russ@mac~]$vagrantssh
TipUsingabackslash
Aswehavealotoptionstopasstothedockerruncommand,weareusing\tosplitthecommandovermultiplelinessoit’seasiertofollowwhatisgoingon.
Onceyouhaveaccesstothehostmachine,runthefollowingcommand:
dockerrun\
--detach=true\
--volume=/:/rootfs:ro\
--volume=/var/run:/var/run:rw\
--volume=/sys:/sys:ro\
--volume=/var/lib/docker/:/var/lib/docker:ro\
--publish=8080:8080\
--privileged=true\
--name=cadvisor\
google/cadvisor:latest
YoushouldnowhaveacAdvisorcontainerupandrunningonyourhostmachine.Beforewestart,let’slookatcAdvisorinmoredetailbydiscussingwhywehavepassedalltheoptionstothecontainer.
ThecAdvisorbinaryisdesignedtorunonthehostmachinealongsidetheDockerbinary,sobylaunchingcAdvisorinacontainer,weareactuallyisolatingthebinaryinitsdownenvironment.TogivecAdvisoraccesstotheresourcesitrequiresonthehostmachine,wehavetomountseveralpartitionsandalsogivethecontainerprivilegedaccesstoletthecAdvisorbinarythinkitisbeingexecutedonthehostmachine.
![Page 89: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/89.jpg)
NoteWhenacontainerislaunchedwith--privileged,Dockerwillenablefullaccesstodevicesonthehostmachine;also,DockerwillconfigurebothAppArmororSELinuxtoallowyourcontainerthesameaccesstothehostmachineasaprocessrunningoutsidethecontainerwillhave.Forinformationonthe--privilegedflag,seethispostontheDockerblogathttp://blog.docker.com/2013/09/docker-can-now-run-within-docker/.
![Page 90: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/90.jpg)
![Page 91: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/91.jpg)
CompilingcAdvisorfromsourceAsmentionedintheprevioussection,cAdvisorreallyoughttobeexecutedonthehostmachine;thismeans,youmayhavetouseacasetocompileyourowncAdvisorbinaryandrunitdirectlyonthehost.
TocompilecAdvisor,youwillneedtoperformthefollowingsteps:
1. InstallGoandMercurialonthehostmachine—version1.3orhigherofGoisneededtocompilecAdvisor.
2. SetthepathforGotoworkfrom.3. GrabthesourcecodeforcAdvisorandgodep.4. SetthepathforyourGobinaries.5. BuildthecAdvisorbinaryusinggodeptosourcethedependenciesforus.6. Copythebinaryto/usr/local/bin/.7. DownloadeitheranUpstartorSystemdscriptandlaunchtheprocess.
Ifyoufollowedtheinstructionsintheprevioussection,youwillalreadyhaveacAdvisorprocessrunning.Beforecompilingfromsource,youshouldstartwithacleanhost;let’slogoutofthehostandlaunchafreshcopy:
[vagrant@centos7~]$exit
logout
Connectionto127.0.0.1closed.
[russ@mac~]$vagrantdestroy
default:Areyousureyouwanttodestroythe'default'VM?[y/N]y
==>default:ForcingshutdownofVM…
==>default:DestroyingVMandassociateddrives…
==>default:Runningcleanuptasksfor'shell'provisioner…
[russ@mac~]$vagrantup
Bringingmachine'default'upwith'virtualbox'provider…
==>default:Importingbasebox'russmckendrick/centos71'...
==>default:MatchingMACaddressforNATnetworking…
==>default:Checkingifbox'russmckendrick/centos71'isuptodate…
.....
==>default:=>Installingdocker-engine…
==>default:=>Configuringvagrantuser…
==>default:=>Startingdocker-engine…
==>default:=>Installingdocker-compose…
==>default:=>FinishedinstallationofDocker
[russ@mac~]$vagrantssh
TobuildcAdvisorontheCentOS7host,runthefollowingcommand:
sudoyuminstall-ygolanggitmercurial
exportGOPATH=$HOME/go
goget-dgithub.com/google/cadvisor
gogetgithub.com/tools/godep
exportPATH=$PATH:$GOPATH/bin
cd$GOPATH/src/github.com/google/cadvisor
![Page 92: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/92.jpg)
godepgobuild.
sudocpcadvisor/usr/local/bin/
sudowgethttps://gist.githubusercontent.com/russmckendrick/f647b2faad5d92c96
771/raw/86b01a044006f85eebbe395d3857de1185ce4701/cadvisor.service-O
/lib/systemd/system/cadvisor.service
sudosystemctlenablecadvisor.service
sudosystemctlstartcadvisor
OntheUbuntu14.04LTShost,runthefollowingcommand:
sudoapt-get-yinstallsoftware-properties-common
sudoadd-apt-repositoryppa:evarlast/golang1.4
sudoapt-getupdate
sudoapt-get-yinstallgolangmercurial
exportGOPATH=$HOME/go
goget-dgithub.com/google/cadvisor
gogetgithub.com/tools/godep
exportPATH=$PATH:$GOPATH/bin
cd$GOPATH/src/github.com/google/cadvisor
godepgobuild.
sudocpcadvisor/usr/local/bin/
sudowgethttps://gist.githubusercontent.com/russmckendrick/f647b2faad5d92c96
771/raw/e12c100d220d30c1637bedd0ce1c18fb84beff77/cadvisor.conf-O
/etc/init/cadvisor.conf
sudostartcadvisor
YoushouldnowhavearunningcAdvisorprocess.Youcancheckthisbyrunningpsaux|grepcadvisorandyoushouldseeaprocesswithapathof/usr/local/bin/cadvisorrunning.
![Page 93: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/93.jpg)
![Page 94: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/94.jpg)
CollectingmetricsNow,youhavecAdvisorrunning;whatdoyouneedtodotoconfiguretheserviceinordertostartcollectingmetrics?Theshortansweris,nothingatall.WhenyoustartedthecAdvisorprocess,itinstantlystartedpollingyourhostmachinetofindoutwhatcontainersarerunningandgatheredinformationonboththerunningcontainersandyourhostmachine.
![Page 95: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/95.jpg)
![Page 96: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/96.jpg)
TheWebinterfacecAdvisorshouldberunningonthe8080port;ifyouopenhttp://192.168.33.10:8080/,youshouldbegreetedwiththecAdvisorlogoandanoverviewofyourhostmachine:
Thisinitialpagestreamslivestatsaboutthehostmachine,thougheachsectionisrepeatedwhenyoustarttodrilldownandviewthecontainers.Tostartwith,let’slookateachsectionusingthehostinformation.
![Page 97: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/97.jpg)
OverviewThisoverviewsectiongivesyouabird’s-eyeviewofyoursystem;itusesgaugessoyoucanquicklygetanideaofwhichresourcesarereachingtheirlimits.Inthefollowingscreenshot,thereisverylittleinthewayofCPUutilizationandthefilesystemusageisrelativelylow;however,weareusing64%oftheavailableRAM:
![Page 98: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/98.jpg)
ProcessesThefollowingscreenshotdisplaysacombinedviewoftheoutputofthepsaux,dockerpsandtopcommandsweusedinthepreviouschapter:
Hereiswhateachcolumnheadingmeans:
User:ThisshowswhichuserisrunningtheprocessPID:ThisistheuniqueprocessIDPPID:ThisisthePIDoftheparentprocessStartTime:ThisshowswhattimetheprocessstartedCPU%:ThisisthepercentageoftheCPUtheprocessiscurrentlyconsumingMEM%:ThisisthepercentageoftheRAMtheprocessiscurrentlyconsumingRSS:ThisshowshowmuchofthemainmemorytheprocessisusingVirtualSize:ThisshowshowmuchofthevirtualmemorytheprocessisusingStatus:Thisshowsthecurrentstatusoftheprocess;thisarethestandardLinuxprocessstatecodesRunningTime:ThisshowshowlongtheprocesshasbeenrunningCommand:ThisshowswhichcommandtheprocessisrunningContainer:Thisshowswhichcontainertheprocessisattachedto;thecontainerlistedas/isthehostmachine
Astherecouldbeseveralhundredprocessesactive,thissectionissplitintopages;youcannavigatetothesewiththebuttonsonthebottom-left.Also,youcansorttheprocessesbyclickingonanyoftheheadings.
![Page 99: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/99.jpg)
CPUThefollowinggraphshowstheCPUutilizationoverthelastminute:
Hereiswhateachtermmeans:
TotalUsage:ThisshowsanaggregateusageacrossallcoresUsageperCore:ThisgraphbreaksdowntheusagepercoreUsageBreakdown(notshownintheprecedingscreenshot):Thisshowsaggregateusageacrossallcores,butbreaksitdowntowhatisbeingusedbythekernelandwhatisbeingusedbytheuser-ownedprocesses
![Page 100: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/100.jpg)
MemoryTheMemorysectionissplitintotwoparts.Thegraphtellsyouthetotalamountofmemoryusedbyalltheprocessesforthehostorcontainer;thisisthetotalofthehotandcoldmemory.TheHotmemoryisthecurrentworkingset:pagesthathavebeentouchedbythekernelrecently.TheColdmemoryisthepagethathasn’tbeentouchedforawhileandcouldbereclaimedifneeded.
TheUsageBreakdowngivesavisualrepresentationofthetotalmemoryinthehostmachine,orallowanceinthecontainer,alongsidethetotalandhotusage:
![Page 101: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/101.jpg)
NetworkThissectionshowstheincomingandoutgoingtrafficoverthelastminute.Youcanchangetheinterfaceusingthedrop-downboxonthetop-left.Thereisalsoagraphthatshowsanynetworkingerrors.Typically,thisgraphshouldbeflat.Ifitisn’t,thenyouwillbeseeingperformanceissueswithyourhostmachineorcontainer:
![Page 102: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/102.jpg)
FilesystemThefinalsectiongivesabreakdownofthefilesystemusage.Inthefollowingscreenshot,/dev/sda1isthebootpartition,/dev/sda3isthemainfilesystem,and/dev/mapper/docker-8…isanaggregateofthewritefilesystemsofyourrunningcontainers:
![Page 103: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/103.jpg)
![Page 104: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/104.jpg)
ViewingcontainerstatsAtthetopofthepage,thereisalinkofyourrunningcontainers;youcaneitherclickonthelinkorgodirectlytohttp://192.168.33.10:8080/docker/.Oncethepageloads,youshouldseealistofallyourrunningcontainers,andalsoadetailedoverviewofyourDockerprocess,andfinallyalistoftheimagesyouhavedownloaded.
![Page 105: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/105.jpg)
SubcontainersSubcontainersshowsalistofyourcontainers;eachentryisaclickablelinkthatwilltakeyoutoapagethatwillgiveyouthefollowingdetails:
Isolation:
CPU:ThisshowsyoutheCPUallowancesofthecontainer;ifyouhavenotsetanyresourcelimits,youwillseethehost’sCPUinformationMemory:Thisshowsyouthememoryallowancesofthecontainer;ifyouhavenotsetanyresourcelimits,yourcontainerwillshowanunlimitedallowance
Usage:
Overview:ThisshowsgaugessoyoucanquicklyseehowclosetoanyresourcelimitsyouareProcesses:ThisshowstheprocessesforjustyourselectedcontainerCPU:ThisshowstheCPUutilizationgraphsisolatedtojustyourcontainerMemory:Thisshowsthememoryutilizationofyourcontainer
![Page 106: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/106.jpg)
DriverstatusThedrivergivesthebasicstatsonyourmainDockerprocess,alongwiththeinformationonthehostmachine’skernel,hostname,andalsotheunderlyingoperatingsystem.
Italsogivesinformationonthetotalnumberofcontainersandimages.Youmaynoticethatthetotalnumberofimagesisamuchlargerfigurethanyouexpectedtosee;thisisbecauseitiscountingeachfilesystemasanindividualimage.
NoteFormoredetailsonDockerimages,seetheDockeruserguideathttps://docs.docker.com/userguide/dockerimages/.
Italsogivesyouadetailedbreakdownofyourstorageconfiguration.
![Page 107: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/107.jpg)
ImagesFinally,yougetalistoftheDockerimageswhichareavailableonthehostmachine.ItliststheRepository,Tag,Size,andwhentheimagewascreated,alongwiththeimages’uniqueID.Thisletsyouknowwheretheimageoriginatedfrom(Repository),whichversionoftheimageyouhavedownloaded(Tag)andhowbigtheimageis(Size).
![Page 108: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/108.jpg)
![Page 109: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/109.jpg)
Thisisallgreat,what’sthecatch?Soyouaremaybethinkingtoyourselfthatallofthisinformationavailableinyourbrowserisreallyuseful;beingabletoseereal-timeperformancemetricsinaneasilyreadableformatisareallyplus.
ThebiggestdrawbackofusingthewebinterfaceforcAdvisor,asyoumayhavenoticed,isthatitonlyshowsyouoneminute’sworthofmetrics;youcanquiteliterallyseetheinformationdisappearinginrealtime.
Asapaneofglassgivesareal-timeviewintoyourcontainers,cAdvisorisabrillianttool;ifyouwanttoreviewanymetricsthatareolderthanoneminute,youareoutofluck.
Thatis,unlessyouconfiguresomewheretostoreallofyourdata;thisiswherePrometheuscomesin.
![Page 110: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/110.jpg)
![Page 111: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/111.jpg)
PrometheusSowhat’sPrometheus?Itsdevelopersdescribeitasfollows:
Prometheusisanopen-sourcesystem’smonitoringandalertingtoolkitbuiltatSoundCloud.Sinceitsinceptionin2012,ithasbecomethestandardforinstrumentingnewservicesatSoundCloudandisseeinggrowingexternalusageandcontributions.
OK,butwhatdoesthathavetodowithcAdvisor?Well,Prometheushasquiteapowerfuldatabasebackendthatstoresthedataitimportsasatimeseriesofevents.
Wikipediadescribesatimeseriesasfollows:
“Atimeseriesisasequenceofdatapoints,typicallyconsistingofsuccessivemeasurementsmadeoveratimeinterval.Examplesoftimeseriesareoceantides,countsofsunspots,andthedailyclosingvalueoftheDowJonesIndustrialAverage.Timeseriesareveryfrequentlyplottedvialinecharts.”
https://en.wikipedia.org/wiki/Time_series
OneofthethingscAdvisordoes,bydefault,isexposeallthemetricsitiscapturingonasinglepageat/metrics;youcanseethisathttp://192.168.33.10:8080/metricsonourcAdvisorinstallation.Themetricsareupdatedeachtimethepageisloaded:
Asyoucanseeintheprecedingscreenshot,thisisjustasinglelongpageofrawtext.ThewayPrometheusworksisthatyouconfigureittoscrapethe/metricsURLatauser-definedinterval,let’ssayeveryfiveseconds;thetextisinaformatthatPrometheusunderstandsanditisingestedintothePrometheus’stimeseriesdatabase.
Whatthismeansisthat,usingPrometheus’spowerfulbuilt-inquerylanguage,youcanstarttodrilldownintoyourdata.Let’slookatgettingPrometheusupandrunning.
![Page 112: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/112.jpg)
LaunchingPrometheusLikecAdvisorthereareseveralwaysyoucanlaunchPrometheus.Tostartwith,wewilllaunchacontainerandinjectourownconfigurationfilesothatPrometheusknowswhereourcAdvisorendpointis:
dockerrun\
--detach=true\
--
volume=/monitoring_docker/Chapter03/prometheus.yml:/etc/prometheus/promethe
us.yml\
--publish=9090:9090\
--name=prometheus\
prom/prometheus:latest
Onceyouhavelaunchedthecontainer,PrometheuswillbeaccessibleonthefollowingURL:http://192.168.33.10:9090.WhenyoufirstloadtheURL,youwillbetakentoastatuspage;thisgivessomebasicinformationonthePrometheusinstallation.Theimportantpartofthispageisthelistoftargets.ThisliststheURLthatPrometheuswillbescrappingtocapturemetrics;youshouldseeyourcAdvisorURLlistedwithastateofHEALTHY,asshowninthefollowingscreenshot:
Anotherinformationpagecontainsthefollowing:
Runtimeinformation:ThisdisplayshowlongPrometheushasbeenupandpollingdata,ifyouhaveconfiguredanendpointBuildinformation:ThiscontainsthedetailsoftheversionofPrometheusthatyouhavebeenrunningConfiguration:ThisisacopyoftheconfigurationfileweinjectedintothecontainerwhenitwaslaunchedRules:Thisisacopyofanyrulesweinjected;thesewillbeusedforalertingStartupflags:Thisshowsalltheruntimevariablesandtheirvalues
![Page 113: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/113.jpg)
QueryingPrometheusAsweonlyhaveafewcontainersupandrunningatthemoment,let’slaunchonethatrunsRedissowecanstarttolookatthequerylanguagebuiltintoPrometheus.
WewillusetheofficialRedisimageforthisandasweareonlygoingtousethisasanexamplewewon’tneedtopassitanyuservariables:
dockerrun--namemy-redis-server-dredis
Wenowhaveacontainercalledmy-redis-serverrunning.cAdvisorshouldalreadybeexposingmetricsaboutthecontainertoPrometheus;let’sgoaheadandsee.InthePrometheuswebinterface,gototheGraphlinkinthemenuatthetopofthepage.Here,youwillbepresentedwithatextboxintowhichyoucanenteryourquery.Tostartwith,let’slookattheCPUusageoftheRediscontainer.
Inthebox,enterthefollowing:
container_cpu_usage_seconds_total{job="cadvisor",name="my-redis-server"}
Then,afterclickingonExecute,youshouldhavetworesultsreturned,listedintheConsoletabofthepage.Ifyouremember,cAdvisorrecordstheCPUusageofeachoftheCPUcoresthatthecontainerhasaccessto,whichiswhywehavetwovaluesreturned,onefor“cpu00”andonefor“cpu01”.ClickingontheGraphlinkwillshowyouresultsoveraperiodoftime:
Asyoucanseeintheprecedingscreenshot,wenowhaveaccesstotheusagegraphsforthelast25minutes,whichisabouthowlongagoIlaunchedtheRedisinstancebefore
![Page 114: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/114.jpg)
generatingthegraph.
![Page 115: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/115.jpg)
DashboardAlso,whencreatingoneofthegraphsusingthequerytoolinthemainapplication,youcaninstallaseparateDashboardapplication.ThisrunsinasecondcontainerthatconnectstoyourmainPrometheuscontainerusingtheAPIasadatasource.
BeforewestarttheDashboardcontainer,weshouldinitializeaSQLite3databasetostoreourconfiguration.Toensurethatthedatabaseispersistent,wewillstorethisonthehostmachinein/tmp/prom/file.sqlite3:
dockerrun\
--volume=/tmp/prom:/tmp/prom\
-eDATABASE_URL=sqlite3:/tmp/prom/file.sqlite3\
prom/promdash./bin/rakedb:migrate
Oncewehaveinitializedthedatabase,wecanlaunchtheDashboardapplicationproperly:
dockerrun\
--detach=true\
--volume=/tmp/prom:/tmp/prom\
-eDATABASE_URL=sqlite3:/tmp/prom/file.sqlite3\
--publish=3000:3000\
--name=promdash\
prom/promdash
Theapplicationshouldnowbeaccessibleathttp://192.168.33.10:3000/.Thefirstthingweneedtodoissetupthedatasource.Todothis,clickontheServerslinkatthetopofthescreenandthenclickonNewServer.Here,youwillbeaskedtoprovidethedetailsofyourPrometheusserver.NametheserverandenterthefollowingURL:
Name:cAdvisorURL:http://192.168.33.10:9090ServerType:Prometheus
OnceyouclickonCreateServer,youshouldreceiveamessagesayingServerwassuccessfullycreated.Nextup,youneedtocreateadirectory;thisiswhereyourdashboardswillbestored.
ClickontheDashboardslinkinthetopmenuandthenclickonNewdirectoryandcreateonecalledTestdirectory.Now,youarereadytostartcreatingDashboards.ClickonNewDashboard,callitMyDashboard,placeitinTestdirectory.OnceyouclickonCreateDashboard,youwillbetakentothepreviewscreen.
Fromhere,youcanbuildupdashboardsusingthecontrolinthetopright-handsideofeachsection.Toadddata,yousimplyenterthequeryyouwouldliketoseeinthedashboardsection:
![Page 116: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/116.jpg)
NoteFordetailedinformationonhowtocreateDashboards,seethePROMDASHsectionofthePrometheusdocumentationathttp://prometheus.io/docs/visualization/promdash/.
![Page 117: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/117.jpg)
ThenextstepsAtthemoment,wearerunningPrometheusinasinglecontaineranditsdataisbeingstoredwithinthatsamecontainer.Thismeans,ifforanyreasonthecontaineristerminated,ourdataislost;italsomeansthatwecan’tupgradewithoutloosingoutdata.Togetaroundthisproblem,wecancreateadatavolumecontainer.
NoteAdatavolumecontainerisaspecialtypeofcontainerthatonlyexistsasstorageforothercontainers.Formoredetails,seetheDockeruserguideathttps://docs.docker.com/userguide/dockervolumes/#creating-and-mounting-a-data-volume-container.
Firstofall,let’smakesurewehaveremovedalltherunningPrometheuscontainers:
dockerstopprometheus&&dockerrmPrometheus
Nextup,let’screateadatacontainercalledpromdata:
dockercreate\
--volume=/promdata\
--name=promdata\
prom/prometheus/bin/true
Finally,launchPrometheusagain,thistime,usingthedatacontainer:
dockerrun\
--detach=true\
--volumes-frompromdata\
--
volume=/monitoring_docker/Chapter03/prometheus.yml:/etc/prometheus/promethe
us.yml\
--publish=9090:9090\
--name=prometheus\
prom/prometheus
Thiswillensurethat,ifyouhavetoupgradeorrelaunchyourcontainer,themetricsyouhavebeencapturingaresafeandsound.
WehaveonlytouchedonthebasicsofusingPrometheusinthissectionofthebook;forfurtherinformationontheapplication,Irecommendthefollowinglinksasagoodstartingpoint:
Documentation:http://prometheus.io/docs/introduction/overview/Twitter:https://twitter.com/PrometheusIOProjectpage:https://github.com/prometheus/prometheusGooglegroups:https://groups.google.com/forum/#!forum/prometheus-developers
![Page 118: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/118.jpg)
![Page 119: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/119.jpg)
Alternatives?TherearesomealternativestoPrometheus.OnesuchalternativeisInfluxDBthatdescribesitselfasfollows:
Anopen-sourcedistributedtimeseriesdatabasewithnoexternaldependencies.
However,atthetimeofwriting,cAdvisorisnotcurrentlycompatiblewiththelatestversionofInfluxDB.TherearepatchesinthecodebaseforcAdvisor;however,theseareyettomakeitthroughtotheGoogle-maintainedDockerImage.
FormoredetailsonInfluxDBandit’snewvisualizationcomplainapplicationChronograf,seetheprojectwebsiteathttps://influxdb.com/andformoredetailsonhowtoexportcAdvisorstatisticstoInfluxDB,seethesupportingdocumentationforcAdvisorathttps://github.com/google/cadvisor/tree/master/docs.
![Page 120: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/120.jpg)
![Page 121: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/121.jpg)
SummaryInthischapter,welearnedhowtotaketheviewingreal-timestatisticsofourcontainersoffthecommandlineandintothewebbrowser.WeexploredsomedifferentmethodstoinstallGoogle’scAdvisorapplicationandalsohowtouseitswebinterfacetokeepaneyeonourrunningcontainers.WealsolearnedhowtocapturemetricsfromcAdvisorandstorethemusingPrometheus,amoderntimeseriesdatabase.
Thetwomaintechnologieswehavecoveredinthischapterhaveonlybeenpublicallyavailableforlessthantwelvemonths.Inthenextchapter,wewilllookatusingamonitoringtoolthathasbeeninaSysAdminstoolboxforover10years—Zabbix.
![Page 122: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/122.jpg)
![Page 123: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/123.jpg)
Chapter4.ATraditionalApproachtoMonitoringContainersSofar,wehavelookedatonlyafewtechnologiestomonitorourcontainers,sointhischapter,wewillbelookingmoreatatraditionaltoolformonitoringservices.Bytheendofthischapter,youshouldknowyourwayaroundZabbixandthevariouswaysyoucanmonitoryourcontainers.Wewillcoverthefollowingtopicsinthischapter:
HowtorunaZabbixServerusingcontainersHowtolaunchaZabbixServeronavagrantmachineHowtoprepareourhostsystemformonitoringcontainersusingtheZabbixagentHowtofindyourwayaroundtheZabbixwebinterface
![Page 124: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/124.jpg)
ZabbixFirstthingsfirst,whatisZabbixandwhyuseit?
Ihavepersonallybeenusingitsinceversion1.2;theZabbixsitedescribesitasfollows:
“WithZabbix,itispossibletogathervirtuallylimitlesstypesofdatafromthenetwork.High-performancereal-timemonitoringmeansthattensofthousandsofservers,virtualmachines,andnetworkdevicescanbemonitoredsimultaneously.Alongwithstoringthedata,visualizationfeaturesareavailable(overviews,maps,graphs,screens,andsoon),aswellasveryflexiblewaysofanalyzingthedataforthepurposeofalerting.
Zabbixoffersgreatperformancefordatagatheringandcanbescaledtoverylargeenvironments.DistributedmonitoringoptionsareavailablewiththeuseofZabbixproxies.Zabbixcomeswithaweb-basedinterface,secureuserauthentication,andaflexibleuserpermissionschema.Pollingandtrappingissupported,withnativehigh-performanceagentsgatheringdatafromvirtuallyanypopularoperatingsystem;agent-lessmonitoringmethodsareavailableaswell.”
AtthetimeIstartedusingZabbix,theonlyrealviableoptionswereasfollows:
Nagios:https://www.nagios.org/Zabbix:http://www.zabbix.com/Zenoss:http://www.zenoss.org/
Outofthethesethreeoptions,Zabbixseemedtobethemoststraightforwardoneatthetime.ItwasdoingenoughworktomanagetheseveralhundredserversIwasgoingtomonitorwithouthavingtohavetheextraworkoflearningthecomplexitiesofsettingupNagiosorZenoss;afterall,giventhetaskthesoftwarehad,IneededtobeabletotrustthatIhadsetitupcorrectly.
Inthischapter,whileIamgoingtogointosomedetailaboutthesetupandthebasicsofusingZabbix,wewillonlybetouchingonsomeofthefunctionalities,whichcandoalotmorethanjustmonitoryourcontainers.Formoreinformation,Iwouldrecommendthefollowingasagoodstartingpoint:
Zabbixblog:http://blog.zabbix.comZabbix2.4manual:https://www.zabbix.com/documentation/2.4/manualFurtherreading:https://www.packtpub.com/all/?search=zabbix
![Page 125: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/125.jpg)
![Page 126: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/126.jpg)
InstallingZabbixAsyoumayhavenoticedfromthelinksintheprevioussection,therearealotofmovingpartsinZabbix.Itleveragesseveralopensourcetechnologies,andaproduction-readyinstallationneedsalittlemoreplanningthanwecangointointhischapter.BecauseofthiswearegoingtolookattwowaysofinstallingZabbixquicklyrathergointotoomuchdetail.
![Page 127: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/127.jpg)
UsingcontainersAtthetimeofwriting,thereareoverahundredDockerimagesavailableontheDockerHub(https://hub.docker.com)thatmentionsZabbix.Theserangefromfullserverinstallationstojustthevariousparts,suchastheZabbixagentorproxyservices.
Outoftheoneslisted,thereisonethatisrecommendbyZabbixitself.So,wewilllookatthisone;itcanbefoundatthefollowingURLs:
DockerHub:https://hub.docker.com/u/zabbix/Projectpage:https://github.com/zabbix/zabbix-community-docker
TogettheZabbixServercontainerupandrunning,wemustfirstlaunchadatabasecontainer.Let’sstartafreshwithourvagrantinstancebyrunningthefollowingcommand:
[russ@mac~]$cd~/Documents/Projects/monitoring-docker/vagrant-centos/
[russ@mac~]$vagrantdestroy
default:Areyousureyouwanttodestroythe'default'VM?[y/N]y
==>default:ForcingshutdownofVM…
==>default:DestroyingVMandassociateddrives…
==>default:Runningcleanuptasksfor'shell'provisioner…
[russ@mac~]$vagrantup
Bringingmachine'default'upwith'virtualbox'provider…
==>default:Importingbasebox'russmckendrick/centos71'...
==>default:MatchingMACaddressforNATnetworking…
==>default:Checkingifbox'russmckendrick/centos71'isuptodate…
.....
==>default:=>Installingdocker-engine…
==>default:=>Configuringvagrantuser…
==>default:=>Startingdocker-engine…
==>default:=>Installingdocker-compose…
==>default:=>FinishedinstallationofDocker
[russ@mac~]$vagrantssh
Now,wehaveacleanenvironmentandit’stimetolaunchourdatabasecontainer,asfollows:
dockerrun\
--detach=true\
--publish=3306\
--env="MARIADB_USER=zabbix"\
--env="MARIADB_PASS=zabbix_password"\
--name=zabbix-db\
million12/mariadb
Thiswilldownloadthemillion12/mariadbimagefromhttps://hub.docker.com/r/million12/mariadb/andlaunchacontainercalledzabbix-db,runningMariaDB10(https://mariadb.org)withausercalledzabbixwhohasapasswordzabbix_password.WehavealsoopenedtheMariaDBport3306uponthecontainer,butaswewillbeconnectingtoitfromalinkedcontainer,thereisnoneedtoexposethatportonthehostmachine.
![Page 128: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/128.jpg)
Now,wehavethedatabasecontainerupandrunning,wenowneedtolaunchourZabbixServercontainer:
dockerrun\
--detach=true\
--publish=80:80\
--publish=10051:10051\
--link=zabbix-db:db\
--env="DB_ADDRESS=db"\
--env="DB_USER=zabbix"\
--env="DB_PASS=zabbix_password"\
--name=zabbix\
zabbix/zabbix-server-2.4
Thisdownloadstheimage,whichatthetimeofwritingisover1GBsothisprocesscouldtakeseveralminutesdependingonyourconnection,andlaunchesacontainercalledzabbix.Itmapsthewebserver(port80)andtheZabbixServerprocess(port10051)onthehosttothecontainer,createsalinktoourdatabasecontainer,setsupthealiasdb,andinjectsthedatabasecredentialsasenvironmentvariablessothatthescriptsthatlaunchwhenthecontainerbootscanpopulatethedatabase.
Youcanverifythateverythingworkedasexpectedbycheckingthelogsonthecontainer.Todothis,enterdockerlogszabbix.Thiswillprintdetailsofwhathappenedwhenthecontainerlaunchedonscreen:
![Page 129: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/129.jpg)
Now,oncewehavethecontainerupandrunning,itistimetomovetothebrowserforourfirsttasteofthewebinterface.Gotohttp://192.168.33.10/inyourbrowserandyouwillbegreetedbyawelcomepage;beforewecanstartusingZabbix,weneedtocompletetheinstallation.
Onthewelcomepage,clickonNexttobetakentothefirststep.ThiswillverifythateverythingweneedtorunaZabbixServerisinstalled.Aswehavelauncheditinacontainer,youshouldseeOKnexttoalloftheprerequisites.ClickonNexttomoveontothenextstep.
Now,weneedtoconfigurethedatabaseconnectionforthewebinterface.Here,youshouldhavethesamedetailsasyoudidwhenyoulaunchedthecontainer,asillustratedinthefollowingscreenshot:
![Page 130: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/130.jpg)
Onceyouhaveenteredthedetails,clickonTestconnectionandyoushouldreceiveanOKmessage;youwillnotbeabletoproceeduntilthistestcompletessuccessfully.OnceyouhaveenteredthedetailsandhaveanOKmessage,clickonNext.
Nextup,arethedetailsontheZabbixServerthatthewebinterfaceneedstoconnectto;clickonNexthere.Nextup,youwillreceiveasummaryoftheinstallation.Toproceed,clickonNextandyouwillbegetconfirmationthatthe/usr/local/src/zabbix/frontends/php/conf/zabbix.conf.phpfilehasbeencreated.ClickonFinishtobetakentotheloginpage.
![Page 131: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/131.jpg)
UsingvagrantWhilewritingthischapter,IthoughtalotaboutprovidinganothersetofinstallationinstructionsfortheZabbixServerservice.WhilethebookisallaboutMonitoringDockercontainers,havingaserviceasresourceintensiveasZabbixrunninginsideacontainerfeelsalittlecounterintuitive.Becauseofthis,thereisavagrantmachinethatusesPuppettobootstrapaworkinginstallationofZabbixServer:
[russ@mac~]$cd~/Documents/Projects/monitoring-docker/vagrant-zabbix/
[russ@mac~]$vagrantup
Bringingmachine'default'upwith'virtualbox'provider…
==>default:Importingbasebox'russmckendrick/centos71'...
==>default:MatchingMACaddressforNATnetworking…
==>default:Checkingifbox'russmckendrick/centos71'isuptodate…
.....
==>default:Debug:Receivedreporttoprocessfromzabbix.media-glass.es
==>default:Debug:Evictingcacheentryforenvironment'production'
==>default:Debug:Cachingenvironment'production'(ttl=0sec)
==>default:Debug:Processingreportfromzabbix.media-glass.eswith
processorPuppet::Reports::Store
Asyoumayhavenoticed,thereisalotofoutputstreamedtotheterminal,sowhatjusthappened?Firstofall,aCentOS7vagrantinstancewaslaunchedandthenaPuppetagentwasinstalled.Onceinstalled,theinstallationwashandedofftoPuppet.UsingtheZabbixPuppetmodulebyWernerDijkerman,ZabbixServerwasinstalled;formoredetailsonthemodule,seeitsPuppetForgepageathttps://forge.puppetlabs.com/wdijkerman/zabbix.
UnlikethecontainerizedversionofZabbixServer,thereisnoadditionalconfigurationrequired,soyoushouldbeabletoaccesstheZabbixloginpageathttp://zabbix.media-glass.es/(anIPaddressof192.168.33.11ishardcodedintotheconfiguration).
![Page 132: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/132.jpg)
PreparingourhostmachineFortheremainderofthischapter,IwillassumethatyouareusingtheZabbixServerthatisrunningonitsownvagrantinstance.ThishelpstoensurethatyourenvironmentisconsistentwiththeconfigurationoftheZabbixagentwewillbelookingat.
TopassthestatisticsfromourcontainerstotheZabbixagent,whichwilltheninturnexposethemtotheZabbixServer,wewillbeinstallingusingtheZabbix-Docker-MonitoringZabbixagentmodulethathasbeendevelopedbyJanGaraj.Formoreinformationontheproject,seethefollowingURLs:
TheProjectpage:https://github.com/monitoringartist/Zabbix-Docker-Monitoring/TheZabbixsharepage:https://share.zabbix.com/virtualization/docker-containers-monitoring
Togettheagentandmoduleinstalled,configured,andrunning,weneedtoexecutethefollowingsteps:
1. InstalltheZabbixpackagerepository.2. InstalltheZabbixagent.3. Installtheprerequisitesforthemodule.4. AddtheZabbixagentusertotheDockergroup.5. Downloadtheauto-discoverybashscript.6. Downloadtheprecompiledzabbix_module_dockerbinary.7. ConfiguretheZabbixagentwiththedetailsofourZabbixServerandalsotheDocker
module.8. Setthecorrectpermissionsonallthefileswehavedownloadedandcreated.9. StarttheZabbixagent.
WhilethestepsremainthesameforbothCentOSandUbuntu,theactionstakentodotheinitialpackageinstallationdifferslightly.Ratherthangoingthroughtheprocessofshowingthecommandstoinstallandconfiguretheagent,thereisascriptforeachofthehostoperatingsystemsinthe/monitoring_docker/chapter04/folder.Toviewthescripts,runthefollowingcommandfromyourterminal:
cat/monitoring_docker/chapter04/install-agent-centos.sh
cat/monitoring_docker/chapter04/install-agent-ubuntu.sh
Now,youhavetakenalookatthescriptsitstimetorunthem,todothistypeoneofthefollowingcommands.IfyouarerunningCentOS,runthiscommand:
bash/monitoring_docker/chapter04/install-agent-centos.sh
ForUbuntu,runthefollowingcommand:
bash/monitoring_docker/chapter04/install-agent-ubuntu.sh
Toverifythateverythingranasexpected,checktheZabbixagentlogfilebyrunningthefollowingcommand:
![Page 133: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/133.jpg)
cat/var/log/zabbix/zabbix_agentd.log
Youshouldseethattheendofthefileconfirmsthattheagenthasstartedandthatthezabbix_module_docker.somodulehasbeenloaded:
BeforewemoveontotheZabbixwebinterface,let’slaunchafewcontainersusingthedocker-composefilefromChapter2,UsingtheBuilt-inTools:
[vagrant@docker~]$cd/monitoring_docker/chapter02/02-multiple/
[vagrant@docker02-multiple]$docker-composeup-d
[vagrant@docker02-multiple]$docker-composescaleweb=3
[vagrant@docker02-multiple]$docker-composeps
WeshouldnowhavethreewebservercontainersrunningandarunningZabbixagentonthehost.
![Page 134: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/134.jpg)
TheZabbixwebinterfaceOnceyouhaveZabbixinstalledyoucanopentheZabbixwebinterfacebygoingtohttp://zabbix.media-glass.es/inyourbrowser,thislinkwillonlyworkwhenyouhavetheZabbixvagrantboxupandrunning,ifyoudon’thaveitrunningthepagewilltimeout.Youshouldbepresentedwithaloginscreen.Enterthedefaultusernameandpasswordhere,whichisAdminandzabbix(notethattheusernamehasacapitalA),tologin.
Onceloggedin,youwillneedtoaddthehosttemplates.ThesearepreconfiguredenvironmentsettingsandwilladdsomecontextaroundthestatisticsthattheZabbixagentissendingtotheserver,alongwiththeauto-discoveryofcontainers.
Toaddthetemplates,gototheConfigurationtabinthetopmenuandselectTemplate;thiswillbringupalistofallthetemplatesthatarecurrentlyinstalled.ClickontheImportbuttonintheheaderanduploadacopyofthetwotemplatefilesyoucanfindinthe~/Documents/Projects/monitoring-docker/chapter04/templatefolderonyourmainmachine;thereisnoneedtochangetheruleswhenuploadingthetemplates.
Oncebothtemplateshavebeensuccessfullyimported,itistimetoaddourDockerhost.Again,gototheConfigurationtab,butthistimeselectHosts.Here,youneedtoclickonCreatehost.Then,enterthefollowinginformationintheHosttab:
![Page 135: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/135.jpg)
Herearethedetailsoftheprecedinginformation:
Hostname:ThisisthehostnameofourDockerhostVisiblename:Here,thenameserverwillappearasinZabbixGroups:WhichgroupwithinZabbixtheserveryouwouldliketheDockerhosttobepartofAgentInterfaces:ThisistheIPaddressortheDNSnameofourDockerhostEnabled:Thisshouldbeticked
BeforeclickingonAdd,youshouldclickontheTemplatestabandlinkthefollowingtwotemplatestothehost:
TemplateAppDockerTemplateOSLinux
Hereisthescreenshotofthehost:
![Page 136: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/136.jpg)
Onceyouhaveaddedthetwotemplates,clickonAddtoconfigureandenablethehost.Toverifythatthehosthasbeenaddedcorrectly,youshouldgototheMonitoringtabandthenLatestdata.Fromhere,clickonShowfilterandenterthehostmachineintheHostsbox.Youshouldthenstarttoseeitemsappearing:
![Page 137: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/137.jpg)
Don’tworryifyoudon’tseetheDockersectionimmediately,bydefault,Zabbixwillattempttoauto-discovernewcontainerseveryfiveminutes.
![Page 138: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/138.jpg)
![Page 139: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/139.jpg)
DockermetricsForeachcontainer,Zabbixdiscoversthefollowingmetricsthatwillberecorded:
Container(yourContainersname)isrunningCPUsystemtimeCPUusertimeUsedcachememoryUsedRSSmemoryUsedswap
Apartfrom“Usedswap”,thesearethesamemetricsrecordedbycAdvisor.
![Page 140: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/140.jpg)
CreatecustomgraphsYoucanaccessatime-basedgraphforanyofthemetricscollectedbyZabbix;youcanalsocreateyourowncustomgraphs.Inthefollowinggraph,IhavecreatedagraphthatplotsalltheCPUSystemstatsfromthethreewebcontainerswelaunchedearlierinthechapter:
Asyoucansee,IperformedafewtestsusingApacheBenchtomakethegraphalittlemoreinteresting.
Formoreinformationonhowtocreatecustomgraphs,seethegraphssectionofthedocumentationsiteathttps://www.zabbix.com/documentation/2.4/manual/config/visualisation/graphs.
![Page 141: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/141.jpg)
ComparecontainerstoyourhostmachineAsweaddedtheLinuxOStemplateandtheDockertemplatetothehostandwearealsorecordingquitealotofinformationaboutthesystem,herewecantelltheeffectthetestingwithApacheBenchhadontheoverallprocessorload:
Wecandrilldownfurthertogetinformationontheoverallutilization:
![Page 142: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/142.jpg)
TriggersAnotherfeatureofZabbixistriggers:youcandefineactionstohappenwhenametricmeetsacertainsetofcriteria.Inthefollowingexample,ZabbixhasbeenconfiguredwithatriggercalledContainerDown;thischangesthestatusofthemonitoreditemtoProblemwithaseverityofDisaster:
Thischangeinstatusthentriggersane-mailtoinformthat,forsomereasonthecontainerisnolongerupandrunning:
Thiscouldhavealsotriggeredothertasks,suchasrunningacustomscript,sendinganinstantmessageviaJabber,oreventriggeringathird-partyservicesuchasPagerDuty(https://www.pagerduty.com)orSlack(https://slack.com).
![Page 143: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/143.jpg)
FormoreinformationonTriggers,Events,andNotifications,seethefollowingsectionsofthedocumentation:
https://www.zabbix.com/documentation/2.4/manual/config/triggershttps://www.zabbix.com/documentation/2.4/manual/config/eventshttps://www.zabbix.com/documentation/2.4/manual/config/notifications
![Page 144: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/144.jpg)
![Page 145: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/145.jpg)
SummarySo,howdoesthistraditionalapproachtomonitoringfitintoacontainer’slifecycle?
GoingbacktothePetsversusCattleanalogy,atfirstglance,ZabbixseemstobegearedmoretowardsPets:itsfeaturesetisbestsuitedtomonitoringservicesthatarestaticoveralongperiodoftime.Thismeansthatthesameapproachtomonitoringapetcanalsobeappliedtolong-runningprocessesrunningwithinyourcontainers.
Zabbixisalsotheperfectoptionformonitoringmixedenvironments.Maybeyouhaveseveraldatabaseserversthatarenotrunningascontainers,butyouhaveseveralhostsrunningDocker,andhaveequipmentsuchasswitchesandSANsthatyouneedtomonitor.Zabbixcanprovideyouwithasinglepaneofglassshowingyoumetricsforallyourenvironments,alongwithbeingabletoalertyoutoproblems.
Sofar,wehavelookedatusingAPIsandmetricsprovidedbyDockerandLXC,butwhataboutothermetricscanweuse?Inthenextchapter,wewilllookatatoolthathooksstraightintothehostmachine’skerneltogatherinformationonyourcontainers.
![Page 146: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/146.jpg)
![Page 147: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/147.jpg)
Chapter5.QueryingwithSysdigTheprevioustoolswehavelookedathaveallreliedonmakingAPIcallstoDockerorreadingmetricsfromLXC.Sysdigworksdifferentlybyhookingitselfintothehostsmachine’skernelwhilethisapproachdoesgoagainstDocker’sphilosophyofeachservicebeingruninitsownisolatedcontainer,theinformationyoucangetbyrunningSysdigonlyforafewminutesfaroutweighsanyargumentsaboutnotusingit.
Inthischapter,wewilllookatthefollowingtopics:
HowtoinstallSysdigandCsysdigonthehostmachineBasicusageandhowtoqueryyourcontainersinrealtimeHowtocapturelogssotheycanbequeriedlater
![Page 148: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/148.jpg)
WhatisSysdig?BeforewestarttogetintoSysdig,let’sfirstunderstandwhatitis.WhenIfirstheardaboutthetool,Ithoughttomyselfthatitsoundedtoogoodtobetrue;thewebsitedescribesthetoolasfollows:
“Sysdigisopensource,system-levelexploration:capturesystemstateandactivityfromarunningLinuxinstance,thensave,filterandanalyze.SysdigisscriptableinLuaandincludesacommandlineinterfaceandapowerfulinteractiveUI,csysdig,thatrunsinyourterminal.Thinkofsysdigasstrace+tcpdump+htop+iftop+lsof+awesomesauce.Withstateoftheartcontainervisibilityontop.”
Thisisquiteaclaimasallthetoolsthatitisclaimingtobeaspowerfulwereallinasetofgotocommandstorunwhenlookingintoproblems,soIwasalittleskepticalatfirst.
Asanyonewhohashadtotryandtrackdownahaywireprocessoftryandtrackdownanissuethatisn’tbeingveryverboseinitserrorlogsonaLinuxserverwillknowthatusingtoolssuchasstrace,lsof,andtcpdumpcangetcomplicatedveryquicklyanditnormallyinvolvescapturingawholelotofdataandthenusingacombinationofseveraltoolstoslowly,andmanually,tracetheproblembyreducingtheamountofdatayoucaptured.
ImaginemydelightwhenSysdig’sclaimsturnedouttobetrue.ItmademewishIhadthetoolbackwhenIwasafrontlineengineer;itwouldhavemademylifealoteasier.
Sysdigcomesintwodifferentflavors,firstistheOpenSourceversionavailableathttp://www.sysdig.org/;thiscomeswithanncursesinterfacesothatyoucaneasilyaccessandquerydatafromaterminal-basedGUI.
NoteWikipediadescribesncurses(newcurses)asaprogramminglibrarythatprovidesanAPIthatallowstheprogrammertowritetext-baseduserinterfacesinaterminal-independentmanner.Itisatoolkitfordeveloping“GUI-like”applicationsoftwarethatrunsunderaterminalemulator.Italsooptimizesscreenchangesinordertoreducethelatencyexperiencedwhenusingremoteshells.
ThereisalsoacommercialservicethatallowsyoutostreamyourSysdigtotheirexternallyhostedservice;thisversionhasaweb-basedinterfaceforviewingandqueryingyourdata.
Inthischapter,wewillbeconcentratingontheopensourceversion.
![Page 149: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/149.jpg)
![Page 150: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/150.jpg)
InstallingSysdigConsideringhowpowerfulSysdigis,ithasoneofthemoststraightforwardinstallationandconfigurationprocessesIhavecomeacross.ToinstallSysdigoneitheraCentOSorUbuntuserver,typethefollowingcommand:
curl-shttps://s3.amazonaws.com/download.draios.com/stable/install-sysdig
|sudobash
Afterrunningtheprecedingcommand,youwillgetthefollowingoutput:
That’sit,youarereadytogo.Thereisnothingmoretoconfigureordo.Thereisamanualinstallationprocessandalsoawayofinstallingthetoolusingcontainerstobuildthenecessarykernelmodules;formoredetails,seetheinstallationguideasfollows:
http://www.sysdig.org/wiki/how-to-install-sysdig-for-linux/
![Page 151: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/151.jpg)
![Page 152: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/152.jpg)
UsingSysdigBeforewelookathowtouseSysdig,let’slaunchafewcontainersusingdocker-composebyrunningthefollowingcommand:
cd/monitoring_docker/chapter05/wordpress/
docker-composeup–d
ThiswilllaunchaWordPressinstallationrunningadatabaseandtwowebservercontainersthatareloadbalancedusinganHAProxycontainer.YouwillbeabletoviewtheWordPressinstallationathttp://docker.media-glass.es/oncethecontainershavelaunched.Youwillneedtoentersomedetailstocreatetheadminuserbeforethesiteisvisible;followtheon-screenpromptstocompletethesesteps.
![Page 153: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/153.jpg)
ThebasicsAtitscore,Sysdigisatoolforproducingastreamofdata;youcanviewthestreambytypingsudosysdig(toquit,pressCtrl+c).
Thereisalotinformationtheresolet’sstarttofilterthestreamdownandrunthefollowingcommand:
sudosysdigevt.type=chdir
Thiswilldisplayonlyeventsinwhichauserchangesdirectory;toseeitinaction,openasecondterminalandyouwillseethatwhenyoulogin,youseesomeactivityinthefirstterminal.Asyoucansee,itlooksalotlikeatraditionallogfile;wecanformatoutputtogiveinformationsuchastheusername,byrunningthefollowingcommand:
sudosysdig-p"user:%user.namedir:%evt.arg.path"evt.type=chdir
Then,inyoursecondterminal,changethedirectoryafewtimes:
Asyoucansee,thisisaloteasiertoreadthantheoriginalunformattedoutput.PressCtrl+ctostopfiltering.
![Page 154: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/154.jpg)
CapturingdataIntheprevioussection,welookedatfilteringdatainrealtime;itisalsopossibletostreamSysdigdatatoafilesothatyoucanquerythedataatalatertime.Exitfromyoursecondterminalandrunthefollowingcommandonyourfirstone:
sudosysdig-w~/monitoring-docker.scap
Whilethecommandisrunningonthefirstterminal,logintothehostonthesecondoneandchangethedirectoryafewtimes.Also,whilewearerecording,clickaroundtheWordPresssitewestartedatthebeginningofthissection,theURLishttp://docker.media-glass.es/.Onceyouhavedonethat,stoptherecordingbypressingCrtl+c;youshouldhavenowdroppedbacktoaprompt.YoucancheckthesizeofthefilecreatedbySysdigbyrunningthefollowing:
ls-lha~/monitoring-docker.scap
Now,wecanusethedatathatwehavecapturedtoapplythesamefilteraswedidwhenlookingatthereal-timestream:
sudosysdig-r~/monitoring-docker.scap-p"user:%user.name
dir:%evt.arg.path"evt.type=chdir
Byrunningtheprecedingcommand,youwillgetthefollowingoutput:
Noticehowwegetsimilarresultstowhenwewereviewingthedatainrealtime.
![Page 155: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/155.jpg)
ContainersOneofthethingsthatwasrecordedin~/monitoring-docker.scapwasdetailsonthesystemstate;thisincludesinformationonthecontainerswelaunchedatthestartofthechapter.Let’susethisfiletogetsomestatsonthecontainers.Tolistthecontainersthatwereactiveduringthetime,wecapturedthedatafilerun:
sudosysdig-r~/monitoring-docker.scap-clscontainers
ToseewhichofthecontainersutilizedtheCPUmostofthetime,wewereclickingaroundtheWordPresssiterun:
sudosysdig-r~/monitoring-docker.scap-ctopcontainers_cpu
Tohavealookatthetopprocessesineachofthecontainersthathave“wordpress”intheirnames(whichisalloftheminourcase),runthefollowingcommand:
sudosysdig-r~/monitoring-docker.scap-ctopprocs_cpucontainer.name
containswordpress
Finally,whichofourcontainerstransferredthemostamountofdata?:
sudosysdig-r~/monitoring-docker.scap-ctopcontainers_net
Byrunningtheprecedingcommand,youwillgetthefollowingoutput:
Asyoucansee,wehaveextractedquiteabitofinformationonourcontainersfromthedatawecaptured.Also,usingthefile,youcanremovethe-r~/monitoring-docker.scappartofthecommandtoviewthecontainermetricsinrealtime.
![Page 156: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/156.jpg)
It’salsoworthpointingoutthattherearebinariesforSysdigthatworkonbothOSXandWindows;whilethesedonotcaptureanydata,theycanbeusedtoreaddatathatyouhaverecordedonyourLinuxhost.
![Page 157: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/157.jpg)
FurtherreadingFromthefewbasicexercisescoveredinthissection,youshouldstarttogetanideaofjusthowpowerfulSysdigcanbe.TherearemoreexamplesontheSysdigwebsiteathttp://www.sysdig.org/wiki/sysdig-examples/.Also,Irecommendyoutoreadtheblogpostathttps://sysdig.com/fishing-for-hackers/;itwasmyfirstexposuretoSysdiganditreallydemonstratesitsusefulness.
![Page 158: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/158.jpg)
![Page 159: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/159.jpg)
UsingCsysdigAseasyasitistoviewdatacapturedbySysdigusingthecommandlineandmanuallyfilteringtheresults,itcangetmorecomplicatedasyoustarttostringmoreandmorecommandstogether.TohelpmakethedatacapturedbySysdigasaccessibleaspossible,SysdigshipswithaGUIcalledCsysdig.
LaunchingtheCsysdigisdonewithasinglecommand:
sudocsysdig
Oncetheprocesshaslaunched,itshouldinstantlylookfamiliartoanyonewhohasusedtoporcAdvisor(minusthegraphs);itsdefaultviewwillshowyoureal-timeinformationontheprocessesthatarerunning:
Tochangethisview,knownastheProcessesview,pressF2toopentheViewsmenu;fromhere,youcanusetheupanddownarrowsonyourkeyboardtoselectaview.Asyoumayhavealreadyguessed,wewouldliketoseetheContainersview:
![Page 160: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/160.jpg)
However,beforewedrilldownintoourcontainers,let’squitCsysdigbypressingqandloadupthefilewecreatedintheprevioussection.Todothis,typethefollowingcommand:
sudocsysdig-r~/monitoring-docker.scap
OnceCsysdigloads,youwillnoticethatSourcehaschangedfromLiveSystemtothefilepathofourdatafile.Fromhere,pressF2andusetheuparrowtoselectcontainersandthenhitEnter.Fromhere,youcanusetheupanddownarrowstoselectoneofthetwowebservers,thesewouldbeeitherwordpress_wordpress1_1orwordpress_wordpress2_1asshowninthefollowingscreen:
NoteTheremainingpartofthischapterassumesthatyouhaveCsysdigopenin-frontofyou,itwilltalkyouthroughhowtonavigatearoundthetool.Pleasefeelfreetoexploreyourselfaswell.
![Page 161: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/161.jpg)
Onceyouhaveselectedaserver,hitEnterandyouwillbepresentedwithalistofprocessesthatthecontainerwasrunning.Again,youcanusethearrowkeystoselectaprocesstodrilldownfurtherinto.
IsuggestedlookingatoneoftheApacheprocessesthathasavaluelistedintheFilecolumn.Thistime,ratherthanpressingEntertoselecttheprocess,let’s“Echo”whattheprocesswasuptoatthetimewecapturedthedata;withtheprocessselected,pressF5.
Youcanusetheupanddownarrowstoscrollthroughtheoutput:
Tobetterformatthedata,pressF2andselectPrintableASCII.Asyoucanseefromtheprecedingscreenshot,thisApacheprocessperformedthefollowingtasks:
AcceptedanincomingconnectionAccessedthe.htaccessfileReadthemod_rewriterulesGotinformationfromthehostsfileMadeaconnectiontotheMySQLcontainerSenttheMySQLpassword
Byscrollingthroughtheremainderofthedatainthe“Echo”resultsfortheprocess,youshouldbeabletoeasilyfollowtheinteractionswiththedatabaseallthewaythroughtothepagebeingsenttothebrowser.
Toleavethe“Echo”screen,pressBackspace;thiswillalwaystakeyoualevelback.
Ifyouwantamoredetailedbreakdownonwhattheprocesswasdoing,thenpressF6toentertheDigview;thiswilllistthefilesthattheprocesswasaccessingatthetime,alongwiththenetworkinteractionandhowitisaccessingtheRAM.
![Page 162: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/162.jpg)
Toviewafulllistofcommandsandformorehelp,youcanpressF1atanytime.Also,togetabreakdownonanycolumnsthatareonscreen,pressF7.
![Page 163: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/163.jpg)
![Page 164: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/164.jpg)
SummaryAsImentionedatthestartofthischapter,SysdigisprobablyoneofthemostpowerfultoolsIhavecomeacrossinrecentyears.
Partofitspoweristhewaythatitexposesalotofinformationandmetricsinawaythatneverfeelsoverwhelming.It’sclearthatthedevelopershavespentalotoftimeensuringthatboththeUIandthewaythatcommandsarestructuredfeelnaturalandinstantlyunderstandable,evenbythenewestmemberofanoperationsteam.
Theonlydownsideisthat,unlessyouwanttoviewtheinformationinrealtimeorlookintoaproblemindevelopmentstoringtheamountofdatathatisbeinggeneratedbySysdig,itcanbequitecostlyintermsofdiscspacebeingused.
ThisissomethingthatSysdighasrecognized,andtohelpwiththis,thecompanyoffersacloud-basedcommercialservicecalledSysdigCloudforyoutostreamyourSysdigdatainto.Inthenextchapter,wewilllookatthisserviceandalsosomeofitscompetitors.
![Page 165: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/165.jpg)
![Page 166: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/166.jpg)
Chapter6.ExploringThirdPartyOptionsSofar,wehavebeenlookingatthetoolsandservicesyouhostyourself.Alongwiththeseself-hostedtools,alargeamountofcloud-basedsoftwarehasdevelopedaroundDockerasaserviceecosystem.Inthischapter,wewilllookatthefollowingtopics:
WhyuseaSaaSserviceoverself-hostedorreal-timemetrics?Whatservicesareavailableandwhatdotheyoffer?InstallationofagentsforSysdigCloud,Datadog,andNewReliconthehostmachinesConfigurationoftheagentstoshipmetrics
![Page 167: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/167.jpg)
AwordaboutexternallyhostedservicesSofar,toworkthroughtheexamplesinthisbook,wehaveusedlocallyhostedvirtualserversthatarelaunchedusingvagrant.Duringthischapter,wearegoingtouseservicesthatneedtobeabletocommunicatewithyourhostmachine,soratherthantryingtodothisusingyourlocalmachine,itsabouttimeyoutookyourhostmachineintothecloud.
Aswearegoingtostartandstoptheremotehostswhilewelookattheservices,itpaystouseapubliccloud,asweonlygetchargedforwhatweuse.
Thereareseveralpubliccloudservicesthatyoucanusetoevaluatethetoolscoveredinthischapter,whichoneyouchoosetouseisuptoyou,youcoulduse:
DigitalOcean:https://www.digitalocean.com/AmazonWebServices:https://aws.amazon.com/MicrosoftAzure:https://azure.microsoft.com/VMwarevCloudAir:http://vcloud.vmware.com/
Oruseyourownpreferredprovider,theonlypre-requisiteisthatyourserverispublicallyaccessible.
ThischapterassumesthatyouarecapableoflaunchingeitheraCentOS7orUbuntu14.04cloudinstanceandyouunderstandthatyouwilllikelyincurchargeswhilethecloudinstanceisupandrunning.
![Page 168: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/168.jpg)
DeployingDockerinthecloudOnceyouhavelaunchedyourcloudinstance,youcanbootstrapDockerinthesamewaythatyouinstalledusingvagrant.Inthechapter6folderoftheGitrepository,therearetwoseparatescriptstodownloadandinstalltheDockerengineandcomposeitonyourcloudinstance.
ToinstallDocker,ensurethatyourcloudinstanceisupdatedbyrunning:
sudoyumupdate
FortheCentOSinstanceofyourUbuntu,runthefollowingcommand:
sudoapt-getupdate
Onceupdated,runthefollowingcommandtoinstallthesoftware.Duetothedifferencesinthewaydifferentcloudenvironmentsareconfigured,itisbesttoswitchovertotherootusertoruntheremainderofthecommands,todothis,run:
sudosu-
Nowyouwillbeabletoruntheinstallscriptusingthefollowingcommand:
curl-fsShttps://raw.githubusercontent.com/russmckendrick/monitoring-
docker/master/chapter06/install_docker/install_docker.sh|bash
Tocheckthateverythingworksasexpected,runthefollowingcommand:
dockerrunhello-world
Youshouldseesomethingsimilartotheterminaloutput,asshowninthefollowingscreenshot:
![Page 169: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/169.jpg)
WecanstarttolookattheSasSservicesonceyouhaveDockerupandrunning.
![Page 170: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/170.jpg)
![Page 171: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/171.jpg)
WhyuseaSaaSservice?Youmayhavenoticedwhileworkingwiththeexamplesinthepreviouschaptersthatthetoolswehaveusedcanpotentiallyusemanyresourcesifweneededtostartcollectingmoremetrics,especiallyiftheapplicationswewanttomonitorareinproduction.
TohelpshiftthisloadfrombothstorageandCPU,anumberofcloud-basedSaaSoptionshavestartedofferingsupporttorecordmetricsforyourcontainers.Manyoftheseserviceswerealreadyofferingservicestomonitorservers,soaddingsupportforcontainersseemedanaturalprogressionforthem.
Thesetypicallyrequireyoutoinstallanagentonyourhostmachine,onceinstalled,theagentwillsitinthebackgroundandreporttotheservices,normallycloud-basedandAPIservices.
AfewoftheservicesallowyoutodeploytheagentsasDockercontainers.Theyoffercontainerizedagentssothattheservicecanrunonstrippeddownoperatingsystems,suchas:
CoreOS:https://coreos.com/RancherOS:http://rancher.com/rancher-os/Atomic:http://www.projectatomic.io/UbuntuSnappyCore:https://developer.ubuntu.com/en/snappy/
Theseoperatingsystemsdifferfromtraditionalones,asyoucannotinstallservicesonthemdirectly;theironlypurposeistorunaservice,suchasDocker,sothatyoucanlaunchtheservicesorapplicationsyouneedtoberunascontainers.
Aswearerunningfulloperatingsystemsasourhostsystems,wedonotneedthisoptionandwillbedeployingtheagentsdirectlytothehosts.
TheSaaSoptionsthatwearegoingtolookatinthischapterareasfollows:
SysdigCloud:https://sysdig.com/product/Datadog:https://www.datadoghq.com/NewRelic:http://newrelic.com
Theyallofferfreetrialsandtwoofthemofferfreecut-downversionsofthemainservice.Onthefaceofit,theymightallappeartooffersimilarservices;however,whenyoustarttousethem,youwillimmediatelynoticethattheyareinfactallverydifferentfromeachother.
![Page 172: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/172.jpg)
![Page 173: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/173.jpg)
SysdigCloudInthepreviouschapter,wehadalookattheopensourceversionofSysdig.WesawthatthereisagreatncursesinterfacecalledcSysdiganditallowsustonavigatethroughallthedatathatSysdigiscollectingaboutourhost.
ThesheeramountofmetricsanddatacollectedbySysdigmeansthatyouhavetotrytostayontopofiteitherbyshippingyourfilesofftheserver,maybetoAmazonSimpleStorageService(S3),ortosomelocalsharedstorage.Inaddition,youcanquerythedatainthecommandlineonthehostitselforonyourlocalmachineusinganinstallationofthecommand-linetools.
ThisiswhereSysdigCloudcomesintoplay;itoffersaweb-basedinterfacetothemetricsthatSysdigcapturesalongwiththeoptionstoshiptheSysdigcapturesoffyourhostmachineeithertoSysdig’sownstorageortoyourS3bucket.
Sysdigcloudoffersthefollowingfunctionality:
ContainerVision™Real-TimeDashboardHistoricalReplayDynamicTopologyAlerting
Aswellas,theoptiontotriggeracaptureonanyofyourhostsandatanytime.
SysdigdescribesContainerVisionas:
“SysdigCloud’spatent-pendingcoretechnology,ContainerVision,istheonlymonitoringtechnologyonthemarketdesignedspecificallytorespecttheuniquecharacteristicsofcontainers.ContainerVisionoffersyoudeepandcomprehensivevisibilityintoallaspectsofyourcontainerizedenvironment-applications,infrastructures,servers,andnetworks-allwithouttheneedtopolluteyourcontainerswithanyextrainstrumentation.Inotherwords,ContainerVisiongivesyou100%visibilityintotheactivityinsideyourcontainers,fromtheoutside.”
BeforewedelveintoSysdigCloudanyfurther,Ishouldpointoutthatthisisacommercialserverandatthetimeofwriting,itcosts$25perhostpermonth.Thereisalsoa14-dayfullyfeaturedtrialavailable.Ifyouwishtoworkthroughtheagentinstallationandfollowtheexampleinthischapter,youwillneedanactiveaccountthatrunseitheronthe14-daytrialorapaidsubscription.
Signupfora14-dayfreetrial:https://sysdig.com/Detailsonpricing:https://sysdig.com/pricing/Introductiontothecompany:https://sysdig.com/company/
![Page 174: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/174.jpg)
InstallingtheagentTheagentinstallationissimilartoinstallingtheopensourceversion;youneedtoensurethatyourcloudhostisrunninganup-to-datekernelandthatyouarealsobootedintothekernel.
Somecloudproviderskeepatightcontrolonthekernelsyoucanbootinto(forexample,DigitalOcean),andtheydonotallowyoutomanageyourkernelonthehostitself.Instead,youneedtochoosethecorrectversionthroughtheircontrolpanel.
Onceyouhavethecorrectkernelinstalled,youshouldbeabletorunthefollowingcommandtoinstalltheagent.Ensurethatyoureplacetheaccesskeyattheendofthecommandwithyourownaccesskey,whichcanbefoundonyourUserProfilepageorontheagentinstallationpages;youcanfindtheseat:
UserProfile:https://app.sysdigcloud.com/#/settings/userAgentInstallation:https://app.sysdigcloud.com/#/settings/agentInstallation
Thecommandtorunis:
curl-shttps://s3.amazonaws.com/download.draios.com/stable/install-agent|
sudobash-s—--access_keywn5AYlhjRhgn3shcjW14y3yOT09WsF7d
Theshelloutputshouldlooklikethefollowingscreen:
Oncetheagenthasbeeninstalled,itwillimmediatelystarttoreportthedatabacktoSysdigCloud.IfyouclickonExplore,youwillseeyourhostmachineandtherunning
![Page 175: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/175.jpg)
containers:
Asyoucanseehere,IhavemyhostmachineandfourcontainersrunningaWordPressinstallationsimilartotheoneweusedinthepreviouschapter.Fromhere,wecanstarttodrilldownintoourmetrics.
TolaunchtheWordPressinstallationonyourcloud-basedmachine,runthefollowingcommandsastherootuser:
sudosu-
mkdir~/wordpress
curl-Lhttps://raw.githubusercontent.com/russmckendrick/monitoring-
docker/master/chapter05/wordpress/docker-compose.yml>~/wordpress/docker-
compose.yml
cd~/wordpress
docker-composeup-d
![Page 176: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/176.jpg)
ExploringyourcontainersTheSysdigCloudwebinterfacewillfeelinstantlyfamiliar,asitsharesasimilardesignandoverallfeelingwithcSysdig:
Onceyoustarttodrilldown,youcanseethatabottompaneopensupandthisiswhereyoucanviewthestatistics.OneofthethingsIlikedaboutSysdigCloudisthatitopensupawealthofmetricsandthereshouldbeverylittlethatyouneedtoconfigurefromhere.
Forexample,ifyouwanttoknowwhatprocesseshavebeenconsumingthemostCPUtimeinthelast2hours,clickon2HinthesecondarymenuandthenfromtheViewstabinthebottom-leftclickonSystem:TopProcesses;thiswillgiveyouatableoftheprocesses,orderedbytheonesthathaveusedthemosttime.
Toapplythisviewtoacontainer,clickonacontainerinthetop-sectionandthebottom-sectionwillbeinstantlyupdatedtoreflectthetopCPUutilizationforjustthatcontainer;asmostcontainerswillonlyrunoneortwoprocesses,thismaynotbethatinteresting.So,let’shaveadeeplookattheprocessesthemselves.Let’ssay,weclickedonourdatabase
![Page 177: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/177.jpg)
containerandwewantedinformationonwhatishappeningwithinMySQL.
SysdigCloudcomeswithapplicationoverlays,thesewhenselectedgiveyoumoregranularinformationontheprocesseswithinthecontainer.SelectingtheApp:MySQL/PostgreSQLviewgivesyouaninsightintowhatyourMySQLprocessesarecurrentlydoing:
Here,youcanseethatviewinthebottomsectionhasinstantlyupdatedtogiveawealthofinformationonwhathasbeenhappeninginthelast5minuteswithinMySQL.
SysdigCloudsupportsanumberofapplicationviews,including:
ApacheHAProxy
![Page 178: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/178.jpg)
NGINXRabbitMQRedisTomcat
Eachonegivesyouimmediateaccesstometrics,whicheventhemostexperiencedSysAdminswillfindvaluable.
Youmayhavenoticedthatatthetopofthesecondpaneltherearealsoafewicons,theseallowyouto:
AddAlert:Createsanalertbasedontheviewyouhaveopen;itletsyoutweakthethresholdandalsochoosehowyouarenotified.SysdigCapture:Pressingthisbringsupadialog,whichletsyourecordaSysdigsession.Oncerecorded,thesessionistransferredtoSysdigCloudoryourownS3bucket.Oncethesessionisavailable,youdownloaditorexploreitwithinthewebinterface.SSHConnect:GetsaremoteshellontheserverfromtheSysdigCloudwebinterface;itisusefulifyoudonothaveimmediateaccesstoyourlaptopordesktopmachineandyouwanttodosometroubleshooting.Pintodashboard:Addsthecurrentviewtoacustomdashboard.
Outtheseoptionsicons,the“AddAlert”and“SysdigCapture”optionsareprobablytheonesthatyouwillendupusingthemost.OnefinalviewthatIfoundinteresting,isthetopologyone.Itgivesyouabird’seyeviewofyourhostandcontainers,thisisusefultooseetheinteractionbetweencontainersandhosts:
![Page 179: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/179.jpg)
Here,youcanseemerequestapagefromtheWordPresssite(it’sintheboxontheleft),thisrequesthitsmyhostmachine(theboxontheright).Onceit’sonthehostmachine,itisroutedtotheHAProxycontainer,whichthenpassesthepagerequesttotheWordpress2container.Fromhere,theWordpress2containerinteractswiththedatabasethatisrunningontheMySQLcontainer.
![Page 180: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/180.jpg)
SummaryandfurtherreadingAlthoughSysdigCloudisquiteanewservice,itfeelsinstantlyfamiliarandfullyfeaturedasitisbuiltontopofanalreadyestablishedandrespectedopensourcetechnology.Ifyoulike,thelevelofdetailyougetfromtheopensourceversionofSysdig,thenSysdigCloudisanaturalprogressionforyoutostartstoringyourmetricsoffsiteandalsotoconfigurealerts.SomegoodstartingpointsforlearningmoreaboutSysdigCloudare:
VideoIntroduction:https://www.youtube.com/watch?v=p8UVbpw8n24SysdigCloudBestPractices:http://support.sysdigcloud.com/hc/en-us/articles/204872795-Best-PracticesDashboards:http://support.sysdigcloud.com/hc/en-us/articles/204863385-DashboardsSysdigblog:https://sysdig.com/blog/
TipIfyouhavelaunchedacloudinstanceandarenolongerusingit,nowwouldbeagoodtimetopowertheinstancedownorterminateitaltogether.Thiswillensurethatyoudonotgetbilledforservicesthatyouarenotusing.
![Page 181: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/181.jpg)
![Page 182: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/182.jpg)
DatadogDatadogisafullmonitoringplatform;itsupportsvariousservers,platforms,andapplications.Wikipediadescribestheserviceas:
“DatadogisaSaaS-basedmonitoringandanalyticsplatformforITinfrastructure,operationsanddevelopmentteams.Itbringstogetherdatafromservers,databases,applications,toolsandservicestopresentaunifiedviewoftheapplicationsthatrunatscaleinthecloud.”
Itusesanagentthatisinstalledonyourhostmachine;thisagentsendsmetricsbacktotheDatadogserviceperiodically.Italsosupportmultiplecloudplatforms,suchasAmazonWebServices,MicrosoftAzure,andOpenStacktonameafew.
Theaimistobringallofyourservers,applications,andhostprovidermetricsintoasinglepaneofglass;fromhere,youcancreatecustomdashboardsandalertssothatyoucanbenotifiedofanyproblematanylevelwithinyourinfrastructure.
Youcansignupforafreetrialofthefullserviceathttps://app.datadoghq.com/signup.Youwillneedatleastatrialaccounttoconfigurethealtering,andifyourtrialhasalreadyexpiredtheliteaccountwilldo.FormoredetailonDatadog’spricingstructure,pleaseseehttps://www.datadoghq.com/pricing/.
![Page 183: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/183.jpg)
InstallingtheagentTheagentcanbeinstalledeitherdirectlyonthehostmachineorasacontainer.Toinstalldirectlyonthehostmachine,runthefollowingcommandandmakesurethatyouuseyourownuniqueDD_API_KEY:
DD_API_KEY=wn5AYlhjRhgn3shcjW14y3yOT09WsF7dbash-c"$(curl-L
https://raw.githubusercontent.com/DataDog/dd-
agent/master/packaging/datadog-agent/source/install_agent.sh)"
Toruntheagentasacontainer,usethefollowingcommandandagainmakesurethatyouuseyourownDD_API_KEY:
sudodockerrun-d--namedd-agent-h`hostname`-v
/var/run/docker.sock:/var/run/docker.sock-v
/proc/mounts:/host/proc/mounts:ro-v/sys/fs/cgroup/:/host/sys/fs/cgroup:ro
-eAPI_KEY=wn5AYlhjRhgn3shcjW14y3yOT09WsF7ddatadog/docker-dd-agent
Oncetheagenthasbeeninstalled,itwillcallbacktoDatadogandthehostwillappearinyouraccount.
IftheagenthasbeeninstalleddirectlyonthehostmachinethenwewillneedtoenabletheDockerintegration,ifyouinstalledtheagentusingthecontainerthenthiswillhavebeendoneforyouautomatically.
Todothis,youfirstneedtoallowtheDatadogagentaccesstoyourDockerinstallationbyaddingthedd-agentusertotheDockergroupbyrunningthefollowingcommand:
usermod-a-Gdockerdd-agent
Thenextstepistocreatethedocker.yamlconfigurationfile,luckilytheDatadogagentshipswithanexampleconfigurationfilethatwecanuse;copythisinplaceandthenrestarttheagent:
cp-pr/etc/dd-agent/conf.d/docker.yaml.example/etc/dd-
agent/conf.d/docker.yaml
sudo/etc/init.d/datadog-agentrestart
Nowtheagentonourhostmachinehasbeenconfiguredandthefinalstepistoenabletheintegrationthroughthewebsite.Todothis,gotohttps://app.datadoghq.com/andclickonIntegrations,scrolldownandthenclickoninstallonDocker:
![Page 184: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/184.jpg)
Onceyouclickinstall,youwillbepresentedwithanoverviewoftheintegration,clickontheConfigurationtab,thisgivesinstructionsonhowtoconfiguretheagent;aswehavealreadydonethisstep,youcanclickonInstallIntegration.
YoucanfindmoreinformationoninstallingtheagentandtheintegrationsatthefollowingURLs:
https://app.datadoghq.com/account/settings#agenthttps://app.datadoghq.com/account/settings#integrations
![Page 185: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/185.jpg)
ExploringthewebinterfaceNow,youhaveinstalledtheagentandenabledtheDockerintegration,youcanstarttohavealookaroundthewebinterface.Tofindyourhost,clickon“Infrastructure”intheleft-handsidemenu.
Youshouldbetakentoascreenthatcontainsamapofyourinfrastructure.Likeme,youprobablyonlyhaveasinglehostmachinelisted,clickonitandsomebasicstatsshouldappearatthebottomofthescreen:
Ifyoudon’talreadyhavethecontainerslaunched,nowwouldbeagoodtimetodoso,letslaunchtheWordPressinstallationagainusing:
sudosu-
mkdir~/wordpress
curl-Lhttps://raw.githubusercontent.com/russmckendrick/monitoring-
docker/master/chapter05/wordpress/docker-compose.yml>~/wordpress/docker-
compose.yml
cd~/wordpress
docker-composeup-d
![Page 186: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/186.jpg)
Now,gobacktothewebinterface,fromthereyoucanclickonanyoftheserviceslistedonthehexagon.Thiswillbringupsomebasicmetricsfortheserviceyouhaveselected.Ifyouclickondocker,youwillseealinkforaDockerDashboardamongthevariousgraphsandsoon;clickingthiswilltakeyoutoamoredetailedviewofyourcontainers:
Asyoucansee,thisgivesusournowfamiliarbreakdownoftheCPUandmemorymetrics,alongwithinthetoprightofthedashboardabreakdownofthecontaineractivityonthehostmachine;thislogsevents,suchasstoppingandstartingcontainers.
Datadogcurrentlyrecordsthefollowingmetrics:
docker.containers.running
docker.containers.stopped
docker.cpu.system
docker.cpu.user
docker.images.available
docker.images.intermediate
docker.mem.cache
![Page 187: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/187.jpg)
docker.mem.rss
docker.mem.swap
FromtheMetricsexploreroptionintheleft-handsidemenu,youcanstarttographthesemetricsandonceyouhavethegraphs,youcanthenstarttoaddthemtoyourowncustomdashboardsorevenannotatethem.Whenyouannotateagraph,asnapshotiscreatedandthegraphshowsupintheeventsqueuealongwiththeotherevents,thathavebeenrecorded,suchascontainerstoppingandstarting:
Also,withinthewebinterfaceyoucanconfiguremonitors;theseallowyoutodefinetriggers,whichalertyouifyourconditionsarenotmet.Alertscanbesentase-mailsorviathirdpartyservices,suchasSlack,Campfire,orPagerDuty.
![Page 188: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/188.jpg)
SummaryandfurtherreadingWhileDatadog’sDockerintegrationonlygivesyouthebasicmetricsonyourcontainers,itdoeshaveawealthoffeaturesandintegrationwithotherapplicationsandthirdparties.IfyouneedtomonitoranumberofdifferentservicesalongsideyourDockercontainers,thenthisservicecouldbeforyou:
Homepage:https://www.datadoghq.comOverview:https://www.datadoghq.com/product/MonitoringDockerwithDatadog:https://www.datadoghq.com/blog/monitor-docker-datadog/Twitter:https://twitter.com/datadoghq
TipPleaseRemember
Ifyouhavelaunchedacloudinstanceandarenolongerusingitthennowwouldbeagoodtimetopowertheinstancedownorterminateitaltogether.Thiswillensurethatyoudonotgetbilledforanyservicesyouarenotusing.
![Page 189: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/189.jpg)
![Page 190: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/190.jpg)
NewRelicNewReliccouldbeconsideredthegranddaddyofSaaSmonitoringtools,chancesarethatifyouareadeveloperyouwillhaveheardofNewRelic.IthasbeenaroundforawhileanditisthestandardtowhichotherSaaStoolscomparethemselves.
NewRelichasgrownintoseveralproductsovertheyear,currently,theyoffer:
NewRelicAPM:Themainapplicationperformance-monitoringtool.ThisiswhatmostpeoplewillknowNewRelicfor;thistollgivesyouthecodelevelvisibilityofyourapplication.NewRelicMobile:Asetoflibrariestoembedintoyournativemobileapps,givingAPMlevelsofdetailforyouriOSandandroidapplication.NewRelicInsights:Ahigh-levelviewofallofthemetricscollectedbyotherNewRelicservices.NewRelicServers:Monitorsyourhostservers,recordingmetricsaroundCPU,RAM,andstorageutilization.NewRelicBrowser:Givesyouaninsightintowhathappenswithyourweb-basedapplicationsoncetheyleaveyourserversandenteryourenduser’sbrowserNewRelicSynthetics:Monitorsyourapplicationsresponsivenessfromvariouslocationsaroundtheworld.
RatherthanlookingatalloftheseofferingsthatgiveusaninsightintowhatishappeningwithourDocker-basedcode,asthat’sprobablyawholebookonitsown,wearegoingtotakealookattheserverproduct.
TheservermonitoringserviceofferedbyNewRelicisavailablefreeofcharge,youjustneedanactiveNewRelicaccount,youcansignupforanaccountathttps://newrelic.com/signup/detailsonNewRelicspricingcanbefoundattheirhomepageathttp://newrelic.com/.
![Page 191: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/191.jpg)
InstallingtheagentLiketheotherSaaSofferingswehavelookedatinthischapter,NewRelicServershasahost-basedclient,whichneedstobeabletoaccesstheDockerbinary.ToinstallthisonaCentOSmachine,runthefollowing:
yuminstallhttp://download.newrelic.com/pub/newrelic/el5/i386/newrelic-
repo-5-3.noarch.rpm
yuminstallnewrelic-sysmond
ForUbuntu,runthefollowingcommand:
echo'debhttp://apt.newrelic.com/debian/newrelicnon-free'|sudotee
/etc/apt/sources.list.d/newrelic.list
wget-O-https://download.newrelic.com/548C16BF.gpg|sudoapt-keyadd-
apt-getupdate
apt-getinstallnewrelic-sysmond
Nowthatyouhavetheagentinstalled,youneedtoconfiguretheagentwithyourlicensekey.Youcandothiswiththefollowingcommandandmakesurethatyouaddyourlicense,whichcanbefoundinyoursettingspage:
nrsysmond-config--setlicense_key=wn5AYlhjRhgn3shcjW14y3yOT09WsF7d
Nowthattheagentisconfigured,weneedtoaddthenewrelicusertothedockergroupsothattheagenthasaccesstoourcontainerinformation:
usermod-a-Gdockernewrelic
Finally,weneedtostarttheNewRelicServeragentandrestartDocker:
/etc/init.d/newrelic-sysmondrestart
/etc/init.d/dockerrestart
TipRestartingDockerwillstoptherunningcontainersthatyouhave;makesurethatyoumakeanoteoftheseusingdockerpsandthenstartthemmanuallyandbackuponcetheDockerservicerestarts.
YoushouldseeyourserverappearonyourNewReliccontrolpanelafterafewminutes.
![Page 192: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/192.jpg)
ExploringthewebinterfaceOnceyouhavetheNewRelicserveragentinstalled,configured,andrunningonyourhostmachine,youwillseesomethingsimilartothefollowingscreenshotwhenclickingonServersinthetopmenu:
Selectingtheserverwillallowyoutostartexploringthevariousmetricsthattheagentisrecording:
![Page 193: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/193.jpg)
Fromhere,youhavetheoptiontodrilldownfurther:
Overview:GivesaquickoverviewofyourhostmachineProcesses:ListsalloftheprocessesthatarerunningbothonthehostmachineandwithinyourcontainersNetwork:LetsyouseethenetworkactivityforyourhostmachineDisks:GivesyoudetailsonhowmuchspaceyouareusingDocker:ShowsyoutheCPUandmemoryutilizationforyourcontainers
Asyoumayhaveguessed,wearegoingtobelookingattheDockeritemnext,clickonitandyouwillseealistofyouractiveimages:
![Page 194: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/194.jpg)
YoumayhavenoticedadifferencebetweenNewRelicandtheotherservices,asyoucanseeNewRelicdoesnotshowyoutherunningcontainers,insteaditshowsyoutheutilizationbyDockerimage.
Intheprecedingscreenshot,IhavefourcontainersactiveandrunningtheWordPressinstallationwehaveusedelsewhereinthebook.IfIwantedabreakdownpercontainer,thenIwouldbeoutofluck,asdemonstratedbythefollowingscreen:
![Page 195: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/195.jpg)
It’saprettydullscreen,butitgivesyouanideaaboutwhatyouwillseeifyouarerunningmultiplecontainersthathavebeenlaunchedusingthesameimage.Sohowisthisuseful?Well,coupledwiththeotherservicesofferedbyNewRelic,itcangiveyouanindicationofwhatyourcontainerswereuptowhenaproblemoccurredwithinyourapplication.IfyourememberthePetsversusCattleversusChickensanalogyfromChapter1,IntroductiontoDockerMonitoring,wedon’tnecessarilycarewhichcontainerdidwhat;wejustwanttoseetheimpactithadduringtheissuewearelookinginto.
![Page 196: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/196.jpg)
SummaryandfurtherreadingDuetotheamountofproductsitoffers,NewReliccanbealittledauntingatfirst,butifyouworkwithadevelopmentteamthatactivelyusesNewRelicwithintheirday-to-dayworkflow,thenhavingalloftheinformationaboutyourinfrastructurealongsidethisdatacanbebothvaluableandnecessary,especiallyduringanissue:
NewRelicServermonitoring:http://newrelic.com/server-monitoringNewRelicandDocker:http://newrelic.com/docker/Twitter:https://twitter.com/NewRelic
TipIfyouhavelaunchedacloudinstanceandarenolongerusingitthen,nowisagoodtimetopowertheinstancedownorterminateitaltogether,thiswillensureyoudonotgetbilledforanyservicesyouarenotusing.
![Page 197: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/197.jpg)
![Page 198: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/198.jpg)
SummaryWhichSaaSserviceyouchoosedependsonyourcircumstances,thereareanumberofquestionsyoushouldaskyourselfbeforeyoustartevaluatingtheSaaSofferings:
Howmanycontainerswouldyouliketomonitor?Howmanyhostmachinesdoyouhave?Isthereanon-containerizedinfrastructureyouneedtomonitor?Whatmetricsdoyouneedfromthemonitoringservice?Howlongshouldthedataberetainedfor?Couldotherdepartments,suchasdevelopmentandutilizetheservice?
WecoveredjustthreeoftheavailableSaaSoptionsinthischapter,thereareotheroptionsavailable,suchas:
Ruxit:https://ruxit.com/docker-monitoring/Scout:https://scoutapp.com/plugin_urls/19761-docker-monitorLogentries:https://logentries.com/insights/server-monitoring/Sematext:http://sematext.com/spm/integrations/docker-monitoring.html
Monitoringserversandservicesareonlyasgoodasthemetricsyoucollect,ifpossibleandifyourbudgetallows,youshouldtakefulladvantageoftheservicesofferedbyyourchosenproviders,asmoredatabeingrecordedbyasingleproviderwillonlybenefityouwhenitcomestoanalyzingproblemswithnotonlyyourcontainerizedapplications,butalsowithyourinfrastructure,codeandevenyourcloudprovider.
Forexample,ifyouaremonitoringyourhostmachineusingthesameserviceasyouusetomonitoryourcontainers,thenbyusingthecustomgraphingfunctions,youshouldbeabletocreateoverlaygraphsofCPUloadspikesofbothyourhostmachineandyourcontainer.Thisisalotmoreusefulthantryingtocomparetwodifferentgraphsfromdifferentsystemssidebyside.
Inthenextchapter,wewilllookatanoften-overlookedpartofmonitoring:shippingyourlogfilesawayfromyourcontainers/hoststoasinglelocationsothattheycanbemonitoredandreviewed.
![Page 199: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/199.jpg)
![Page 200: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/200.jpg)
Chapter7.CollectingApplicationLogsfromwithintheContainerOneofthemostoverlookedpartsofmonitoringarelogfilesgeneratedbytheapplicationorservicessuchasNGINX,MySQL,Apache,andsoon.SofarwehavelookedatvariouswaysofrecordingtheCPUandRAMutilizationoftheprocesseswithinyourcontainersareatapointintime,nowitstimetodothesameforthelogfiles.
IfyouarerunningyourcontainersasCattleorChickens,thenthewayyoudealwiththeissuestodestroyandrelaunchyourcontainereithermanuallyorautomaticallyisimportant.Whilethisshouldfixtheimmediateproblem,itdoesnothelpwithtrackingdowntherootcauseoftheissueandifyoudon’tknowthatthenhowcanyouattempttoresolveitsothatitdoesnotreoccur.
Inthischapter,wewilllookathowwecangetthecontentofthelogfilesfortheapplicationsrunningwithinourcontainerstothecentrallocationsothattheyareavailable,evenifyouhavetodestroyandreplaceacontainer.Wearegoingtocoverthefollowingtopicsinthischapter:
Howtoviewcontainerlogs?Deployingan“ELK”stackusingaDockercontainersstacktoshipthelogstoReviewingyourlogsWhatthirdpartyoptionsareavailable?
![Page 201: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/201.jpg)
ViewingcontainerlogsLikethedockertopcommand,thereisaverybasicwayofviewinglogs.Whenyouusethedockerlogscommand,youareactuallyviewingtheSTDOUTandSTDERRoftheprocessesthatarerunningwithinthecontainer.
NoteFormoreinformationonStandardStreams,pleaseseehttps://en.wikipedia.org/wiki/Standard_streams.
Asyoucanseefromthefollowingscreenshot,thesimplestthingyouhavetodoisrundockerlogsfollowedbyyourcontainername:
Toseethisonyourownhost,let’slaunchtheWordPressinstallationfromchapter05usingthefollowingcommands:
cd/monitoring_docker/chapter05/wordpress/
docker-composeup–d
dockerlogswordpress_wordpress1_1
Youcanextendthedockerlogscommandbyaddingthefollowingflagsbeforeyourcontainername:
-for--followwillstreamthelogsinrealtime-tor--timestampswillshowatimestampatthestartofeachline--tail="5"willshowthelastxnumberoflines--since="5m00s"willshowonlytheentriesforthelast5minutes
UsingtheWordPressinstallationthatwehavejustlaunched,tryrunningthefollowingcommands:
dockerlogs--tail="2"wordpress_wordpress1_1
Thiswillshowthelasttwolinesofthelogs,youcanaddtimestampsusing:
![Page 202: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/202.jpg)
dockerlogs--tail="2"–timestampswordpress_wordpress1_1
Asyoucanseeinthefollowingterminaloutput,youcanalsostringcommandstogethertoformaverybasicquerylanguage:
Thedownsideofusingdockerlogsisexactlythesameasusingdockertop,inthatitisonlyavailablelocallyandthelogsareonlypresentforthetimethecontainerisaround,youcanviewthelogsofastoppedcontainer,butoncethecontainerisremoved,soarethelogs.
![Page 203: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/203.jpg)
![Page 204: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/204.jpg)
ELKStackSimilartosomeofthetechnologiesthatwehavecoveredinthisbook,anELKstackreallydeservesabookbyitself;infact,therearebooksforeachoftheelementsthatmakeanELKstack,theseelementsare:
Elasticsearchisapowerfulsearchserver,whichhasbeendevelopedwithmodernworkloadsinmindLogstashsitsbetweenyourdatasourceandElasticsearchservices;ittransformsyourdatainrealtimetoaformat,whichElasticsearchcanunderstand.KibanaisinfrontofyourElasticsearchservicesandallowsyoutoqueryyourdatainafeature-richweb-baseddashboard.
TherearealotofmovingpartswithanELKstack,sotosimplifythings,wewilluseaprebuiltstackforthepurposeoftesting;however,youprobablydon’twanttousethisstackinproduction.
![Page 205: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/205.jpg)
StartingthestackLet’slaunchafreshvagranthostonwhichtoruntheELKstack:
[russ@mac~]$cd~/Documents/Projects/monitoring-docker/vagrant-centos/
[russ@mac~]$vagrantup
Bringingmachine'default'upwith'virtualbox'provider…
==>default:Importingbasebox'russmckendrick/centos71'...
==>default:MatchingMACaddressforNATnetworking…
==>default:Checkingifbox'russmckendrick/centos71'isuptodate…
.....
==>default:=>Installingdocker-engine…
==>default:=>Configuringvagrantuser…
==>default:=>Startingdocker-engine…
==>default:=>Installingdocker-compose…
==>default:=>FinishedinstallationofDocker
[russ@mac~]$vagrantssh
Now,wehaveacleanhostthatisupandrunning,wecanstartthestackbyrunningthefollowingcommands:
[vagrant@docker~]$cd/monitoring_docker/chapter07/elk/
[vagrant@dockerelk]$docker-composeup-d
Asyoumayhavenoticed,itdidmorethatjustpulldownsomeimages;whathappenedwas:
AnElasticsearchcontainerwaslaunchedusingtheofficialimagefromhttps://hub.docker.com/_/elasticsearch/.ALogstashcontainerwaslaunchedusingtheofficialimagefromhttps://hub.docker.com/_/logstash/,itwasalsolaunchedwithourownconfiguration,whichmeansthatourinstallationlistensforlogssentfromLogspout(moreaboutthatinaminute).AcustomKibanaimagewasbuiltusingtheofficialimagefromhttps://hub.docker.com/_/kibana/.AllitdidwasaddasmallscripttoensurethatKibanadoesn’tstartuntilourElasticsearchcontainerisfullyupandrunning.Itwasthenlaunchedwithacustomconfigurationfile.AcustomLogspoutcontainerwasbuiltusingtheofficialimagefromhttps://hub.docker.com/r/gliderlabs/logspout/andthenweaddedacustommodulesothatLogspoutcouldtalktoLogstash.
Oncedocker-composehasfinishedbuildingandlaunchingthestackyoushouldbeabletoseethefollowingwhenrunningdocker-composeps:
![Page 206: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/206.jpg)
WenowhaveourELKstackupandrunning,asyoumayhavenoticed,thereisanadditionalcontainerrunningandgivingusanELK-Lstack,sowhatisLogspout?
![Page 207: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/207.jpg)
LogspoutIfweweretolaunchElasticsearch,Logstash,andKibanacontainers,weshouldhaveafunctioningELKstackbutwewillhavealotofconfigurationtodotogetourcontainerlogsintoElasticsearch.
SinceDocker1.6,youhavebeenabletoconfigureloggingdrivers,thismeantthatitispossibletolaunchacontainerandhaveitsenditsSTDOUTandSTDERRtoaSyslogServer,whichwillbeLogstashinourcase;however,thismeansthatyouwillhavetoaddsomethingsimilartothefollowingoptionseachtimewelaunchacontainer:
--log-driver=syslog--log-optsyslog-address=tcp://elk_logstash_1:5000
ThisiswhereLogspoutcomesin,ithasbeendesignedtocollectalloftheSTDOUTandSTDERRmessagesonahostmachinebyinterceptingthemessagesthatarebeingcollectedbytheDockerprocessandthenitroutesthemtoourLogstashinstanceinaformatthatisunderstoodbyElasticsearch.
Justasthelog-driver,itsupportsSyslogoutofthebox;however,thereisathirdpartymodulethattransformstheoutputtoJSON,whichLogstashunderstands.Asapartofourbuildwedownloaded,compiledandconfiguredthemodule.
YoucanfindoutmoreaboutLogspoutandloggingdriversatthefollowing:
OfficialLogspoutimage:https://hub.docker.com/r/gliderlabs/logspout/LogspoutProjectpage:https://github.com/gliderlabs/logspoutLogspoutLogstashmodule:https://github.com/looplab/logspout-logstashDocker1.6releasenotes:https://blog.docker.com/2015/04/docker-release-1-6/DockerLoggingDrivers:https://docs.docker.com/reference/logging/overview/
![Page 208: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/208.jpg)
ReviewingthelogsSonow,wehaveourELKrunningandamechanisminplacetostreamalloftheSTDOUTandSTDERRmessagesgeneratedbyourcontainersintoLogstash,whichinturnroutesthedataintoElasticsearch.NowitstimetoviewthelogsinKibana.ToaccessKibanagotohttp://192.168.33.10:8080/inyourbrowser;whenyouaccessthepage,youwillbeaskedtoConfigureanindexpattern,thedefaultindexpatternwillbefineforourneedssojustclicktheCreatebutton.
Onceyoudo,youwillseealistoftheindexpatterns,thesearetakendirectlyfromtheLogspoutoutput,andyoushouldnoticethefollowingitemsintheindex:
docker.name:Thenameofcontainerdocker.id:ThefullcontainerIDdocker.image:Thenameoftheimageusedtolaunchtheimage
Fromhere,ifyouweretoclickonDiscoverinthetopmenuyouwouldseesomethingsimilartothefollowingpage:
![Page 209: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/209.jpg)
Inthescreenshot,youwillseethatIhaverecentlylaunchedtheWordPressstackandwehavebeenusingitthroughoutthebook,usingthefollowingcommands:
[vagrant@dockerelk]$cd/monitoring_docker/chapter05/wordpress/
[vagrant@dockerwordpress]$docker-composeup–d
Togiveyouanideaofwhatisbeinglogged,hereistherawJSONtakenfromElasticseachforrunningtheWordPressinstallationscript:
{
"_index":"logstash-2015.10.11",
"_type":"logs",
"_id":"AVBW8ewRnBVdqUV1XVOj",
"_score":null,
"_source":{
"message":"172.17.0.11--[11/Oct/2015:12:48:26+0000]\"POST/wp-
![Page 210: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/210.jpg)
admin/install.php?step=1HTTP/1.1\"2002472\"http://192.168.33.10/wp-
admin/install.php\"\"Mozilla/5.0(Macintosh;IntelMacOSX10_11)
AppleWebKit/601.1.56(KHTML,likeGecko)Version/9.0Safari/601.1.56\"",
"docker.name":"/wordpress_wordpress1_1",
"docker.id":
"0ba42876867f738b9da0b9e3adbb1f0f8044b7385ce9b3a8a3b9ec60d9f5436c",
"docker.image":"wordpress",
"docker.hostname":"0ba42876867f",
"@version":"1",
"@timestamp":"2015-10-11T12:48:26.641Z",
"host":"172.17.0.4"
},
"fields":{
"@timestamp":[
1444567706641
]
},
"sort":[
1444567706641
]
}
Fromhere,youcanstarttousethefreetextsearchboxandbuildupsomequitecomplexqueriestodrilldownintoyourcontainer’sSTDOUTandSTDERRlogs.
![Page 211: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/211.jpg)
Whataboutproduction?Asmentionedatthetopofthissection,youprobablydon’twanttorunyourproductionELKstackusingthedocker-composefile,whichaccompaniesthischapter.Firstofall,youwillwantyourElasticsearchdatatobestoredonapersistentvolumeandyoumorethanlikelywantyourLogstashservicetobehighlyavailable.
TherearenumerousguidesonhowtoconfigureahighlyavailableELKstack,aswellas,thehostedservicesfromElastic,whichisthecreatorofElasticsearch,andalsoAmazonWebServices,whichoffersanElasticsearchservice:
ELKtutorial:https://www.youtube.com/watch?v=ge8uHdmtb1MFoundfromElastic:https://www.elastic.co/foundAmazonElasticsearchService:https://aws.amazon.com/elasticsearch-service/
![Page 212: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/212.jpg)
![Page 213: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/213.jpg)
LookingatthirdpartyoptionsThereareafewoptionswhenitcomestohostingcentralloggingforyourcontainersexternaltoyourownserverinstances.Someoftheseare:
LogEntries:https://logentries.com/Loggly:https://www.loggly.com/
Bothoftheseservicesofferafreetier.LogEntriesalsooffersa“LogentriesDockerFree”accountthatyoucanfindoutmoreaboutathttps://logentries.com/docker/
NoteAsrecommendedintheExploringThirdPartyOptionschapter,itisbesttouseacloudservicewhenevaluatingthirdpartyservices.Theremainderofthischapterassumesthatyouarerunningacloudhost.
Let’slookatconfiguringtheLogEntriesonanexternalserver,firstofallyouneedtohavesignedupforanaccountathttps://logentries.com/.Onceyouhavesignedup,youshouldbetakentoapageinwhichyourlogswilleventuallybedisplayed.
Tostart,clickontheAddnewlogbuttoninthetop-rightcornerofthepageandthenclicktheDockerlogointhePlatformssection.
YouhavetonameyoursetoflogsintheSelectsetsection,sogiveanametoyourlogset.YounowhavethechoiceofbuildingyourowncontainerlocallyusingtheDockerfilefromhttps://github.com/logentries/docker-logentries:
gitclonehttps://github.com/logentries/docker-logentries.git
cddocker-logentries
dockerbuild-tdocker-logentries.
Afterrunningtheprecedingcommand,youwillgetthefollowingoutput:
![Page 214: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/214.jpg)
Beforeyoustartyourcontainer,youwillneedtogenerateanaccesstokenforyourlogsetbyclickingonGenerateLogToken.Onceyouhavethis,youcanlaunchyourlocallybuiltcontainersusingthefollowingcommand(replacethetokenwiththeoneyouhavejustgenerated):
dockerrun-d-v/var/run/docker.sock:/var/run/docker.sockdocker-
logentries-twn5AYlh-jRhgn3shc-jW14y3yO-T09WsF7d-j
YoucandownloadtheimagestraightfromtheDockerhubbyrunning:
dockerrun-d-v/var/run/docker.sock:/var/run/docker.sock
logentries/docker-logentries-twn5AYlh-jRhgn3shc-jW14y3yO-T09WsF7d–j
It’sworthpointingoutthattheautomaticallygeneratedinstructionsgivenbyLogEntrieslaunchesthecontainerintheforeground,ratherthandetachingfromthecontaineronceithasbeenlaunchedliketheprecedinginstructions.
Onceyouhavethedocker-logentriescontainerupandrunning,youshouldstarttoseelogsfromyourcontainerstreamedinreal-timetoyourdashboard:
![Page 215: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/215.jpg)
Fromhere,youwillbeabletoqueryyourlogs,createdashboards,andcreatealertsdependingontheaccountoptionyougofor.
![Page 216: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/216.jpg)
![Page 217: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/217.jpg)
SummaryInthischapter,wehavecoveredhowtoquerytheSTDOUTandSTDERRoutputfromyourcontainersusingthetoolbuiltintoDocker,howtoshipthemessagestoanexternalsource,ourELKstack,andhowtostorethemessagesevenafterthecontainerhasbeenterminated.Finally,wehavelookedatafewofthethird-partyserviceswhoofferservicestowhichyoucanstreamyourlogs.
Sowhygotoallofthiseffort?Monitoringisn’tjustaboutkeepingandqueryingCPU,RAM,HDD,andNetworkutilizationmetrics;thereisnopointinknowingiftherewasaCPUspikeanhouragoifyoudon’thaveaccesstothelogfilestoseeifanyerrorswerebeinggeneratedatthattime.
Theserviceswehavecoveredinthischapterofferthequickestandmostefficientinsightsintowhatcanquicklybecomeacomplexdataset.
Inthenextchapter,wewilllookatalloftheservicesandconceptswehavecoveredinthebookandapplythemtosomerealworldscenarios.
![Page 218: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/218.jpg)
![Page 219: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/219.jpg)
Chapter8.WhatAretheNextSteps?Inthisfinalchapter,wewilllookatthenextstepsyoucantaketomonitoryourcontainers,bytalkingaboutthebenefitsofaddingalertstoyourmonitoring.Also,wewillcoversomedifferentscenariosandalsowhichtypeofmonitoringisappropriateforeachofthem:
Commonproblems(performance,availability,andsoon)andwhichtypeofmonitoringisbestforyoursituation.Whatarethebenefitsofalertingonthemetricsyouarecollectingandwhataretheoptions?
![Page 220: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/220.jpg)
SomescenariosTolookatwhichtypeofmonitoringyoumightwanttoimplementforyourcontainer-basedapplications,weshouldworkthroughafewdifferentexampleconfigurationsthatyourcontainer-basedapplicationscouldbedeployinginto.First,let’sremindourselvesaboutPets,Cattle,Chickens,andSnowflakes.
![Page 221: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/221.jpg)
Pets,Cattle,Chickens,andSnowflakesBackintheChapter1,IntroductiontoDockerMonitoring,wespokeaboutPets,Cattle,Chickens,andSnowflakes;inthatchapter,wedescribedwhateachtermmeantwhenitwasappliedtomodernclouddeployments.Here,wewillgointoalittlemoredetailabouthowthetermscanbeappliedtoyourcontainers.
PetsForyourcontainerstobeconsideredaPet,youwillbemorethanlikelytoberunningeitherasingleorasmallnumberoffixedcontainersonadesignatedhost.
Eachoneofthesecontainerscouldbeconsideredasinglepointoffailure;ifanyoneofthemgoesdown,itwillmorethanlikelyresultinerrorsforyourapplication.Worststill,ifthehostmachinegoesdownforanyreason,yourentireapplicationwillbeoffline.
ThisisatypicaldeploymentmethodformostofourfirststepswithDocker,andinnowayshoulditbeconsideredbad,frownedupon,ornotrecommend;aslongasyouareawareofthelimitations,youwillbefine.
Thispatterncanalsobeusedtodescribemostdevelopmentenvironments,asyouareconstantlyreviewingitshealthandtuningasneeded.
YouwillmorethanlikelybehostingthemachineonyourlocalcomputeroronahostingservicesuchasDigitalOcean(https://www.digitalocean.com/).
CattleForthebulkofproductionorbusinesscriticaldeployments,youshouldaimtolaunchyourcontainersinaconfigurationthatallowsthemtoautomaticallyrecoverthemselvesafterafailure,or,whenmorecapacityisneeded,additionalcontainersarelaunchedandthenterminatedwhenthescalingeventisover.
Youwillmorethanlikelybeusingapubliccloud-basedserviceasfollows:
AmazonEC2ContainerService:https://aws.amazon.com/ecs/GoogleContainerEngine:https://cloud.google.com/container-engine/JoyentTriton:https://www.joyent.com/blog/understanding-triton-containers/
Alternatively,youwillbehostingonyourownserversusingaDocker-friendlyandcluster-awareoperatingsystemasfollows:
CoreOS:https://coreos.com/RancherOS:http://rancher.com/rancher-os/
Youwon’tcaresomuchastowhereacontainerislaunchedwithinyourclusterofhosts,aslongasyoucanroutetraffictoit.Toaddmorecapacitytothecluster,youwillbebringingupadditionalhostswhenneededandremovingthemfromtheclusterwhennotneededinordertosaveoncosts.
Chickens
![Page 222: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/222.jpg)
Itsmorethanlikelyyouwillbeusingcontainerstolaunch,processdata,andthenterminate.Thiscanhappenanytimefromonceadaytoseveraltimesaminute.Youwillbeusingadistributedschedulerasfollows:
KubernetesbyGoogle:http://kubernetes.io/ApacheMesos:http://mesos.apache.org/
Becauseofthis,youwillhavealargenumberofcontainerslaunchingandterminatingwithinyourcluster;youdefinitelywon’tcareaboutwhereacontainerislaunchedorevenhowtrafficisroutedtoit,aslongasyourdataisprocessedcorrectlyandpassedbacktoyourapplication.
LiketheclusterdescribedintheCattlesection’sdescription,hostswillbeaddedandremovedautomatically,probablyinresponsetoscheduledpeakssuchasendofmonthreportingorseasonalsalesandsoon.
SnowflakesIhopeoneofthethingsyoutookawayfromthefirstchapteristhatifyouhaveanyserversorservicesthatyouconsiderbeingSnowflakes,thenyoushoulddosomethingtoretirethemassoonaspossible.
Luckily,duetothewaythecontainerizingofyourapplicationsworks,youshouldneverbeabletocreateasnowflakeusingDocker,asyourcontainerizedenvironmentshouldalwaysbereproducible,eitherbecauseyouhavetheDockerfile(everyonemakesbackupsright?)oryouhaveaworkingcopyofthecontainerimagebecauseyouhaveexportedthecontainerasawholeusingthebuilt-intools.
NoteSometimesitmaynotbepossibletocreateacontainerusingaDockerfile.Instead,youcanbackupormigrateyourcontainersbyusingtheexportcommand.Formoreinformationonexportingyourcontainers,seethefollowingURL:
https://docs.docker.com/reference/commandline/export/
Ifyoufindyourselfinthisposition,letmebethefirsttocongratulateyouonmitigatingafuturedisasterbypromotingyourSnowflakeintoaPetorevenCattleaheadofanyproblems.
TipStillrunningaSnowflake?
IfyoufindyourselfstillrunningaSnowflakeserverorservice,Icannotstressenoughthatyoulookatdocumenting,migrating,orupdatingtheSnowflakeassoonaspossible.Thereisnopointinmonitoringaservicethatmaybeimpossibleforyoutorecover.Rememberthattherearecontainersforoldtechnologies,suchasPHP4,ifyoureallyneedtorunthem.
![Page 223: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/223.jpg)
ScenariooneYouarerunningapersonalWordPresswebsiteusingtheofficialcontainersfromtheDockerHub;thecontainershavebeenlaunchedusingaDockerComposefileliketheonewehaveusedseveraltimesthroughoutthisbook.
YouhavetheDockerComposefilestoredinaGitHubrepositoryandyoucantakesnapshotsofthehostmachineasabackup.Asit’syourownblog,youarefinerunningitonasinglecloud-basedhost.
Asuitablemonitoringwillbeasfollows:
DockerstatsDockertopDockerlogscAdvisorSysdig
Asyouarerunningasinglehostmachinethatyouaretreatingasabackup,thereisnorealneedforyoutoshipyourlogfilestoacentrallocationasoddsareyourhostmachines;likethecontainers,itshostingwillbeonlineformonthsorpossiblyevenyears.
Itisunlikelythatyouwillneedtodigtoodeeplyintoyourcontainers’historicalperformancestats,asmostofthetuningandtroubleshootingwillbedoneinrealtimeasproblemsoccur.
Withthemonitoringtoolssuggested,youwillbeabletogetagoodinsightintowhatishappeningwithinyourcontainersinrealtime,andtogetmorethanenoughinformationonprocessesthatareconsumingtoomuchRAMandCPU,alongwithanyerrormessagesfromwithinthecontainers.
YoumaywanttoenableaservicesuchasPingdom(https://www.pingdom.com/)orUptimeRobot(http://uptimerobot.com/).TheseservicespollyourwebsiteeveryfewminutestoensurethattheURLyouconfigurethemto,checkwhetheritsloadingwithinacertaintimeoratall.Iftheydetectanyslowdownorfailureswiththepageloading,theycanbeconfiguredtosendaninitialalerttonotifyyouthatthereisapotentialissue,suchasboththeservicesmentionedhaveafreetier.
![Page 224: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/224.jpg)
ScenariotwoYouarerunningacustome-commerceapplicationthatneedstobehighlyavailableandalsoscaleduringyourpeaktimes.Youareusingapubliccloudserviceandthetoolsetthatcomeswithittolaunchcontainersandroutetraffictothem.
Asuitablemonitoringwillbeasfollows:
cAdvisor+PrometheusZabbixSysdigCloudNewRelicServerMonitoringDatadogELK+LogspoutLogEntriesLoggly
Withthisscenario,thereisabusinessneedtonotonlybenotifiedaboutcontainerandhostfailures,butalsotoholdyourmonitoringdataandlogsawayfromyourhostserverssothatyoucanproperlyreviewhistoricalinformation.YoumayalsoneedtokeeplogsforPCIcomplianceorinternalauditingforafixedperiodoftime.
Dependingonyourbudget,youcanachievethisbyhostingyourownmonitoring(ZabbixandPrometheus)andcentrallogging(ELK)stackssomewherewithinyourinfrastructure.
Youcanalsochoosetorunafewdifferentthird-partytoolssuchascombiningtoolsthatmonitorperformance,forexample,SysdigCloudorDatadog,withacentralloggingservice,suchasLogEntriesorLoggly.
Ifappropriate,youcanalsorunacombinationofself-hostedandthird-partytools.
Whiletheself-hostedoptionmayappeartobethemostbudget-friendlyoption,therearesomeconsiderationstotakeintoaccount,asfollows:
Yourmonitoringneedstobehostedawayfromyourapplication.Thereisnopointinhavingyourmonitoringinstalledonthesamehostasyourapplication;whatwillalertyouifthehostfails?Yourmonitoringneedstobehighlyavailable;doyouhavetheinfrastructuretodothis?Ifyourapplicationneedstobehighlyavailable,thensodoesyourmonitoring.Youneedtohaveenoughcapacity.Doyouhavethecapacitytobeabletostorelogfilesandmetricsgoingbackamonth,6months,orayear?
Ifyouaregoingtohavetoinvestinanyoftheprecedingoptions,thenitwillbeworthweighingupthecostsofinvestinginboththeinfrastructureandthemanagementofyourownmonitoringsolutionagainstusingathird-partythatwilloffertheprecedingoptionsasaservice.
Ifyouareusingacontainer-onlyoperatingsystemsuchasCoreOSorRancherOS,thenyouwillneedtochooseaservicewhoseagentorcollectorcanbeexecutedfromwithinacontainer,asyouwillnotbeabletoinstalltheagentbinariesdirectlyontheOS.
![Page 225: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/225.jpg)
Youwillalsoneedtoensurethatyourhostmachineisconfiguredtostarttheagents/collectorsonboot.Thiswillensurethatassoonasthehostmachinejoinsacluster(whichistypicallywhencontainerswillstarttopopuponthehost),itisalreadysendingmetricstoyourchosenmonitoringservices.
![Page 226: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/226.jpg)
ScenariothreeYourapplicationlaunchesacontainereachtimeyourAPIiscalledfromyourfrontendapplication;thecontainertakestheuserinputfromadatabase,processesit,andthenpassestheresultsbacktoyourfrontendapplication.Oncethedatahasbeensuccessfullyprocessed,thecontaineristerminated.Youareusingadistributedschedulingsystemtolaunchthecontainers.
Asuitablemonitoringwillbeasfollows:
ZabbixSysdigCloudDatadogELK+LogspoutLogEntriesLoggly
Inthisscenario,youmorethanlikelydonotwanttomonitorthingssuchasCPUandRAMutilization.Thesecontainersafterallshouldonlybearoundforafewminutes,andalsoyourschedulerwilllaunchthecontaineronthehostmachinewherethereisenoughcapacityforthetasktoexecute.
Instead,youwillprobablywanttokeeparecordtoverifythatthecontainerlaunchedandterminatedasexpected.YouwillalsowanttomakesurethatyoulogtheSTDOUTandSTDERRfromthecontainerwhileitisactive,asoncethecontainerhasbeenterminated,itwillbeimpossibleforyoutogetthesemessagesback.
Withthetoolslistedintheprecedingpoints,youshouldbeabletobuildsomequiteusefulqueriestogetadetailedinsightintohowyourshortrunprocessesareperforming.
Forexample,youwillbeabletogettheaveragelifetimeofacontainer,asyouknowthetimethecontainerwaslaunchedandwhenitwasterminated;knowingthiswillthenallowyoutosetatriggertoalertyouifanycontainersarearoundforanylongerthanyouwouldexpectthemtobe.
![Page 227: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/227.jpg)
![Page 228: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/228.jpg)
AlittlemoreaboutalertingAlotofthetoolswehavelookedatinthisbookofferatleastsomesortofbasicalertingfunctionality;themillion-dollarquestionisshouldyouenableit?
Alotofthisisdependentonthetypeofapplicationyouarerunningandhowthecontainershavebeendeployed.Aswehavealreadymentionedafewtimesinthischapter,youshouldneverreallyhaveaSnowflakecontainer;thisleavesuswithPets,Cattle,andChickens.
![Page 229: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/229.jpg)
ChickensAsalreadydiscussedintheprevioussection,youprobablydon’tneedtoworryaboutgettingalertsforRAM,CPU,andharddriveperformanceonaclusterthatisconfiguredtorunChickens.
Yourcontainersshouldnotbeuplongenoughtoexperienceanyrealproblems;however,shouldtherebeanyunexpectedspikes,yourschedulerwillprobablyhaveenoughintelligencetodistributeyourcontainerstohoststhathavethemostavailableresourcesatthattime.
Youwillneedtoknowifanyofyourcontainershavebeenrunninglongerthanyouexpectthemtobeup;forexample,aprocessinacontainerthatnormallytakesnomorethan60secondsisstillrunningafter5minutes.
Thisnotonlymeansthatthereisapotentialproblem,italsomeansthatyoufindyourselfrunninghoststhatonlycontainstalecontainers.
![Page 230: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/230.jpg)
CattleandPetsWhenitcomestosettingupalertsonCattleorPets,youhaveafewoptions.
YouwillmorethanlikelywanttoreceivealertsbasedonCPUandRAMutilizationforboththehostmachineandthecontainers,asthiscouldindicateapotentialproblemthatcouldcauseslowdownwithintheapplicationandalsolossofbusiness.
Asmentionedpreviously,youwillprobablyalsowanttobealertedifyourapplicationstartstoservethecontentthatisunexpected.Forexample,ahostandacontainerwillquitehappilysitthereservinganapplicationerror.
YoucanuseaservicesuchasPingdom,Zabbix,orNewRelictoloadapageandcheckforthecontentinthefooter;ifthiscontentismissing,thenanalertcanbesent.
Dependingonhowfluidyourinfrastructureis,inaCattleconfiguration,youwillprobablywanttobealertedwhencontainersspinupanddown,asthiswillindicateperiodsofhightraffic/transactions.
![Page 231: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/231.jpg)
SendingalertsSendingalertsdiffersforeachtool,forexample,analertcouldbeassimpleassendinganemailtoinformyouthatthereisanissuetothesoundingofanaudiblealertinaNetworkOperationsCenter(NOC)whentheCPUloadofacontainergoesabovefive,ortheloadonthehostgoesabove10.
Forthoseofyouwhorequireanon-callteamtobealerted,mostofthesoftwarewehavecoveredhassomelevelofintegrationalertaggregationservicessuchasPagerDuty(https://www.pagerduty.com).
TheseaggregationserviceseitherinterceptyouralertemailsorallowservicestomakeAPIcallstothem.Whentriggered,theycanbeconfiguredtoplacephonecalls,sendSMSmessages,andevenescalatetosecondaryon-calltechnicianifanalerthasnotbeenflaggeddownwithinadefinabletime.
Ican’tthinkofanycaseswhereyoushouldn’tlookatenablingalerting,afterall,it’salwaysbesttoknowaboutanythingthatcouldeffectyourapplicationbeforeyourendusersdo.
Howmuchalertingyouenableisreallydowntowhatyouareusingyourcontainersfor;however,Iwouldrecommendthatyoureviewallyouralertsregularlyandalsoactivelytuneyourconfiguration.
Thelastthingyouwantisaconfigurationthatproducestoomanyfalsepositivesoronethatistootwitchy,asyoudonotwanttheteamwhoreceivesyouralertstobecomedesensitizedtothealertsthatyouaregenerating.
Forexample,ifacriticalCPUalertistriggeredevery30minutesbecauseofascheduledjob,thenyouwillprobablyneedtoreviewthesensitivityofthealert,otherwiseitiseasyfortheengineertosimplydismissacriticalalertwithoutthinkingaboutit,as“thisalertcomeseveryhalfanhourandwillbeokinafewminutes”,whenyourentireapplicationcouldbeunresponsive.
![Page 232: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/232.jpg)
![Page 233: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/233.jpg)
KeepingupWhileDockerhasbeenbuiltontopofwell-establishedtechnologiessuchasLinuxContainers(LXC),thesehavetraditionallybeendifficulttoconfigureandmanage,especiallyfornon-systemadministrators.
Dockerremovesalmostallthebarrierstoentry,allowingeveryonewithasmallamountofcommand-lineexperiencetolaunchandmanagetheirowncontainer-basedapplications.
Thishasforcedalotofthesupportingtoolstoalsolowertheirbarriertoentry.Softwarethatoncerequiredcarefulplanningtodeploy,suchassomeofthemonitoringtoolswecoveredinthisbook,cannowbedeployedandconfiguredinminutesratherthanhours.
Dockerisalsoaveryfast-movingtechnology;whileithasbeenconsideredproduction-readyforawhile,newfeaturesarebeingaddedandexistingfeaturesareimprovedwithregularupdates.
Sofar,in2015,therehavebeen11releasesofDockerEngine;ofthese,onlysixhavebeenminorupdatesthatfixbugs,andtheresthaveallbeenmajorupdates.Detailsofeachreleasecanbefoundintheproject’sChangelog,whichcanbefoundathttps://github.com/docker/docker/blob/master/CHANGELOG.md.
BecauseofthepaceofdevelopmentofDocker,itisimportthatyoualsoupdateanymonitoringtoolsyoudeploy.Thisisnotonlytokeepupwithnewfeatures,butalsotoensurethatyoudon’tlooseanyfunctionalityduetochangesinthewayinwhichDockerworks.
Thisattitudeofupdatingmonitoringclients/toolscanbeabitofachangeforsomeadministratorswhomaybeinthepastwouldhaveconfiguredamonitoringagentonaserverandthennotthoughtaboutitagain.
![Page 234: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/234.jpg)
![Page 235: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/235.jpg)
SummaryAsdiscussedinthischapter,Dockerisafastmovingtechnology.Whilethisbookhasbeeninproduction,therehavebeenthreemajorversionsreleasedfrom1.7to1.9;witheachreleaseDockerhasbecomemorestableandmorepowerful.
Inthischapter,wehavelookedatdifferentwaystoimplementthetechnologiesthathavebeendiscussedinthepreviouschaptersofthisbook.Bynow,youshouldhaveanideaofwhichapproachisappropriatetomonitoryourcontainersandhostmachines,forbothyourapplicationandforthewaytheapplicationhasbeendeployedusingDocker.
Nomatterwhichapproachyouchosetotake,itisimportantthatyoustayup-to-datewithDocker’sdevelopmentandalsothenewmonitoringtechnologiesastheyemerge,thefollowinglinksaregoodstartingpointstokeepyourselfinformed:
DockerEngineeringBlog:http://blog.docker.com/category/engineering/DockeronTwitter:https://twitter.com/dockerDockeronReddit:https://www.reddit.com/r/dockerDockeronStackOverflow:http://stackoverflow.com/questions/tagged/docker
OneofthereasonswhytheDockerprojecthasbeenembracedbydevelopers,systemadministratorsandevenenterprisecompaniesisbecauseitisabletomoveataquickpace,whileaddingmorefeaturesandveryimpressivelymaintainingitseaseofuseandflexibility.
Overthenext12months,thetechnologyissettobeevenmorewidespread;theimportanceofensuringthatyouarecapturingusefulperformancemetricsandlogsfromyourcontainerswillbecomemorecriticalandIhopethatthisbookhashelpedyoustartyourjourneyintomonitoringDocker.
![Page 236: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/236.jpg)
IndexA
alertsenabling/Alittlemoreaboutalertingsettingup,onChickens/Chickenssettingup,onCattle/CattleandPetssettingup,onPets/CattleandPetssending/Sendingalerts
AmazonEC2ContainerServiceURL/Cattle
AmazonWebServicesURL/Awordaboutexternallyhostedservices
ApacheBenchURL/RunningDockerstats
ApacheMesosURL/Chickens
AtomicURL/WhyuseaSaaSservice?
![Page 237: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/237.jpg)
Bbackslash
using/RunningcAdvisorusingacontainer
![Page 238: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/238.jpg)
CcAdvisor
about/WhatiscAdvisor?executing,containerused/RunningcAdvisorusingacontainercompiling,fromsource/CompilingcAdvisorfromsourcereferencelink/Alternatives?
Cattleabout/Cattlecontainers,deployingonto/Cattlealerts,settingup/CattleandPets
Changelog,DockerURL/Keepingup
Chickensabout/Chickenscontainers,deployingonto/Chickensalerts,settingup/Chickens
cloudDocker,deployingin/DeployingDockerinthecloud
CloudComputingreferencelink/Chickens
Cloudscalingreferencelink/Pets,Cattle,Chickens,andSnowflakes
containerused,forexecutingcAdvisor/RunningcAdvisorusingacontainer
ContainerDownabout/Triggers
containerlogsviewing/Viewingcontainerlogs
containersresourceutilization,tracking/Whatjusthappened?comparing,tohostmachine/Comparecontainerstoyourhostmachinedeploying,ontoPets/Petsdeploying,ontoCattle/Cattledeploying,ontoChickens/Chickensdeploying,ontoSnowflakes/Snowflakes
containers,monitoringscenarios/Scenarioone,Scenariotwo,Scenariothree
containerstatsviewing/Viewingcontainerstatssubcontainers/Subcontainersdriverstatus/Driverstatusimages/Images
CoreOS
![Page 239: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/239.jpg)
URL/WhyuseaSaaSservice?,CattleCsysdig
about/UsingCsysdigusing/UsingCsysdig
customgraphscreating/Createcustomgraphsreferencelink/Createcustomgraphs
![Page 240: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/240.jpg)
DDashboard
launching/DashboardDatadog
URL/WhyuseaSaaSservice?,Datadogabout/Datadogagent,installing/Installingtheagentagentinstallation,URL/Installingtheagentwebinterface,exploring/Exploringthewebinterfacereferences/Summaryandfurtherreading
datavolumecontainercreating/Thenextsteps
DigitalOceanURL/Pets
DigitalOceanURL/Awordaboutexternallyhostedservices
Dockerabout/DockerURL,foruserguide/Runningavirtualserver,Thenextstepsdeploying,incloud/DeployingDockerinthecloudadvancements/Keepingup
docker-composefileused,forexecutingELKstack/Whataboutproduction?
DockerComposeabout/Docker
DockerEngineabout/Docker
Dockerexecabout/Dockerexec
dockerexportcommandURL/Snowflakes
DockerHubabout/DockerURL/Docker,WhatiscAdvisor?,Usingcontainers
DockerimagesURL/Driverstatus
Dockerstatsabout/Dockerstatsexecuting/RunningDockerstats
Dockertopabout/Dockertop
driverstatusobtaining/Driverstatus
![Page 241: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/241.jpg)
EElasticsearch
URL,forlaunching/StartingthestackELKstack
about/ELKStackstarting/StartingthestackLogspout/Logspoutlogs,reviewing/Reviewingthelogsexecuting,docker-composefileused/Whataboutproduction?references/Whataboutproduction?
environmentcloning/Cloningtheenvironment
externallyhostedservicesabout/AwordaboutexternallyhostedservicesDocker,deployingincloud/DeployingDockerinthecloud
![Page 242: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/242.jpg)
FFig
about/Docker
![Page 243: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/243.jpg)
GGo
URL/WhatiscAdvisor?GoogleContainerEngine
URL/Cattle
![Page 244: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/244.jpg)
Hhostmachine
containers,comparingto/Comparecontainerstoyourhostmachine
![Page 245: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/245.jpg)
Iimages
listing/ImagesInfluxDB
about/Alternatives?URL/Alternatives?
![Page 246: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/246.jpg)
JJoyentTriton
URL/Cattle
![Page 247: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/247.jpg)
KKibana
URL,forlaunching/StartingthestackKubernetes
URL/Chickens
![Page 248: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/248.jpg)
LLiquidWaxEster(LXE)
about/Keepinguplmctfy
URL/WhatiscAdvisor?localenvironment
launching/LaunchingalocalenvironmentLogEntries
URL/Lookingatthirdpartyoptionsconfiguring/Lookingatthirdpartyoptions
LogglyURL/Lookingatthirdpartyoptions
logsreviewing/Reviewingthelogs
LogspoutURL,forlaunching/Startingthestackabout/Logspoutreferences/Logspout
LogstashURL,forlaunching/Startingthestack
![Page 249: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/249.jpg)
MMariaDB10
URL/Usingcontainersmetrics
collecting/Collectingmetricsrecording/Dockermetricscustomgraphs,creating/Createcustomgraphscontainers,comparingtohostmachine/Comparecontainerstoyourhostmachinetriggers,defining/Triggers
MicrosoftAzureURL/Awordaboutexternallyhostedservices
MonitoringDockerrepositoryURL/Cloningtheenvironment
![Page 250: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/250.jpg)
NNagios
URL/Zabbixncurses
about/WhatisSysdig?NetworkOperationsCenter(NOC)
about/SendingalertsNewRelic
URL/WhyuseaSaaSservice?,NewRelicabout/NewRelicagent,installing/Installingtheagentwebinterface,exploring/Exploringthewebinterfacereferences/Summaryandfurtherreading
![Page 251: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/251.jpg)
P—privilegedflag
about/RunningcAdvisorusingacontainerURL/RunningcAdvisorusingacontainer
PagerDutyURL/Triggers,Sendingalerts
Petsabout/Petscontainers,deployingonto/Petsalerts,settingup/CattleandPets
phpinfoURL/RunningDockerstats
PingdomURL/Scenarioone
processesisolating/Whataboutprocesses?Dockertop/DockertopDockerexec/Dockerexec
Prometheusabout/Prometheuslaunching/LaunchingPrometheusquerying/QueryingPrometheusDashboard,launching/DashboardURL,fordocumentation/Dashboarddatavolumecontainer,creating/Thenextstepsreferences/Thenextsteps
PuppetForgeURL/Usingvagrant
![Page 252: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/252.jpg)
RRancherOS
URL/WhyuseaSaaSservice?,Cattle
![Page 253: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/253.jpg)
SSaaSservice
using/WhyuseaSaaSservice?serverstyles
selecting/Sowhatdoesthisallmean?Slack
URL/TriggersSnowflakes
about/Snowflakesreferencelink/Snowflakescontainers,deployingonto/Snowflakes
StandardStreamsreferencelink/Viewingcontainerlogs
subcontainersabout/Subcontainers
SysAdminabout/Pets,Cattle,Chickens,andSnowflakes
Sysdigabout/WhatisSysdig?installing/InstallingSysdigURL/InstallingSysdigusing/UsingSysdigbasics/Thebasicsdata,capturing/Capturingdatacontainers,listing/Containersreferences/Furtherreading
SysDigCloudURL/WhyuseaSaaSservice?
SysdigCloudabout/SysdigCloudfunctionality/SysdigCloudreferences/SysdigCloud,Summaryandfurtherreadingagent,installing/Installingtheagentagentinstallation,URL/Installingtheagentcontainers,exploring/Exploringyourcontainers
![Page 254: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/254.jpg)
Ttimeseries
about/Prometheusreferencelink/Prometheus
triggersdefining/Triggers
![Page 255: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/255.jpg)
UUbuntuSnappyCore
URL/WhyuseaSaaSservice?UptimeRobot
URL/Scenarioone
![Page 256: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/256.jpg)
VVagrant
URL/Launchingalocalenvironmentvagrantbox
URL/RunningavirtualserverVirtualBox
URL/Launchingalocalenvironmentvirtualserver
executing/Runningavirtualserverhalting/Haltingthevirtualserver
VMwarevCloudAirURL/Awordaboutexternallyhostedservices
![Page 257: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/257.jpg)
WWebinterface
about/TheWebinterfaceoverview/OverviewProcesses/ProcessesCPU/CPUMemory/MemoryNetwork/NetworkFilesystem/Filesystemdrawbacks/Thisisallgreat,what’sthecatch?
WordPressURL,forinstallation/UsingSysdig
![Page 258: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/258.jpg)
YYAML
URL/Docker
![Page 259: Monitoring Docker · choice for web applications. Since 2003, he has used Linux and UNIX-related operational systems, from Slackware to Gobo Linux, Archlinux, CentOS, Debian, and](https://reader034.fdocuments.in/reader034/viewer/2022042203/5ea4f5be8d48fc1335260fc6/html5/thumbnails/259.jpg)
ZZabbix
about/ZabbixURL/Zabbix,Usingvagrant,TheZabbixwebinterfacereferences/Zabbixinstalling/InstallingZabbixcontainers,using/Usingcontainersvagrant,using/Usingvagranthostmachine,preparing/Preparingourhostmachinehosttemplates,adding/TheZabbixwebinterface
ZenossURL/Zabbix