Zabbix Network Monitoring Essentialsinfonetwork.ir/wp-content/uploads/2019/07/Zabbix_Network... ·...

ZabbixNetworkMonitoringEssentials

TableofContents


Credits

AbouttheAuthors

AbouttheReviewers

www.PacktPub.com

Supportfiles,eBooks,discountoffers,andmore

Whysubscribe?

FreeaccessforPacktaccountholders

Preface

Whatthisbookcovers

Whatyouneedforthisbook

Whothisbookisfor

Conventions

Readerfeedback

Customersupport

Downloadingtheexamplecode

Errata

Piracy

Questions

1.InstallingaDistributedZabbixSetup

Zabbixarchitectures

UnderstandingZabbixdataflow

UnderstandingtheZabbixproxies’dataflow

InstallingZabbix

Installingfrompackages

SettingupaZabbixagent

CreatingaZabbixagentpackagewithCheckInstall

Serverconfiguration

Installingadatabase

Consideringthedatabasesize

MySQLpartitioning

InstallingaZabbixproxy

InstallingtheWebGUIinterface

Summary

2.ActiveMonitoringofYourDevices

UnderstandingZabbixhosts

Hostsandhostgroups

Hostinterfaces

Hostinventory

GoingbeyondZabbixagents

Simplechecks

KeepingSNMPsimple

GettingSNMPdataintoZabbix

FindingtherightOIDstomonitor

MappingSNMPOIDstoZabbixitems

Gettingdatatypesright

SNMPtraps

Snmptrapd

TransformingatrapintoaZabbixitem

Gettingnetflowfromthedevicestothemonitoringserver

Receivingnetflowdataonyourserver

MonitoringalogfilewithZabbix

Summary

3.MonitoringYourNetworkServices

MonitoringtheDNS

DNS–responsetime

DNSSEC–monitoringthezonerollover

Apachemonitoring

NTPmonitoring

NTP–whatarewemonitoring?

Squidmonitoring

Summary

4.DiscoveringYourNetwork

FindinghoststheZabbixway

Definingactionconditions

Choosingactionoperations

Remotecommands

Low-leveldiscovery

Summary

5.VisualizingYourTopologywithMapsandGraphs

Creatingcustomgraphs

Maps–aquicksetupforalargetopology

Maps–automatingtheDOTcreation

DraftingZabbixmapsfromDOT

Puttingeverythingtogetherwithscreens

Summary

A.PartitioningtheZabbixDatabase

MySQLpartitioning

Thepartition_maintenanceprocedure

Thepartition_createprocedure

Thepartition_verifyprocedure

Thepartition_dropprocedure

Thepartition_maintenance_allprocedure

Housekeepingconfiguration

B.CollectingSquidMetrics

Squidmetricscript

Index

ZabbixNetworkMonitoringEssentialsCopyright©2015PacktPublishing

Allrightsreserved.Nopartofthisbookmaybereproduced,storedinaretrievalsystem,ortransmittedinanyformorbyanymeans,withoutthepriorwrittenpermissionofthepublisher,exceptinthecaseofbriefquotationsembeddedincriticalarticlesorreviews.

Everyefforthasbeenmadeinthepreparationofthisbooktoensuretheaccuracyoftheinformationpresented.However,theinformationcontainedinthisbookissoldwithoutwarranty,eitherexpressorimplied.Neithertheauthors,norPacktPublishing,anditsdealersanddistributorswillbeheldliableforanydamagescausedorallegedtobecauseddirectlyorindirectlybythisbook.

PacktPublishinghasendeavoredtoprovidetrademarkinformationaboutallofthecompaniesandproductsmentionedinthisbookbytheappropriateuseofcapitals.However,PacktPublishingcannotguaranteetheaccuracyofthisinformation.

Firstpublished:February2015

Productionreference:1210215

PublishedbyPacktPublishingLtd.

LiveryPlace

35LiveryStreet

BirminghamB32PB,UK.

ISBN978-1-78439-976-4

www.packtpub.com

http://www.packtpub.com

CreditsAuthors

AndreaDalleVacche

StefanoKewanLee

Reviewers

RaviBhure

NicholasPier

NicolaVolpini

CommissioningEditor

AmarabhaBanerjee

AcquisitionEditor

NikhilKarkal

ContentDevelopmentEditor

SiddheshSalvi

TechnicalEditor

HumeraShaikh

CopyEditor

SarangChari

ProjectCoordinator

KrantiBerde

Proofreaders

SimranBhogal

LindaMorris

Indexer

HemanginiBari

Graphics

DishaHaria

ProductionCoordinator

AparnaBhagat

CoverWork

AparnaBhagat

AbouttheAuthorsAndreaDalleVaccheisahighlyskilledITprofessionalwithover14yearsofexperienceintheITindustryandbanking.HegraduatedfromUniversitàdegliStudidiFerrarawithaninformationtechnologycertification.ThislaidthetechnologyfoundationthatAndreahasbuiltoneversince.Andreahasacquiredvariousindustry-respectedaccreditations,whichincludeCisco,Oracle,RHCE,ITIL,andofcourse,Zabbix.Throughouthiscareer,hehasworkedinmanylarge-scaleenvironments,ofteninrolesthathavebeenverycomplex,onaconsultantbasis.Thishasfurtherenhancedhisgrowingskillset,addingtohispracticalknowledgebaseandincreasinghisappetitefortheoreticaltechnicalstudying.

Andrea’sloveforZabbixcamefromhistimespentintheOracleworldasadatabaseadministrator/developer.Histimewasspentmainlyonreducingownershipcosts,specializinginmonitoringandautomation.ThisiswherehecameacrossZabbixandtheflexibilityitoffered,bothtechnicallyandadministratively.Withthisasalaunchpad,AndreawasinspiredtodevelopOrabbix,thefirstopensourcesoftwaretomonitorOracle’scompleteintegrationwithZabbix.HehaspublishedanumberofarticlesonZabbix-relatedsoftware,suchasDBforBIX.Hisprojectsarepubliclyavailableathttp://www.smartmarmot.com.Currently,Andreaisworkingasaseniorarchitectforaleadingglobalinvestmentbankinaverydiverseandchallengingenvironment.HedealswithmanyaspectsoftheUnix/Linuxplatformsaswellasmanytypesofthird-partysoftware,whicharestrategicallyalignedtothebank’stechnicalroadmap.Inadditiontothistitle,AndreaDalleVaccheisacoauthorofMasteringZabbix,PacktPublishing.

StefanoKewanLeeisanITconsultantwithmorethan12yearsofexperienceinsystemintegration,security,andadministration.HeisacertifiedZabbixspecialistinlargeenvironmentsholdsaLinuxadministrationcertificationfromtheLPIandaGIACGCFWcertificationfromSANSInstitute.Whenhe’snotbusybreakingwebsites,helivesinthecountrysidewithhistwocatsandtwodogsandpracticesmartialarts.Inadditiontothistitle,StefanoKewanLeeisacoauthorofMasteringZabbix,PacktPublishing.

http://www.smartmarmot.com

AbouttheReviewersRaviBhureisbasicallyanITengineerwithnicheskills,suchasChef,CloudAnsible,SaltStack,Python,Ruby,andShell/Bash.Healsowritescodeforinfrastructure,dailyIToperations,andsoon.Inshort,heisfondofusinghisskillsandknowledgeoffault-tolerantsolutionsfortheday-to-daymaintenanceofmission-criticalproductioninfrastructure.

Ravistartedinteractingwithcomputerssince1996whenhegothisfirstcomputerathome.Thingschangedveryfast,andin1998,heenteredthemagicalworldoftheInternet☺forthefirsttimeever,whichchangedhislife!Hestartedhisowncybercafein1999.In2004,hegothisfirstjobasafieldengineer,hiredtomaintainandsupportVRIUFOsystems.After2years,hemovedtoPuneandworkedwithmanyorganizations,suchasVyomLabs,GlamIndia,Symphony,andDhingana.

Themosthappeningandinterestingfactabouthisdiverseexposureisthatheisfromanartsbackground.Yes,heholdsabachelor’sdegreeinartsfromSRTMUniversity,Nanded,Maharashtra,India.Andweallwillhavetoagreethathehasthearttosolveproblems☺,agreatinspirationforpeoplewhoarenonengineers!

Currently,RaviisassociatedwithOpexSoftwareasaseniorDevOpsengineer.

NicholasPierisanetworkengineerinthemanagedservices/professionalservicesfield.HisexperienceincludesdesigningdatacenternetworkinfrastructureswithvirtualizationandSANsolutions,webdevelopment,andwritingmiddlewareforbusinessapplications.Atthetimeofwritingthis,Nicholasholdsanumberofindustrycertifications,includingtheCiscoCCNP,VMwareVCP5-DCV,andvariousotherCiscoandCompTIAcertifications.Inhisfreetime,heindulgesinhispassionforcraftbeer,distancerunning,andreading.

I’dliketothankPacktPublishingforthisopportunity!

NicolaVolpinihasbeenplayingwithtechnologyfromayoungage,havingahardtimeresistingtheurgetodisassemblecomplextoysorkitchenappliances.

Theloveforcomputersoriginatedaroundhistenthbirthday,whenheaccidentallytoastedhisfirstCPU.Thisepisodeonlyincreasedhisfascinationforcomputers,andtheaccidents,fortunately,stopped.

Forthepast10years,he’sbeenworkingasanITprofessional,specializinginenterprisenetworkingandsystemadministration.ExperimentingwiththemostdiversetechnologiesinthefieldandbeinganavidfanoftheFOSSphilosophy,Linux,and*BSD,hedreamsofseeingthecollaborativethinkingoftheFOSSmovementhelpinspiretheworld.

He’scurrentlyworkingatStockholm,Sweden,whereheresideswithhisgirlfriend.

www.PacktPub.com

Supportfiles,eBooks,discountoffers,andmoreForsupportfilesanddownloadsrelatedtoyourbook,pleasevisitwww.PacktPub.com.

DidyouknowthatPacktofferseBookversionsofeverybookpublished,withPDFandePubfilesavailable?YoucanupgradetotheeBookversionatwww.PacktPub.comandasaprintbookcustomer,youareentitledtoadiscountontheeBookcopy.Getintouchwithusatformoredetails.

Atwww.PacktPub.com,youcanalsoreadacollectionoffreetechnicalarticles,signupforarangeoffreenewslettersandreceiveexclusivediscountsandoffersonPacktbooksandeBooks.

https://www2.packtpub.com/books/subscription/packtlib

DoyouneedinstantsolutionstoyourITquestions?PacktLibisPackt’sonlinedigitalbooklibrary.Here,youcansearch,access,andreadPackt’sentirelibraryofbooks.

http://www.PacktPub.comhttp://www.PacktPub.commailto:[email protected]://www.PacktPub.comhttps://www2.packtpub.com/books/subscription/packtlib

Whysubscribe?FullysearchableacrosseverybookpublishedbyPacktCopyandpaste,print,andbookmarkcontentOndemandandaccessibleviaawebbrowser

FreeaccessforPacktaccountholdersIfyouhaveanaccountwithPacktatwww.PacktPub.com,youcanusethistoaccessPacktLibtodayandview9entirelyfreebooks.Simplyuseyourlogincredentialsforimmediateaccess.

http://www.PacktPub.com

PrefaceNetworkadministratorsarefacinganinterestingchallengethesedays.Ontheonehand,computernetworksarenotsomethingnewanymore.Theyhavebeenaroundforquiteawhile:theirphysicalcomponentsandcommunicationprotocolsarefairlywellunderstoodanddon’trepresentabigmysterytoanincreasingnumberofprofessionals.Moreover,networkappliancesaregettingcheaperandeasiertosetup,tothepointthatitdoesn’ttakeacertifiedspecialisttoinstallandconfigureasimplenetworkorconnectittoothernetworks.Theveryconceptofnetworkingissowidespreadandingrainedinhowusersanddevelopersthinkofacomputersystemthatbeingonlineinsomeformisexpectedandtakenforgranted.Inotherwords,acomputernetworkisincreasinglyseenasacommodity.

Ontheotherhand,theverysameforcesthatarecallingforsimpler,easier,accessiblenetworksaretheonesthatareactuallypushingthemtogrowmoreandmorecomplexeveryday.It’samatterofbothquantityandquality.Thenumberofconnecteddevicesonagivennetworkisalmostalwaysconstantlygrowingandsoistheamountofdataexchanged:mediastreams,applicationdata,backups,databasequeries,andreplicationtendtosaturatebandwidthjustasmuchastheyeatupstoragespace.Asforquality,therearedozensofdifferentrequirementsthatfactorinagivennetworksetup:fromhavingtomanagedifferentphysicalmediums(fiber,cable,radio,andsoon),totheneedtoprovidehighperformanceandavailability,bothontheconnectionandontheapplicationlevel;fromtheneedtoincreaseperformanceandreliabilityforgeographicallinks,toprovidingconfidentiality,security,anddataintegrityatalllevels,andthelistgoeson.

Thesetwocontrasting,yetintertwined,tendenciesareforcingnetworkadministratorstodomore(moreservices,moreavailability,andmoreperformance)withless(lessbudget,butalsolessattentionfromthemanagementcomparedtonewer,flashiertechnologies).Now,morethanever,asanetworkadmin,youneedtobeabletokeepaneyeonyournetworkinordertokeepitinahealthystate,butalsotoquicklyidentifyandresolvebottlenecksandoutagesofanykind—orbetteryet,findwaystoanticipateandworkaroundthembeforetheyhappen.You’llalsoneedtointegrateyoursystemswithdifferenttoolsandenvironments(bothlegacyandstrategicones)thatwillbeoutofyourdirectcontrol,suchasassetdatabases,incidentmanagementsystems,accountingandprofilingsystems,andsoon.Evenmoreimportantly,you’llneedtobeabletoshowyourworkandexplainyourneedsinclear,understandabletermstonontechnicalpeople.

Now,ifweweretosaythatZabbixistheperfect,one-size-fits-allsolutiontoallyournetworkmonitoringandmanagementproblems,wewouldclearlybelying.Tothisday,nosuchtoolexistsdespitewhatmanyvendorswantyoutobelieve.Eveniftheyhavemanyfeaturesincommon,whenitcomestomonitoringandcapacitymanagement,everynetworkhasitsownquirks,specialcases,andpeculiarneeds,tothepointthatanytoolhastobecarefullytunedtotheenvironmentorfacetheriskofbecominguselessandneglectedveryquickly.

WhatistrueisthatZabbixisamonitoringsystempowerfulenoughandflexibleenough

that,withtherightamountofwork,canbecustomizedtomeetyourspecificneeds.Andagain,thoseneedsarenotlimitedtomonitoringandalerting,butalsotoperformanceanalysisandprediction,SLAreporting,andsoon.WhenusingZabbixtomonitoranenvironment,youcancertainlycreateitemsthatrepresentvitalmetricsforthenetworkinordertohaveareal-timepictureofwhat’shappening.However,thosesameitemscanalsoproveveryusefultoanalyzeperformancebottlenecksandtoplannetworkexpansionandevolution.Items,triggers,andactionscanworktogethertoletyoutakeanactiveroleinmonitoringyournetworkandeasilyidentifyandpre-emptcriticaloutages.

Inthisbook,we’llassumethatyoualreadyknowZabbixasageneral-purposemonitoringtool,andthatyoualsousedittoacertainextent.Specifically,wewon’tcovertopicssuchasitem,trigger,oractioncreationandconfigurationwithabasic,step-by-stepapproach.Here,wewanttofocusonafewtopicsthatcouldbeofparticularinterestfornetworkadministrators,andwe’lltrytohelpthemfindtheirownanswerstoreal-worldquestionssuchasthefollowing:

Ihavealargenumberofappliancestomonitorandhavetokeepmonitoringdataavailableforalongtimeduetoregulatoryrequirements.HowdoIinstallandconfigureZabbixsothatitisabletomanageeffectivelythislargeamountofdata?Whatarethebestmetricstocollectinordertobothhaveaneffectivereal-timemonitoringsolutionandleveragehistoricaldatatomakeperformanceanalysisandpredictions?ManyZabbixguidesandtutorialsfocusonusingtheZabbixagent.Theagentiscertainlypowerfulanduseful,buthowdoIleverageinaneffectiveandsecurewaymonitoringprotocolsthatarealreadyavailableonmynetwork,suchasSNMPandnetflow?Loadbalancers,proxies,andwebserverssometimesfallunderagrayareabetweennetworkandapplicationadministration.Ihaveabunchofwebserversandproxiestomonitor.Whatkindofmetricsaremostusefultocheck?Ihaveacomplexnetworkwithhoststhataredeployedanddecommissionedonadailybasis.HowdoIkeepmymonitoringsolutionup-to-datewithoutresortingtolong,error-pronemanualinterventionsasmuchaspossible?NowthatIhavecollectedalargeamountofmonitoringandperformancedata,howcanIanalyzeitandshowtheresultsinameaningfulway?HowdoIputtogetherthegraphsIhaveavailabletoshowhowtheyarerelated?

Inthecourseofthenextfewchapters,we’lltrytoprovidesomepointersonhowtoanswerthosequestions.Wediscussasmanypracticalexamplesandreal-worldapplicationsaswecanaroundthesubjectofnetworkmonitoring,butmorethananything,wewantedtoshowyouhowit’srelativelysimpletoleverageZabbix’spowerandflexibilitytoyourownneeds.

Theaimofthisbookisnottoprovideyouwithasetofprepackagedrecipesandsolutionsthatyoucanapplyuncriticallytoyourownenvironment.Eventhoughweprovidedsomescriptsandcodethataretestedandworking(andhopefullyyou’llfindthemuseful),therealintentionwasalwaystogiveyouadeeperunderstandingofthewayZabbixworksso

thatyouareabletocreateyourownsolutionstoyourownchallenges.

Wehopewehavesucceededinourgoal,andthatbytheendofthebook,you’llfindyourselfamoreconfidentnetworkadministratorandamoreproficientZabbixuser.Evenifthiswillnotbethecase,wehopeyou’llbeabletofindsomethingusefulinthefollowingchapters:wetouchupondifferentaspectsofZabbixandnetworkmonitoringandalsodiscussacoupleoflessknownfeaturesthatyoumightfindveryinterestingnonetheless.

So,withoutfurtherado,let’sgetstartedwiththeactualcontentwewanttoshowyou.

WhatthisbookcoversChapter1,InstallingaDistributedZabbixSetup,teachesyouhowtoinstallZabbixinadistributedsetup,withalargeuseofproxies.Thechapterwillguideyouthroughallthepossiblesetupscenarios,showingyouthemaindifferencesbetweentheactiveandpassiveproxysetup.ThischapterwillexplainhowtoprepareandsetupaZabbixinstallation,whichisreadytobegrownwithinyourinfrastructure,readytosupportyou,andmonitoralargeenvironmentorevenaverylargeone.

Chapter2,ActiveMonitoringofYourDevices,offersyouafewveryusefulexamplesofthedifferentmonitoringpossibilitiesZabbixcanachievebyrelyingondifferentmethodsandprotocols.You’llseehowtoqueryyournetworkfromthelinkleveluptoroutingandnetworkflowusingICMP,SNMP,andlog-parsingfacilitiestocollectyourmeasurements.Youwillalsolearnhowtoextractmeaningfulinformationfromthegathereddatausingaggregatedandcalculateditems,andconfiguringcomplextriggersthatwillalertyouaboutrealnetworkissueswhileminimizingsignalnoiseandfalsepositives.

Chapter3,MonitoringYourNetworkServices,takesyouthroughhowtoeffectivelymonitorthemostcriticalnetworkservices,suchasDNS,DHCP,NTP,Apacheproxy/reverseproxies,andproxycacheSquid.Asitiseasytounderstand,allofthemarecriticalserviceswhereasimpleissuecanaffectyournetworksetupandquicklypropagatetheissuetoyourentirenetwork.Youwillunderstandhowtoextractmeaningfulmetricsandusefuldatafromallthelistedservices,beingablethennotonlytomonitortheirownreliability,butalsotoacquireimportantmetricsthatcanhelpyoutopredictfailuresorissues.

Chapter4,DiscoveringYourNetwork,explainshowtodeeplyautomatethemonitoringconfigurationofnetworkobjects.Itwillmassivelyusethebuilt-indiscoveryfeatureinordertokeepthemonitoringsolutionup-to-datewithinanevolvingnetworkenvironment.ThischapterisdividedintotwocorepartsthatcoverthetwomainlevelsofZabbix’sdiscovery:hostdiscoveryandlow-leveldiscovery.

Chapter5,VisualizingYourTopologywithMapsandGraphs,showsyouhowtocreatecomplexgraphsfromyouritem’snumericalvalues,automaticallydrawmapsthatreflectthecurrentstatusofyournetwork,andbringitalltogetherusingscreensasatooltocustomizemonitoringdatapresentation.ThischapteralsopresentsasmartwaytoautomatetheinitialstartupofyourZabbix’ssetup,makingyouabletodrawnetworkdiagramsusingmapsinafullyautomatedway.Youwillthenlearnaproduction-readymethodtomaintainmapswhileyournetworkisgrowingorrapidlychanging.

AppendixA,PartitioningtheZabbixDatabase,containsalltherequiredsoftwareandstoredprocedurestoefficientlypartitionyourZabbixdatabase.

AppendixB,CollectingSquidMetrics,containsthesoftwareusedtomonitorSquid.

WhatyouneedforthisbookThesoftwarethathasbeenusedandisnecessaryforthisbookis:

LinuxRedHatEnterpriseLinux6.5orhigherZabbix4.2ApacheHTTPD2.2MySQLServer-5.1Netflow1.6.12Nmap

Thisbookalsorequiresanintermediateexperienceinshellscripting,abasic-to-intermediateknowledgeofPython,andanintermediateknowledgeofZabbix.

Anyway,alltheexamplesdiscussedandproposedinthisbookareexplainedwellandcommentedupon.Thesameapproachhasbeenappliedeventothesoftwareusedonthisbookwhereitisexplained,withareasonablelevelofdetail,howtosetupandconfigureeachsoftwarecomponent.

WhothisbookisforThisbookisintendedforexperiencednetworkadministratorslookingforacomprehensivemonitoringsolutionfortheirnetworks.ThereadermusthaveagoodknowledgeofUnix/Linux,networkingconcepts,protocols,andappliancesandabasic-to-intermediateknowledgeofZabbix.Thereaderwillbeguidedstepbysteptomanageandleadalltheimportantpointsyouwillhavetodealwith.Youwillthenbeabletostartupaneffectiveandlarge-environment-readyZabbixmonitoringsolutionthatwillbeaperfectfitwithinyournetwork.

ConventionsInthisbook,youwillfindanumberoftextstylesthatdistinguishbetweendifferentkindsofinformation.Herearesomeexamplesofthesestylesandanexplanationoftheirmeaning.

Codewordsintext,databasetablenames,foldernames,filenames,fileextensions,pathnames,dummyURLs,userinput,andTwitterhandlesareshownasfollows:“OntheZabbixserver-side,youneedtocarefullysetthevalueofStartTrappers=.”

Ablockofcodeissetasfollows:

#FirstofallweneedtoimportcsvandNetworkximportcsvimportnetworkxasnx#ThenweneedtodefinewhoisourzabbixserverandsomeotherdetailtoproperlyproducetheDOTfilezabbix_service_ipaddr="192.168.1.100"main_loop_ipaddr="10.12.20.1"

Whenwewishtodrawyourattentiontoaparticularpartofacodeblock,therelevantlinesoritemsaresetinbold:

#wecanopenourCSVfilecsv_reader=csv.DictReader(open('my_export.csv'),\delimiter=",",\fieldnames=("ipaddress","hostname","oid","dontcare","neighbors"))#Skiptheheadercsv_reader.next()

Anycommand-lineinputoroutputiswrittenasfollows:

#chkconfig--level345zabbix-serveron

Newtermsandimportantwordsareshowninbold.Wordsthatyouseeonthescreen,forexample,inmenusordialogboxes,appearinthetextlikethis:“Thereisaclearwarningonthewebsitethatwarnsuswiththisstatement:TheApplianceisnotintendedforseriousproductionuseatthistime.”

NoteWarningsorimportantnotesappearinaboxlikethis.

TipTipsandtricksappearlikethis.

ReaderfeedbackFeedbackfromourreadersisalwayswelcome.Letusknowwhatyouthinkaboutthisbook—whatyoulikedordisliked.Readerfeedbackisimportantforusasithelpsusdeveloptitlesthatyouwillreallygetthemostoutof.

Tosendusgeneralfeedback,simplye-mail,andmentionthebook’stitleinthesubjectofyourmessage.

Ifthereisatopicthatyouhaveexpertiseinandyouareinterestedineitherwritingorcontributingtoabook,seeourauthorguideatwww.packtpub.com/authors.

mailto:[email protected]://www.packtpub.com/authors

CustomersupportNowthatyouaretheproudownerofaPacktbook,wehaveanumberofthingstohelpyoutogetthemostfromyourpurchase.

DownloadingtheexamplecodeYoucandownloadtheexamplecodefilesfromyouraccountathttp://www.packtpub.comforallthePacktPublishingbooksyouhavepurchased.Ifyoupurchasedthisbookelsewhere,youcanvisithttp://www.packtpub.com/supportandregistertohavethefilese-maileddirectlytoyou.

http://www.packtpub.comhttp://www.packtpub.com/support

ErrataAlthoughwehavetakeneverycaretoensuretheaccuracyofourcontent,mistakesdohappen.Ifyoufindamistakeinoneofourbooks—maybeamistakeinthetextorthecode—wewouldbegratefulifyoucouldreportthistous.Bydoingso,youcansaveotherreadersfromfrustrationandhelpusimprovesubsequentversionsofthisbook.Ifyoufindanyerrata,pleasereportthembyvisitinghttp://www.packtpub.com/submit-errata,selectingyourbook,clickingontheErrataSubmissionFormlink,andenteringthedetailsofyourerrata.Onceyourerrataareverified,yoursubmissionwillbeacceptedandtheerratawillbeuploadedtoourwebsiteoraddedtoanylistofexistingerrataundertheErratasectionofthattitle.

Toviewthepreviouslysubmittederrata,gotohttps://www.packtpub.com/books/content/supportandenterthenameofthebookinthesearchfield.TherequiredinformationwillappearundertheErratasection.

http://www.packtpub.com/submit-erratahttps://www.packtpub.com/books/content/support

PiracyPiracyofcopyrightedmaterialontheInternetisanongoingproblemacrossallmedia.AtPackt,wetaketheprotectionofourcopyrightandlicensesveryseriously.IfyoucomeacrossanyillegalcopiesofourworksinanyformontheInternet,pleaseprovideuswiththelocationaddressorwebsitenameimmediatelysothatwecanpursuearemedy.

Pleasecontactusatwithalinktothesuspectedpiratedmaterial.

Weappreciateyourhelpinprotectingourauthorsandourabilitytobringyouvaluablecontent.

mailto:[email protected]

QuestionsIfyouhaveaproblemwithanyaspectofthisbook,youcancontactusat,andwewilldoourbesttoaddresstheproblem.

mailto:[email protected]

Chapter1.InstallingaDistributedZabbixSetupMostlikely,ifyouarereadingthisbook,youhavealreadyusedandinstalledZabbixasanetworkmonitoringsolution.Now,inthischapter,wewillseehowtoinstallZabbixinadistributedsetup,eventuallymovingontoalargeuseofproxies.Thechapterwilltakeyouthroughallthepossiblescenariosandexplainthemaindifferencesbetweentheactiveandpassiveproxysetup.Usually,thefirstZabbixinstallationisdoneasapartoftheconcepttoseewhethertheplatformisgoodenoughforyou.Here,thecommonerroristostartusingthissetuponalargeproductionenvironment.Afterreadingthischapter,youwillbereadytoinstallandsetupalargeenvironmentreadyinfrastructure.

Inthischapter,wewillexplainhowtoprepareandsetupaZabbixinstallation,whichisreadytobegrownwithinyourinfrastructure,andreadyforalargetoaverylargeenvironment.ThisbookismainlyfocusedonZabbixfornetworkmonitoring.Thischapterwillquicklytakeyouthroughtheinstallationprocess,emphasizingonallthemostimportantpointsyouneedtoconsider.Inthenextchapter,wewillspendmoretimedescribingabetterapproachtomonitoryournetworkdevicesandhowtoretrieveallthecriticalmetricsfromthem.Afterreadingthischapter,youwillbecomeawareofthecommunicationbetweenserverandproxiesbeingabletomixtheactiveandpassivesetupinordertoimproveyourinfrastructure.YoucanextendthestrongcentralZabbixcoresetupwithmanylightweightandeffectiveZabbixproxiesactingasasatelliteinsideyournetworktoimproveyourmonitoringsystem.

ZabbixarchitecturesZabbixwasbornasadistributednetworkmonitoringtoolwithacentralwebinterfacewhereyoucanmanagealmosteverything.Nowadays,withZabbix2.4,thenumberofpossiblearchitectureshasbeenreducedtoasingleserversetupandaZabbix-proxiesdistributedsetup.

NoteFromZabbix2.4,thenode-setupwasdiscontinued.Moreinformationisavailableathttps://www.zabbix.com/documentation/2.4/manual/introduction/whatsnew240#node-based_distributed_monitoring_removed.

Now,thesimplestarchitecture(whichisreadytohandlelargeenvironmentssuccessfully)thatyoucanimplementcomposesofthreeservers:

WebserverRDBMSserverZabbixserver

Topreparethissimplesetupforalargeenvironmentsetting,it’sbettertouseadedicatedserverforeachoneofthesecomponents.

Thisisthesimplestsetupthatcanbeeasilyextendedandisreadytosupportalargeenvironment.

Theproposedarchitectureisshowninthefollowingdiagram:

ThiskindofsetupcanbeextendedbyaddingmanyZabbixproxiesresultinginaproxy-basedsetup.Theproxy-basedsetupisimplementedwithoneZabbixserverandseveralproxies:oneproxyperbranch,datacenteror,inourcase,foreachremotenetworksegmentyouneedtomonitor.

Thisconfigurationiseasytomaintainandofferstheadvantagetohaveacentralizedmonitoringsolution.Thiskindofconfigurationistherightbalancebetweenlargeenvironmentmonitoringandcomplexity.

https://www.zabbix.com/documentation/2.4/manual/introduction/whatsnew240#node-based_distributed_monitoring_removed

TheZabbixproxy,likeaserver,isusedtocollectdatafromanynumberofhostsordevices,acquiringallthemetricsrequestedandactingasaproxy.Thismeansthatitcanretainthisdataforanarbitraryperiodoftime,relyingonadedicateddatabasetodoso.Theproxydoesn’thaveafrontendandismanageddirectlyfromthecentralserver.

NoteTheproxylimitsitselftodatacollectionwithouttriggerevaluationsoractions;allthedataisstoredinitsdatabase.Forthisreason,it’sbettertouseanefficientrobustRDBMSthatcanpreventdatalossincaseofacrash.

AllthesecharacteristicsmaketheZabbixproxyalightweighttooltodeployandoffloadsomechecksfromthecentralserver.Ourobjectiveistocontrolandstreamlinetheflowofmonitoreddataacrossnetworks,andtheZabbixproxygivesusthepossibilitytosplitandsegregateitemsanddataonthedifferentnetworks.Themostimportantfeatureisthattheacquiredmetricsarestoredinitsdatabase.Therefore,incaseofanetworkloss,youwillnotlosethem.

UnderstandingZabbixdataflowThestandardZabbixdataflowiscomposedofseveralactorsthatsenddatatoourZabbixserver.OfallthesourcesthatcansenddatatoourZabbixserver,wecanidentifythreemaindatasources:

ZabbixagentZabbixsenderOtheragents(externalscriptsorcomponentsbuiltinhouse)

Theotheragentsrepresentedinthenextdiagramcanbeoftwomaintypes:

Customand/orthird-partyagentsZabbixproxy

Asthediagramdisplaysthedatathatgetsacquiredfrommanydifferentsourcesintheformofitems.Attheendofthediagram,youseetheGUI,whichpracticallyrepresentstheusersconnectedandthedatabasethatistheplacewhereallthevaluesarestored.

Inthenextsection,wewilldivedeepintotheZabbixproxies’dataflow.

UnderstandingtheZabbixproxies’dataflowZabbixproxiescanoperateintwodifferentmodes,activeandpassive.Thedefaultsetupistheactiveproxy.Inthissetup,theproxyinitiatesallconnectionstotheZabbixserver,theoneusedtoretrieveconfigurationinformationonmonitoredobjects,andtheconnectiontosendmeasurementsbacktotheserver.Here,youcanchangeandtweakthefrequencyofthesetwoactivitiesbysettingthefollowingvariablesintheproxyconfigurationfile:/etc/zabbix/zabbix_proxy.conf:

ConfigFrequency=3600DataSenderFrequency=1

Valuesareexpressedinseconds.OntheZabbixserver-side,youneedtocarefullysetthevalueofStartTrappers=.

Thisvalueneedstobegreaterthanthenumberofallactiveproxiesandnodesyoudeployed.Thetrapperprocesses,indeed,managealltheincominginformationfromtheproxies.

NotePleasenotethattheserverwillforkextraprocessesasrequired,ifneeded,butitisstronglyadvisabletopreforkalltheprocessesthatareneededduringthestartup.Thiswillreducetheoverheadduringthenormaloperation.

Ontheproxyside,anotherparametertoconsideris:

HeartbeatFrequency

Thisparametersetsasortofkeepalive,whichafterthedefinednumberofseconds,willcontacttheserveralthoughitdoesn’thaveanydatatosend.Theproxyavailabilitycanbeeasilycheckedwiththefollowingitem:

zabbix[proxy,"proxyuniquename",lastaccess]

Heretheproxyuniquename,ofcourse,istheidentifieryouassignedtotheproxyduringdeployment.Theitemwillreturnthenumberofsecondsasthelasttimethattheproxywascontacted,avalueyoucanthenusewiththeappropriatetriggeringfunctions.

TipIt’sreallyimportanttohaveatriggerassociatedtothisitem,soyoucanbewarnedincaseofconnectionloss.Lookingatthetrendofthistrigger,youcanlearnaboutaneventualreapingtimesetonthefirewall.Let’slookatapracticalexample:ifyounoticethatafter5minutesyourconnectionsaredropped,settheheartbeatfrequencyto120secondsandcheckforthelastaccesstimeabove300seconds.

Inthefollowingdiagram,youcanseethecommunicationflowbetweentheZabbixserverandtheproxy:

Asyoucanseefromthediagram,theserverwillwaittoreceiverequestsfromtheproxyandnothingmore.

NoteTheactiveproxyisthemostefficientwaytooffloaddutiesfromtheserver.Indeed,theserverwilljustsitherewaitingtobeaskedaboutchangesinconfiguration,ortoreceivenewmonitoringdata.

Ontheotherside,proxiesareusuallydeployedtomonitorsecurenetworksegmentswithstrictoutgoingtrafficpolicies,andareusuallyinstalledonDMZs.Inthesekindofscenarios,normally,itisverydifficulttoobtainpermissionfortheproxytoinitiatethecommunicationwiththeserver.Unfortunately,it’snotjustduetopolicies.DMZsareisolatedasmuchaspossiblefrominternalnetworks,astheyneedtobeassecureastheycan.Generally,it’softeneasierandmoreacceptedfromasecuritypointofviewtoinitiateaconnectionfromtheinternalnetworktoaDMZ.Inthiskindofscenario,thepassiveproxyisveryhelpful.Thepassiveproxyisalmostamirroredimageoftheactiveproxysetup,asyoucanseeinthefollowingdiagram:

Withthisconfiguration,theZabbixserverwillcontacttheproxyperiodicallytodelivertheconfigurationchangesandtorequesttheitemvaluestheproxyisholding.

Thisistheproxyconfigurationtoenabletheproxyyouneedtoset:

ProxyMode=1

Thisparameterspecifiesthepassiveproxy,youdon’tneedtodoanythingelse.Now,ontheserverside,youneedtosetthefollowingparameters:

StartProxyPollers=

Thiswillsetthenumberofprocessesdedicatedtothepassiveproxies

NoteTheStartProxyPollersparametershouldmatchthenumberofpassiveproxiesyouhavedeployed.

ProxyConfigFrequency=

Thisvalueexpressesthefrequencywithwhichtheserversendstheconfigurationtoitsproxy

ProxyDataFrequency=

Thisistheintervalparameterthatexpressesthenumberofsecondsbetweentwoconsecutiverequeststogettheacquiredmetricsfromtheproxy

Theitemusedtocheckapassiveproxy’savailabilityisasfollows:

zabbix[proxy,"proxyuniquename",lastaccess]

Thisisexactlythesameastheactiveone.

Thepassiveproxyenablesustogathermonitoringdatafromotherwiseclosedandlockeddownnetworkswithaslightlyincreasedoverhead.

NoteYoucanmixasmanyactiveandpassiveproxiesasyouwantinyourenvironment.Thisenablesyoutoexpandyourmonitoringsolutiontoreacheachpartofthenetworkandtohandlealargenumberofmonitoredobjects.Thisapproachkeepsthearchitecturesimpleandeasytomanagewithastrongcentralcoreandmanysimple,lightweightsatellites.

Ifyouwouldliketokeeptrackofalltheremainingitemsthattheproxyneedstosend,youcansetuptheproxytorunthisqueryagainstitsdatabase:

SELECT((SELECTMAX(proxy_history.id)FROMproxy_history)-nextid)FROMidsWHEREfield_name='history_lastid'

TipDownloadingtheexamplecode

Youcandownloadtheexamplecodefilesfromyouraccountathttp://www.packtpub.comforallthePacktPublishingbooksyouhavepurchased.Ifyoupurchasedthisbookelsewhere,youcanvisithttp://www.packtpub.com/supportandregistertohavethefilese-maileddirectlytoyou.

ThisquerywillreturnthenumberofitemsthattheproxystillneedstosendtotheZabbix

http://www.packtpub.comhttp://www.packtpub.com/support

server.ConsideringthatyouareusingMySQLasadatabase,youneedtoaddthefollowinguserparameterintheproxyagentconfigurationfile:

UserParameter=zabbix.proxy.items.sync.remaining,mysql-u-p''-e'SELECT((SELECTMAX(proxy_history.id)FROMproxy_history)-nextid)FROMidsWHEREfield_name=history_lastid'2>&1

Now,allyouneedtodoissetanitemontheZabbixserversideandyoucanseehowyourproxyisfreeingitsqueue.

InstallingZabbixZabbix,likealltheothersoftware,canbeinstalledintwoways:

1. Downloadthelatestsourcecodeandcompileit.2. Installitfrompackages.

Actually,thereisanotherwaytohaveaZabbixserverupandrunning:usingthevirtualappliance.TheZabbixserverappliancewillnotbeconsideredinthisbookasZabbixitselfdefinesthisvirtualapplianceasnotreadyforproductiveenvironments.Thisvirtualapplianceisnotaproductionreadysetupformanyreasons:

Itisamonolithwhereeverythingisinstalledonthesameserver.Thereisnoseparationfromthedatabaselayerandthepresentationlayer.Thismeansthateachoneofthesecomponentscanaffecttheperformanceoftheother.Thereisaclearwarningonthewebsitethatwarnsuswiththisstatement:TheApplianceisnotintendedforseriousproductionuseatthistime.

Ontheotherhand,theinstallationfrompackagesgivesussomebenefits:

ThepackagesmakeiteasytoupgradeandupdateDependenciesareautomaticallysortedout

Thesourcecodecompilationalsogivesussomebenefits:

WecancompileonlytheneededfeaturesWecanbuildtheagentstaticallyanddeployondifferentLinuxflavorsCompletecontrolonupdate

It’squiteusualtohavedifferentversionsofLinux,Unix,andMicrosoftWindowsonalargeenvironment.Thiskindofscenarioisquitediffusedonaheterogeneousinfrastructure,andifweusetheZabbix’sagentdistributionpackageoneachLinuxserver,wewillhavedifferentversionsoftheagentforsure,anddifferentlocationsfortheconfigurationfiles.

Themorethethingsarestandardizedacrossourserver,theeasieritwillbecometomaintainandupgradetheinfrastructure.The--enable-staticoptiongivesusawaytostandardizetheagentacrossdifferentLinuxversionsandrelease,whichisastrongbenefit.Theagent,staticallycompiled,canbeeasilydeployedeverywhereand,forsure,wewillhavethesamelocation(andwecanusethesameconfigurationfileapartfromthenodename)fortheagentandhis/herconfigurationfile.Theonlythingthatmightvaryisthestart/stopscriptandhowtoregisteritontherightinitrunlevel,butatleastthedeploymentwillbestandardized.

ThesamekindofconceptcanbeappliedtothecommercialUnix,bearinginmindtocompileitonthetargetenvironmentsothatthesameagentcanbedeployedondifferentUnixreleasesofthesamevendor.

InstallingfrompackagesThefirstthingtodotoinstallZabbixfromrepoistoaddtheyumrepositorytoourlist.Thiscanbedonewiththefollowingcommand:

$rpm-Uvhhttp://repo.zabbix.com/zabbix/2.4/rhel/6/x86_64/zabbix-release-2.4-1.el6.noarch.rpmRetrievinghttp://repo.zabbix.com/zabbix/2.4/rhel/6/x86_64/zabbix-release-2.4-1.el6.noarch.rpmwarning:/var/tmp/rpm-tmp.dsDB6k:HeaderV4DSA/SHA1Signature,keyID79ea5ed4:NOKEYPreparing…###########################################[100%]1:zabbix-release###########################################[100%]

Oncethisisdone,wecantakeadvantageofallthebenefitsintroducedbythepackagemanagerandhavethedependenciesautomaticallyresolvedbyyum.

ToinstalltheZabbixserver,yousimplyneedtorun:

$yuminstallzabbix-server-mysqlzabbix-agentzabbix-javagateway

Now,youhaveyourserverreadytostart.Wecan’tstartitnowasweneedtosetupthedatabase,whichwillbedoneinthenextheading,anyway,whatyoucandoissetupthestart/stoprunlevelforourzabbix_serverandzabbix_agentdaemons:

$chkconfig--level345zabbix-serveron$chkconfig--level345zabbix-agenton

Pleasedoublecheckifthepreviouscommandransuccessfullywiththefollowing:

$chkconfig--list|grepzabbixzabbix-agent0:off1:off2:off3:on4:on5:on6:offzabbix-server0:off1:off2:off3:on4:on5:on6:off

SettingupaZabbixagentNow,asusuallyhappensinalargeserverfarm,itispossiblethatyouhavemanydifferentvariantsofLinux.Here,ifyoucan’tfindthepackageforyourdistribution,youcaneventhinktocompiletheagentfromscratch.Thefollowingarethestepsforthesame:

1. DownloadthesourcecodefromtheZabbixwebsite.2. Unpackthesoftware.3. Satisfyallthesoftwaredependencies,installingalltherelated-develpackages.4. Runthefollowingcommand:$./configure--enable-agent.

TipHere,youcanstaticallylinktheproducedbinarywiththe--enable-staticoption.Withthis,thebinaryproducedwillnotrequireanyexternallibrary.ThisisreallyusefultodistributetheagentacrossdifferentversionsofLinux.

Compileeverythingwith$make.

Now,beforeyourun$makeinstall,youcandecidetocreateyourownpackagetodistributewithCheckInstall.

CreatingaZabbixagentpackagewithCheckInstallTheadviceistonotrunmakeinstall,butuseCheckInstalltoproducetherequiredpackageforyourLinuxOSfromhttp://asic-linux.com.mx/~izto/checkinstall/.

NoteWecanalsouseaprebuiltCheckInstall;thecurrentreleaseischeckinstall-1.6.2-20.2.i686.rpmonRedHat/CentOS.Thepackagewillalsoneedtherpm-buildpackage:

rpm-buildyuminstall

Also,weneedtocreatethenecessarydirectories:

mkdir-p~/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}

Thissoftwareenablesyoutocreateapackageformanydifferentversionsofthepackagemanager,namely,RPM,deb,andtgz.

NoteCheckInstallwillproducepackagesforDebian,Slackware,andRedHat,helpingustopreparetheZabbix’sagentpackage(staticallylinked)anddistributeitaroundourserver.

Now,weneedtoswitchtotherootaccountusing$sudosu–.Also,usethecheckinstallfollowedbytheseoptions:

$checkinstall--nodoc--install=yes-y

Ifyoudon’tfaceanyissue,youshouldgetthefollowingmessage:

******************************************************************Done.Thenewpackagehasbeensavedto/root/rpmbuild/RPMS/i386/zabbix-2.4.0-1.i386.rpmYoucaninstallitinyoursystemanytimeusing:rpm-izabbix-2*.4.0-1.i386.rpm******************************************************************

Rememberthattheserverbinarieswillbeinstalledin/sbin,utilitieswillbein/bin,andthemainpagesunderthe/sharelocation.

TipTospecifyadifferentlocationforZabbixbinaries,weneedtouse--prefixontheconfigureoptions(forexample,--prefix=/opt/zabbix).

http://asic-linux.com.mx/~izto/checkinstall/

ServerconfigurationFortheserverconfiguration,weonlyhaveonefiletocheckandedit:

/etc/zabbix/zabbix_server.conf

Alltheconfigurationfilesarecontainedinthefollowingdirectory:

/etc/zabbix/

Allyouneedtochangefortheinitialsetupisthe/etc/zabbix/zabbix_server.confconfigurationfileandwritetheusername/passwordanddatabasenamehere.

NotePleasetakecaretoprotecttheaccesstotheconfigurationfilewithchmod400/etc/zabbix/zabbix_server.conf.

Thedefaultexternalscriptslocationis:

/usr/lib/zabbix/externalscripts

Also,thealertscriptdirectoryis:

/usr/lib/zabbix/alertscripts

Thiscanbechangedbyeditingthezabbix_server.conffile.

Theconfigurationontheagentsideisquiteeasy;basically,weneedtowritetheIPaddressofourZabbixserver.

InstallingadatabaseThedatabasewewilluseonthisbook,asalreadyexplained,isMySQL.

Now,consideringthatyouhaveaRedHatserver,theproceduretoinstallMySQLfromtheRPMrepositoryisquiteeasy:

$yuminstallmysqlmysql-server

Now,youneedtosetuptheMySQLservicetostartautomaticallywhenthesystemboots:

$chkconfig--levels235mysqldon$/etc/init.d/mysqldstart

TipRemembertosetapasswordfortheMySQLrootuser

Tosetapasswordfortheroot,youcanrunthesetwocommands:

/usr/bin/mysqladmin-urootpassword'new-password'/usr/bin/mysqladmin-uroot-hhostname-of-your.zabbix.dbpassword'new-password'

Alternatively,youcanrun:

/usr/bin/mysql_secure_installation

Thiswillalsohelpyoutoremovethetestdatabasesandanonymoususerdatathatwascreatedbydefault.Thisisstronglyrecommendedforproductionservers.

Now,it’stimetocreatetheZabbixdatabase.Forthis,wecanusethefollowingcommands:

$mysql-uroot-p$mysql>CREATEDATABASEzabbixCHARACTERSETUTF8;QueryOK,1rowaffected(0.00sec)$mysql>GRANTALLPRIVILEGESonzabbix.*to'zabbixuser'@'localhost'IDENTIFIEDBY'zabbixpassword';QueryOK,0rowsaffected(0.00sec)$mysql>FLUSHPRIVILEGES;$mysql>quit

Next,weneedtorestorethedefaultZabbixMySQLdatabasefiles:

$mysql-uzabbixuser-pzabbixpasswordzabbix</usr/share/doc/zabbix-server-mysql-2.4.0/create/schema.sql$mysql-uzabbixuser-pzabbixpasswordzabbix</usr/share/doc/zabbix-server-mysql-2.4.0/create/images.sql$mysql-uzabbixuser-pzabbixpasswordzabbix</usr/share/doc/zabbix-server-mysql-2.4.0/create/data.sql

Now,ourdatabaseisready.Beforewebegintoplaywiththedatabase,it’simportanttodosomeconsiderationaboutdatabasesizeandheavytasksagainstit.

ConsideringthedatabasesizeZabbixusestwomaingroupsoftablestostoreitsdata:

HistoryTrends

Now,thespaceconsumedbythesetablesisinfluencedby:

Items:Thisisthenumberofitemsyou’regoingtoacquireRefreshrate:ThisisthemeanaveragerefreshrateofouritemsSpacetostorevalues:ThisdependsonRDBMS

Thespaceusedtostoredatacanvaryduetothedatabase,butwecanresumethespaceusedbythesetablesinthefollowingtable:

Typeofmeasure Retentionindays Spacerequired

History 30 10.8G

Events 1825(5years) 15.7GB

Trends 1825(5years) 26.7GB

Total NA 53.2GB

Thiscalculationis,ofcourse,doneconsideringtheenvironmentafter5yearsofretention.Anyway,weneedtohaveanenvironmentreadytosurvivethisperiodoftimeandretainthesameshapethatithadwhenitwasinstalled.Wecaneasilychangethehistoryandtrendsretentionpolicyperitem.Thismeansthatwecancreateatemplatewithitemsthathaveadifferenthistoryretentionbydefault.Normally,thehistoryissetto30days,butforsomekindofmeasure(suchasinwebscenarios)orotherparticularmeasures,weneedtokeepallthevaluesformorethanaweek.Thispermitsustochangethisvalueoneachitem.

MySQLpartitioningNowthatweareawareofhowbigourdatabasewillbe,it’seasytoimaginethathousekeepingwillbeaheavytaskandthetime,CPU,andresourceconsumedbythisonewillgrowtogetherwiththedatabasesize.

Housekeepingisinchargetoremovetheoutdatedmetricsfromthedatabaseandtheinformationdeletedbyauser,andaswe’veseenthehistory,trends,andeventstablesare,aftersometime,hugetables.Thisexplainswhytheprocessissoheavytomanage.

Theonlywaywecanimproveperformancesoncewehavereachedthisvolumeofdataisbyusingpartitioninganddisablingthehousekeeperaltogether.

Partitioningthehistoryandtrendtableswillprovideuswithmanymajorbenefits:

Allhistorydatainatableforaparticulardefinedwindowtimeareself-containedinitsownpartition.Thisallowsyoutoeasilydeleteolddatawithoutimpactingthedatabaseperformance.WhenyouuseMySQLwithInnoDB,andifyoudeletedatacontainedinatable,thespaceisnotreleased.Thespacefreedismarkedasfree,butthediskspaceconsumedwillnotchange.Whenyouusepartition,andifyoudropapartition,thespaceisimmediatelyfreed.Queryperformancecanbeimproveddramaticallyinsomesituations,inparticular,whenthereisheavyaccesstothetable’srowsinasinglepartition.Whenaqueryupdatesahugeamountofdataorneedsaccesstoalargepercentageofthepartition,thesequentialscanisoftenmoreefficientthantheindexusagewitharandomaccessorscatteredreadsagainstthisindex.

Unfortunately,Zabbixisnotabletomanagethepartitions.So,weneedtodisablehousekeeping,anduseanexternalprocesstoaccomplishhousekeeping.

Whatweneedtohaveisastoredprocedurethatdoesalltheworkforus.

Thefollowingisthestoredprocedure:

DELIMITER$$CREATEPROCEDURE`partition_maintenance`(SCHEMA_NAMEVARCHAR(32),TABLE_NAMEVARCHAR(32),KEEP_DATA_DAYSINT,HOURLY_INTERVALINT,CREATE_NEXT_INTERVALSINT)BEGINDECLAREOLDER_THAN_PARTITION_DATEVARCHAR(16);DECLAREPARTITION_NAMEVARCHAR(16);DECLARELESS_THAN_TIMESTAMPINT;DECLARECUR_TIMEINT;

Untilhere,wehavedeclaredthevariableweneedafter.Now,onthenextline,wewillcallthestoredprocedureresponsibletocheckwhetherapartitionisalreadypresentandifnot,wewillcreatethem:

CALLpartition_verify(SCHEMA_NAME,TABLE_NAME,HOURLY_INTERVAL);SETCUR_TIME=UNIX_TIMESTAMP(DATE_FORMAT(NOW(),'%Y-%m-%d00:00:00'));

IFDATE(NOW())='2014-04-01'THENSETCUR_TIME=UNIX_TIMESTAMP(DATE_FORMAT(DATE_ADD(NOW(),INTERVAL1DAY),'%Y-%m-%d00:00:00'));ENDIF;SET@__interval=1;create_loop:LOOPIF@__interval>CREATE_NEXT_INTERVALSTHENLEAVEcreate_loop;ENDIF;SETLESS_THAN_TIMESTAMP=CUR_TIME+(HOURLY_INTERVAL*@__interval*3600);SETPARTITION_NAME=FROM_UNIXTIME(CUR_TIME+HOURLY_INTERVAL*(@__interval-1)*3600,'p%Y%m%d%H00');

Nowthatwehavecalculatedalltheparametersneededbythecreate_partitionprocedure,wecanrunit.Thisstoredprocedurewillcreatethenewpartitiononthedefinedschema:

CALLpartition_create(SCHEMA_NAME,TABLE_NAME,PARTITION_NAME,LESS_THAN_TIMESTAMP);SET@__interval=@__interval+1;ENDLOOP;SETOLDER_THAN_PARTITION_DATE=DATE_FORMAT(DATE_SUB(NOW(),INTERVALKEEP_DATA_DAYSDAY),'%Y%m%d0000');

Thesectionthatfollowsisresponsibletoremovetheolderpartitions,usingtheOLDER_TAN_PARTITION_DATEprocedure,whichwehavecalculatedonthelinesbefore:

CALLpartition_drop(SCHEMA_NAME,TABLE_NAME,OLDER_THAN_PARTITION_DATE);END$$DELIMITER;

Thisstoredprocedurewillbethecoreofourhousekeeping.Itwillbecalledwiththefollowingsyntax:

CALLpartition_maintenance('','',,,)

Theprocedureworksbasedon1hourintervals.Next,ifyouwanttopartitiononadailybasis,theintervalwillbe24hours.Instead,ifyouwant1hourpartitioning,theintervalwillbe1.

Youneedtospecifythenumberofintervalsthatyouwantcreatedinadvance.Forexample,ifyouwant2weeksintervaloffuturepartitions,use14.Ifyourintervalis1(forhourlypartitioning),thenthenumberofintervalstocreateis336(24*14).

Thisstoredprocedureusessomeotherstoresprocedures:

partition_create:Thiscreatesthepartitionforthespecifiedtablepartition_verify:Thischeckswhetherthepartitionisenabledonatable,ifnot,thencreateasinglepartitionpartition_drop:Thisdropspartitionsolderthanatimestamp

Forallthedetailsaboutthesestoredprocedures,seeAppendixA,PartitioningtheZabbix

Database.

Onceyou’vecreatedalltherequiredstoredprocedures,youneedtochangetwoindexestoenabletheminordertobereadyforapartitionedtable:

mysql>Altertablehistory_textdropprimarykey,addindex(id),dropindexhistory_text_2,addindexhistory_text_2(itemid,id);QueryOK,0rowsaffected(0.49sec)Records:0Duplicates:0Warnings:0

mysql>Altertablehistory_logdropprimarykey,addindex(id),dropindexhistory_log_2,addindexhistory_log_2(itemid,id);QueryOK,0rowsaffected(2.71sec)Records:0Duplicates:0Warnings:0

Oncethisisdone,youneedtoschedulethepartition_maintenance_allstoredprocedurewithacronjob.Formoredetailsaboutthepartition_maintenance_allprocedure,pleasechecktheinstructionscontainedinAppendixA,PartitioningtheZabbixDatabase.Thecronjobneedstoexecutethefollowingcommand:

mysql-h-u-pzabbixdatabase-e"CALLpartition_maintenance_all('zabbix');"

Oncethishasbeenset,youneedtobearinmindtodisablethehousekeepingforhistoryandtrends.VerifythattheOverrideitemperiodZabbixconfigurationischeckedforbothhistoryandtrends.Here,youneedtosettheDatastorageperiod(indays)boxforhistoryandtrendstothevalueyou’vedefinedinyourprocedure,ourexampleinAppendixA,PartitioningtheZabbixDatabaseisof28and730.

InstallingaZabbixproxyInstallationoftheZabbixproxyfrompackagesisaquitesimpletask.Onceyou’veaddedtheZabbixrepository,youonlyneedtorunthefollowingcommand:

$yuminstallzabbix-proxy-mysql

Thiswillinstalltherequiredpackages:

Installation:zabbix-proxy-mysqlx86_642.4.0-1.el6zabbix390kInstallingfordependencies:zabbix-proxyx86_642.4.0-1.el6zabbix21k

TheZabbixproxyinstallationisquitesimilartotheserverone.Onceyou’veinstalledtheserver,youneedtoinstallMySQL,createthedatabase,andimporttheDBschema:

$mysql-uroot-p$mysql>CREATEDATABASEzabbixCHARACTERSETUTF8;QueryOK,1rowaffected(0.00sec)$mysql>GRANTALLPRIVILEGESonzabbix.*to'zabbixuser'@'localhost'IDENTIFIEDBY'zabbixpassword';QueryOK,0rowsaffected(0.00sec)$mysql>FLUSHPRIVILEGES;$mysql>quit

Next,weneedtorestorethedefaultZabbixMySQLdatabasefiles:

$mysql-uzabbixuser-pzabbixpasswordzabbix</usr/share/doc/zabbix-proxy-mysql-2.4.0/create/schema.sql

Now,weneedtostartthedatabase,configuretheproxy,andstarttheservice.Inthisexample,wehaveconsideredtouseaZabbixproxythatreliesonaMySQLwithInnoDBdatabase.Thisproxycanbeperformedintwodifferentways:

Lightweight(andthenuseSQLite3)Robustandsolid(andthenuseMySQL)

Here,wehavechosenthesecondoption.Inalargenetworkenvironmentwheretheproxy,incaseofissue,needstopreserveallthemetricsacquireduntiltheserveracquiresthemetrics,it’sbettertoreduce,attheminimum,theriskofdataloss.Also,ifyouconsiderthisscenarioinalargenetworkenvironment,youmostlikelywillhavethousandsofsubnetworksconnectedtotheZabbixserverwithallthepossiblenetworkdevicesin-between.Well,exactly,thisisnecessarytouseadatabasethatcanpreventanydatacorruptions.

InstallingtheWebGUIinterfaceTheWebGUIinterfacewillbedoneoncemoreusingtheRPMs.

Toinstallthewebinterface,youneedtorunthefollowingcommand:

$yuminstallzabbix-web-mysql

Yumwilltakecaretoresolveallthedependencies.Onceyou’redone,theprocessofthiscomponentisquiteeasy:weneedtoopenawebbrowser,pointatthefollowingURL:http://your-web-server/zabbix,andfollowtheinstructions.

OnthestandardRedHatsystem,yousimplyneedtochangetheseparametersonyour/etc/php.inifile:

php_valuemax_execution_time300php_valuememory_limit128Mphp_valuepost_max_size16Mphp_valueupload_max_filesize2Mphp_valuemax_input_time300

Also,setyourtimezoneonthesamefile(forexample,php_valuedate.timezoneEurope/Rome).

Now,it’stimetostartupApache,butbeforethis,weneedtocheckwhetherwehaveSELinuxenabledandonwhichmode?TocheckyourSELinuxstatus,youcanrun:

#sestatusSELinuxstatus:enabledSELinuxfsmount:/selinuxCurrentmode:permissiveModefromconfigfile:permissivePolicyversion:24Policyfromconfigfile:targeted

Now,youneedtocheckwhetheryouhavethehttpddaemonenabledtousethenetworkwiththefollowingcommand:

#getseboolhttpd_can_network_connecthttpd_can_network_connect-->off

Mostlikely,youwillhavethesamekindofresult,thenallweneedtodoisenablethehttpd_can_network_connectoptionusingthenextcommandwith–Ptopreservethevalueafterareboot:

#setsebool–Phttpd_can_network_connecton#getseboolhttpd_can_network_connecthttpd_can_network_connect-->on

Now,allthatwestillhavetodoisenablethehttpddaemonandstartourhttpdserver:

#servicehttpdstartStartinghttpd:[OK]

Next,enablethehttpdserverasaservice:

#chkconfighttpdon

Wecancheckthechangedonewiththenextcommand:

#chkconfig--listhttpdhttpd0:off1:off2:on3:on4:on5:on6:off

Onceyou’vedonethis,youonlyneedtofollowthewizard,andinafewclicks,youwillhaveyourwebinterfacereadytostartup.

TipIfyouknowthattheloadagainstthewebserverwillbehigh,duetoahighnumberofaccountsthatwillaccessit,probably,it’sbettertoconsiderusingNginx.

Now,youcanfinallystartyourZabbixserverandthefirstentryinthe/var/log/zabbix/zabbix_server.logfilewilllooksomethinglikethefollowingcode:

37909:20140925:091128.868StartingZabbixServer.Zabbix2.4.0(revision48953).37909:20140925:091128.868******Enabledfeatures******37909:20140925:091128.868SNMPmonitoring:YES37909:20140925:091128.868IPMImonitoring:YES37909:20140925:091128.868WEBmonitoring:YES37909:20140925:091128.868VMwaremonitoring:YES37909:20140925:091128.868Jabbernotifications:YES37909:20140925:091128.868EzTextingnotifications:YES37909:20140925:091128.868ODBC:YES37909:20140925:091128.868SSH2support:YES37909:20140925:091128.868IPv6support:YES37909:20140925:091128.868******************************37909:20140925:091128.868usingconfigurationfile:/etc/zabbix/zabbix_server.conf******************************

Next,youcanstarttoimplementandacquirealltheitemscriticalforyournetwork.

SummaryInthischapter,wecoveredalargenumberofcomponents.Westartedwithdefiningwhatalargeenvironmentis.Wealsosawhowthenetworksetupcanbedesignedandhowitcanevolvewithinyourinfrastructure.Wesawtheheaviesttaskontheserverside(housekeeping)andhowtoavoidperformancedegradationduetothis.WediscussedMySQLpartitioningin-depth.Wealsobrieflydiscussedthedifferencesbetweenactiveandpassiveproxies;youwillnowbeabletodecidehowtosetthemupandwhichonetochooseonceyouknowyournetworktopology.Also,wesawhowtoacquiresomecriticalmetricstomonitortheZabbixproxyconnectionandtheamountofitemsthatitstillneedstosendus.

Asyoucansee,wecoveredalotofargumentsinjustonechapter;wedidthisbecausewewouldliketousemorespaceintheupcomingchapters.Inthenextchapter,wewillexplorethedifferentappliancesandprotocolsatlayer2andlayer3oftheISO/OSIstack.Also,youwillseehowtobestextrapolatemeaningfulmonitoringdatafromthecollectedmeasurefortheprotocollayers2and3.

Chapter2.ActiveMonitoringofYourDevicesNowthatyouhaveaworkingZabbixsetup,it’stimetotakealookatyournetworkandfigureoutthecomponentsthatyouwanttomonitor,thekindofdatayouwanttocollect,andtheconditionsunderwhichyouwanttobenotifiedaboutproblemsandstatechanges.

Itwouldbeimpossibleforanybookonthistopictofullycoverallthedifferentkindsofnetworkappliancesandtopologiesandallthedifferentmonitoringscenariosthatanetworkadministratormightneedaseveryenvironmenthasitsownspecificquirksthatagoodmonitoringsolutionhastoaccountfor.ThischapterwillofferyouafewexamplesofthedifferentmonitoringpossibilitiesZabbixcanachievebyrelyingondifferentmethodsandprotocols.You’llseehowtoqueryyournetworkfromthedatalinklayeruptoroutingandnetworkflowusingICMP,SNMP,andlogparsingfacilitiestocollectyourmeasurements.

You’lllearnhowtoextractmeaningfulinformationfromthedatayougatheredusingaggregatedandcalculateditemsandhowtoconfigurecomplextriggersthatwillalertyouaboutrealnetworkissueswhileminimizinguninterestingornonrelevantdata.

Bytheendofthechapter,you’llhaveagoodoverviewofZabbix’snetworkmonitoringpossibilities,andyou’llbereadytoadaptwhatyoulearnedforyourspecificrequirements.Butlet’sfirsthaveaquickoverviewofhowZabbixorganizesmonitoringdatawithhosts,templates,items,andtriggers.

UnderstandingZabbixhostsOneofZabbix’sgreatstrengthsisitsflexibilitywhenitcomestoorganizingmonitoringdata.Evenwithoutconsideringitspowerfultemplatinganddiscoveryfeatures,whichwillbecoveredinChapter4,DiscoveringYourNetwork,thereisalotthatyoucandowithstandardhosts,items,andtriggers.Hereareafewtipsonhowyoucanusethemeffectively.

HostsandhostgroupsZabbixhostsusuallyrepresentasingle,specificboxorapplianceinyournetwork.Theycanalsobeapartofoneormorehostgroups.

HostgroupsareveryusefulastheymakeiteasytonavigateZabbix’sinterface,separatinghostsintocategoriesandallowingyoutoorganizeandmanageahugeamountofapplianceswithouthavingtodealwithimpossiblylonglistsofhostnames.Thesamehostcanbepartofdifferenthostgroups,andthiscanbeveryusefulasyoumightwant,forexample,tohaveagroupforallyourrouters,agroupforallyourswitches,andagroupforeverysubnetyoumanage.So,asinglerouterwillbepartoftheroutersgroupandallthesubnetgroupsithasaninterfaceon,whileaswitchwillbepartoftheswitchesgroupandofthesubnetit’spartof,andsoon.

Whilethisiscertainlyagoodwaytoorganizeyourhosts,bothtovisualizeandtomanageyourmonitoringdata,thereareacoupleofnot-too-obviouspitfallsyoushouldbeawareofifyoudecidetoputthesamehostinmultiplegroups:

Calculateditemsshowaggregatemonitoringdatabasedonhostgroupmembership.Ifyouconfigureanaggregateditemthatusesmorethanonecalculateditemfromdifferenthostgroups,youcanendupusingthesamehost’sdatamorethanonce,introducingasignificanterrorinyourcalculations.Actionsareusuallyfilteredbasedonhostgroups.Thismeansthatthesametriggereventcouldfireupmorethanoneactionifthehostispartofmorethanonehostgroup,leadingtopotentiallyduplicatemessagesandalerts.Useraccesspermissionsarehost-group-based.Thismeansthatsomeuserscouldbeabletoseemorehostsandmonitoringdatathantheyactuallyneedtoifahostendsupinahostgrouptheyhaveaccessto.

Thisisbynomeansanattempttodiscouragethepracticeofassigningmultiplehostgroupstothesamehost.Justbeawareoftheramificationsofsuchapracticeanddon’tforgettotakeintoconsiderationtheaddedcomplexitywhenyouconfigureyouritems,actions,andaccesspermissions.

HostinterfacesEachhostiscomposedofacollectionofitemsthatrepresenttherawmonitoringdata,andtriggers,whichrepresentZabbix’smonitoringintelligencebasedonthedatagathered.It’salsocomposedofaseriesofinterfacesthattelltheZabbixserverorproxyhowtocontactthehosttocollecttheaforesaidmonitoringdata.Mostnetworkapplianceshavemorethanoneinterface,soyouwouldwanttomakesurethatallhoststhatrepresentrouters,firewalls,proxies,gateways,andwhatnot,arelistingallthoseappliances’interfacesandtheiraddresses.Theadvantagesareobvious:

You’llbeabletoquicklyreviewwhataddressesareconfiguredonaspecifichostwhilelookingatmonitoringdataYou’llbeabletodifferentiateyourchecksbyqueryingdifferentaddressesorportsofthesamehostbasedonyourneeds

Yourmapsandtopologieswillbemoreconsistentwithwhat’sactuallydeployed

Addinginterfacestoahostisfairlystraightforward.AllyouneedtodoisnavigatetoConfiguration|Hostsandthenselectthehostyouwanttoedit.Theinterfacessectionisinthemainconfigurationtab,asshowninthefollowingscreenshot:

Asyoucanseeintheaboveexample,therearethreeagentinterfacesthatshowallthenetworkstherouterisconnectedtoandjustoneSNMPinterface.AgentinterfacesareusednotonlyforZabbixagentitems,butalsoforsimpleandexternalchecks.Ontheotherhand,you’lluseSNMPinterfacestosendSNMPqueriestoyourhost.Theprecedingexampleassumesthatyou’llonlyuseSNMPontherouter’sinterfacethatisconnectedtoamanagementnetwork(192.168.1.0inthisexample),whileyou’llalsouseICMP,TCP,andexternalchecksonitstwoproductioninterfaces.Ofcourse,youarefreetoconfiguredifferentIPaddressesforAgentandSNMPinterfacesdependingonwhatprotocolsandchecksyouplantoactivateonwhichinterfaces.

HostinventoryHavinginventorydatadirectlyavailableinyourmonitoringsolutionhasalotofobviousadvantageswhenitcomestoattachingusefulinformationtoyouralertsandalarms.Unfortunately,themorehostsyouhavetomanage,themoreessentialitistohaveup-to-dateinventoryinformation,andtheharderitistomaintaintheaforesaidinformationinareliableandtimelymanner.Manuallyupdatingahost’sinventorydatacanquicklybecomeanimpossibletaskwhenyouhavetensorhundredsofhoststomanage,andit’snotalwayspossibletowriteautomatedscriptsthatwilldothejobforyou.Fortunately,Zabbixoffers

anautomaticinventoryfeaturethatcanatleastpartiallyfillininventorydatabasedonactualmonitoringdata.Toactivatethisfeature,firstyou’llneedtoselectAutomaticintheHostinventorytabofahostconfigurationpageandthenmovetotheitemsthatyou’llusetopopulatetheinventorydata.

Whenconfiguringanitem,youshouldassignitsdatatoaspecificinventoryfieldsothattheaforesaidfield’svaluewillbesetandautomaticallyupdatedbasedontheitem’smeasurements,asshowninthefollowingscreenshot:

Asyoucanseeintheprecedingexample,ahost’slocationinventoryvaluewillbepopulatedbasedonthecorrespondingSNMPquery.Thismeansthatifyouchangeadevice’slocationinformation,thatchangewillbereflectedinZabbixassoonastheitem’svalueispolledonthedevice.Dependingonthedataavailableonthedevice,you’llbeabletopopulateonlyafewinventoryfieldsormostofthem,whilefallingbackonmanualupdatesofthefieldsthatfalloutsideofyourdevice’sreportingpossibilities.

Speakingofitems,let’snowfocusonthedifferentmonitoringpossibilitiesthatZabbixitemsofferandhowtoapplythemtoyourenvironment.

GoingbeyondZabbixagentsTherearecertainlymanyadvantagesinusingZabbix’sownagentsandprotocolwhenitcomestomonitoringWindowsandUnixoperatingsystemsortheapplicationsthatrunonthem.However,whenitcomestonetworkmonitoring,thevastmajorityofmonitoredobjectsarenetworkappliancesofvariouskinds,whereit’softenimpossibletoinstallandrunadedicatedagentofanytype.Thisbynomeansimpliesthatyou’llbeunabletofullyleverageZabbix’spowertomonitoryournetwork.Whetherit’sasimpleICMPechorequest,anSNMPquery,anSNMPtrap,netflowlogging,oracustomscript,therearemanypossibilitiestoextractmeaningfuldatafromyournetwork.Thissectionwillshowyouhowtosetupthesedifferentmethodsofgatheringdata,andgiveyouafewexamplesonhowtousethem.

SimplechecksLet’sstartwiththesimplestcase.Atfirstglance,simplechecksdon’tlookthatinteresting:excludingalltheVMwareHypervisorchecksthatareincludedinthiscategory,simplechecksarereducedtoacoupleofgenericTCP/IPconnectionchecksandthreeICMPechochecks,asfollows:

Checkname Description

Icmpping Thisreturns1ifthehostrespondstoanICMPping;0otherwise

Icmppingloss ThisreturnsthepercentageoflostICMPpingpackets

Icmppingsec ThisreturnstheICMPresponsetimeinseconds

Net.tcp.service Thisreturns1ifthehostacceptsconnectionsonaspecifiedTCPport;0otherwise

Net.tcp.service.perf ThisreturnsthenumberofsecondsspenttoobtainaconnectiononaspecifiedTCPport

Generallyspeaking,thesechecksprovemoreusefulasthedistancebetweenthemonitoringprobeandthemonitoredhostincreases,bothintermsofphysicaldistance(ageographicallinktoanothercityforexample)andintermsofhopsapackethastogothrough.Thismeansthatifyouareinterestedinyournetwork’sperformance,itwouldmakesensetoassignhostswithsimplecheckstoZabbixproxiesthatarenotinthesamesubnet,butaresituatedwheretheywillmimicascloselyaspossibleyouractualnetworktraffic.Net.tcp.serviceisparticularlyusefulfromthispointofview,notjusttocheckthestatusoftheavailabilityofspecificserviceswhenyoucannotuseZabbixagents,butalsotocheckgeneralhostavailabilityacrossrestrictivefirewallsthatblockICMPtraffic.

TipInordertoreducenetworktrafficandtomakemoreefficientICMPchecks,Zabbixusesfpinginsteadoftheregularpingwhenexecutingicmpping,icmppingloss,andicmppingsecitemchecks.

MakesureyouhavefpinginstalledonyourZabbixserverandalsoonalltheZabbixproxiesthatmightneedit.Ifyoudon’thaveit,asimpleyuminstallfpingwillusuallybeenoughfortheZabbixdaemonstofinditanduseit.

Whilebothnet.tcp.serviceandnet.tcp.service.perfdosupportsomewell-knownprotocols,suchasSSH,FTP,HTTP,andsoon,thesetwoitems’mostusefuloptionisprobablytheonethatallowsyoutoperformasimpleTCPhandshakeconnectionandcheckwhetheraspecificIPisreachableonaspecificport.Thesekindofchecksareusefulbecause,justlikeICMPpings,theywillmostlyinvolvethenetworkstack,reducingapplicationoverheadtoaminimum,thusgivingyoudatathatmorecloselymatchesyouractualnetworkperformance.Ontheotherhand,unlikeICMPpings,theywillallowyoutocheckforTCPportavailabilityforagivenhost.Obvioususecasesincludemakinglightweightservicechecksthatwillnotimpactverybusyhostsorappliancestoomuch,

andmakingsurethatagivenfirewallisallowingtrafficthrough.

Aslightlylessobvioususecaseisusingoneormorenet.tcp.serviceitemstomakesurethatsomeservicesarenotrunningonagiveninterface.Takeforexample,thecaseofaborderrouterorfirewall.Unlessyouhavesomeveryspecialandspecificneeds,you’lltypicallywanttomakesurethatnoadminconsolesareavailableontheexternalinterfaces.Youmighthavedouble-checkedtheappliance’sinitialconfiguration,butasystemupdate,acarelessadmin,orasecuritybugmightchangetheaforesaidconfigurationandopenyourappliance’sadmininterfacestoafarwideraudiencethanintended.AsecuritybreachlikethisonecouldpassunobservedforalongtimeunlessyouconfigureafewsimpleTCP/IPchecksonyourappliance’sexternalinterfacesandthensetupsometriggersthatwillreportaproblemifthosechecksreportanopenandresponsiveport.

Let’staketheexampleoftherouterwithtwoproductioninterfacesandamanagementinterfaceshowninthesectionabouthostinterfaces.Iftherouter’sHTTPSadminconsoleisavailableonTCPport8000,you’llwanttoconfigureasimplecheckitemforeveryinterface:

Itemname Itemkey

management_https_console net.tcp.service[https,192.168.1.254,8000]

zoneA_https_console net.tcp.service[https,10.10.1.254,8000]

zoneB_https_console net.tcp.service[https,172.16.7.254,8000]

Allthesecheckswillreturn1iftheserviceisavailable,and0iftheserviceisnotavailable.Whatchangesishowyouimplementthetriggersontheseitems.Forthemanagementitem,you’llhaveaproblemiftheserviceisnotavailable,whilefortheothertwo,you’llhaveaproblemiftheserviceisindeedavailable,asshowninthefollowingtable:

Triggername Triggerexpression

Managementconsoledown {it-1759-r1:net.tcp.service[http,192.168.1.254,8000].last()}=0

ConsoleavailablefromzoneA {it-1759-r1:net.tcp.service[http,10.10.1.254,8000].last()}=1

ConsoleavailablefromzoneB {it-1759-r1:net.tcp.service[http,172.16.7.254,8000].last()}=1

Thisway,you’llalwaysbeabletomakesurethatyourdevice’sconfigurationwhenitcomestoopenorclosedportswillalwaysmatchyourexpectedsetupandbenotifiedwhenitdivergesfromthestandardyouset.

Tosummarize,simplechecksaregreatforallcaseswhereyoudon’tneedcomplexmonitoringdatafromyournetworkastheyarequitefastandlightweight.Forthesamereason,theycouldbethepreferredsolutionifyouhavetomonitoravailabilityforhundredstothousandsofhostsastheywillimpartarelativelylowoverheadonyour

overallnetworktraffic.

Whenyoudoneedmorestructureandmoredetailinyourmonitoringdata,it’stimetomovetothebreadandbutterofallnetworkmonitoringsolutions:SNMP.

KeepingSNMPsimpleTheSimpleNetworkMonitoringProtocol(SNMP)isanexcellent,generalpurposeprotocolthathasbecomewidelyusedbeyonditsoriginalpurpose.Whenitcomestonetworkmonitoringthough,it’salsooftentheonlyprotocolsupportedbymanyappliances,soit’softenaforced,albeitnaturalandsensible,choicetointegrateitintoyourmonitoringscenarios.Asanetworkadministrator,youprobablyalreadyknowallthereistoknowaboutSNMPandhowitworks,solet’sfocusonhowit’sintegratedintoZabbixandwhatyoucandowithit.

Firstofall,we’llneedtotalkaboutSNMPgetsandSNMPtrapsintwodifferentdiscussionsastheyareimplementedandusedindifferentwaysbyZabbix.ThereasonforthisseparationisintheverynatureofSNMPgetsasopposedtoSNMPtraps.AnSNMPgetrepresentsasingle,discretepieceofinformationthatrepresentsthecurrentstatusofametric,andit’snottiedtoanyspecificevent.Whetherit’sacounterwiththetotalnumberofbytesthatpassedthroughaninterface,aBooleanvaluethatwilltellifalinkisupordown,orastringwithanappliance’slocationorcontactinformation,anSNMPvaluewillbeavailableatanymoment,anditwillbepossibletopollitwithanarbitraryfrequency.

ThismapsnicelytoZabbixitems.JustlikeSNMPgetvalues,theyalsorepresentsingle,discretevaluesthatcanbepolledwitharbitraryfrequency.ThismakesitreallystraightforwardtouseregularSNMPqueriestopopulateZabbixitemssincetheonlythingsyouhavetoworryaboutaretheSNMPOID,thedatatype,andthecommunitystringorauthenticationinformation.We’llseeafewexamplesinthenextparagraph.

AnSNMPtraprepresentsaspecificeventthathappensataspecificpointintime.Itmightrepresentalinkstatechange,arebootevent,orauserlogin.Inanycase,youcannotquerythestateofanSNMPtrap;youjusthavetowaittoreceiveone,anditwillnotrepresentasingle,discretevaluebutachangefromonevaluetoanother.Theyresemble,inmanyways,Zabbixeventsinsteadofrawdata.ThiscomplicatesthingsalittlesinceZabbixeventsaretheresultofevaluatingtriggersagainstcollecteddata,whileSNMPtrapscanonlyenterZabbixasitemvalues,thatis,ascollecteddata.Sowe’llneedtoresolvethisapparentmismatchinordertofullyleveragetheinformationcontainedinSNMPtraps.We’llseehowinashortwhile,butfirstlet’slookatafewdetailsconcerningregularSNMPqueriesexecutedfromZabbix.

GettingSNMPdataintoZabbixAZabbixserverusuallycomeswithgoodSNMPsupportoutofthebox.Notonlydoesitsupportthequeryingprotocolnatively,butitalsocomesequippedwithanumberofSNMPtemplatesthatcangetyoustartedintherightdirection.ThismeansthatformostdevicesyouonlyhavetolinktheTemplateSNMPDevicetemplate,andyou’llimmediatelybeabletogetsomebasicinformationaboutit,asshowninthefollowingscreenshot:

We’vealreadyseenhowtheDevicelocationitemcanbeusedtopopulateahost’sinventorylocationrecord,butthereareacoupleofotherusefulbitsofinformationintheabovepicture.

Firstofall,there’salow-leveldiscoveryruletoexplore.We’lldelvemoredeeplyintodiscoveryrulesinChapter4,DiscoveringYourNetwork,butfornow,we’lljustseethatit’saboutdynamicallycreatingnetworkinterfaceitems:

Foreveryinterface,eightitemswillbecreated,includingtheinterfacename,operationalstatus,incomingandoutgoingtraffic,andsoon.Thismeansthatthesametemplatewillbeusefulforthebasicmonitoringofnetworkapplianceswithanynumberofnetworkinterfaces.

Thesecondthingtonotice,lookingatbothimages,istheupdateinterval,andhistoryandtrendretentionperiodsfortheitems.Zabbixtriestosetsomesensibledefaults,butyou’llprobablyneedtoupdatesomeofthosevaluesbasedonthenumberofmonitoredhostsyouhaveinyourenvironment,yourstoragespaceavailability,andthenetworkloadofyourmonitoringtraffic.

NoteAnotherparameterthatisrelatedtoZabbix’sperformanceistheinitial(andminimum)numberofpollersthattheserverkeepsactiveatanygiventime.Ifyoufindthatyourpollingqueueisgettinglonger,youmightwanttoincreasethenumberofpollersinzabbix_server.conf.Theavailabledefaultoptionsare:

#StartPollers=5#StartIPMIPollers=0#StartPollersUnreachable=1#StartTrappers=5#StartPingers=1#StartDiscoverers=1#StartHTTPPollers=1

Workyourwayupslowly,oryou’lljustendupwithunnecessaryprocessesbeingcreatedwhenZabbixisstarted.

Ifyouhavehundredsofhoststomonitor,andforeveryhost,youcollecttensofsinglemeasurementseveryminute,youwouldreachapointwhereyourZabbixserver’snetworkloadorCPUloadwillstarttoimpactontheserver’sperformance,leadingtodelaysinitempollingordroppedconnections.Ifyoucannotjustupgradetomorepowerfulhardware,youmighthavetotweakthepollingintervalofyourtemplatessothattheystrikeagoodbalancebetweengranularityofdetailandperformance.

Adevice’sname,contactdetails,description,location,andsuchlike,willrarelychangeoncethedevicehasbeendeployed,soitwouldbeawastetopollforthosevalueseveryhour(3,600seconds).Bychangingtheintervalto6hoursorevenaday,you’llautomaticallyreduceyournetworktrafficrelatedtoessentiallyfixedinformationbyafactorof6,upto24.

Raisingthepollingintervalforsomeoftheinterfacecounterscanhaveanevenmoredramaticimpactonyoursystemandnetworkload.Whileyou’llprobablywanttochecktheadminandoperationalstatusofaninterfaceasoftenaspossible—otherwiseyouruntheriskofnotgettingnotifiedaboutpossibleproblemsinatimelymanner—ontheotherhand,you’llprobablybeabletolivewithpollingincomingandoutgoingtrafficanderrorseveryfiveminutes(300seconds)insteadofeveryminute.Yourgraphswillstillbeverydetailed,butyournetworkwillbemuchlessfloodedwithSNMPrequests.Keepinmindthatchangeslikethesemightnotseemmuchwhenreferredtoasinglehost,butasthenumberofyourmonitoredobjectsgrow,youcanveryquicklyrunuptohundredsoreventhousandsofnewmonitoringvaluespersecondcomingintoyourZabbixserver.

Thesamecanbesaidwhenitcomestoretentionperiodsandstoragespace.Inthiscase,keepinmindthattrendsstoreaboutthreevaluesperhour(min,maxandaverage)overthetimerangespecified,whilehistorystoresallvaluescollectedinthespecifiedtimerange.Thismeansthatbasedonyourpollinginterval,it’susuallycheapertoextendatrendretentionvaluethanahistoryone.Thisis,ofcourse,validonlyfornumericalvaluesasstringonescan’treallyhavetrends,justhistory.

OnelastthingtonoticeintheaboveimagesisthatthemonitoringprotocolforallitemsissettoSNMPv2.JustlikeSNMPv1,SNMPv2doesn’tofferrealsecurityforthemonitoringdatathatcrossesthenetworkbetweenanapplianceandthemonitoringserver:alltrafficissentandreceivedintheclear,andtheSNMPcommunityisjustastring,easilyparsablefrominterceptedtraffic.Whileit’scertainlytruethatafewnetworkappliancesdon’tsupportSNMPv3becauseeithertheyaretoooldortheyaretoosimple,It’salsotruethat

thenewversionoftheprotocolhasbeenaroundforquiteawhilenowandanumberofappliancesdosupportit.ThemainadvantagesofSNMPv3areitsauthenticationandencryptioncapabilities.Thesecanhelpmakesurethatallmonitoringtrafficisnotbogusorcorrupted,andthatit’skeptconfidentialfrompryingeyes.Thisisparticularlyimportantifyouneedtomonitorsomehostsoveranetworklinkyouhavenorealcontrolover,suchasaWANconnectionthroughathird-partyprovider.ItwouldalwaysbenicetouseSNMPv3acrossyournetwork,butincaseslikethese,youarestronglyencouragedtodosoasthere’sarealpossibilitythatyourtrafficcanbeindeedinterceptedandtappedinto.

Let’staketheexampleofaCiscorouter,andlet’sseehowtoconfigureSNMPv3onitbeforemovingontotheZabbixside.

Firstofall,let’screateamonitoringgroup.Thisisusedtodefineaccesstothedevice’sMIBs.OntheCiscorouter,openaconsolesessionandgointoconfigurationmode.Thenissuethefollowingcommand:

R1(config)#snmp-servergroupMonitoringGroupv3priv

Thev3keywordspecifiesthatwewanttouseSNMPv3,whiletheprivkeywordspecifiesthatwewanttousebothauthenticationandencryption.It’spossibletopassmoreoptionstotheprecedingcommandinordertodefineanaccesslistifyouwanttolimitaccesstospecificMiBs,butwe’llkeepthingssimplehereandletourZabbixprobeaccessallMIBs.

Nowthatwehaveagroup,wecancreateauser,asfollows:

R1(config)#snmp-serveruserzabbixMonitoringGroupv3authshazbxpassprivaes128zbxpriv

Asyoucansee,weassignedtheZabbixusertothepreviouslycreatedgroupanddefinedtheauthenticationandencryptionpassphrases.Takenoteofalltheseelementsasyou’llneedtospecifyallofthemonZabbix’ssideandtheywillneedtomatchwhatyouusedhere.Tosummarize,hereiswhatyou’llinputlaterwhenconfiguringanSNMPv3Zabbixitem:

Field Value

User zabbix

Authenticationprotocol sha

Authenticationpassphrase zbxpass

Privacyprotocol aes

Privacypassphrase zbxpriv

NotePleasedon’tusethepassphrasesshownhere.Theseareintentionallyweak,andweusedthemforillustrationpurposesonly.

Thisisallthereistoit.Later,we’lladdsomeinformationabouttellingtheappliance

wheretosendSNMPtraps,butfornowyou’rereadytogetSNMPvaluesfromyourappliance,solet’sfocusonthatforawhile.

FindingtherightOIDstomonitorWhileZabbix’sdefaultSNMPtemplateswillhelpyougetstartedwithbasicmonitoring,you’llsoonfindtheneedtopollyourdevicesformoreinformation.Todothat,you’llneedtoknowtheOIDofthemetricyouwanttomonitoraswellasthedatatypeitwillyield.Afirstoptionistoconsultyourvendor’sdocumentationonthedeviceandfindoutwhichMIBsandOIDsareexposedbytheSNMPagent.Another,moreinteractive,optionistofindthemusingthesnmpwalkutilityanddirectlyaskingyourdeviceforthem.

NoteIfyoudon’talreadyhavesnmpwalk(andtheotherSNMPutilitiesforLinux)installed,youcanquicklydosowithasimplecommand:

#yuminstallnet-snmp-utils

OIDsaresentandreceivedbySNMPagentsandserversasdottedsequencesofnumbers.JustlikeIPaddresses,thisisconvenientformachine-to-machinecommunication,buthardtoreadforhumans.Inordertomakethemostfromtheexplorationofyourdeviceusingsnmpwalk,makesureyouhavealltheMIBsyouneedinstalled.MIBsessentiallymapOIDstoreadableandunderstandabledescriptionsofthemselves.Inotherwords,theytakeoutputlikethisone:

.1.3.6.1.2.1.2.2.1.1.1=INTEGER:1

.1.3.6.1.2.1.2.2.1.1.2=INTEGER:2

.1.3.6.1.2.1.2.2.1.1.3=INTEGER:3

.1.3.6.1.2.1.2.2.1.1.5=INTEGER:5

.1.3.6.1.2.1.2.2.1.2.1=STRING:lo

.1.3.6.1.2.1.2.2.1.2.2=STRING:eth1

.1.3.6.1.2.1.2.2.1.2.3=STRING:tap0

.1.3.6.1.2.1.2.2.1.2.5=STRING:br0

.1.3.6.1.2.1.2.2.1.3.1=INTEGER:softwareLoopback(24)

.1.3.6.1.2.1.2.2.1.3.2=INTEGER:ethernetCsmacd(6)



.1.3.6.1.2.1.2.2.1.4.1=INTEGER:16436

.1.3.6.1.2.1.2.2.1.4.2=INTEGER:1500

.1.3.6.1.2.1.2.2.1.4.3=INTEGER:1500

.1.3.6.1.2.1.2.2.1.4.5=INTEGER:1500

.1.3.6.1.2.1.2.2.1.5.1=Gauge32:10000000

.1.3.6.1.2.1.2.2.1.5.2=Gauge32:1000000000

.1.3.6.1.2.1.2.2.1.5.3=Gauge32:10000000

.1.3.6.1.2.1.2.2.1.5.5=Gauge32:0

.1.3.6.1.2.1.2.2.1.6.1=STRING:

.1.3.6.1.2.1.2.2.1.6.2=STRING:0:c:29:24:15:50

.1.3.6.1.2.1.2.2.1.6.3=STRING:2:10:f7:72:77:50

.1.3.6.1.2.1.2.2.1.6.5=STRING:0:c:29:24:15:50

.1.3.6.1.2.1.2.2.1.7.1=INTEGER:up(1)

.1.3.6.1.2.1.2.2.1.7.2=INTEGER:up(1)

.1.3.6.1.2.1.2.2.1.7.3=INTEGER:up(1)

.1.3.6.1.2.1.2.2.1.7.5=INTEGER:up(1)

.1.3.6.1.2.1.2.2.1.8.1=INTEGER:up(1)

.1.3.6.1.2.1.2.2.1.8.2=INTEGER:up(1)

.1.3.6.1.2.1.2.2.1.8.3=INTEGER:up(1)

.1.3.6.1.2.1.2.2.1.8.5=INTEGER:up(1)

Then,theyturnitintoamuchmorereadableform:

IF-MIB::ifIndex.1=INTEGER:1IF-MIB::ifIndex.2=INTEGER:2IF-MIB::ifIndex.3=INTEGER:3IF-MIB::ifIndex.5=INTEGER:5IF-MIB::ifDescr.1=STRING:loIF-MIB::ifDescr.2=STRING:eth1IF-MIB::ifDescr.3=STRING:tap0IF-MIB::ifDescr.5=STRING:br0IF-MIB::ifType.1=INTEGER:softwareLoopback(24)IF-MIB::ifType.2=INTEGER:ethernetCsmacd(6)IF-MIB::ifType.3=INTEGER:ethernetCsmacd(6)IF-MIB::ifType.5=INTEGER:ethernetCsmacd(6)IF-MIB::ifMtu.1=INTEGER:16436IF-MIB::ifMtu.2=INTEGER:1500IF-MIB::ifMtu.3=INTEGER:1500IF-MIB::ifMtu.5=INTEGER:1500IF-MIB::ifSpeed.1=Gauge32:10000000IF-MIB::ifSpeed.2=Gauge32:1000000000IF-MIB::ifSpeed.3=Gauge32:10000000IF-MIB::ifSpeed.5=Gauge32:0IF-MIB::ifPhysAddress.1=STRING:IF-MIB::ifPhysAddress.2=STRING:0:c:29:24:15:50IF-MIB::ifPhysAddress.3=STRING:2:10:f7:72:77:50IF-MIB::ifPhysAddress.5=STRING:0:c:29:24:15:50IF-MIB::ifAdminStatus.1=INTEGER:up(1)IF-MIB::ifAdminStatus.2=INTEGER:up(1)IF-MIB::ifAdminStatus.3=INTEGER:up(1)IF-MIB::ifAdminStatus.5=INTEGER:up(1)IF-MIB::ifOperStatus.1=INTEGER:up(1)IF-MIB::ifOperStatus.2=INTEGER:up(1)IF-MIB::ifOperStatus.3=INTEGER:up(1)IF-MIB::ifOperStatus.5=INTEGER:up(1)

IfyouhavetherightMIBs,youwon’thavetoguessthemeaningofeachOIDfromitsvalueasmostofthetime,itwillbeclearenoughfromitsname.ToaddanewMIBtoyourSNMPtools,youhavetoobtainitfromthevendorofyourdeviceandtheninstallitonyoursystem.VendorsusuallymaketheirMIBsfreelyavailable,soyoushouldn’thaveanyproblemsfindingthem.

HerearesomeofthemajorvendorsofMIBsources,compiledatthetimeofwriting:

Vendor MIBs

Cisco http://www.cisco.com/public/sw-center/netmgmt/cmtk/mibs.shtml

Juniper http://www.juniper.net/techpubs/software/index_mibs.html

Barracudanetworks https://techlib.barracuda.com/search/go/global?q=MIB

http://www.cisco.com/public/sw-center/netmgmt/cmtk/mibs.shtmlhttp://www.juniper.net/techpubs/software/index_mibs.htmlhttps://techlib.barracuda.com/search/go/global?q=MIB

NoteAveryusefulresourceisOIDView’sfreeMIBdatabasethatyoucanfindhere:

http://www.oidview.com/mibs/detail.html

Atthetimeofwritingthis,thedatabasehadmorethan7,000MIBs,sochancesareyou’llbeabletofindaMIBforthemostobscurenetworkdeviceyoumighthavetomonitor.

MIBsareplaintextfiles,soifyouhaveacompressedarchive,youwillneedtounpackitbeforeyoucaninstallitscontents.OnceyouhavetheplaintextMIBS,it’sasimplematterofcopyingtheminto/usr/share/snmp/mibsandthenusingthe-moptiontotheSNMPcommandstospecifywhichMIByouwanttoloadinadditiontothedefaultones.

ShouldyourMIBscollectionbecometoobigandyouwantedtoorganizethemindifferentdirectories,thenyou’llneedtotellyourtoolswheretofindthem.Youhavetwooptions:eitherspecifyfromthecommandlinethedirectoriesyouwantyourcommandtosearchforMIBs,orputthisinformationinaconfigurationfilesothatyourcommandsalwaysknowtheMIBs’location.Theoptionsarediscussedasfollows:

Thefirstoptionisusefulifyou’rejusttryingoutanewMIBandseeingwhetherthat’stheoneyouneed.EveryNet-SNMP-basedcommandwilltakea-moptionthatyoucanusetospecifyaspecificMIBtoloadfromthemibsdirectory.Here’sacommandforexample:

$snmpwalk-m+CISCO-STUN-MIB-v3-uzabbix-aSHA-Azbxpassword-lAuthPriv-xAES-Xprivpassword10.10.1.9

ThiscommandwilluseSNMPv3tocontacttheSNMPagentat10.10.1.9withthespecifiedcredentialsandwillloadtheCISCO-STUN-MIBthatitwillfindinthe/usr/share/snmp/mibsdirectory,inadditiontothosealreadyloadedasdefault.

Thesecondoptionismorepermanentandinvolvesediting(orcreating,ifit’snotalreadythere)the/etc/snmp/snmp.conffile.JustaddalinewiththelistofdirectoriestosearchformibsandanotherlinethatspecifieswhichMIBsthecommandsshouldactuallyload(inthiscase,we’llloadallofthem),asfollows:

mibdirs/usr/share/snmp/mibs:/usr/share/snmp/mibs/cisco:/usr/share/snmp/mibs/juniper:/mnt/remote/shared_mibs/mibs+ALL

Asyoucansee,evenifyoukeepyoursubdirectoriesin/usr/share/snmp/mibs,you’llhavetospecifyeachoneyouwantautomaticallyincluded.OnceyouhaveyourMIBsinstalledandloaded,you’llbereadytofullyexploreyourdevices’SNMPagents.ToperformacompletesnmpwalkonadevicecantakequitealotoftimeandproducealotofoutputdependingonhowmanyOIDsitexposes.Aroutercanhavethousandsofthem,soit’sadvisabletoredirectthecommand’soutputtoafilesothatyouareabletoreferenceitandexploreitatanytimeyouwantwithouthavingtoperformacompletewalkonthedeviceitself,asfollows:

$snmpwalk-v3-uzabbix-aSHA-Azbxpassword-lAuthPriv-xAES-X

http://www.oidview.com/mibs/detail.html

privpassword10.10.1.9>router-R1-snmp_baseline.txt

AnotheradvantageofhavingtheMIBsyouneedisthatit’llbeeasiertocreatenewSNMPitemsinZabbixasyou’llbeabletospecifythestringversionofanOIDandnotonlyitsnumericalvalue.ZabbixreliesontheNet-SNMPlibrary,soitwillalsoreferenceanyMIBsinstalledinyoursystem’sdefaultdirectories.

Solet’sseehowyoucanusetheoutputofsnmpwalktocreatenewZabbixitems.

MappingSNMPOIDstoZabbixitemsAnSNMPvalueiscomposedofthreedifferentparts:theOID,thedatatype,andthevalueitself.WhenyouusesnmpwalkorsnmpgettogetvaluesfromanSNMPagent,theoutputlookslikethis:

SNMPv2-MIB::sysObjectID.0=OID:CISCO-PRODUCTS-MIB::cisco3640DISMAN-EVENT-MIB::sysUpTimeInstance=Timeticks:(83414)0:13:54.14SNMPv2-MIB::sysContact.0=STRING:SNMPv2-MIB::sysName.0=STRING:R1SNMPv2-MIB::sysLocation.0=STRING:Upperfloorroom13SNMPv2-MIB::sysServices.0=INTEGER:78SNMPv2-MIB::sysORLastChange.0=Timeticks:(0)0:00:00.00...IF-MIB::ifPhysAddress.24=STRING:c4:1:22:4:f2:fIF-MIB::ifPhysAddress.26=STRING:IF-MIB::ifPhysAddress.27=STRING:c4:1:1e:c8:0:0IF-MIB::ifAdminStatus.1=INTEGER:up(1)IF-MIB::ifAdminStatus.2=INTEGER:down(2)…

Andsoon.

Thefirstpart,theonebeforethe=signis,naturally,theOID.ThiswillgointotheSNMPOIDfieldintheZabbixitemcreationpageandistheuniqueidentifierforthemetricyouareinterestedin.SomeOIDsrepresentasingleanduniquemetricforthedevice,sotheyareeasytoidentifyandaddress.Intheaboveexcerpt,onesuchOIDisDISMAN-EVENT-MIB::sysUpTimeInstance.IfyouareinterestedinmonitoringthatOID,you’donlyhavetofillouttheitemcreationformwiththeOIDitselfandthendefineanitemname,adatatype,andaretentionpolicy,andyouarereadytostartmonitoringit.Inthecaseofanuptimevalue,time-ticksareexpressedinseconds,soyou’llchooseanumericdecimaldatatype.We’llseeinthenextsectionhowtochooseZabbixitemdatatypesandhowtostorevaluesbasedonSNMPdatatypes.You’llalsowanttostorethevalueasisandoptionallyspecifyaunitofmeasure.Thisisbecauseanuptimeisalreadyarelativevalueasitexpressesthetimeelapsedsinceadevice’slatestboot.Therewouldbenopointincalculatingafurtherdeltawhengettingthismeasurement.Finally,you’lldefineapollingintervalandchoosearetentionpolicy.Inthefollowingexample,thepollingintervalisshowntobe5minutes(300seconds),thehistoryretentionpolicyas3days,andthetrendstorageperiodasoneyear.Theseshouldbesensiblevaluesasyoudon’tnormallyneedtostorethedetailedhistoryofavaluethateitherresetstozero,or,bydefinition,growslinearlybyonetickeverysecond.

Thefollowingscreenshotencapsulateswhathasbeendiscussedinthisparagraph:

Rememberthattheitem’skeyvaluestillhastobeuniqueatthehost/templatelevelasitwillbereferencedtobyallotherZabbixcomponents,fromcalculateditemstotriggers,maps,screens,andsoon.Don’tforgettoputtherightcredentialsforSNMPv3ifyouareusingthisversionoftheprotocol.

ManyofthemoreinterestingOIDs,though,areabitmorecomplex:multipleOIDscanberelatedtooneanotherbymeansofthesameindex.Let’slookatanothersnmpwalkoutputexcerpt:

IF-MIB::ifNumber.0=INTEGER:26IF-MIB::ifIndex.1=INTEGER:1IF-MIB::ifIndex.2=INTEGER:2IF-MIB::ifIndex.3=INTEGER:3…IF-MIB::ifDescr.1=STRING:FastEthernet0/0

IF-MIB::ifDescr.2=STRING:Serial0/0IF-MIB::ifDescr.3=STRING:FastEthernet0/1…IF-MIB::ifType.1=INTEGER:ethernetCsmacd(6)IF-MIB::ifType.2=INTEGER:propPointToPointSerial(22)IF-MIB::ifType.3=INTEGER:ethernetCsmacd(6)…IF-MIB::ifMtu.1=INTEGER:1500IF-MIB::ifMtu.2=INTEGER:1500IF-MIB::ifMtu.3=INTEGER:1500…IF-MIB::ifSpeed.1=Gauge32:10000000IF-MIB::ifSpeed.2=Gauge32:1544000IF-MIB::ifSpeed.3=Gauge32:10000000…IF-MIB::ifPhysAddress.1=STRING:c4:1:1e:c8:0:0IF-MIB::ifPhysAddress.2=STRING:IF-MIB::ifPhysAddress.3=STRING:c4:1:1e:c8:0:1…IF-MIB::ifAdminStatus.1=INTEGER:up(1)IF-MIB::ifAdminStatus.2=INTEGER:down(2)IF-MIB::ifAdminStatus.3=INTEGER:down(2)…IF-MIB::ifOperStatus.1=INTEGER:up(1)IF-MIB::ifOperStatus.2=INTEGER:down(2)IF-MIB::ifOperStatus.3=INTEGER:down(2)…IF-MIB::ifLastChange.1=Timeticks:(1738)0:00:17.38IF-MIB::ifLastChange.2=Timeticks:(1696)0:00:16.96IF-MIB::ifLastChange.3=Timeticks:(1559)0:00:15.59…IF-MIB::ifInOctets.1=Counter32:305255IF-MIB::ifInOctets.2=Counter32:0IF-MIB::ifInOctets.3=Counter32:0…IF-MIB::ifInDiscards.1=Counter32:0IF-MIB::ifInDiscards.2=Counter32:0IF-MIB::ifInDiscards.3=Counter32:0…IF-MIB::ifInErrors.1=Counter32:0IF-MIB::ifInErrors.2=Counter32:0IF-MIB::ifInErrors.3=Counter32:0…IF-MIB::ifOutOctets.1=Counter32:347968IF-MIB::ifOutOctets.2=Counter32:0IF-MIB::ifOutOctets.3=Counter32:0

Asyoucansee,foreverynetworkinterface,thereareseveralOIDs,eachonedetailingaspecificaspectoftheinterface:itsname,itstype,whetherit’supordown,theamountoftrafficcominginorgoingout,andsoon.ThedifferentOIDsarerelatedthroughtheirlastnumber,theactualindexoftheOID.Lookingattheprecedingexcerpt,weknowthatthedevicehas26interfaces,ofwhichweareshowingsomevaluesforjustthefirstthree.Bycorrelatingtheindexnumbers,wealsoknowthatinterface1iscalledFastEthernet0/0,itsMACaddressisc4:1:1e:c8:0:0,theinterfaceisupandhasbeenupforjust17

seconds,andsometrafficalreadywentthroughit.

Now,onewaytomonitorseveralofthesemetricsforthesameinterfaceistomanuallycorrelatethesevalueswhencreatingtheitems,puttingthecompleteOIDintheSNMPOIDfield,andmakingsurethatboththeitemkeyanditsnamereflecttherightinterface.Thisprocessisnotonlypronetoerrorsduringthesetupphase,butitcouldalsointroducesomeinconsistenciesdowntheroad.Thereisnoguarantee,infact,thattheindexwillremainconsistentacrosshardwareorsoftwareupgradesorevenacrossconfigurationswhenitcomestomorevolatilestateslikethenumberofVLANsorroutingtablesinsteadofnetworkinterfaces.FortunatelyZabbixprovidesafeature,calleddynamicindexes,thatallowsyoutoactuallycorrelatedifferentOIDsinthesameSNMPOIDfieldsothatyoucandefineanindexbasedontheindexexposedbyanotherOID.

ThismeansthatifyouwanttoknowtheadminstatusofFastEthernet0/0,youdon’tneedtofindtheindexassociatedwithFastEthernet0/0(inthiscaseitwouldbe1)andthenaddthatindextoIF-MIB::ifAdminStatusofthebaseOID,hopingthatitwon’teverchangeinthefuture.Youcaninsteadusethefollowingcode:

IF-MIB::ifAdminStatus["index","IF-MIB::ifDescr","FastEthernet0/0"]

UponusingtheprecedingcodeintheSNMPOIDfieldofyouritem,theitemwilldynamicallyfindtheindexoftheIF-MIB::ifDescrOIDwherethevalueisFastEthernet0/0andappendittoIF-MIB::ifAdminStatusinordertogettherightstatusfortherightinterface.

Ifyouorganizeyouritemsthisway,you’llalwaysbesurethatrelateditemsactuallyshowtherightrelatedvaluesforthecomponentyouareinterestedinandnotthoseofanotheronebecausethingschangedonthedevice’ssidewithoutyourknowledge.Moreover,we’llbuildonthistechniquetodeveloplow-leveldiscoveryofadeviceaswe’llseeinChapter4,DiscoveringYourNetwork.

Youcanusethesametechniquetogetotherinterestinginformationoutofadevice.Consider,forexample,thefollowingexcerpt:

ENTITY-MIB::entPhysicalVendorType.1=OID:CISCO-ENTITY-VENDORTYPE-OID-MIB::cevChassis3640ENTITY-MIB::entPhysicalVendorType.2=OID:CISCO-ENTITY-VENDORTYPE-OID-MIB::cevContainerSlotENTITY-MIB::entPhysicalVendorType.3=OID:CISCO-ENTITY-VENDORTYPE-OID-MIB::cevCpu37452fe

ENTITY-MIB::entPhysicalClass.1=INTEGER:chassis(3)ENTITY-MIB::entPhysicalClass.2=INTEGER:container(5)ENTITY-MIB::entPhysicalClass.3=INTEGER:module(9)

ENTITY-MIB::entPhysicalName.1=STRING:3745chassisENTITY-MIB::entPhysicalName.2=STRING:3640ChassisSlot0ENTITY-MIB::entPhysicalName.3=STRING:c3745MotherboardwithFastEthernetonSlot0

ENTITY-MIB::entPhysicalHardwareRev.1=STRING:2.0ENTITY-MIB::entPhysicalHardwareRev.2=STRING:

ENTITY-MIB::entPhysicalHardwareRev.3=STRING:2.0

ENTITY-MIB::entPhysicalSerialNum.1=STRING:FTX0945W0MYENTITY-MIB::entPhysicalSerialNum.2=STRING:ENTITY-MIB::entPhysicalSerialNum.3=STRING:XXXXXXXXXXX

Itshouldbeimmediatelycleartoyouthatyoucanfindthechassis’sserialnumberbycreatinganitemwith:

ENTITY-MIB::entPhysicalSerialNum["index","ENTITY-MIB::entPhysicalName","3745chassis"]

Thenyoucanspecify,inthesameitem,thatitsho

Zabbix Network Monitoring Essentialsinfonetwork.ir/wp-content/uploads/2019/07/Zabbix_Network... ·...

Documents

Transcript of Zabbix Network Monitoring Essentialsinfonetwork.ir/wp-content/uploads/2019/07/Zabbix_Network... ·...