Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only...
Transcript of Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only...
![Page 1: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/1.jpg)
ManySolutions,OneGoal.
SplunkandWindowsEventLog:BestPractices,Reductionand
EnhancementDavidShpritzAplura,LLC
BaltimoreAreaSplunkUserGroupJune,2017
![Page 2: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/2.jpg)
ManySolutions,OneGoal.
Agenda
• GettingWindowsEventsintoSplunk:PatternsandPractices• TURNDOWNTHEVOLUME:Licensereductiontips• Makingthemmoreuseful:Improvingknowledgeobjects
![Page 3: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/3.jpg)
ManySolutions,OneGoal.
GroundRules
• Fidelitylevels• Howcompletearetheevents?
• WindowsEventinterpretation• Thesearebinaryrecords• AgentscanreadthemdirectlyorasktheWindowsAPI• Thismeansthatyouaren’treallygettingtheeventlog,justarepresentationofit
![Page 4: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/4.jpg)
ManySolutions,OneGoal.
GettingWindowsEventsintoSplunk
![Page 5: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/5.jpg)
ManySolutions,OneGoal.
DifferentWaystoSkinaCat
• BesttoWorst• UniversalForwarder• WindowsEventForwarding• WMI• EVTXImport• ThirdPartySyslogAgent(Snare,forexample)
![Page 6: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/6.jpg)
ManySolutions,OneGoal.
UniversalForwarder
• ThebestwaytogetWindowsevents(ofcoursewe’rebiased)• Pros• Highfidelity• CanbecontrolledbyDeploymentServer• CanfilterWindowsevents• Canrunscripts(batch,exe,PS)• Canalsogetadmon (greatforassetsandidentities)
• Cons• “Anotheragent!?!?”• Securityconcerns
![Page 7: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/7.jpg)
ManySolutions,OneGoal.
WindowsEventForwarding
• NativetoWindows(2008R2andup)• Pros• NativetoWindows,noagent• CanbeconfiguredwithGPO
• Cons• Almosthighfedlity• Slower• CustomertestingshowsitconsumesmoreresourcesthanaUF
![Page 8: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/8.jpg)
ManySolutions,OneGoal.
WMI
• UsedbyaSplunksystemtocollectWindowsEventsfromaremotesystem• Pros• Remote,noagent
• Cons• Slow• Alotofoverhead• Limitedcollectionavailability(mayneedmultiplesystemstopullallofyourWindowshosts)• Lowfidelity• Dealingwithpermissions
![Page 9: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/9.jpg)
ManySolutions,OneGoal.
EVTXImport
• Canbeusedtoexporteventlogsfromasystemandthenimporttherawfilesonanothersystem• Oftenseenin”air-gapped”environments• Pros• Nonetworkconnectionneededfromtheclientsystemstothetargetindexers
• Cons• Lowfidelity(rememberthat“interpretation”thingearlier?)• Movingandremovingthefilesisamanualprocess• Opentoeventduplication
![Page 10: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/10.jpg)
ManySolutions,OneGoal.
ThirdPartySyslogAgent(Snare)
• It’sathing,theseagentsexist• Pros• Canworkwithyourexistingsysloginfrastructure
• Cons• Superlowfidelity• Unreliable(syslogneverdies)• Remoteconfiguration?
![Page 11: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/11.jpg)
ManySolutions,OneGoal.
TURNDOWNTHEVOLUME:Licensereductiontips
![Page 12: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/12.jpg)
ManySolutions,OneGoal.
• Splunkestimatesbetween200-300mbperday,persystem• Ofcourse,thatcanvarywildly• Lotsofrepeatedeventswithlittletonovalue(lookingatyou4662)• Dowereallyneedallofthese?• Doweneedeverypartofallofthese?
Thesethingsarechatty
![Page 13: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/13.jpg)
ManySolutions,OneGoal.
Stratergery
• Pickyoursystemscarefully• Pickyourinputscarefullyonthosesystems• WhitelistandBlacklistcarefully• Resolvingobjects• Baseline?• Current_only?Start_from?• XmlWinEventLog• Filteringandcleaningup
![Page 14: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/14.jpg)
ManySolutions,OneGoal.
Whichsystems?
• JustActiveDirectoryservers?• Endpoints?• Servers?• Sorry,thisisonacasebycasebasis
![Page 15: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/15.jpg)
ManySolutions,OneGoal.
Pickingyourinputs(notyournose)
• SetabaselineforwhichlogsALLofyoursystemsshouldbesending• Forothereventlogs,useanindividualappforturningonthatinput(DS-Input-wineventlog_application)• Doyouneedadmon fromallofyoursystems?Probablynot,justonafewADsystems• Makesureyouaren’tusinglegacyinputs(WMIvsPerfmon)• LookoutforWindowsFirewallEvents(maybeStreaminstead?)
![Page 16: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/16.jpg)
ManySolutions,OneGoal.
WhitelistingandBlacklisting
• Canhaveabigimpactonyourlicenseusage• Investingthetimein“whichevents”canpayoffbig• Carefulwithawhitelist-onlyapproach• Notethatthereisalimittothenumberoflists• Performedattheforwarder,sodoesnotusenetworktraffic
![Page 17: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/17.jpg)
ManySolutions,OneGoal.
Someniceblacklistoptionstostartwith
• https://gist.github.com/automine/a3915d5238e2967c8d44b0ebcfb66147
![Page 18: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/18.jpg)
ManySolutions,OneGoal.
ADObjectResolution
• ResolvesthingslikeSIDsandGIUDs• YoucantellSplunkwhichDCstousetoresolvethese• Canaddsomeoverhead(CPUandMemory),butusuallylowimpact• Recommendationistoresolvethem(lookattheevt_*)optionsininputs.conf forWindowsEventLogs
![Page 19: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/19.jpg)
ManySolutions,OneGoal.
BaseliningAD
• WillcollectyourwholeADschema• CantakeupalotofmemoryonADcontrollers• ButbaseliningisusefulforAssetsandIdentitiesinES• Sobecarefulwhichsystemsyoubaselineon
![Page 20: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/20.jpg)
ManySolutions,OneGoal.
Current_only vs.start_from
• Current_only tellsSplunktoonlygrabthelatestevents(liketail–f,ifWindowshadsuchathing)• Usefultomakesureyoudon’tgetallofthehistoricaldata• Maywanttosetthatto“true”oninitialdeployment• Thensetto“false”,restart,anditshouldpickupfromthecheckpoint• Start_from shouldbe“oldest”• Settingitto“newest”canbeusedtogrababacklogofevents• I’veneverseenthisinthewild
![Page 21: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/21.jpg)
ManySolutions,OneGoal.
XmlWinEventLog
• Shouldreducelicenseusage(claimsareupto70%)• ItwillalwaysbeinEnglish(pro?Con?)• Hardertoread,Imean,it’sXML• QualityofCIMcompliancehasbeenvariedinthepast• Itdoesn’t”looklikeWindowsevents”andsomeauditorsarenotbright• Whatifyoucouldgetthesamelogsavingsandthereadability
![Page 22: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/22.jpg)
ManySolutions,OneGoal.
Filteringandcleaningup
• Don’tuse“suppress_text”• It’stempting,buttheregoesthebabywiththebathwater• Maybejustcleanupthetextyoudon’tneed
![Page 23: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/23.jpg)
ManySolutions,OneGoal.
Filteringandcleaningup
• IPv6supportineventlogsresultsinalotof“::”and“ffff”andothergarbage• Let’scleanupalot(thankstoalotofpeopleforthis)
• https://gist.github.com/automine/5c8ef5b50e1df38249dfba01a70f2875
![Page 24: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/24.jpg)
ManySolutions,OneGoal.
MakingThemMoreUseful
![Page 25: Splunk and Windows Event Log: Best Practices, Reduction ... · •Current_onlytells Splunk to only grab the latest events (like tail –f, if Windows had such a thing) •Useful to](https://reader034.fdocuments.in/reader034/viewer/2022042123/5e9f241b4c3c5a38b15af647/html5/thumbnails/25.jpg)
ManySolutions,OneGoal.
Sorry,Iranoutoftime
• GotES?TakealookatRyanFaircloth’sSecKit work• https://splunkbase.splunk.com/app/3059/• https://bitbucket.org/SPLServices/seckit_sa_idm_windows
• AlternativeTAs• ShouldhelpwithKOoverhead• https://github.com/my2ndhead/TA-microsoft-windows (candoXMLevents)• https://bitbucket.org/SPLServices/seckit_ta_microsoft_windows (forusewithSecKit)