Making a Scalable Automated Hacking System by Artem Dinaburg

MakingaScalableAutomatedHackingSystem

ShakaConVIIIArtemDinaburg

artem@trailo=its.com

•  PrincipalSecurityEngineeratTrailofBits– artem@trailo=its.com/[email protected]– hFp://blog.trailo=its.com

AboutMe

AboutThisTalk•  Automatedvulnerabilitydiscoveryisaverybroadtopic.


•  Thistalkisgoingtobetechnical,butaccessible.


•  Thistalkisgoingtobetechnical,butaccessible.– WewilltalkaboutbuildingapracLcalsystem

Outline•  Part1:Learnhowourautomatedbugfindingsystemworks&howtomakeyourown


•  Part2:Understandourapproachtoautomatedbinarypatching


•  Part2:Understandourapproachtoautomatedbinarypatching

•  What’sNext:thefutureofautomatedvulnerabilitydiscovery

IntroducLon•  TrailofBitswasacompeLtorinDARPA’sCyberGrandChallenge


•  WebuiltCyberdyne,anautomatedbugfindingandpatchingsystem



•  ItworkedpreFywell…

2ndinBugsFound

77

65

57

57

44

39

23

12

12

9

0 10 20 30 40 50 60 70 80

TeamACyberdyne

TeamBTeamCTeamDTeamETeamFTeamGTeamHTeamI

ConfirmedBugsFound

5thinPatchingEffecLveness



•  ItworkedpreFywell…•  Butdidn’tqualify 😢

LastinPerformance

AutomatedBugFinding

©flickruserHenriSivonen

VulnerabilityDiscoveryTheory•  Notoolwillfindallthebugs.•  Provablyimpossible.

VulnerabilityDiscoveryTheory•  OverApproximateAnalyses– PointsTo– AliasAnalysis

VulnerabilityDiscoveryTheory•  UnderApproximateAnalyses– Fuzzing,SymbolicExecuLon

Under-ApproximateAnalyses:Roadblocks

HardForFuzzing,EasyforSymbolicExecuLon

if(input[0]==0xBADFOOD)

Under-ApproximateAnalyses:Roadblocks

HardforSymbolicExecuLon,EasyforFuzzing

if(hash(input[0])==hash(input[1]))

Under-ApproximateAnalyses:Theory

•  Alltoolsoperateoverthesamedomain

•  Alldiscoveriesareequallytrue

•  Whatiftoolscouldsharediscoveries?

©flickruserJean-PierreDalbéra

AnalysisBoosLng•  Sharingdiscoveriesacrosstoolscreatesavirtuouscyclethatremovesroadblocks

if(input[0]==0xBADFOOD)if(hash(input[0])==hash(input[1]))BUG();

AnalysisBoosLng

• Howdoyoucombineexis1nganalysistools?

AnalysisBoosLng

• Howdoyoucombineexis1nganalysistools?•  ‘Universal’Knowledge:Inputs!– Kindoflame,butitworks!

AnalysisBoosLng•  Inputsgeneratedbyonetoolfeedintoallothers

Inputs


Inputs

Analyses


Inputs

Analyses New Inputs


Inputs

Analyses New Inputs

Merge Knowledge

AnalysisBoosLng:ImplementaLon•  TwosymbolicexecuLonengines– KLEE(viaanx86toLLVMtranslator)– PySymEmu


•  Grr:areallyawesomeDBT-basedfuzzer


•  Grr:areallyawesomeDBT-basedfuzzer•  MergeknowledgeviaMinSet– Minimalsetofmaximalcoverageinputs

PracLcalConsideraLons:MinSet•  MinimizingtheinputsetisessenLalforanalysisboosLngtobepracLcal.


•  MinSetneedsafastmethodtomeasureandcomparecoverage.


•  MinSetneedsafastmethodtomeasureandcomparecoverage.

•  Doesn’thavetobeperfect.

AutomatedBugFinding:BuildingIt

©flickruserJuliaManzerova

BuildingYourOwn

A NB

•  Step1:Findthetoolstocombine

BuildingYourOwn•  Step2:Wrapyourtoolswithamanagerscript

A NB

BuildingYourOwn•  Step3•  WriteTests

• WriteTests

• WriteTests

BuildingYourOwn•  Step4:SharedResources

A NB

MessageQueueDistributed Storage

BuildingYourOwn•  Step5:MinSet

A NB


BuildingYourOwn•  Step6•  WriteTests

• WriteTests

• WriteTests

BuildingYourOwn•  Step7:MonitoringandStaLsLcs

A NB


BuildingYourOwn•  Step8:ControlandProvision









Control & Provision

ObligatoryAFLComparison

78

69

68

65

61

16

14

0 10 20 30 40 50 60 70 80

Boosted(current)

Boosted(CQE)

AFL(DrillerPaper)

Union(KLEE,PSE,Grr)

Grr(Fuzzing)

PSE(Symbolic)

KLEE(Symbolic)

BugsFoundinCQEBinariesByMethod

(approximate)

Patching

©USArmy

Patching•  Makethebinarynotcrash

Patching•  Makethebinarynotcrash•  PatchingMethod– Whatdoyou“fix”?


•  PatchingMechanics– Howdoyouapplythe“fix”?


•  PatchingMechanics– Howdoyouapplythe“fix”?

•  PatchLocaLon– Wheredoyoufix,ifyouaren’tsureofthebug?

Patching•  PatchingMethod– Preventaccesstoinvalidmemory


•  PatchingMechanics– TranslatebinarytoLLVM,patchthebitcode

PatchingviaLLVM

Bin LLVMPatchBin

PatchLLVM


•  PatchingMechanics– TranslatebinarytoLLVM,patchthebitcode

•  PatchLocaLon– Specificvs.Generic

PatchLocaLon:SpecificPatching•  So,wehavethisthingthatfindsbugs…– Letsjustpatchthose

•  Insertcheckformemoryvalidity•  Problems– Wrongbug– MulLplecodepaths

PatchLocaLon:GenericPatching•  Fixeverybugofacertainclass•  Examples:– StackCookies– ASLR– CFI

•  Problem:slow

PatchingProblems•  Slow.Bloated.

Patching:EntertheDragon

PatchingMontage•  Cross-BlockDeadStoreEliminaLon

PatchingMontage•  Cross-BlockDeadStoreEliminaLon•  Unusedregisteranalysis

PatchingMontage•  Cross-BlockDeadStoreEliminaLon•  Unusedregisteranalysis•  DominatorTreeTraversal

PatchingMontage•  Cross-BlockDeadStoreEliminaLon•  Unusedregisteranalysis•  DominatorTreeTraversal•  DataFlowfrominputs

PatchingMontage•  Cross-BlockDeadStoreEliminaLon•  Unusedregisteranalysis•  DominatorTreeTraversal•  DataFlowfrominputs•  CombinaLonsoftheabove

Reminder:LastinPerformance

What’sNext?•  CyberGrandChallengeconLnues!•  FinalsareAugust4th,co-locatedwithDEFCON– Freetoenter

•  WinnerwillplaythewinninghumanCTFteamatDEFCON

What’sNext?•  Integratemoretoolsandknowledge!– AFL(viaNCC’sprojecttriforce)– Grammar-basedfuzzers–  InformaLonfromcompiler/sourcecode

•  Broaderlibrarysupport

What’sNext?•  Letsautomate(orsemi-automate)boringassessmentwork.

•  WearedoingapilotauditofzlibfortheMozillaFoundaLon.

•  EventuallywewouldliketoautomaLcallyauditopensourcesosware.

QuesLons?ContactInformaLon:

artem@trailo=its.comhFp://blog.trailo=its.com

FurtherReading:hFp://blog.trailo=its.com/2015/07/15/how-we-fared-in-the-cyber-grand-challenge/hFp://blog.trailo=its.com/2016/01/13/hacking-for-charity-automated-bug-finding-in-libotr/hFps://github.com/trailo=its/mcsemahFps://www.usenix.org/conference/usenixsecurity15/technical-sessions/presentaLon/walker

Making a Scalable Automated Hacking System by Artem Dinaburg

Technology

Transcript of Making a Scalable Automated Hacking System by Artem Dinaburg