Cyber Grand Challenge and CodeJitsufree.eol.cn/edu_net/edudown/spkt/zhangchao.pdf · § PPP: embed...
48
Title Cyber Grand Challenge and CodeJitsu Chao Zhang References: h9ps://cgc.darpa.mil/ h9ps://www.cybergrandchallenge.com/
Transcript of Cyber Grand Challenge and CodeJitsufree.eol.cn/edu_net/edudown/spkt/zhangchao.pdf · § PPP: embed...
Title
CyberGrandChallengeandCodeJitsu
ChaoZhang
References:h9ps://cgc.darpa.mil/h9ps://www.cybergrandchallenge.com/
Title
AboutMe§ Experience• PekingUniversity,Ph.D.(2008.9~2013.7)• UCBerkeley,Postdoc(2013.9~2016.9)• TsinghuaUniversity,AssociateProfessor(2016.9~)
§ ResearchInterests• systemsecurity,programanalysis,reverseengineering
§ Hackforfun• 2012MicrosoXBlueHatPrizeContest• 2015/2016DEFCONCTF• 2015/2016DARPACGC
Title
CyberSecurity:Defense§ n
Title
CyberSecurity:A9ackhoursanddaystofindvulnerabili`esandwriteexploits
DEFCONCTF2015(Blue-Lotus)
Title
Ques`on
Canmachineautoma`callyperforma9ackanddefense,andevenbeathuman?
Title
We’veBeenHereBefore
h9ps://cgc.darpa.mil/ISSTA_2014_r2.pdf
Title
CyberGrandChallenge
AnewDARPAChallenge…
Title
DARPA’sGrandChallenges§ 2004GrandChallenge• robotvehicles,target150miles,max7.32miles
§ 2005GrandChallenge• robotvehicles,target132miles,5teamspassed• focusonphysicalchallenges
§ 2007UrbanChallenge• autonomouscars,target60milesin6hours,6teamspassed• focusonsoXware:trafficlights,stopsigns,distance• industry:Googleself-driving,TeslaAutopilot,etc.
§ 2012Robo`csChallenge• humanoidrobo`cs,executecomplexac`onsincomplexenvironments
• industry:BostonDynamicsRobot(ModelAtlas,2016)
Title
CyberGrandChallengeAgenda7FundingTeams
(0.75M)97OpenTrack
Teams
CQE(2015/6)
7Finalists(0.75M)
CFE(2016/8)
CGCChampion(2M)
DEFCONCTFHumanTeams
Machinevs.Human(2016/8)
Title
HowisCGC(CFE)operated?
• Reputa`onvs.Chea`ng• Fairness(noprioriknowledge)• Closetoreal-worldenvironment
Title
CGCRoles
DARPA
organizer&coordinator
Compe``onFrameworkBuilders
ChallengeBinary
Developers
Howdoteamsinteract?
Run`meBuilders
Howdoprogramsrun?
vulnerabili`es&referenceexploits&polls
Team1
analyzeCBs
Team7
analyzeCBs
…
Title
Compe``onFramework§ RESTAPIs
• h9ps://github.com/CyberGrandChallenge/cgc-release-documenta`on/blob/master/`-api-spec.txt
§ Submission(HTTPPOST)• RCB:ReplacementCBs• IDSrules• PoV:ProofofVulnerabili`es
§ Download(HTTPGET)• status
§ round,scoreboard• consensusevalua`on
§ opponents’RCBs§ opponents’IDSrules§ noopponents’PoVs
• feedback§ performance: `meandmemory§ security: CBcrashinforma`on,notprecisea3ackinforma7on§ evalua`on: whethersubmi9edexploitsworkornot
§ Networktraffic(incomingandoutgoing)• aspecialtapinterface
Compe``onFrameworkBuilders
Title
DECREE§ DARPAExperimentalCyberResearchEvalua`onEnvironment• Linuxkernelwithslightmodifica`ons
§ SpeciallyDesignedEnvironment• 7SystemCalls[Garfinkel2003]
§ terminate–endprogram(exit)§ transmit–writedatatoanfd(write)§ receive–readdatafromanfd(read)§ fdwait–waitforfds(select)§ allocate–allocatesmemory(mmap)§ deallocate–releasesallocatedmemory(munmap)§ random–populateabufferwithrandombytes
§ RestrictedInter-ProcessCommunica`on• Nosharedmemory• Onlysocket-pairs
§ Cleanbidirec`onalcommunica`on§ Automa`callycreatedbysystemonstartup§ SharedbetweenallprocessesinanIPCChallengeBinary
Run`meBuilders
Title
ChallengeBinary(CB)§ CGCformat• minormodifica`ontoELF• aspecialloader
§ Nofilesystemaccess,nonetworkaccess• communicateviacontrolledfdsocket-pairs
§ Userspaceonlyandsta`callylinked§ Nocode-reuseexceptacommon“libc”• 7syscallswrappers• commonmathfunc`ons
§ CompiledBinariesonly(nothandcoded)• alwaysavailableinrealworld• groundtruth(withoutnoiseofcompilerop`miza`onetc.)
ChallengeBinary
Developers
Title
Teams
IDSrules
Patches
exploits
polls
Title
CyberReasoningSystem(CRS)§ EachteamisresponsibletobuildanautomatedCRS,abletoconnecttothecompe``onframework,andcompetewithotherCRSsystems
Input:• OriginalCB• Opponents’RCBs• Opponents’IDS• networktraffic• status• feedback
Output:• PoVs(exploits)• RCBs(patches)• IDSrules
Tasks:• interactwithcompe``onframework• analyzeCBs• analyzetraffic• findvulnerabili?es• generatePoVs(exploits)• generateRCBs(patches)• generateIDSrules(networkdefenses)• tes`ng(func`onalityandperformance)
Teams
Title
ProofofVulnerability(PoV)§ Type1:control-flowhijacking• crashatanego`atedEIP• oneextrageneralregisterhasanego`atedvalue
§ Type2:informa`onleakage• leak4bytesatanego`atedaddressinaflagpage• Theflagpageisatafixedaddress,withrandombytes
§ Note:a9ackerscanusecontrol-flowhijackingtoleakflagpage.
Teams
Title
Scoring
§ availability(0~1)• func`onality• performance
§ memoryoverhead5%~50%§ `meoverhead5%~50%§ filesizeoverhead20%~200%
§ Security(1or2)• abletodefeatalla9acks?
§ Evalua`on(1~2)• linearfunc`on• howmanyteamscanwea9ack?
Title
Round-Accumula`ngScoring
§ Eachroundisabout4.5minutes• CBscouldbereleasedandrevokeddynamicallybyDARPA
§ IfwesubmitaRCB(orIDS)inroundN• wewillgetascoreof0inroundN+1• opponentscoulddownloaditinroundN+1• itwillbedeployedinroundN+2
§ IfwesubmitaPoVinroundN• itwilltakeeffectinroundN+1
Title
CodeJitsu
Title
OurTeam
HengYinDawnSong GeorgeCandea ChaoZhang
UCBerkeleyBitBlazae
Syracuse(UCRiverside)TEMU/DECAF
EPFL(CyberHeaven)S2E
UCBerkeley(TsinghuaUniv.)
Title
Title
CGCMachines
Title
OurCRS:Galac`ca
Highlightedinskyblue,theCRSthatleadsapowerfulfleetofselecMvesymbolicexecuMonengines,binaryinstrumentaMontools,andfuzzersonaheroicquesttofindcybersecurityformankind.
Title
design:cloud-basedarchitecture§ HPC:64nodes,eachwith20cores,256Gmem,1TBdisk• toanalyzeatmost30CBsata`me
Title
deployment§ Unifiedstorage:• glusterfs+postgres
§ Automateddeployment:• ansible
§ Self-containedapplica`ons:• docker
§ resourcemanagement:• mesos
§ taskscheduling:• custommesosscheduler
§ healthmonitoringandautomatedrecovery:• monit
Title
design:coreanalysiscomponents§ s
Title
Analysis§ disassembly• customdisassemblerbasedonIDAPro• conserva`velyscancodepointersindatasec`ons• integratecodeinforma`onfromdynamicanalysiscomponents(AFL,S2E)
§ defensemetadata• iden`fysuspiciousfunc`on,e.g.,prinx• iden`fyindirectlycalledfunc`ons• JITmemoryalloca`onsite• JITcodecallsites
Title
VulnerabilityDetec`on§ SmartFuzzing:improvedversionofAFL• IPCsupport• cookiehandling• seedmetrics:• throughputimprovement:AFLFast(CCS’2016)
§ SymbolicExecu`on:S2E• exploreprogramstatesandsolveconstraintstofindvulnerabili`es• statemergingandpriori`zing
§ Fusionofdifferentsolu`ons• seedsharing:fuzzer+S2E+trafficreplay• pathexplora`on:S2EhelpsFuzzertobreakthroughsomebranches
Title
ExploitGenera`on§ Crashsamples
§ Dynamicanalysis• Trackprogramstates:e.g.,memoryobjects.• Detecterrorevents:e.g.,memoryviola`ons.• Reportexploitablescenarios:e.g.,symbolicEIP.
§ Exploitgenera`on• fromexploitablescenarios,tryandsolveknownexploitpa9erns
§ PoVformathandling• Howtoembedthe(dynamic)nego`atedvaluesintothe(sta`c)exploits?
• S2Ewillembedtheformulaofthenego`atedvalueinthePoV• MayhemembedsaPythoninterpreterintoeachPoV!
Title
Defense§ CFI:controlflowintegrity§ Shadowstacks§ DEP§ Randomiza`on§ Dataleakagedefense§ op`miza`on
Trade-off:• Security• Func`onality• Performance
ShellPhishfindsabuginQEMU,andembedsspecialinstrucMonsintheirRCBs,topreventopponentteamsanalyzingthem.
Title
defensecorpus
Title
Results
Title
FinalScore§ green:availability,blue:security,red:evalua`on(a9ack)
Title
Evalua`on:A9acksout
Foreachteam,howmanysuccessfula9acks(team*round*CB)?
3.5.4.2.7.1.6.
Title
Evalua`on:Firstblood
Foreachteam,howmanyCBsdoesita9ackfirst?
5.3.1.2.4.7.6.
Title
Evalua`on:SolvedCBs
Foreachteam,howmanyCBsdoesitexploit?
3.1.5.7.4.2.6.
Title
Security:A9acksin
2.3.4.5.7.1.6.
Title
Availability:`meoverhead
1.5.6.4.3.2.7.
Title
Availability:memoryoverhead
3.5.1.6.7.4.2.
Title
Availability:func`onality
1.2.6.3.7.4.5.
Title
Availability:SubmissionsofRCBs
Eachsubmissionwillcausenextround’sscoretobe0!
1.6.2.7.4.5.3.
Title
Lessonslearned§ AvailabilityscoreismoreimportantthanSecurityscoreandEvalua`onscoreinCGC.
§ Opponentteamsarenotgoodatexploits,soit’ssafetokeeporiginalCBswithoutanypenalty.• Allteamsexploited26/82CBstogether.
Title
Machinevs.Human
Title
DEFCONCTF2016§ Day1:last§ Day2:3rdtolast§ Day3:last
§ Human• copyopponents’patches
§ PPP:embedbackdoorsintheirRCBs• reconstructopponents’exploits
§ Machine• firsttogenerateexploitsagainsta“arbitrarywritebyte0”vulnerability• firsttogenerateexploitsagainstanobfuscatedCB
Title
SomeThoughts§ Machinesaregoodat• findinglow-levelbugs• a9ack:defeatsimpleobfusca`on• defense:deploygenericdefensesquickly• defense:generatevaria`onsofprograms(movingtargets)
§ Machinearenotgoodat• findhigh-levelbugs• a9ack:generateadvancedexploits• defense:deployvulnerability-specificpatches
§ Futureofmachines• machinelearning?
Title
Conclusion§ CGCisagreatpioneerprojectinmakingautomateddefenseanda9ackintoprac`ce.
§ Itisthefirsta9empttomakesuchasystemwork.It’sreasonablesomegamerulesarenotproperlyset.
§ Itsuccessfullys`mulatesthecrea`onof7prototypesystems,andprovesautomateddefenseanda9ackispossible.Itwillleadawaveofresearchandindustryefforts.
§ Themachineisrising!
Title
Thanks!Q&A
