StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including...

95
StarCraft: Remastered Emulating a buffer overflow for fun and profit

Transcript of StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including...

Page 1: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

StarCraft:RemasteredEmulatingabufferoverflowforfunandprofit

Page 2: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

AnotebeforewebeginBlizzardEntertainmentinnowayendorsesorcondonesreverseengineeringofourproperties.

Theexerciseshereinwereconductedtounderstandthemethodsusedtocreateunlicensedbehaviors.

Page 3: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Anti-CheatEngineer,BlizzardEntertainment

• PreviouslyworkedatHex-RaysandMicrosoft

• Technicalwriter:• PracticalReverseEngineering,AntivirusHackers

Handbook• Batchography

• Passionateaboutreverseengineeringandlow-levelprogrammingonMS-Windows

• Interestedindebuggers,emulators,APIhooking,dynamicbinaryinstrumentationandvirtualizationtechnologies

• Contact• Email:ebachaalany atblizzard.com• Twitter:@0xeb

AboutMe:EliasBachaalany

Page 4: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

Mycolleagues• GuillaumeBreuil,YiDeng,ChrisGenova,Mark

Chandler,JamesTouton,PeteStilwell,ZakBennettandGrantDavies

Tools• SCMDraft2mapeditor- HenrikArlinghaus• trgk (TriggerKing)- https://github.com/phu54321/• MPQtools– Ladislav Zezula• BWAPI- AdamHeinermann• IDAPro- Hex-Rays• Diaphora – Joxean Koret• EUDEnabler andtheEUDDB- Farty1Billion-

http://farty1billion.dyndns.org/EUDDB/

SouthKoreanmapmakersandtoolscommunity• Kongze1004– RandomTowerDefensemapauthor• Sksljh2091– MarioExodusmapauthor• Jacksell12,Deation,Sato

CommunitySites• TeamLiquid,StarEdit Network,Naver.com

Comradesontheadventure

SorryifImissedanyone!

Page 5: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• StarCraftisasciencefictionRTS(real-timestrategy)

• ReleasedforPCandMaconMarch31,1998

• StarCraft:BroodWar- ExpansionpackreleasedonNovember30,1998

• Significantpatchestothistalk:• 1.16.1- 01/21/2009– Lastpatchfor8years• 1.18.0- 04/18/2017– Firstmodernpatch• 1.20.0– 08/14/2017– StarCraft:Remastered• 1.21.0– 12/07/2017– EUDreintroducedvia

emulation

Backstory/1

Page 6: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• StarCrafthadvariousbufferoverflowbugs,butonewasrelatedtoaparticulartriggerconditionandaction:• TheExtendedUnitDeathtriggerØ Orsimply:EUD

• BlizzarddidnotupdateStarCraftbetween2009andearly2017• Thecommunityre-enabledthebugwith

customlaunchersandtools

• Patch1.17wasslatedforreleasebutwasheldbackbecauseitwouldbreakmods,tools,andlaunchers:• wMode• wLauncher,ChaosLauncher• BWAPI– PlugintowriteAIbotsthatplay

StarCraft

Backstory/2

Page 7: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• StarCraftmapsbasedonEUDtriggersthrivedamongtheSouthKoreanmapmakerscommunity

• TheEUDtriggers:• Areencodedinthemapfile• Allowedarbitrarymemoryreadandwrite:

• ThemajorityofthepublicEUDmapsincirculationhavehardcodedaddressescompatiblewithStarCraft1.16.1 onWindows

Ø IamnotawareofanyEUDmapsfortheMacOS versionofthegame

• TheEUDexploitallowedmodders toauthormapsthatmodifythegameradically:• RandomTowerDefense• MarioExodusMap• Etc.

Backstory/3

Page 8: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

RandomTowerDefense– EUDmap

Page 9: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

BouncingBallEUDmap(SC1.16.1)

Page 10: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

BouncingBallEUDmap(SC:Rw/emulation)

Page 11: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique
Page 12: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• TheMarioExodusmapauthorcreatedaleveleditor!

• Themapwasdevelopedusingtrgk’s epScriptlanguageandcompiler

Page 13: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• TheyarejustMPQarchives• TheMPQformathasbeenextensivelyreverse

engineeredanddocumentedbythecommunity

• Theycontainvariousfiles:• TheycontaincustomWAVaudiousedbythe

map• staredit/scenario.chkß Theactualmapchunk

file• Thisfilecontainsthetriggerschunk• Itcontainsstringstablechunk• Itcontainsachunkdescribingbuildings

andunits• Etc.

StarCraftmapfileformat

Page 14: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

MapfileinMPQEditor• Ladik’s MPQeditorcanbeusedto

viewormodifythecontentsofanMPQmapfilehttp://zezula.net/en/mpq/download.html

Notethechunkfile:“staredit/scenario.chk”

Page 15: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Madeofoneormorechunks: • Chunkheaderisfollowedbythechunkbody• ThegameparseseachchunkbasedonitsID:

Scenariochunkfile/1

Page 16: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Somechunksmighthavetheirownsub-headers

• Thestringschunkissuchanexample:

Scenariochunkfile/2

Page 17: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Thestringschunkcanbeusedtohidedatanotusedbythegamedirectly• WhenCK_HDR.ckSize >(sizeof(thecompleteTStrTbl header)+∑strlen(ofallstringsinthetable))

• Themodders hideadditionaltriggersinthecaveareaofthestringchunk

Scenariochunkfile/3

Page 18: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Thisscreenshotshowsthelaststringinthestringstable• That’snotthechunk’sendthough,itisjustthestringtable’send• TheremainingbytesareadditionaltriggersinsertedbytheEUDtriggercompiler

Scenariochunkfile/4

• https://github.com/phu54321/

Page 19: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Theyareasetofconditionsandactionsthatgetevaluatedduringthegameloop

• Therearetriggerconditionsthattellyouwhen:• Acertaintimeperiodhaselapsed(timers)• Playerresourcesreachedacertainamount• Amaplocationhasbeenreached• Etc.

• Whenallthetriggerconditionsarefulfilled,thenyoucandoactionssuchas:• PlayWAVfile• Displayamessage• Create,kill,moveaunit,etc.• Changeunitownerandhealthpoints• Giveplayerresources• Etc.

Whataretriggers?/1

Page 20: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Triggersarestoredinsidethemapchunkfile

• Thetriggerschunkissimplyanarrayof_trigger structs

• EachtriggerhasanarrayoftheCONDITION andACTION structures

• ThedwPlayer andwType fieldsareusercontrolledØ Theyareusedtoread/write

out-of-boundsinsideanarray

• ThebOpCode fielddictatesthetriggerconditionandactiontype

Whataretriggers?/2

Page 21: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• ThebOpCode fieldisusedtoselectwhichconditionoractiontoexecute:

Whataretriggers?/3

Page 22: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Eachtriggerconditionisevaluated,thentheactionsareperformedifallconditionssucceed:

Whataretriggers?/4

Page 23: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

Whataretriggers?/5

Page 24: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Classic(visual)triggereditor(SCMDraft 2.0– byHenrikArlinghaus)

• Notethelargevalues:• UnitID• Deathtableindex• Etc.

Whataretriggers?/6

Page 25: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Texttriggereditor

• AprivatebuildofSCMDraftshowstheEUDoverflowaddresses

Whataretriggers?/7

Page 26: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Thebufferoverflowbuginquestionisfoundinthe“ExtendedUnitDeath”triggercode:• Thedeath_count()triggercondition

• à Readanywhereprimitive

• Theset/add/sub_death_count()triggeraction• àWriteanywhereprimitive

• Triggersarereadas-isfromthechunkfileandstoredinadoubly-linkedlist:

Thebufferoverflow/1

Page 27: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Adeathconditionwithout-of-boundsunittype(wType)orplayernumber(dwPlayer)causesthereadanywhereprimitive

Thebufferoverflow/2

Page 28: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Asetdeathactioncausesawriteanywhere andprovidethefollowingprimitives:• [mem]+=lQuantity• [mem]-=lQuantity• [mem]=lQuantity

Thebufferoverflow/3

Page 29: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• AnexampleofEUDtriggersfoundinsideanEUDmap:

Thebufferoverflow/4

Page 30: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• GivenaStarCraftmapthatcontainsmalformedinputthattriggersaread/writeanywhere:• Isthereisawaytoemulatethebufferoverflow

inanewergameversionwhere:• Thebufferoverflowbugisfixed• Someaddressesnolongerexistinthe

newgameversion• Someaddressesrefertonew/different

datastructureformat?

• Cantheemulatorworkondifferentarchitecturesandoperatingsystems?

EUDmapemulation– Problemstatement

Page 31: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

1. Identify• Identify/tracealltheaddressesusedbyan

EUDmap• Buildatableoftheaddressesandidentify

whattheyrepresentinthegamesourcecode

2. Intercept• Interceptallout-of-boundsaccess• Redirectaccessusingatranslationtable

• Oldaddressà Newaddress

3. Emulate1. Missingmemoryaddressesshouldbe

handledbycode2. Dangerousmemorychangesshouldbe

filtered/changedaccordingly(pointers,functioncallbacks,etc.)

Threestepssolution

Page 32: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

1. Identify• Unfortunately,wedidnothaveprivateor

publicsymbolsforStarCraft1.16.1.Ihadtostartreversingthegameexecutablefromscratch

• HowcanItellwhataddressesthemapsareaccessing?

• Whatisthegoal/intentbehindamemoryaccess?

2. Intercept1. Noproblemshere.Luckily,wecanfunnelall

theout-of-boundsread/writestotheemulationlayer

3. Emulate1. Handlebasicmemoryaccessemulation2. Emulateaddressesthatarenolongerpresent3. Emulateincompatiblestructuretypes

Implementationchallenges

Page 33: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

1. Reverseengineeringeffortswereimpededbythelackofdebuggingsymbols:• Reverseengineeredthegameclientfrom

scratch• Usedtheclosestsourcecodesnapshotfor

1.16.1• Foundtherightcompiler(VS2003)andthe

approximateoptimizationswitchesØ NowIhavedebuggingsymbolsfora

binarythatisveryclosetothepublicbuild

2. IusedbinarydiffingpluginsforIDAPro1. PatchDiff2- TenableNetworkSecurity,Inc2. Diaphora - http://diaphora.re/

Identify– Reversingthegame/1

Page 34: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Binarydiffingwaslimited:• Mismatchedfunctionsbetweenthediffed

binaries• Globalvariableswerenotidentified• Optimizedcodeandinlined functionsmade

diffingharder

• ResortedtomanualreverseengineeringtobridgethelimitationsfromBinDiffing

• Usedscriptingtoautomatethereversingtask• LotsofIDAPython scriptingwasinvolved

Identify– Reversingthegame/2

Page 35: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

SourcecodevsDisassemblyview

Page 36: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

SourcecodevsHex-Rayspseudo-code

Page 37: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

Automatingdatastructurerecovery

Page 38: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• StarCraftRemasteredcollectsgametelemetry(includingmapinformation,etc.)

• AsofOctober2017,wehadaround~603,773totaluniquemapsplayed• Ofwhich17,916wereEUDmaps(i.e.containedoutofbounds

indices)

• AfterImanagedtoreverseengineerenoughofthegame,Iwroteatooltoprocessallthemaps,identifyEUDmapsanddumptheout-of-boundsEUDaddresses

Identify– Staticallyidentifyalladdresses/1

Page 39: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

Identify– Staticallyidentifyalladdresses/2

Page 40: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• AfteraggregatingtheuniqueEUDaddressesacrossallofthe17kEUDmaps,Iendedupwitharound~800variablesusedbypopularEUDmaps

• IwroteanIDAPython scripttoemitatableforalltheuniqueaddresses,theirnamesandsizes

Identify– Staticallyidentifyalladdresses/3

Page 41: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Staticaddressdiscoverywasnotenough:• SomeEUDmapsweredereferencing

pointersandreachingintotheheap• Somestructuresarecomplicatedand

linkedtootherstructures(linkedlists,TCtrl*,TDialog*,etc.)

• Needmoretools:• IrealizedtheneedforadynamicEUD

addresstracer• Ialsoneededawaytosinglestep/debug

triggers

• IdevelopedanEUDTracer,aDLLthathooksthegameandinstrumentsalltherelevanttriggerhandlingcode

Identify– Staticallyidentifyalladdresses/4

Page 42: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• TheinstrumentedgamebinarycallsintothetracerDLLuponeachread/write

Identify– Dynamictracer/1

Page 43: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• ThePythontablecontainingEUDaddressesispassedtoasourcecodegeneratortoemitCcodeandtables

• Thetracerusesthattabletoaccountformemoryaccess

Identify– Dynamictracer/2

Page 44: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• WhenthegameloadsanEUDmap,thetracerDLLinterceptsallout-of-boundsaccess

• Anyunknownaddresstriggersabreakpointforfurtheranalysisandidentification

• AfterIidentifyanunknownaddress,IaddittothePythontablewhichisusedtoupdatethetracer’sEUDitemstable

Identify– Dynamictracer/3

Page 45: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Thetracer’smainroleistoguaranteethatalltheaddressesreferredtofromtheEUDmapareaccountedfor

Identify– Dynamictracer/4

Page 46: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• HavingawaytorecordallaccessedEUDaddresseswasnotenoughtounderstandtheintentbehindtheaccess

• IhadnorealwaytodebuganEUDmap:• IneededawaytonicelyrepresentanEUD

address• Ineededtosinglestepaftereachtrigger• Ineededawaytoconvertaseriesof

read/writeprimitivestopseudo-code

Identify– Moredebuggingtools

Page 47: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• IfIwantedtotracetriggers,Ineededtohaveawaytoconvertanaddresstoanicevariablerepresentation

• Sowhatisthesymbolicrepresentationof:• 0x5187E8+(0xC*3)+4?Ø gCards[3].pBtns

Identify– EUDaddresstosymbolicname/1

Page 48: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• WiththehelpoftheHex-Raysdecompilerandothermetadata,Iwrotethefunction“R”toresolveanaddressintoanicesymbolicname

Identify– EUDaddresstosymbolicname/2

Ø Ifthearray’sindicesarebasedonenums,then“R”willproperlyshowtheenum nameinsteadofanumericindex

Page 49: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• SCMDraft triggereditortextuallyrepresentsthetriggerscript:

Identify– Staticpseudocodegenerator/1

Page 50: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• IwroteaconverterfromthetriggerstexttoCpseudo-code(converttriggerstoanASTandthenemitasCpseudo-code)

Identify– Staticpseudocodegenerator/2

Page 51: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• TriggertextconvertedtoCpseudo-code(trig2cpp()):

Identify– Staticpseudocodegenerator/3

Page 52: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• WithIDA’sconditionalbreakpointsandtheAppcall feature,Iwroteadynamicpseudocodegenerator:• Ithelpsdebugthemaptriggerlogicduringruntime• Ithelpsinthediscoveryandunderstandingof

dynamictriggers(generatedbytheEUDcompilerfromtrgk)

• Conditionalbreakpointsaresetatstrategicentrypoints(pre,inandposttriggerexecution)

Identify– Dynamicpseudocodegenerator/1

Page 53: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• ConditionalbreakpointsdynamicallybuildtheASTonaccess

Identify– Dynamicpseudocodegenerator/2

Page 54: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Thedebugscripthasa‘Singlestep’switchtobreakaftereachtrigger• Pseudocodeisemittedonthefly

Demo– Dynamicpseudocodegenerator/1

Page 55: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• The“Singlestep”switchcanbeconfiguredtoprintthepseudocodeontheflyasthemaptriggersexecuteswithoutsuspendingthegame

Demo– Dynamicpseudocodegenerator/2

Page 56: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

Inthefirststep(identify):1. Webuiltalltherequiredstaticanddynamictracers

2. WecreatedtheEUDtablewithallknownaddressesandtheirsymbolicnames

3. Wehaveenoughtoolstoidentifyanyaddressandtracewhereitcamefrom

Nowweneedtointercepttheout-of-boundsaccessinthenewcodebase

Intercept/1

Page 57: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

Readprimitivesinterception Writeprimitivesinterception

Intercept/2

Page 58: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Fromtheemulator’sperspective,allEUDmaplogicboilsdowntotwoactions:

1. Readanywhereà value=read_vmem(eud_addr)2. Writeanywhereà write_vmem(eud_addr,value)

Intercept/3

Page 59: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

Inbasicscenarios,theemulationisverysimple:1. Computethefullvirtualaddress(EUD

address)fromthedwPlayer andwTypeindices

2. FromtheEUDaddress,findtheequivalentnewaddress(backingdata)inthecurrentgameversion

3. Computetheoffsetandreadorwritefrom/tothenewaddress

Emulate

Page 60: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Let’sextendthepreviousPythontableandattachthesourcefilenamewereeachvariableislocated

• Thetabledefines:virtualaddress,itemsize,sourcefilename,emulationflags,andbackingvariablename

Emulate– Variablesmapping/1

Page 61: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

RunningtheEUDtablegenerationscriptpatchesthesourcecodeandexportsallreferencedvariables:

Emulate– Variablesmapping/2

Page 62: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

Exportedvariablesexample:

Emulate– Variablesmapping/3

Page 63: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

Noneedtomakestaticvariablesglobal:• Thegeneratorhasanoptionthatletsyoupickanamefortheexportedvariable

Emulate– Variablesmapping/4

Page 64: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• The“eud_table.cpp”isautogeneratedfromthePythontable.Itreferstoalltheexportedvariablesfromvarioussourcecodefiles

• Itisusedtopopulatetheemulator’svirtualmemorylayout

• ItemsalsohaveassociatedflagsthatinstructtheemulatorwhichEUDadapterhandleswhichaddress

• Note:the“g_nothing”variablesarealignmentbytesinSC1.16.1.Themapmakersusethatspaceforstoringvariables

• A“nullptr”backingdataalmostalwaysindicatesthatthevariableistobehandledpurelybyanadaptercode

Emulate– TheEUDtable/1

Page 65: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• The“eud_extern.h”isautogeneratedfromthePythontable

• ItexposesalltheknownEUDvariables• Veryhandyforaccessingstaticvariablesfrom

anywhereinthecodewhenneeded

Emulate– TheEUDtable/2

Page 66: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

EUDTable{addr1,size1,backing_data1,handler_flags1}{addr2,size2,backing_data2,handler_flags2}{addr3,size3,backing_data3,handler_flags3}…

EUDEmulator

Shadowtable

VirtualMemoryEUDaddress

ÛHandlersmapping

table

EUDAdapters

Datastructure#1adapter

Datastructure#nadapter…

Duetothenatureoftheoverflow,thefollowingrestrictionsapply:• AnEUDaddressisalways4bytesaligned• AnEUDvalueisa32bitsinteger

StarCraftRemastered

VirtualSC1.16.1memory

Realgamememory

Emulatorarchitecture/1

Page 67: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

Shadowtable• Itcontainstheneededmemorycontents

fromtheSC1.16.1binary

Virtualmemory• Itusestheaddress-to-handlerslookuptable• ItmapsanEUDaddressrangetoanEUD

tableentryà EUDhandler/adapter

• ThetableentryforanEUDitemdescribes:• Thebackingdata(thenewvariable

address,ifpresent)• Theflagswhichtelltheemulatorwhich

EUDadapter(handler)touseforemulation

Emulatorarchitecture/2

Page 68: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

AspecializedEUDadapterisneededwhen:• Handlingnon-standarddatatypes• WhendealingwithEUDaddressesthatnolongermap

toanythinginthenewgameclient

Thefollowing5virtualmethodsareexposed• read_vmem() à Returna32bitsvalue• write_vmem() àWritea32bitsvalue• backup() à Itemspecificbackupcode• restore() à Itemspecificrestorecode• deferred_write() à Invokedafterallthetriggers

haveexecuted.Givesachancetobatchprocesswrites

Emulatorarchitecture/3

Page 69: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

ThebasicEUDadapter(eud_vmemitem_t class)handlesbasicdatatypes:1. TheemulatorcomputesthefullEUDaddress2. Findsthenewvariable’sbaseaddressandconvertstheEUDaddresstoanoffset3. Theappropriateadapteristhencalledwiththedesiredoffsettoread/writefrom/to

Thissimpletranslationapproachworksnicelyforbasictypes

EUDadapters– Basic/1

Page 70: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

Thebasic(pass-thru)adapterisgoodformostcases:• Byte,Word,Dword• Theemulatorcancrossboundariesbetweentwoitems• Basictypesarraysarealsosupported

UWORDa[2] UWORDb[4]

Readingavaluefromtheendinvolvesreadingfromtwodifferent adapters(handlers)

EUDadapters– Basic/2

Page 71: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Wecoveredtwoprimitives:

1. *memasg_op =const• asg_opà +=,=,-=

2. if(*memcmp_op const){actions…}• cmp_opà ==,>=,<=

• Howdowegetthefollowingprimitive?• *mem1asg_op *mem2

Usingbinarysearch!

Waitaminute,weneedonemoreprimitive!

Page 72: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Triggercondition:1. Probesthevalueofsrc_var

• Triggeraction:1. Incrementsthevalueofdst_var2. Decrementthevalueofsrc_var3. src_var’s valueeventuallyreaches

zero4. Backupchangesintovar_copy

Thesameprimitiveisrepeatedtocopyvar_copy backtodst_var

The*a=*bprimitive

Thisprimitiveisexpensiveandgenerateslotsoftriggers

Page 73: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Pointersare32bitsinSC1.16.1

• Obviously,wecannotjustusethepass-thrubasicemulation• Pointershavetobetranslatedfrom

EUDvirtualaddressestorealaddresses

• Theprimitive“*ptr1=*ptr2”invokedfromtheEUDtriggerswillspoilthepointervalueuntilthebinarysearchisover• Whattodowithincompletepointer

values?

EUDadapters– Pointers/1

Page 74: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Changestoaphysicalpointervalueshouldnottakeeffectunlessthevirtualpointervaluepassesa“pointervaliditycheckfunction”à Doesthevirtualpointerhaveaproperrealpointerequivalent?

• Relyontheshadowpointervaluewhenworkingwithincompletevirtualpointervaluesforfuturereads/writes:

Realmemory EUDvirtualmemory

void*game_ptr;

uint32_tgame_ptr_shadow;boolgame_ptr_dirty;

uint32_tgame_ptr;

EUDadapters– Pointers/2

Page 75: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Theeud_cobject_ptr_adapter_t isconstructedwithbackingdatapointingtoareferencetoarealpointerthatwewanttoexposetotheEUDemulator

EUDadapters– Pointers/3

Page 76: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• WhataboutEUDlogicthatdoesfunctionpointerarithmetic?

EUDadapters– Functionpointers/1

Page 77: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• PointerarithmeticmakesenseonlyintheEUDvirtualmemoryaddressingspace

• Fortherealpointeraddressingwehavetotranslatetoproperpointersandaccountforfunctionprototypecompatibility

• Basicimplementationidea:1. vaddr +=voffs2. paddr =find_real_fptr(vaddr,function_prototype_id)3. if(paddr !=nullptr)à struct.pFn =paddr;

• Intheemulator,suchcasesarehandledwiththeeud_struct_with_ptr_adapter_t

Virtualfunctionpointersandtheirprototypestable

EUDadapters– Functionpointers/2

Page 78: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• VariousdatastructureshavechangedbetweenSC1.16.1andSC:R

• Pass-thruadaptersarenothelpfulinthiscase

• Aspecializedadapterisneededtoconvertbetweenbothstructures:• Readoperation: translatesfromphysicalstructuretovirtual

structure• Writeoperation: translatesfromvirtualstructuretophysical

structure

EUDadapters– Incompatiblestructures/1

Page 79: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

EUDadapters– Incompatiblestructures/2

Page 80: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• InSC1.16.1• TriggerswerestoredinaStorm linked

listdatastructure• Storm isalibrarythatprovides

containersandplatformindependentfunctionality

• InSC:R• Triggersarestoredasblz::list<_trigger>• ‘blz’istheequivalentofSTL’sstd

namespace

• OtherstructuresintheoldgamealsouseStorm listswhilethenewgameusesdifferentcontainers

EUDadapters– Linkedlists

Page 81: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

Becausetriggersarehardtoprogram,theSouthKoreanhacker(nicknamedTriggerKing/trgk)wroteatriggercompiler:

1. YouwriteproperlogicinaJavaScript/PythonlikelanguagecalledepScript

2. TheepScript getscompiledintoabunchoftriggersandistheninjectedintotheappropriatemapchunks

3. MapcontainingtriggerscompiledwithepScript canbeidentifiedusingthebootstrapcodethatlinksregulartriggersintothedynamictriggers(insidethestringstable)

EUDadapters– Triggers/1

Page 82: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• epScript isaverypowerfullanguage:• TheMarioExodusEUDmapwaswritten

inthatlanguage

• Itscompilerhidesadditionaltriggersinthecaveareaofthestringschunk:

Ø Makingithardtoreverse-engineercompiledtriggers

Ø Oneneedstowriteatriggersdecompilertorecoverthelogic

• Compiledtriggersareself-modifyingandveryoptimized:

Ø Loops,functioncallsandothercontrolflowrelatedfunctionalityareimplementusingself-modifyingtriggersthatchangethetriggernodelinks(nextandprev links)

EUDadapters– Triggers/2

Page 83: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• EUDmapslocatethepointertothestringtable(gpMapStr)andaddsaconstantoffsetpointingtotheadditionaldynamictriggersinsidethestringtable(seeslide17)

• EUDmapsthenpatchthem_prevlink andm_next linksasneededtointroduceasmanytriggersasneeded• Insertingnewtriggersdynamicallywasnever

supportedinStarCraft.OnlytheEUDemulatorallowssuchactivity.

• Compiled/dynamictriggersarethebasisofcomplexandelaborateEUDmaps• Therefore,supportingdynamictriggerswasthefirst

thingaddedtotheEUDemulator

EUDadapters– Triggers/3

Page 84: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Fromtheemulator’sperspective,therearetwokindsoftriggers:• Initialtriggersoriginatingfromthetriggerschunk• Dynamictriggerslinkedtothetriggerslistbypatchingtheirnodelinks

• WhenStarCraftneedstoexecutetriggersaftereachgameloop:• TheemulatorknowshowtoservebothstatictriggersanddynamicEUDtriggers• Theemulatordoesnotreplicatethebackingdata(thetriggernodedata)whenever

possible

EUDadapters– Triggers/4

SC:Rà blz::list<_trigger>: _trigger0 _trigger1 … _triggerN

SC1.16:stormlist<_trigger>: _trigger0 _trigger1 … _triggerN

shadow:prev|next shadow:prev|next

Stringtable:

(Dynamictriggersinsertedattheendofthestringstable)

Stringschunkdata

Actualstringtable(TStrTbl) Extrachunkdata:dynamictriggers

shadow:prev|next

Page 85: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

TheStormnodeEUDadapterhoststhenodelinksasshadowvariables

EUDadapters– Triggers/5

Page 86: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• TheStormlistadapterimplementsanSTLcompatibleiterator

• Fromtheiterator’sperspective,anynodepointersoutsidethelisthastheirnodelinksanddatainthevirtualmemory

EUDadapters– Triggers/6

Page 87: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Partialbuffersadaptersareusedwheneverthevirtualitemsizeisgreaterthanthephysicalitemsize:

SC1.16.1item(virtual):

SC:Ritem(physical):

data

smallerdata unmapped

• Theadapterservesthemappeddatawhentheaccessoffsetiswithinthemappedrange

• Itwillservezerosw/ofailingwhentheunmappedareaisaccessed

EUDadapters– Partialbuffers

Page 88: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

1. Certainadaptersresorttousingdeferredwritesasmeanstospeed-uptheemulation

2. TheEUDmapwritesinchunksof4bytesatatimeØ Wedon’twanttore-constructrealgamedata

whiletheEUDmapisstillwritingthechanges

3. Instead,awritehandlersimplypasses-thruthewritestoatemporarybufferandmarkstheadapterasdirty• (Readsfromdirtyoffsetsareservedfromthe

temporarybufferforconsistency)

4. Afteralltriggersareexecutedinthatgameloop,theemulatorinvokesallthedirtyadapters’deferredwritecallbacks

5. Insidethedeferredwritecallback,thetemporarybufferisthenusedtoreconstructtherealstructuresusedbythegame.Theadapterdirtyflagisthencleared.

EUDadapters– Deferredwrites/1

Page 89: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

1. ThestatustextadapterletstheEUDmapswritetoatemporarybuffer

2. Afterwards,theadapterre-constructstheproperstatustextstructuresthatarecompatiblewiththenewgame(SC:R)code

EUDadapters– Deferredwrites/2

Deferredwriteexampleadapter:

Page 90: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• Variousgamedatavariablesareintegerarrays

• Sometimes,theelementsinthearraymusthaveboundedvalues• Naturally,thepass-thru(basic)adapterisnot

suitable(becausenovalidationtakesplace)

• Theboundedarrayadapteralsoleverageashadowarraytableforalltheelementsthathaveincomplete/invalidvalues

• Onlyafterthewrittenvaluesarevalid(withinthespecifiedbounds)thenchangesarereflectedintothebackingdata

EUDadapters– Boundedarrayelements/1

Page 91: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• TheUnitFlingy array’svalueshaveanupperboundof209EUDadapters– Boundedarrayelements/2

Page 92: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

ThroughoutthecreationoftheEUDemulator,variousadaptersweredevisedwheneveranewproblemisencountered:

• eud_adapter_cards• Supportstotalcustomizationofunits

commandcards

• eud_adapter_csprites andeud_adapter_cunit• Allowscontrolledmodificationsintothe

CSprite andCUnit structures

• eud_adapter_group• Allowsbitmapshufflinginsidecertaingame

animationframes

• eud_adapter_keytable• AllowsEUDmapstointerceptkeypresses(‘a’,

‘s’,‘w’,‘d’,keyupandkeydownforexample)

EUDadapters– Fulladapterslist/1

Page 93: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• eud_adapter_mpq• Allowssupportforprotectedmaps.• RefertoMPQfrozenmaps:

https://github.com/phu54321/euddraft/tree/master/freeze

• eud_adapter_msgtbl• Readaccessintothein-gamechatmessages

(“ChattingWar”EUDmaps)

• eud_adapter_partial_buffer• Variousnon-emulatedornolongerexistent

variablesarehandledwiththisadapter

• eud_adapter_playerdata• LetsEUDmapsreadplayerinformation

(name,race,color,etc.)

EUDadapters– Fulladapterslist/2

Page 94: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

• eud_adapter_pointers• Allpointerrelatedadaptioncode• Supportspartialpointers(backedbyshadow

values)

• eud_adapter_stattxt• Unitstatustextandhotkeysmanipulation

• eud_adapter_stormlist• Allowshigh-levelemulationofStormlists

• eud_adapter_structwithptr• Usedtoemulatestructuresthatcontainamixof

basictypes(pass-thru)andpointers(incompletepointers+virtual<->physicalconversion)

• eud_adapter_triggers• Supportsdynamictriggersemulation

EUDadapters– Fulladapterslist/3

Page 95: StarCraft: Remastered - Shortjump! · • StarCraft Remastered collects game telemetry (including map information, etc.) • As of October 2017, we had around ~603,773 total unique

Questions?