1

Darkode:Recruitmentpatternsandtransactionalfeaturesof“themostdangerouscybercrimeforumintheworld”BenoîtDupont1,Anne-MarieCôté1,Jean-IanBoutin2,andJoséFernandez3AmericanBehavioralScientist,2017,Vol.61(11),pp.1219-1243,ReprintedbypermissionofSAGEPublications,http://journals.sagepub.com/doi/abs/10.1177/0002764217734263. AbstractThisarticleexploresthesocialandmarketdynamicsofDarkode,aninvitation-onlycybercrimeforumthatwasdismantledbytheFBIinJuly2015andwasdescribedbyaU.S.Attorneyas“themostsophisticatedEnglish-speakingforumforcriminalcomputerhackersintheworld”.Basedonaleakeddatabaseof4788discussionthreads,weexaminetheselectionprocessthroughwhich344potentialnewmembersintroducedthemselvestothecommunityinordertobeacceptedintothisexclusivegroup.Usingaqualitativeapproach,weattempttoassesswhetherthisrigorousproceduresignificantlyenhancedthetrustbetweentraders,andthereforecontributedtotheefficiencyofthisonlineillicitmarketplace.Wefindthattrustremainedelusiveandinteractionswereoftenfraughtwithsuspicionandaccusations.Evenhackerswhowereconsideredsuccessfulfacedsignificantchallengesintryingtoprofitfromthesaleofmalicioussoftwareandstolendata.KeywordsInternet,malicioushackers,malware,illicitonlinemarkets,trustCorrespondingAuthorBenoîtDupont,CICC/UdeM,PavillonLionelGroulx,CP6128succursaleCentre-Ville,Montreal(QC)H3C3J7,CanadaEmail:benoit.dupont@umontreal.ca

1UniversitédeMontréal,Montreal,QC,Canada2ESET,Montreal,QC,Canada3ÉcolePolytechniquedeMontréal,Montreal,QC,Canada

2

IntroductionOn 15 July 2015, the FBI and the U.S. Department of Justice announced the takedown of acomputerhackingforumknownasDarkode,whichledtotheindictmentof12suspectsandthearrest of 70 other members across 20 countries (Zetter, 2015). U.S. Attorney David HicktondescribedDarkodeas“…oneofthegravestthreatstotheintegrityofdataoncomputersintheUnitedStatesandaroundtheworldand…themostsophisticatedEnglish-speaking forumforcriminalcomputerhackersintheworld”(FBI,2015).Europol’sslightlylessdramaticmediareleasestated that Darkode was “the most prolific English-speaking cybercriminal forum to date”(Europol, 2015). Darkode was certainly not the first online illicit marketplace to attract theinterestoflawenforcementagenciesandtobetakendownfollowingalong-lastinginfiltrationoperation(Ablonetal.,2014),butthefactthatitwasaccessibleonlybyinvitationandclaimedtocatertoasmallbutexclusivecommunityofelitemalicioushackersmakesituniquelyinterestingforresearchers.

Mostofthescientificliteratureonmalicioushackersandtheillicitdigitalmarketplacesonwhich they converge to exchange knowledge, find new co-offenders, and trade maliciousmalware,criminalservices,andstolendatareliesheavilyondataculledfromeasilyaccessiblepublic or semi-public online forums. Lacking technical skills and criminal contacts, aspiringmalicioushackersenduponforumsthatareeasytofindandwelcomeanyone;theseforumsarealsoeasierforacademics,boundbytherigorousconstraintsofresearchethicsboards,tostudy.Unfortunately,mostsuchforumssufferfromastructuraltrustdeficit(Dupontetal.,2016)andservelargelyasfertilehuntinggroundswherecunning“rippers”takeadvantageofthegullibilityofnovices(HerleyandFlorêncio,2010).Themostexperienced,skilled,andsuccessfulhackersplytheirtradeoncloselyguardedinvitation-onlyforums,whicharealmostimpossibleforacademicstostudyethically,makingitextremelydifficulttolearnmoreaboutthedynamicsofthesethrivingmarketplaces.Mostoftheknowledgeaboutthemcomesfromjournalisticinvestigations,whichemphasizethehumaninterestaspectsofthisrapidlyexpandingundergroundeconomy(Glenny,2011;Poulsen,2011;Krebs,2014)andareunderstandablymore interested in chronicling theexperiencesofhighprofilehackersthanprovidingacomprehensiveanalysisofthenatureandstructureoftheirillicitexchanges.Thisknowledgegapisregrettable,asthevolumeandimpactofonlineharmsareclearlyontheincreaseaccordingtothelateststatisticaldataavailablefromtheU.K.,which istheonly jurisdictionthathasaddedonlinecrimesto itsvictimizationsurvey(OfficeforNationalStatistics,2016),andnowrepresentthemainformofcriminaloffenceagainstbothorganizationsandindividuals.

However,onrareoccasionshighlysecuredillicitonlineforumsarehackedbycompetitorsorvigilantesandthecontentoftheseforumsispubliclyreleased.On1April2013,aFrenchbloggerusing the alias “Xylitol” released a cache of 4788 files taken fromDarkode that exposed theforum’smembership,productsandservicesforsale,andvariousdiscussionsoverthepreviousfouryearsbetweensomeoftheworld’smostprolificmalicioushackersandprogrammers.Thisarticle provides the first analysis to date of these files and examines the specific social andbusinessdynamicsofwhatwasforatimeaveryactivedigitalconvergencesettingforsuccessfulonlineoffenders.Therecoursetosuch“found”datadoesraisesomeethicalanddata-reliabilityissues(McCoyetal.,2012).However,althoughit’simpossibletoentirelyexcludethepossibilitythat some of the files were forged in an effort to implicate particular members, the efforts

3

required to generate hundreds or thousands of fake discussion threads would bedisproportionate to theexpectedbenefits. Furthermore,ahighly respected journalistwithanintimateknowledgeofthecybercrimeundergroundreviewedthefilesandfoundnoreasontoquestiontheirreliability(Krebs,2013a).Ethicaluseofthisdatawasgreatlyfacilitatedbytheirformat,which ensured thatmetadata or other types of identifying information could not beinadvertently obtained and shared. As well, Darkode members used only aliases and wereextremelyprudentintheiroperationalsecuritypractices,makingitimpossibletoguesstheirrealidentity,exceptinthecaseofthosewhowerearrestedintheFBItakedownandnamedintheindictmentsthatfollowed.

WewereparticularlyinterestedintheDarkodeselectionprocess,duringwhichpotentialnewmembers introduced themselves to the communityasa first step tobeingaccepted into thisexclusivegroup.Usingaqualitativeapproach,weattemptedtoassesswhetherthisapparentlystringentprocedure significantlyenhanced trustbetween traders, therebycontributing to theefficiencyof this illicitonlinemarketplace. In the first section,weprovideanoverviewof thecriminology and computer science literature on online illicitmarkets and the trust dilemmasafflictingthem.Inthesecondsection,wegiveashorthistoryofDarkodebeforedescribinginathirdsectionthedataweusedandtheanalysesweperformed.Thefourthsectionexaminesthepresentational strategiesof344candidatesattempting to join the forumand theoutcomeoftheseapplications.Therelevanceofpersonalconnections,pastexperiences,technicalskills,andbusinessinterests,includingproductsandservicesavailableforsale,arediscussed.Finally,basedontwocasestudies,afifthandfinalsectioncastsnewlightonthechallengesfacedbycybercrimeentrepreneursdealingwithdemandingcustomerswhodonothesitatetoleakthemalwaretheyhavejustpurchasedtothebroaderhackingcommunity,significantlyerodingtheprofitabilityofsuchendeavors.Malicioushackers,illicitmarkets,andtrustasacooperativeenablerforcriminalachievementWhilemostcriminalonlinemarketsoperateasvirtualopen-airbazaars,withverylowentrybarriersinanattempttoattractanyonewithaninterestinbuyingorsellingmalwareorstoleninformation,afewofthemhaveadoptedadifferentmodelandfunctionasprivateclubs,accessibletomembersbyinvitationonly(Holt,2013).Abroadoverviewofwhatweknowaboutonlineillicitmarketsandtheirfailuresisneededtounderstandwhythiscounter-intuitiveapproachissensible. Themainpurposeofanonlineillicitmarketistoconnectsellersandbuyerstoallowthemtotradeinthebroadrangeofproductsandservicesthatallowthemtoexecuteandprofitfromtheircriminalprojects.Theglobalnatureofonlinecrimegeneratestremendousopportunitiesformalicioushackersbyprovidingaccesstoanunprecedentedlylargepoolofvictims(bothmachinesandhumans).However,identifyingandrecruitingco-offenderswhomasterthehighlyspecializedtechnicalskillsrequiredtocarryoutcomplexdigitaltheftandfraudschemesrepresentsachallenge.Toconnectthesupplyanddemandofsuchexpertise,onlineillicitmarketsoffervirtualconvergencesettingswhereoffenderscancongregate,developrapport,andforgeprofitablebusinesstieswithaccomplices(Felson,2003;Soudijn&Zegers,2012;Leukfeldtetal.,2016a;Macdonald&Frank,2016).OnlineillicitmarketsoperateontechnologicalplatformsthatincludeInternetRelayChat(IRC)channels(synchronous),web

4

forums(asynchronous),and,morerecently,theOnionRouter(Tor)network(DécaryHétuandGiommoni,2016).Transactionsareoftencompletedviaprivatemessagingtoolstoensuretheconfidentialityofnegotiations(Holt,2013;Yipetal.,2013).Theseplatformsalsoplayasignificantknowledgetransferfunction(Soudijn&Zegers,2012),replacingprisonsasthe“universityforcybercriminals”(Leukfeldtetal.,2016a). Overthepastfewyears,agrowingbodyofknowledgehasbeguntoprovideabetterunderstandingofthesemarkets’structureandsocialorganization,withaparticularemphasisonforums.Yipetal.(2013a;2013b)identifiedfourmainfeaturesthatmakeforumssoattractivetocybercriminals:formalcontrolandcoordinationmechanisms,socialnetworkingopportunities,andmethodstohelpmitigatebothidentityandqualityuncertainty.HoltandLampke(2010)usedqualitativeanalysistodescribethetypesofinformationandservicesforsaleonsuchmarkets,thepriceandquantitiesavailable,andtheforces(communications,price,quality,andservice)thatinfluencetransactions.Holt(2013)appliedtheorganizationalframeworkdevelopedbyBestandLuckenbill(1994)toshowhowtheorganizationalcomplexityofcybercrimeforumsvaries,rangingfrominformalgroupsofcolleaguestomorestructuredorganizations,andhowtheyfacilitateadivisionoflabor.Severalresearchershavealsousedsocialnetworkanalysis(SNA)methodologiestounderstandthestructuralpropertiesofcybercrimeforumsandthetiesthatbindtheirmembers,oftenusingtheseinsightstosuggestoptimizeddisruptionstrategies(Luetal.,2010;Motoyamaetal.,2011;Yipetal.,2012;Monsmaetal.,2013;Décary-HétuandLaferrière,2015;MacdonaldandFrank,2016).Finally,afewresearchershaveappliedthecrimescriptanalysisapproach,usedbysituationalcrimepreventionscholarstobreakdowntheflowofactionsinvolvedincommittinganoffence,totheonlinesettingsofcybercrimeforums(SoudijnandZegers,2012;HutchingsandHolt,2015;HutchingsandHolt,2016),shiftingthefocusofdisruptionstrategiesfromindividualnodestospecifictasksandfunctions. Oneofthemajorfeaturesofcybercrimeforumsistheinherentmistrustthatcharacterizesinteractionsbetweenmemberswhotradeindeception.HerleyandFlorêncio(2010)wereamongthefirsttoexpressskepticismabouttheprofitabilityofthemostcommonformsofhackingandonlinefraud,notingthatillicitmarketsarecrowdedwithrippers–marketparticipantswhodonotdelivertheproductsandservicesforwhichtheyhavebeenpaid,orwhosupplyproductsofalesserqualitythanwhattheyhadpromisedbuyers.Rippersareubiquitousonillicitmarketsandcreateuncertaintyandparanoiathathindersthenaturalflowoftransactions.Theremaybeahighlevelofactivityonopenonlineillicitmarkets,butthelevelofactivitydoesnotensuregreateconomicperformance.SuchmarketshavebeencomparedtothefamousmarketforlemonsfirsttheorizedaboutbyAkerlof(1970),whereinformationasymmetrybetweenbuyersandsellersdistortsthepricesandproducessuboptimaloutcomesforhonesttraders. Inordertofacilitatedetectionofripperssothattheycanbeexcludedfromcybercrimeforums,administratorshaveimplementedabroadrangeofcontrols,regulations,andreputationmanagementtoolsinspiredbythesolutionsdevelopedbye-commerceplatforms(Lusthaus,2012;SoudijnandZegers,2012;Yipetal.,2013b;Holtetal.,2015).Theserisk-reductionstrategiesareintendedtobuttresstrustandmakemarketsmoreefficient,butpreliminaryevidencefromtheworld’slargesthackingforumindicatesthattheydonottranslateintothesemarketsaseasilyashadbeenhoped(Dupontetal.,2016).Ithasbeensuggestedthat

5

themosteffectivewaytoovercomethistrustdilemmawouldbetoeitherraisethecostofparticipationinordertodeterrippers(Afrozetal.,2013)or,moredrastically,tolimitmembershiptoasmallgroupofreliableparticipants(Yipetal.,2013b).Alimitednumberofcybercrimeforumshaveadoptedanexclusivemodel(Ablonetal.,2014).Whileweknowrelativelylittleabouttheeconomicperformanceofcybercrimeforums,thecriminalachievementsofthosewhopatronizethem(Franklinetal.,2007;McCoyetal.,2012;Allodietal.,2016;Décary-HétuandLeppänen,2016;Holtetal.,2016),orwhatmakesthesemarketssustainable(Afrozetal.,2013),itcertainlyseemspossiblethatahand-pickedgroupofskilledandexperiencedhackerswouldtrademuchmoreeffectivelyandefficientlythanalargecommunityofself-selectedmemberscomprisedmainlyofnovicesorindividualswithverylimitedtechnicalandmonetaryskills.Mostoftheavailableliteratureononlineillicitmarketshasreliedonempiricaldatacollectedfrompubliclyaccessibleforums,withasmallersampleofstudiesusingregistration-onlyforums,whichrequireaself-selectedpasswordandarenotindexedbymainstreamsearchengines(Holt,2016).Tothebestofourknowledge,noresearchhasyetanalyzedtheadmissionandmarketdynamicsofaninvitation-onlyforum,themaincontributionsofthepresentarticle,whichdiscussesdataobtainedfromDarkode.AshorthistoryofDarkodeIntheabsenceofofficialrecordsandreliablearchives,documentingthehistoryofonlineillicitforumsisachallenge.TheiradministratorsgenerallytrytoevadetheattentionofoutsidersandlawenforcementinvestigatorsbylimitingaccesstocontentpostedbymembersandavoidingindexingbysearchenginesthroughtheuseofanindustrystandardknownastheRobotsExclusionProtocol(or‘robots.txt’command),whichletssearchenginesknowthatthecollectionofdatafromsomewebpagesshouldbeavoided(Koster,1996),orbyusingpassword-protectedlandingpagesthatblockaccesstoawebsite.Unlessaresearcherhasbeengrantedaccesstoanillicitforumfromearlyinitsexistenceorhasbeenabletoretrospectivelyandcomprehensivelydownloaditscontents,shemustrelyonthirdparties,suchasjournalists,securitybloggers,andeventuallyhackersthemselves,inanyattempttounderstandthereasonsandconditionsthatledtothecreationanddevelopmentofsuchonlinecriminalmarketplaces. Accordingtosuchthird-partysources,DarkodewaslaunchedinMarch2008byaSlovenianhackernamedMatjažŠkorjancandanAmericanhackernamedDanielPlacek.ŠkorjanchadcodedandmarketedtheMariposabotnet,apowerfulpieceofmalwarethatatitspeakphasemanagedtoenslavecloseto13millioncompromisedcomputers(BBC,2013),whilePlacekwasmoreinterestedinprogrammingcredential-sniffingsoftware(Hrodey,2015).AccordingtoPlacek’sverycandidrecollection,theirintentwasto:“Startalittlecommunity,invite-only…wherewecouldgetsomelike-mindedpeopletogetherandreallyjusttalk[aboutmalicious]code…Wedon’twantthescriptkiddies,peoplewhoarejustusingthesetoolbutdon’treallyunderstandthem.Let’sgetthepeoplewhoarereallymakingthestuff…Westarteditupandinvitedafewpeoplethatwealreadyknew…chattingaboutcode,sharingalittlebitofcode…Initially,itwasaprettysmallgroup,lessthan25.Dayone,itwasfivepeopleorsomething,anditgrewovertime.Wetalkedabouttheprojectswewereworkingon,wetalkedaboutideas,talkedaboutsomeofthedifferenttechnologies”(Placek,2016). Thecommunitybenefitedindirectlyfromtakedownsofanumberofpubliccybercrimeforums

6

thatlefthackerswithlimitedoptionsforforumstotradetheirwares.ThefactthatDarkodehadadoptedaninvitation-onlypolicybecameanattractivefeaturethatgaveitaveneerofexclusivityandcontributedtoaquickriseinpopularity.Simultaneously,existingmembersdecidedtobringinbuyers.InPlacek’swords,“wehadthesepeoplewhowerecreatingthings,andsomeofthemhadsomecustomersthattheyworkedwithalready,andtheywantedtobringthemonthereandbeabletoselltothemthroughthatplatformaswell”(Placek,2016).Effortsweremadetodividethesiteintosectionsthatwereaccessibletomembersaccordingtotheirleveloftechnicalexpertise,withsomesectionsreservedfortheprogrammerswhoweremostskilledatdevelopingmalware.Onhisownadmission,PlacekwasnotassuccessfulanentrepreneurashisassociateŠkorjanc,whomanagedtosellhisbotnetcodetoafewhundredpeoplefor$500to$2,000apiece(Krebs,2015).Bothofthemdisengagedfromtheforumin2010:PlacekafewmonthsbeforehisarrestbytheFBI(hisarrestwasnotmadepublicuntil2015,andhecollaboratedwiththelawenforcementagencyduringthosefiveyears)(Hrodey,2015),andŠkorjancfollowinghiscapturebytheSlovenianpoliceinJuly2010(FBI,2010). ASwedishhackernamedJohanAndersGudmunds,whousedtheonlinealiasesofMafi,CrimandSynthet!c,tookovertheforum’sadministrationresponsibilities,withthehelpofanothermemberwhousedthealiasFubar.Bothhackershaddevelopedandweresellingmalwarethatallowedotherstobuildandoperatebotnets(theCrimepackexploitkitforCrimandtheNgrbotmalwareforFubar)(Krebs,2015).Theforum’scontinuousgrowthattractedsuccessfulRussianhackerssuchasAlexUdakov,Gribodemon,orPaunch,whohaddevelopedpopularandeasy-to-usemalwarepackagessuchasthePhoenixexploitkit,theSpyEyetrojan,ortheBlackholeexploitkit.Butthishighprofilemembershipalsoattractedtheattentionoflawenforcementinvestigatorsandsecurityresearchers,whoinfiltratedtheforumtocollectintelligence,creatingasenseofparanoiaamongmembersandleadingitsadministratorstoaggressivelybansuspiciousaccountsandtightenadmissioncriteria(MalwareTech,2014;Krebs,2015). InJanuary2013,anewadministrator,nicknamedSp3cial1st,whohadbeenoneoftheforum’searlymembers,haddonebusinesswithasignificantshareoftheforum,andhadareputationforspendingagreatmanyhoursonline,wasvotedin.Heproceededtovastlyexpandtheforum’smembershipbyadvertisingonbeginnerforumssuchasHackForumsandbysendingunsolicitedemailstothemembersofoldhackingforums(Xylitol,2013;MalwareTech,2014).ThisbroadrecruitmentdriveattractedamorediversesetofhackerstoDarkode,includingsomewhoactivelysoughtthemedia’sattentionthroughveryhighprofileattacks.TheLizardSquadcrew,whichgainednotorietyinDecember2014foritsDistributedDenialofServiceattacksagainstMicrosoftXboxandSonyPlaystationservers,wreckingChristmasformillionsofvideogameplayers,isrepresentativeofthisnewwaveofDarkodemembers(Turton,2015). Asitsstatusroseamongelitehackers,Darkodewasregularlyinfiltratedbysecurityprofessionals,whousedtheiraccesstomonitormembersandtheirdealings.On1April2013,aFrenchwhite-hathackerknownasXylitol,whohadestablishedareputationasatechnicallysophisticatedvigilantebentondisruptingcybercrimeactivities,releasedmostoftheforum’scontentsafteroneofitsmembersusedXylitol’shandletoconductillegalbusinessonline(Krebs,2011;Xylitol,2013;Pauli,2013;Hrodey,2015).Xylitol’sleakdidnot,however,preventDarkodefromremainingathrivingmarketplaceuntilitstakedownbytheFBI.Followingtheforum’stakedowninJuly2015,Sp3cial1st,whohadavoidedarrest,attemptedtomovetheforumtoamoresecureinfrastructurethatreliedontheobfuscationtechnologiesofthedarkweb,suchas

7

theTornetwork,buttheresurrectedforumwaspoorlysecuredandfailedtoregainthetrustofpastmembers(Kharouni,2015). UsinghackerleakstostudycybercrimeThedataleakedbyXylitolprovidesauniquewindowintothesustainedinteractionsofacommunityofveryactiveandundeniablymalicioushackers.Althoughthematerialwasinitiallymeanttoexposeandembarrassthemembersofthiscommunity,suchaleakalsoprovidesresearcherswithhighqualitysecond-handmaterialthattheywouldhavedifficultycollectingthemselves—forbothtechnicalandethicalreasons—enablingthemtounderstandthesocialandbusinessdynamicsofthesegroups.Thedatasetconsistsof4788screenshotfilesextractedfromtheforum’sdiscussionthreadsandcoversafive-yearperiod,from2009toMarch2013.Itamountsto819.69megabytes(Mb)ofdataandcanbedownloadedfromhttp://darkode.cybercrime-tracker.net.Thefilesareorganizedinfoldersthatreflectthestructureoftheforum’ssections:postsincludeamembershiplist,productsforsale,transactionreportsaboutnewproductsofferedtothecommunity,malwareanalysisreports,tutorialsandprogrammingtips,questionsaboutspecificproblems,anda“Hallofshame”sectionwherecomplaintswereairedandconflictswereadjudicatedbytheadministrators.ThismaterialisinthePortableNetworkGraphics(PNG)fileformat,soweattemptedtobatchprocessthedatabaseusingpowerfulOpticalCharacterRecognition(OCR)programsandcustomizedsolutionsofferedbycomputersciencecolleagues.Unfortunately,noneoftheseautomatedtechniquesforcontentanalysisweresuccessful,forcingustomanuallyparseandcodeeveryimagetoextracttheinformationitcontained.Figure1illustratestheappearanceofatypicalscreenshot,whereonlythefilenameissearchable.Eachdiscussionthreadcontainsmultiplecontributionspostedbyforummemberswhosealias,membershiplevel,accessiondate,numberofposts,reputationlevel,andlocationarepublished,althoughthislastpieceofinformationisnotablyunreliable.NoteforexampleMafi’slocationinFigure1’sfirstmessage:“Siberia,Igloo36b”,althoughhewaseventuallyfoundtoliveinSweden.[INSERTFIGURE1.ABOUTHERE] Forthispartofourstudy,aftercarefullyconsideringthequantityofpostsavailableforanalysis,theresourcesatourdisposal,andtheneedtobetterunderstandthemembershipofthisforumanditsstructure,wedecidedtofocusinitiallyonasubsetofdiscussionsthatseemedtoprovidethebestinformationtoeffortratio:theintroductionsprovidedbyaspiringnewmembers.Asdiscussedabove,Darkodewasaninvitation-onlyforum.Onceaprospectivememberhadsecuredaninvitationfromoneoftheforum’sexistingmembers(whousuallyreceivedanallowanceoftwoinvitationsbutcouldalwaysreplenishthembyaskingtheadministrators),he(asmemberswereexclusivelymale)wasadmittedtotheunverifiedsection(Level-1)wherehehadtocompletetheverificationandaccessionprocessbyintroducinghimselftothecommunity.AsoutlinedinapostbySp3cial1stfromJuly2010(seeFigure2),introductionswereintendedtohighlighttheskillset,recentexperience,ongoingactivities,andmotivationsofanapplicant.Aswell,candidatesusuallydisclosedwhohadinvitedthemtojointheforum.Forpeoplewhohadnopriorcontactsinthecommunity,aninterviewwithtrusted

8

members(Level1or2intheforumhierarchy)wasalsorequired.Eachintroductionwasthencommentedonbyexistingmembers,whoassessedthevalueofthecandidateandvotedtoacceptorrejecttheapplication.Thesecommentsoftenrevealpriorcollaborationsandbusinessexchangesbetweenthecandidateandestablishedmembers,usuallycarriedoutonotherundergroundforums.TheintroductionsandthediscussionsthatfollowthusoperateasatypicalrecruitmentinterviewwhereahackeruseshisintroductorymessagetoprovideacriminalCVthatmustconvincepotentialco-offendersofhistechnicalandbusinessworth,whiletheresultingevaluationsrevealpriorcriminallinks,aswellasthecurrentpreferencesandneedsofthislargecommunityofelitehackers. Onceaccepted,newmembers(designatedasFreshFish,probablyinreferencetoaslangtermusedsincethemid-18thCenturytodescribenewprisoninmatesandpopularizedinthe1994movieTheShawshankRedemption)gainedaccesstoLevel0oftheforum,wheretheycouldbuycertainproductsandparticipateinvariousconversations.Aftertheyearnedthetrustoftheirpeers,theywereadmittedtoLevel1,wherebusinessdealingswerelessrestricted,andeventuallytoLevel2,openonlytohighlytrustedmemberssuchasadministratorsandinfluentialhackers.Inoneoftheadministrator’sownwords,“thepointofthelevelsystemistobelessstrictontheinvitation,wheremorepeoplewillhaveachancetocontributeandeventuallybecomelevel1”(Mafi),whileatthesametimeshieldingthemostsensitivecontentsandtransactionsfromnewentrantswhosetrustworthinesswasuncertain.However,inathreaddiscussingalimitedleakbyXylitolinOctober2012,oneofthecommentersremindedhispeersthatsucharigidhierarchyprovedhardtoenforceinpracticewhenparticipantswantedtoexpandtheirmarket:“everyoneselltheirproduct’sintothelevel0lol…level1/2usersmuststopmakingsalesintothelevel0systemandtostartfinallytobeactiveintothelevel1”(Pwdot).Asthesameuserstatedmorebluntlyinafollow-uppost,“themainideawastoseparatethegoodmembersfromthedumbassandtokeepsecurethewholeforum...butinsteadofthat,everyonemovedintolevel0keepingdeadthelevel1section.”[INSERTFIGURE2ABOUTHERE] Theintroductionsectionoftheavailabledatacontains344applications(476screenshots)fromnewprospectsorformermemberswhohadremainedinactiveforextendedperiodsoftimeandhadtobere-accreditedbythegroup.Thecodingwasdonemanuallybytworesearchassistantswhousedacodebookdesignedbytheprincipalinvestigatorandreviewedeachother’sworkforconsistency.Ambiguousmaterialwasdiscussedwiththeprincipalinvestigatorandthecodebookeditedaccordinglytoensureinternalhomogeneity(Saldaña,2009:21).Eachofthe344applications(seeFigure3foranexample)wasprocessedasasingleeventandenteredintoacodingdatabasewherewerecordedthealiasofthecandidate,thememberwhosponsoredhim,hisparticipationinotherforums,thetechnicalskillsheclaimedhehadmastered,hisbusinessinterests(forexample,whetherhewasasellerorabuyer),hismotiveinjoiningtheforum,andtheproductshewasofferingtotradewithothermembers.Wealsocodedeachresponsetotheseinitialintroductions,inwhichexistingmemberswelcomedcandidates,askedthemquestionsortoclarifyspecificskillsorexperience,orpubliclydiscussedthepotentialvaluethataprospectivememberwouldbringtothecommunity.Foreachcomment,werecordedthenicknameofeachmemberwhoparticipatedintheevaluation

9

process,thenatureofhisassessment(whathadtriggeredapositiveornegativecomment),aswellasthegeneraloutcomeoftheapplication.Overall,thereare404discretealiasesinourdatabase(344candidatesand60“historical”members).Inotherwords,wetriedtosimultaneouslycapturethequalitativeandquantitativedimensionsoftheseinteractionsinordertounderstandhowthisgroupofhackersselecteditsmembersandwhatfeaturesmentionedbyrecruitswereparticularlyvalued.Wethenusedthisdatasettoperformtargetedqualitativeanalysisconcerningsomeindividualsandproductsthatappearedtobeofparticularinterest.Inthefollowingsection,wedescribethemainargumentsusedbyapplicantstogainacceptanceintothiscommunity,aswellasthetypesofresponsesgeneratedbydifferenttypesofskillsandexperience.[INSERTFIGURE3ABOUTHERE] Whathackerstalkaboutwhentheytalkabouthacking:PresentationalstrategiesThedetailedcodingofintroductions,whichwereverydiverseinlengthandformatalthoughtheygenerallyfollowedthescriptoutlinedbySp3cial1stinFigure2,followedaninductiveprocessadaptedfrompreviousworkonreputationandtrustworthinessinonlinecybercrimeforums(Dupontetal.,2016).Intheiranalysisof25,000reputationratings,Dupontetal.(2016:14)identifyfivecategoriesoffeedbackthatjustifypositiveofnegativeratings:thelevelofsatisfactionwithapastbusinessrelationship,thetypeofgeneralcontributiontothecommunity,aspecificbehaviordirectedatthefeedbackprovider,thequalityoftechnicalskills,andsarcasm.Afirsthigh-levelreadingoftheintroductionsledustoremovethe“sarcasm”category,irrelevantinthatcontext,andtomakeminoradjustmentsthefourothercategoriestoclassifythesignalsoftrustworthinesssentbycandidatestothecommunity:whotheyknewintheforum(sponsors),mentionsoftheirtrackrecordonotherforumsorwithparticularhackingteams(experience),adescriptionoftheirhackingabilities,intermsofbothuniquenessandrelevance(technicalskills),andtheroletheyexpectedtoplayinthemarket(businessinterests).Whentheinformationwasavailable,wealsorecordedthetypesofmalwareandservicestheywereselling.Weusedthesamecategoriestoclassifyresponsesbyestablishedmembers,whichallowedustocomparewhatapplicantsthoughtthecommunityvaluedmostwithwhatactuallyattractedattentionorscornfromactivemembers.Table1providesthedescriptivestatisticsforthedistributionofintroductionsacrossthefourcategoriesdescribedabove.Inthenextparagraphs,weprovideadditionaldetailsoneachofthosefourdimensions,frombothcandidatesandestablishedmembers’perspectives.Toillustratehoweachcategoryoftrustworthinessargumentwasusedbyapplicantsandwhattypeofresponsesitelicited,weselectedanumberofquotesthatwebelievearemostrepresentativeofoursample,evenifsuchclaimsarealwayssubjectivewhenqualitativedataisanalyzed.Beforewegoanyfurther,weshouldnotethat,amongthe277applicationstojointheforumforwhichweknowtheoutcomewithcertainty,94.5%weresuccessful,whichwascounterintuitiveconsideringtheclaimstoexclusivitymadebyDarkodeadministrators.[INSERTTABLE1ABOUTHERE]

10

Sponsors:90.7%ofintroductionstotheforummentionthenameofthesponsorwhoprovidedtheinitialinvitation,reflectingtheimportanceofpersonaltiesinadmissiontothisgroup.Veryfewofthecandidatesprovidemoredetailedcontextualinformationthatrevealsthenatureoftheselinkages,butasmallgroupofforumadministratorsappearedtoberesponsibleforalargeshareofsponsorships.The286introductionsthatacknowledgeinvitationsmention119Darkodemembers,withanaverageof2.4invitationsconvertedintoapplicationsperreferrer(median:1,range:1-46).However,thefourmostinfluentialrecruiters(Sp3cial1st,G0dlike,Mafi,andFubar),whowerealsoforumadministrators,accountedfor38%ofinvitations.Withouttheirconstanteffortstopromotetheforumandscoutpotentialnewmembers,thegrowthofthisnetworkthroughregularmembers’referralswouldnothavebeensufficienttosustainthecommunity’sexpansion.Forexample,highprofilememberssuchasGribodemon,Paunch,orBx1broughtinonlyacoupleofnewmemberseach,focusingtheirenergiesonmarketingtheirownsuccessfulproductsratherthanongrowingthecommunityofpurchasers.Althoughtrustworthinesswasthemostfrequentlycitedargumentforadmittingorrefusingmembership,veryfewmembersassessingnewcandidatescommentedontheidentityortrackrecordoftheirsponsor(amere19.5%),seemingtotakethetransferabilityoftrustworthinessforgranted.ThefollowingquotefromaLevel1memberperfectlysummarizesthisvicariousformoftrust:““Iwasinvitedherebymafi.”ugotmewhenIsawthis.thisguysoundscool”. Technicalskills:Thesecondstrategytogainacceptancewastodemonstrateone’spotentialcontribution,withparticularemphasisontheuniqueandrelevanttechnicalskillsthatdifferentiate“scriptkiddies”fromthemoreadvancedprogrammerswhodesignandbuildthemalwareusedbytheformer.69.5%ofapplicantslistedtheirtechnicalskills,withamajorityclaimingtohavemasteredgenericcodingtechniquesandcommonprogramminglanguagessuchasC/C++,Javascript,Python,orPerl(60%),whileasmallergroupadvertisedmorespecializedskillssuchasreverseengineering(12%),obfuscationandencryptiontechniques(6%),sqlinjection(5%),ortraffictheft(2%).Interestingly,onlysevencandidates(2%)claimedtheyhadtheexpertisetofind0-dayexploits,thehighlyprizedundisclosedvulnerabilitiesagainstwhichnocomputersystemisprotected(Bilge&Dumitras,2012).Technicalskillsargumentselicitedonly15.1%ofcomments,usuallytoconfirmthatacandidatehadindeedprogrammedcertainproductstothesatisfactionofexistingmembers,sometimesnotingtheoutstandingqualityofthecodedelivered. Experience:Beforetheyapplied,manycandidateshadbeenactiveonothercybercrimeforumsandusedthesetypesofexperiencetogainadmissiontoDarkode,especiallywhentheyhadworkedtheirwayupandheldverifiedstatusoradministrativerolesinotherforums.Somecandidateswhohaddevelopedandmarketedpopularmalwarealsomadesuretomentiontheseintheirintroduction.Experienceaccumulatedonotherforumsorsellingparticularproductswasmentionedin49.7%ofintroductionsandtriggeredthelargestshareofresponses(48.5%),oftenconfirmingthataparticularmemberusingthesamealiashadbeenactiveonsaidforumandhadbehavedreliably.Theexperiencefactorwasthecontentcategorythatresonatedmostwithexistingmembers,whoseemedtobereassuredbythefactthatacandidate’strackrecordcouldbeverifiedindependently. Business:49.4%ofcandidatesincludedintheirintroductionthetypesoftransactionstheyexpectedtoconductonDarkode,eitherassellersorbuyersofproducts,services,orstolendata.

11

Buyerssignificantlyoutnumberedsellers,with31%ofapplicantswhoemphasizedbusinesscredentialsidentifyingthemselvesassellersand69%asbuyers.Productslistedasavailableincludedbotnets,malwaretools,databasesofstolenpersonalinformationoraccounts,proxyservices,encryptionsolutionstoevadedetection,aswellInternettrafficthatcouldleadtocriminalexploitation.Mostbusinessstatementsmentionedspecificproductsthatcouldbeobtainedfromthemorthattheywereseekingtobuy.Forexample,ExmanoizeclaimedinhisintroductiontobethesellerandauthoroftheEleonoreexploitkit,whichbecamepopularamonghackersin2009(Chen&Li,2015).Suchstatementswouldseemtobebeneficialbecauseacandidatewhohadestablishedarecognizedbrandthroughapopularproductcouldincreasehischancesofbeingacceptedintothecommunitythroughpublicsupportfrompastcustomers.Lessthanonefifthofcomments(18%)addressedthesebusinesscredentials,oftenbyconfirmingthatamemberhadsmoothlyconductedtransactionswithanapplicantandthattheproductsandserviceswereoftheadvertisedquality.Thepurchasingpowerofpotentialnewmemberswasalsoahighlyratedfeature. Hence,whilecandidatestriedtoearnthetrustoftheirpeersbyassociatingthemselveswithestablishedparticipants,showcasingabroadpaletteofattractivetechnicalskills,andleveragingreputationalcapitalaccumulatedonotherundergroundforums,existingmembersseemedmostresponsivetotheFreshFish’spreviousexperienceandtheirbusinesspotential.Butintheend,thesefourpresentationalstrategiesdidnotseemtoenableverydiscriminatoryselectionpatternsamongvotingmembers,consideringthatonly7%ofcommentsintheintroductionswereviewedexpresseddistrust.Asaresult,avastmajorityofapplicantsweregrantedaccesstoDarkodeandallowedtointeractwithhighprofilehackerseagertoexpandtheircustomerbase.ThebusinesschallengesfacedbyprolificsellersOnceadmittedtotheforum,Level0membershadtheopportunitytobuy,sell,andtradeabroadrangeofcybercrimeproductsandservicesandtocommentontheirqualityandaffordability.Theywerealsofreetoparticipateintechnicalproblem-solvingconversationsorinoff-topicdiscussionsaboutaverybroadrangeofsubjects,fromhighprofilearreststopornography,psychoactivesubstances,religion(whenRamadanstartedforexample),orevenArea51,thesecretmilitarybaseintheNevadadesert.Aswell,administratorsorganizedhackingchallengesthatallowedmemberstodisplaytheirtechnicalskills.Inthissection,wediscusstwoareascloselyconnectedtotheday-to-dayoperationsofthisillicitmarketandshowhow,despitetheselectionproceduredescribedabove,whichlookedexactingonlyinappearance,manyinteractionsbetweenbuyersandsellersweredysfunctional,underminingtheperformanceofthemarket.WestartbyanalyzingthetradesconductedbyBx1andcomparetheoutcomeofoneofhislargesttransactionswiththefinanciallossesattributedtohimbytheJusticeDepartment.Weselectedthisparticularmemberforthreemainreasons:hewasatthetimetheprincipalmarketerforoneofthemosteffectivebankingmalwareeverdesigned(Kirk,2011),maintainedaveryactiveprofileontheforum,andwassubsequentlyarrested,prosecutedandsentencedbytheU.S.government.ThisproducedarichtrailofpubliclyavailablelegaldocumentsthatmadethecomparisoninterestingbetweentheinteractionshehadonDarkodewithco-offendersandhowhiscasewaspresentedtopublicopinion.Wethenshiftourfocusfromthetradersoperatinginthismarkettotheproductsbeingexchangedin

12

ordertohighlightthespecificchallengesassociatedwiththesaleofhackingtoolstomalicioushackerswhooftendonothesitatetoleakthem,therebycompromisingthebusinessopportunitiesoftheirdesigners.Toillustratethispoint,weusetheexampleofCrimepack,apieceofmalwaredevelopedandmarketedbyoneofDarkode’sadministratorsthatwasleakedshortlyafterthereleaseofatechnicalupdate.Thiscasestudyshowshowevenoneoftheforum’smostpowerfulmemberscouldnotpreventothersfromundermininghisbusiness.WhatcriminalachievementlookslikefromtheU.S.GovernmentandDarkode’sperspectivesBx1wasoneofthemostactiveandsuccessfulmembersoftheDarkodeforum,wherehesoldapopularbankingtrojancalledSpyEyethathehadhelpeddevelop.SpyEyestoletheonlinebankingcredentialsofitsvictimsandhijackedwebsessionssothatitsoperatorscouldeasilyandstealthilytakeovertheirvictims’accounts.Althoughsuchnumbersarealwayshighlycontroversial,theU.S.JusticeDepartmentestimatedthatSpyEyehadinfectedmorethan50millioncomputers—targeting253discretefinancialinstitutions–andhadcausedclosetoabilliondollarsinfinancialharm.KnownasthesmilinghackerforhisrelaxedattitudeinpicturestakenfollowinghisarrestbytheThaipoliceon5January2013,whilehewasintransitfromMalaysiatoAlgeria,Bx1’srealidentitywasrevealedtobeHamzaBendelladj,a24-year-oldAlgeriannational(Krebs,2013).HewasextraditedtotheU.S.inMayofthesameyear,pleadedguiltytoall23countsofhisindictment,andwassentencedto15yearsinjailinApril2016(U.S.Attorney’sOffice,2016).EvenifhewasnotSpyEye’smaindesigner,heplayedaninstrumentalroleindevelopingcustomizedmodulesandmarketingthemalware.HealsousedSpyEyehimselftocollectlargequantitiesofstolenbankingcredentials,whichhealsosoldonDarkode. ThesentencingmemorandumfiledbytheU.S.AttorneyprovidedadetailedaccountofBendelladj’sdealingsandrequestedanexemplarysentencebasedonincurredfinanciallossesestimatedtohavereached$100million.TheDepartmentofJustice(DOJ)arrivedatsuchanimpressivenumberafterhavingrevealedthatBx1’sseizedlaptopscontainedmorethan200,000fullcreditcardrecords(includingnumbers,owners’nameandaddress,andcardCVV–thethreedigitsecuritynumberfoundatthebackofthesecards)andthathehad“cashedoutmillionsofdollarsstolenfrombankaccountsacrosstheworld”(Hornetal.,2016).Althoughthesentencingmemorandumnotedthatcreditcardissuersandbankshaddocumentedonlyabout$3.25milliondollarsinattemptedfraudand$878,000ineffectivelosses,theU.S.Attorney’sOfficestillappliedsentencingguidelinesthatvaluedlossesataminimumof$500percard,producinganimpressiveglobalamountthatwouldcaptureanyjudge’sattention.AcloserlookatthediscussionsinitiatedbyBx1andhispeers’responsesillustrateclearlyhowsuchcalculationsmighthavedistortedtheprofitabilityofhisbusinessandbeenmisleading. On3December2011,Bx1startedathreadontheforumadvertisingthesaleofafreshlyhacked“shopadmindatabase”containingmorethan140,000orders.A“shopadmin”isthecommontechnicaldesignationofawebinterfaceusedbyonlinemerchantstomanagetheirstore,keeptrackofcustomersandtheirorders,andmanagepaymentsanddeliveries.MostordersinthisdatabasewereshippedtotheU.S.andCanada,soBx1wasabletoofferhighlyvaluedcreditcardnumbersfromthosetwocountries,includingtheirexpirationdate,CVV,anassociatedbillingandshippingaddress,andtheemailandpasswordusedbycustomerstoregisteranaccountonthecompromisedwebsitefromwhichthedatahadbeenstolen.

13

AskedbyaLevel2memberwhatthestartingbidforthisdatabasewas,Bx1suggestedopeningthebiddingat$20,000.ThisisfarlessthanwhattheDOJformulapresentedabovesuggestsastheprojectedprofitsthatcanbegeneratedfromthesetypesoffrauds.UnfortunatelyforBx1,thefirstoffermadebyaforummembernamedDonchichoseriouslydampenedhisinitialhopes,offering$300forthewholedatabase.Bx1repliedthat:“ifIsell0.5$eachcc[creditcard]Iget50kguaranteed.”EvenifweignoreBx1’sshakymath,wearestillfourordersofmagnitudebelowtheDOJ’scalculationsfortheaveragefinanciallossassociatedwithastolencreditcard.SwayedbyBx1’splea,MrGold,anotherLevel2member,madea$2,000bid,whichwaspromptlyrejectedondifferentgrounds:“Itested6outof160kDiffe[different]Dates.Meansfrom2008-2011.Andallapproved.IcantestforinterestedbuyerandIshowthemVIATeamViewer[asoftwareallowingdesktopsharing].”Butthisargument,intendedtohighlightthequalityandreliabilityofthestolendata,wasinturndisputedbySven,aLevel2member,whoexplained:“Youcantest100and100outof100work.Whenyouuseabout6-8kofthetotal160k,allbase[anti-fraudbankingdatabases]willgonutsandyouget~20%approvals.”PerhapssensingBx1’sweakeningnegotiatingposition,MrGoldmadeafinal$3,000offer.ItmayverywellbethatanotherhackereventuallyconcludedamoregenerousdealwithBx1throughprivatechannels(acommonoccurrence),buttheexchangesbetweensellerandpotentialbuyersonthisthreadstillgiveasenseofthewildlyfluctuatingpricingmechanismsatwork,whichprobablyreflectthedifficultyofcashinginonthesetypesofstolendatabases. ThechallengefacedbyBx1wasnotonlytoobtainthepriceheexpectedforthestolendatabutalsotomaintainthesatisfactionofhisclientsand,byextension,hisreputationasareliablehackertodobusinesswith.Inanenlighteningexchangestartedon27May2011,Bx1advertiseda“spreader,”apieceofsoftwarethatautomatesthedisseminationofmaliciouscodeneededtoenrollvulnerablemachinesintoabotnetthroughpopularonlineservicessuchasFacebook,Twitter,Gmail,orHotmail.Afterafewflatteringcommentsfrommembersnotingtheeffectivenessoftheproductforsale,theconversationtookamorepersonalturnwhenSolotech,aLevel1member,complainedthathehadnotreceivedthemostrecenttechnicalupdateforthisspreader,towhichhefelthewasentitled.Additionally,hevoicedhisdispleasureaboutBx1’sunresponsiveness.Lessthantwelvehourslater,Bx1’sansweracknowledgedthathehadindeednotsenttheupdatebutthatthisdecisionhadbeenmotivatedbySolotech’sveiledthreatstopubliclyreleasethecodeofthespreader,whichwouldhavethreatenedBx1’sbusiness.ThisspurredGonzo,anadministratorwhohadpositivelycommentedonthemalwareforspreading“likeAids,”tocometoSolotech’sdefense:“Bro,IknowSolotechforawhilenow.Heisastandupguy,maybehewassayingthatoutofanger.”Anotheradministrator,Sp3cial1st,tookamoreconfrontationalapproachtowardBx1:“Beenwaitingapprox1weekforareplyfromyoubx1!Messagedyouwithsomequestionsabouthowitworksandsoforthbutnoreply.”By25June2011,athirdadministrator,Fubar,hadalsopubliclytakensideswithSolotech,tryingtojustifyhiserraticbehaviorbythefactthatSolotechhadpurchasedtheoriginalversionofthismalwarefromanotherhacker(Jam3s)withexclusiveuserightsfor$10,000andwasirritatedtofindoutBx1hadtakenoverthisprojectandhehadtopayanextra$1,500forthenewerandmorestableversion.Meanwhile,potentialbuyers’queriesweredrownedoutintheintensifyingexchangestryingtoassessthevalidityofBx1andSolotech’sarguments.Inotherwords,whatstartedasaroutinemarketingpostmorphedintoapublicdiscussiondebatingthelegitimacyofBx1’sbusinesspracticesandquestioninghiscommitment

14

tocustomerservice.ApostfromTuxreflectedthatfrustration:Willyoubebackbytheendofthisyear,causeIrememberwhensimilarmemberssaidgoingtovacation…andtheydisappearedforlike4-7monthscausingmealotofpainintheasswhiletimetocompletebusinesswouldn’ttakemorethen30minutestomaxcoupleofhours.Inthattimetheywererelaxing,swimmingorgettinghighIlostallotofmoneyIcouldearn,basicallyIhadopportunitycostcauseoftheirvacation…

Bx1’spositionhadshiftedtoadefensivepostureinwhichhewascompelledtojustifyhisbusinesspracticesandclarifyhisdisputewithSolotechinordertomaintainhisreputation.Alengthypostdated2July2011,summarizesthisdamagecontroloperation: Iaskedaboutu[Solotech]and1/10saysu’regoodrestallsaysNo.AndIstillhaveyourconversationwhenyousaidYougonnamakeitpublicandIdon’ttakeshitofthis.…Youcanpostmeonscammer[alistofuntrustworthytraders]oranywhereyoulike,everyoneknowsmehereIgaveallwhattheypurchaseandalsoIwasgivingthemgiftsinvcc’ssales[virtualcreditcards]andifIdidn’tthatorscamsomeoneheposthereorpostmeonscammer,andifanycoderisonmyplacehewilldosamelikeme.Justletsseepeoplewhattheysay.Guysjustsay,Doyougivesomeoneupdateifhesayshegonnamakeitpublic?YesorNo

Nohonoramongthieves:leaksandthedilemmasofprotectingcybercriminalintellectualpropertyThisfinalquestionisnotarhetoricalone,astheleakingofproprietaryhackingsoftwareisacommonoccurrencethatunderminestheprofitsofmalwaremarketers.Mafiforexample,theforumadministratorindictedintheJuly2015takedown,complainedbitterlytothecommunityon26September20101abouttheleakofhisownCrimepackexploitkitontheContagioMalwareDumpblog:2 Crimepackleak? Whatthefuckistheproblemwithyouguys?Theonlypeoplethathaveversion3.1.3ispeopleonthisboardandhowcomeasecurityresearchergetsaholdofacopyofit?YouguysbetterstartactingasfuckingprofessionalssometimealltheleaksofCrimepacksofarhasbeenthankstopeopleonthisforumandwearesupposedtoholdahigherfuckingstandard.IsupposeinthefutureIwillonlybeabletosell&giveupdatestoverylimitedpeopleandruleouttherest.

Theconversationthenproceededtotrytoidentifythesourceoftheleak,withthepossibility

15

beingraisedthatamolehadinfiltratedthegroup.Amoredisturbingexplanationwasthepossibilitythatoneoftheforum’smembershadbeencompromised,somehowdiscreditingthemythologyofDarkodeasanexclusivecommunityofelitehackers.Mafihimselfwasnotexemptfromquestionsabouthisoperationalsecurityskills,onemember(TheRogue)asking:“Whydidn’tyouputvarioustagsinthecodeandsuch?Likerearrangeblocklistorsomething.Thenwhenthey[securityresearchers]postinfo,youcantellwholeakedit?”Sp3cial1stalsomadehisdispleasureknowninnouncertainterms:“Iliterallyjustgotaroundtofinallyfuckinguploadingandinstallingthenondomainlockedcrimepackandnowitisbasicallyuseless.Goddamitsomeoneneedstobefuckingmurdered!”Toreassureforummembers,Mafistressedthattheobfuscationtechniqueheusedshouldpreventsecurityresearchersfromlearningtoomuchaboutthemalware’sfunctionsandthatitthereforeremainedausefulhackingtoolworthpurchasing.Andindeed,thiswassubsequentlyconfirmedbyacademicresearcherswhonotedthatCrimepackhadbeenencodedusingaveryeffectivecommercialprotectionsoftwarecalledIonCube(KotovandMassacci,2013).However,asaresultoftheuncertaintycreatedaboutthesourceoftheleakandtheneedtopreventfurtherexposure,Mafilimitedthedistributionofupdatestoafewtrustedcustomers,therebyrestrictingnewsalesopportunitiesandprofits.Furthermore,numerouspostsdiscussedwhocouldhavebetrayedMafi’strust,howhecouldbeidentified(someonesuggestedthatthelistofCrimepackbuyersshouldbedisclosedandthentherewouldbeavoteonpotentialsuspects),whathismotivationswere(greedorstupidity),whetherfederalagentswereinvolvedintheforum’sinfiltration,andhowthepotentialsuspectshouldbedealtwithonceidentified(somecalledfor“goodoldfashionrussiandismemberment,”whileothersfeltmoreforgiving,recommending“rapehimalittle”). Evenifoneshouldnotinterpretthecontentsofthesepoststooliterally,thesenseofdramaandparanoiacreatedbythiskindofbusinesstransgressionsandtheiraftermathproveddetrimentaltotheeffectivefunctioningofthismarket.Thesituationbecamesoproblematicthata“noleaking”policywasannouncedinMarch2011byGodlike,oneoftheforum’sadministrators: Leakingwillnotbetoleratedanymore.Pleaserespectmemberswork. Whenyouleak,youmakeauthorsthinkagainbeforereleasingtheirtools.Ifyoudecidetoleakyouwillbebannedwithoutwarning,thisappliestotoolsthatwilleffectmembersoftheforum.

Predictably,inanumberofpoststhatgrewexponentially,somememberswonderedhowthisnewpolicywouldworkasthecomplexityofdesigning,implementaingandenforcinganeffectiveanti-leakingpolicybecameobvious.Thequestionsthatweredebatedamongmembersincludedthefollowing,withadministrators’answersinparenthesis:WouldthebanbecomprehensiveorlimitedtoactivitiesonDarkodewhiletoleratingleakingonotherforums?(Noclearanswer.)Wouldthepolicystillbeenforceableonceleakedmalwarehadbeenwidelydistributedbyothers?(Yes,inordertoavoidpubliclydisrespectingmembers.)Shouldmalwareprogrammedbymembersofbeginners’forumsbeexemptedfromthenoleakingpolicy?(Yes.)Ifyes,whatwouldhappeniftheirdeveloperswereinvitedtojoinDarkodeatalaterdate?(Theleakwouldthenberemovedfromtheforum.)Thesequestionshighlightedtheinherenttensionbetweenhackers’naturalpropensitytoprobe,reverseengineer,anddisclosesoftwarecode,on

16

onehand,andtheneedforanymarket,includinganillicitonewithrestrictedaccess,toadoptandenforceregulationsprotectingtheintellectualpropertyrightsofsellers,ontheother.ThefrustrationexpressedbyDarkode’sadministratorsreflectedthechaoticnatureofexchangesinthisparticularcommunity. Conclusion

This paper’s primary contribution is to analyze the outcomes of an elite hacking forum’sselectionprocess,andtodevelopabetterunderstandingofhowtrust isestablished inonlinemarketswhereparticipantsarevettedbytheirpeers.This is the first time, to thebestofourknowledge,thatdataaboutsuchanexclusiveandclosedgroupofcybercriminalsisexploredindepth.However,wedidnotanticipatethatwewoulduncoverasituationwherethevastmajorityofcandidateswereletinforprofitpurposes,incontradictionwiththedisciplinedandsecurity-mindedimageprojectedbytheforumadministrators.Despitethedetailedadmissionproceduresdesigned by administrators, the forum seems to have faced the same trust and reliabilityproblemsasthosedocumentedinpreviousresearchonpubliccybercrimechannelsandforums(HerleyandFlorêncio,2010;HoltandLampke,2010;Yipetal.,2013b;Dupontetal.,2016).Basedontheevidencederivedfromapartialanalysisofthisdataset,wefindahighacceptancerate(94.5%)fornewmembersthatcontradictsthenarrativeofaneliteforumaccessibleonlytohighlyskilledhackers.Providedthatacandidatewasabletoobtainaninvitation—andadministratorsmade sure that many were extended to members of less-exclusive forums (MalwareTech,2014)—the chances of gaining entry were overwhelmingly positive. This unexpectedly highacceptance ratewas sustained to a large extent by the belief that a proven track record onanotherhackingforumandawillingnesstoconductbusinessonDarkodeweremoredesirablefeatures thantheability todemonstrateadvancedtechnical skills,ademandthatwouldhaveproducedmorediscriminatoryoutcomes.Analysisofintroductionsconfirmedthatamajorityofapplicantsclaimedtohavemasteredonlyverycommonprogrammingtechniquesandthatmorecomplexskillssuchasobfuscation,cryptography,andsqlinjectionwereinmuchshortersupply.Asoneof its co-founders candidlyacknowledged, sucha configurationwasalmost inevitable,given a core group of talented hackers designing powerfulmalwarewho needed to find andcultivatebuyersfortheirproducts(Placek,2016).Inotherwords,whentheadministratorshadtochoosebetweenaclose-knitcommunityoftechnicalexpertssharingcommonvaluesandamoreopenmarketcatering tocybercrimeentrepreneurs, theyputprofitsbefore trust,whichpossiblyledtotheirdemise.

Asaresult,anddespitewhatmayarguablyhavebeenthemostelaborateattemptsyetbyanonlineillicitmarkettoshutoutrippersanderadicatedeceptivepractices,trustremainedelusiveandinteractionswereoftenfraughtwithsuspicionandaccusations.Thissenseofparanoiawasmademoreacutebyevidencethattheforumhadbeenrepeatedlyinfiltratedbylawenforcementinvestigators, security researchers, and theoccasional investigative journalist, putting a lot ofpressureonmembers.Manythreadsthatbeganastechnicalandbusinessdiscussionsescalatedrapidlyintosmearcampaignsthatmobilizedconsiderableamountsofenergyandbecamemajordistractions for a community that had been designed initially to improve the quality ofcollaborationamongproficienthackers.So,wedidnotprogressasexpectedinestablishingwhatfeaturesandtraitsenhancethereputationandtrustworthinessofnewentrantsinonlineillicit

17

markets,butour results suggest that frequentexpressionsofdistrustanddefianceare foundacrossthewholespectrumofhackercommunities,frombeginners’forums(Dupontetal.,2016)toahigh-endmarketsuchasDarkode.Thisinsightisanewcontributiontotheliteraturethatwillneedtobeconfrontedwithfurtherempiricalevidence inordertotest itsgeneralizability.Forexample,caninnovativetechnicalsolutionssuchastheautomatedcryptographic,reputationalandescrowmechanisms foundoncryptomarketsovercomethis trustdeficitandusheranewtransformativeeraofcybercrimeeffectiveness(AldridgeandDécary-Hétu,2014)?Alternatively,dohybridcybercrimenetworksthatblendofflineandonlinesocialtiesprovemoreresilientthantheirpurelyonlinecounterpartsasfarastrustisconcerned(Leukfeldtetal.,2016b)?

ForatraditionalonlineforumsuchasDarkode,oneoftheconsequencesofsuchhighlevelsofdistrustarethedifficultiesandresistancethatevenoneofitsmostsuccessfulmembers,suchasBx1, encounteredwhen he tried to sell his products, services, or stolen data. The estimatedrevenuefromthebigvolumesaleweanalyzeinthispaperprovedtobeverydifferentfromthefinancial loss estimated by the Department of Justice in Bx1’s sentencing memorandum—different by several orders of magnitude.While we acknowledge that our analysis does notincludethecostsoftheharmcausedbytheidentityfraudthatthissalemadepossible,andthatwewerenotabletoperformacompleteanalysisofallthetransactionsconductedbyBx1onthisforumandinotheronlineillicitmarkets,thediscrepancyremainssignificantanddeservestobeinvestigated further in future research, particularly given the lengthy jail sentences that havebeen imposed in similar cases. We have not found in the literature any other contributionattemptingacomparisonbetweenthefinancialharmcausedbyonlineoffendersandtheclaimedharm publicized by law enforcement investigators and prosecutors following high profileconvictions, and we therefore believe that researchers should continue their work onmethodologiesthatproducemorerobustnumbers.

Thequalitativeanalysisofseveralthreadsdiscussingleaksofmalwareandthebestwaystoprevent them highlights how challenging it was for malicious developers to protect theirintellectualpropertyandtomaintainsustainablerevenuestreams.Whiletheystruggledtokeeptheirficklecustomershappy,disloyalcompetitorsorcybercrimewannabesdidnothesitatetocrackcodetheyhadwrittenandshareitforfreeoratbargainprices.Infuturestudies,weplantodigmuchdeeperintothedatatounderstandhowdisputesanddistrustarose,howtheformerwereadjudicatedtolimitthelatter,andwhatwastheimpactonforum’soperations.ThefinancialharmcausedbyDarkodememberswascertainlynottrivial,butresultstakenfromasmallsampleof high profile hackers show that their criminal experiences and achievements divergedsignificantly from themyth of the lone super-hacker (Ohm, 2008) that is often generated bysecurityfirmsandlawenforcementagencies,andobliginglyamplifiedbythemassmedia.Liketheirofflinecriminalentrepreneurcounterparts, their success seemed todependasmuchontheir ability to prevent or overcome the malfeasance, mistakes, or failures (von Lampe andJohansen,2004;Tilly,2005)thatinvariablypunctuatedtheirdealingswithotherhackersasontheirtechnicalexpertise.

Notes

18

1Althoughthepostisdated26September2010,thefirstmentionofaCrimepackleakinthemediaandonspecializedsecurityblogsappearedonlyinMay2011.Thisdiscrepancymaybeattributabletoadelayinpublicizingtheleakortoatimestampingerror.2http://contagiodump.blogspot.ca

19

ReferencesAblon,L.,Libicki,M.,&Golay,A.(2014).Marketsforcybercrimetoolsandstolendata:Hacker’sbazaar.SantaMonica,CA:RANDCorporation.Afroz,S.,Garg,V.,McCoy,D.,&Greenstadt,R.(2013).Honoramongthieves:Acommon’sanalysisofcybercrimeeconomics.2013eCrimeResearchersSummit,SanFrancisco,17-18September2013.Retrievedfromhttp://www1.icsi.berkeley.edu/~sadia/papers/ecrime13.pdf.Akerlof,G.(1970).Themarketfor“lemons”:Qualityincertaintyandthemarketmechanism.TheQuarterlyJournalofEconomics,84,488-500.Aldridge,J.,&Décary-Hétu,D.(2014,May13).Notan'Ebayfordrugs':Thecryptomarket'SilkRoad'asaparadigmshiftingcriminalinnovation.Retrievedfromhttps://ssrn.com/abstract=2436643.Allodi,L.,Corradin,M.,&Massaci,F.(2016).Thenandnow:Onthematurityofthecybercrimemarkets–Thelessonsthatblack-hatmarketeerslearned.IEEETransactionsonEmergingTopicsinComputing,4,35-46.BBC(2013,December24).Mariposabotnet‘mastermind’jailedinSlovenia.Retrievedfromhttp://www.bbc.com/news/technology-25506016Best,J.,&Luckenbill,D.(1994).Organizingdeviance.UpperSaddleRiver,NJ:PrenticeHall.Bilge,L.,&Dumitras,T.(2012).Beforeweknewit:Anempiricalstudyofzero-dayattacksintherealworld.InProceedingsofthe2012ACMconferenceonComputerandcommunicationssecurity(pp.833-844).NewYork,NY:ACM.Chen,J.,&Li,B.(2015).Evolutionofexploitkits:Exploringpasttrendsandcurrentimprovements.Irving,TX:TrendMicro.Décary-Hétu,D.,&Dupont,B.(2012).Thesocialnetworkofhackers.GlobalCrime,13,160-175.Décary-Hétu,D.,&Laferrière,D.(2015).Discreditingvendorsinonlinecriminalmarkets.InA.Malm&G.Bichler(Eds.),Disruptingcriminalnetworks:Networkanalysisincrimeprevention(pp.129-152).Boulder,CO:LynneRienner.Décary-Hétu,D.,&Leppänen,A.(2016).Criminalsandsignals:Anassessmentofcriminalperformanceinthecardingunderworld.SecurityJournal,29,442-460.Décary-Hétu,D.,&Giommoni,L.(2016).Dopolicecrackdownsdisruptdrugcryptomarkets?AlongitudinalanalysisoftheeffectsofOperationOnymous.Crime,LawandSocialChange,doi:10.1007/s10611-016-9644-4.

20

Dupont,B.,Côté,A.-M.,Savine,C.,&Décary-Hétu,D.(2016).Theecologyoftrustamonghackers.GlobalCrime,17,129-151.Europol(2015,July15).CybercriminalDarkodeforumtakendownthroughglobalaction.Retrievedfromhttps://www.europol.europa.eu/content/cybercriminal-darkode-forum-taken-down-through-global-actionFBI(2010,July28).FBI,SlovenianandSpanishpolicearrestMariposabotnetcreator,operators.Retrievedfromhttps://archives.fbi.gov/archives/news/pressrel/press-releases/fbi-slovenian-and-spanish-police-arrest-mariposa-botnet-creator-operatorsFBI(2015,July15).Majorcomputerhackingforumdismantled.Retrievedfromhttps://www.fbi.gov/contact-us/field-offices/pittsburgh/news/press-releases/major-computer-hacking-forum-dismantledFelson,M.(2003).Theprocessofco-offending.InM.Smith&D.Cornish(Eds.),Theoryforpracticeinsituationalcrimeprevention(pp.149-168).Monsey,NY:CriminalJusticePress.Franklin,J.,Paxson,V.,Perrig,A.,&Savage,S.(2007).AninquiryintothenatureandcauseofthewealthofInternetmiscreants.InProceedingsofthe14thACMConferenceonComputerandCommunicationsSecurity(pp.375-388).NewYork,NY:ACM.Glenny,M.(2011).DarkMarket:Cyberthieves,cybercopsandyou.NewYork,NY:AlfredA.Knopf.Herley,C.,&Florêncio,D.(2010).Nobodysellsgoldforthepriceofsilver:Dishonesty,uncertaintyandtheundergroundeconomy.InT.Moore,D.Pym&C.Ioannidis(Eds.),Economicsofinformationsecurityandprivacy(pp.33-53).NewYork,NY:Springer.Holt,T.,&Lampke,E.(2010).Exploringstolendatamarketsonline:Productsandmarketforces.CriminalJusticeStudies:ACriticalJournalofCrime,LawandSociety,23,33-50.Holt,T.(2013).Exploringthesocialorganisationandstructureofstolendatamarkets.GlobalCrime,14,155-174.Holt,T.,Smirnova,O.,Chua,Y.T.,&Copes,H.(2015).Examiningtheriskreductionstrategiesofactorsinonlinecriminalmarkets.GlobalCrime,16,81-103.Holt,T.(2016).Identifyinggapsintheresearchliteratureonillicitmarketson-line.GlobalCrime,doi:10.1080/17440572.2016.1235821.Holt,T.,Smirnova,O.,&Chua,Y.T.(2016).Exploringandestimatingtherevenuesandprofitsofparticipantsinstolendatamarkets.DeviantBehavior,37,353-367.

21

Horn,J.,Ghali,K.,&Grimberg,S.(2016).UnitedStatesofAmericav.HamzaBendelladj(A.K.A.“Bx1”)CriminalActionNo.1:11-CR-557-AT-2SentencingMemorandum.Retrievedfromhttp://krebsonsecurity.com/wp-content/uploads/2016/04/bx1-gribboSM.pdfHrodey,M.(2015,October12).DarkSide.MilwaukeeMagazine.Retrievedfromhttps://www.milwaukeemag.com/2015/10/12/dark-side-darkode-fbi/Hutchings,A.,&Holt,T.(2015).Acrimescriptanalysisoftheonlinestolendatamarket.BritishJournalofCriminology,55,596-614.Hutchings,A.,&Holt,T.(2016).Theonlinestolendatamarket:disruptionandinterventionapproaches.GlobalCrime,doi:10.1080/17440572.2016.1197123.Kharouni,L.(2015,December1).Darkodereloaded–Newforumgets“F”grade.DayBeforeZeroBlog.Retrievedfromhttps://www.damballa.com/darkode-reloaded-new-forum/Kirk,J.(2011,July26).SpyEyeTrojandefeatingonlinebankingdefenses.Computerworld.Retrievedfromhttp://www.computerworld.com/article/2509482/security0/spyeye-trojan-defeating-online-banking-defenses.html.Koster,M.(1996).Amethodforwebrobotscontrol.Retrievedfromhttp://www.robotstxt.org/norobots-rfc.txtKotov,V.,&Massacci,F.(2013).Anatomyofexploitkits:Preliminaryanalysisofexploitkitsassoftwareartefacts.InJ.Jürjens,B.Livshits&R.Scandariato(Eds.),Engineeringsecuresoftwaresystems(pp.181-196).Berlin:SpringerBerlinHeidelberg.Krebs,B.(2011,October17).Softwarepiratecrackscybercriminalwares.KrebsonSecurity.Retrievedfromhttps://krebsonsecurity.com/2011/10/software-pirate-cracks-cybercriminal-wares/Krebs,B.(2013a,April2).Foolmeonce…KrebsonSecurity.Retrievedfromhttps://krebsonsecurity.com/2013/04/fool-me-once/Krebs,B.(2013b,May3).AllegedSpyEyeseller‘Bx1’extraditedtoU.S.KrebsonSecurity.Retrievedfromhttps://krebsonsecurity.com/2013/05/alleged-spyeye-seller-bx1-extradited-to-u-s/Krebs,B.(2014).Spamnation:Theinsidestoryoforganizedcybercrime–fromglobalepidemictoyourfrontdoor.Naperville,IL:Sourcebooks.Krebs,B.(2015).TheDarkodecybercrimeforum,upclose.Retrievedfromhttps://krebsonsecurity.com/2015/07/the-darkode-cybercrime-forum-up-close/

22

Leukfeldt,E.R.,Kleemans,E.,&Stol,W.(2016a).Cybercriminalnetworks,socialtiesandonlineforums:Socialtiesversusdigitaltieswithinphishingandmalwarenetworks.BritishJournalofCriminology,doi:10.1093/bjc/azw009.Leukfeldt,E.R.,Kleemans,E.,&Stol,W.(2016b).Origin,growthandcriminalcapabilitiesofcybercriminalnetworks.Aninternationalempiricalanalysis.Crime,LawandSocialChange,67,39-53.Lu,Y.,Luo,X.,Polgar,M.,&Cao,Y.(2010).Socialnetworkanalysisofacriminalhackercommunity.JournalofComputerInformationSystems,51,31-41.Lusthaus,J.(2012).Trustintheworldofcybercrime.GlobalCrime,13,71-94.Macdonald,M.,&Frank,R.(2016).Thenetworkstructureofmalwaredevelopment,deploymentanddistribution.GlobalCrime,doi:10.1080/17440572.2016.1227707.MalwareTech(2014).Darkode–OdetoLizardSquad(theriseandfallofaprivatecommunity).Retrievedfromhttps://www.malwaretech.com/2014/12/darkode-ode-to-lizardsquad-rise-and.htmlMcCoy,D.,Pitsillidis,A.,Jordan,G.,Weaver,N.,Kreibich,C.,Krebs,B.,Voelker,G.,Savage,S.,&Levchenko,K.(2012).PharmaLeaks:Understandingthebusinessofonlinepharmaceuticalaffiliateprograms.21stUSENIXSecuritySymposium,Bellevue,WA,8-10August2012.Retrievedfromhttps://www.usenix.org/system/files/conference/usenixsecurity12/sec12-final204.pdf.Monsma,E.,Buskens,V.,Soudijn,M.,&Nieuwbeerta,P.(2013).Partnersincybercrime.InD.F.Hsu&D.Marinucci(Eds.),Advancesincybersecurity:Technology,operationsandexperiences(pp.146-172).Bronx,NY:FordhamUniversityPress.Motoyama,M.,McCoy,D.,Levchenko,K.,Savage,S.,&Voelker,G.(2011).Ananalysisofundergroundforums.Proceedingsofthe2011ACMSIGCOMMconferenceonInternetmeasurementconference,NewYork,NY:ACM.OfficeforNationalStatistics(2016).CrimeinEnglandandWales:YearendingJune2016.London:ONS.Ohm,P.(2008).Themythofthesuperuser:Fear,risk,andharmonline.UCDavisLawReview,41,1327-1402.Pauli,D.(2013,September12).Theriseofthewhitehatvigilante.ITNews.Retrievedfromhttp://www.itnews.com.au/news/the-rise-of-the-white-hat-vigilante-356543/page0

23

Placek,M.(2016).DanielPlacekonDarkode.LawfarePodcast,episode157.Retrievedfromhttps://www.lawfareblog.com/lawfare-podcast-daniel-placek-darkodePoulsen,K.(2011).Kingpin:Howonehackertookoverthebillion-dollarcybercrimeunderground.NewYork,NY:CrownPublishers.Saldaña,J.(2009).Thecodingmanualforqualitativeresearchers.London:SAGEPublications.Soudijn,M.,&Zegers,B.(2012).Cybercrimeandvirtualoffenderconvergencesettings.TrendsinOrganizedCrime,15,111-129.Tilly,C.(2005).Trustandrule.Cambridge:CambridgeUniversityPress.Turton,W.(2015,August9).TheyearoftheLizardSquad.TheKernel.Retrievedfromhttp://kernelmag.dailydot.com/issue-sections/features-issue-sections/13941/who-is-lizard-squad-history/U.S.Attorney’sOffice(2016,April20).Twomajorinternationalhackerswhodevelopedthe“SpyEye”malwaregetover24yearscombinedinfederalprison.Retrievedfromhttps://www.justice.gov/usao-ndga/pr/two-major-international-hackers-who-developed-spyeye-malware-get-over-24-years-combinedVonLampe,K.,&Johansen,P.(2004).Organizedcrimeandtrust:Ontheconceptualizationandempiricalrelevanceoftrustinthecontextofcriminalnetworks.GlobalCrime,6,159-184.Xylitol(2013,April1).Darkodeleak.Retrievedfromhttp://www.xylibox.com/2013/04/darkode-leak.htmlYip,M.,Shadbolt,M.,&Webber,C.(2012).Structuralanalysisofonlinecriminalsocialnetworks.InternationalConferenceonIntelligenceandSecurityInformatics,Arlington,11-14June2012.Retrievedfromhttp://eprints.soton.ac.uk/337076/1/yip_isi2012_final.pdf.Yip,M.,Shadbolt,N.,&Webber,C.(2013a).Whyforums?Anempiricalanalysisintothefacilitatingfactorsofcardingforums.ACMWebScience2013,Paris,2-4May2013.Retrievedfromhttp://eprints.soton.ac.uk/349819/1/yip_websci13_final.pdf.Yip,M.,Webber,C.,&Shadbolt,N.(2013b).Trustamongcybercriminals?Cardingforums,uncertaintyandimplicationsforpolicing.PolicingandSociety,23,516-539.Zetter,K.(2015,July15).DozensnabbedintakedownofcybercrimeforumDarkode.Wired.Retrievedfromhttps://www.wired.com/2015/07/dozens-nabbed-takedown-cybercrime-forum-darkode/

24

Figure1.Darkodescreenshot.

25

Figure2.Introductionguidelinesscreenshot.

26

Figure3.Introductionscreenshot.

27

Table1.Frequencydistributionforfourcategoriesofargumentsusedbycandidatesintheirintroductionandbymembersintheirassessment(N=344)Categories Introductions Answersfromexistingmembers % N % NSponsor 90.7 312 19.5 67Technicalskills 69.5 239 15.1 52Experience 49.7 171 48.5 167Business 49.4 170 18.0 62Note:aseachintroductioncontainedsignalsoftrustworthinessbelongingtodifferentcategories,thesumisgreaterthan100%.