1

Adversarial Detection of Flash Malware:Limitations and Open Issues

Davide Maiorca, Member, IEEE, Battista Biggio, Senior Member, IEEE,Maria Elena Chiappe, and Giorgio Giacinto, Senior Member, IEEE,

Abstract—During the past two years, Flash malware has become one of the most insidious threats to detect, with almost 600 criticalvulnerabilities targeting Adobe Flash Player disclosed in the wild. Research has shown that machine learning can be successfully usedto tackle this increasing variability and sophistication of Flash malware, by simply leveraging static analysis to extract information fromthe structure of the file or from its bytecode. However, the robustness of such systems against well-crafted evasion attempts - alsoknown as adversarial examples - has never been investigated. In this paper, we first discuss how to craft adversarial Flash malwareexamples, and show that it suffices to only slightly manipulate them to evade detection. We then empirically demonstrate that populardefense techniques proposed to mitigate such threat, including re-training on adversarial examples, may not always be effective. Weargue that this occurs when the feature vectors extracted from adversarial examples become indistinguishable from those of benigndata, meaning that the given feature representation is intrinsically vulnerable. In this respect, we are the first to formally define andquantitatively characterize this vulnerability, highlighting when an attack can be countered by solely improving the security of thelearning algorithm, or when it requires also considering additional features. We conclude the paper by suggesting alternative researchdirections to improve the security of learning-based Flash malware detectors.

Index Terms—Flash-based Malware Detection, Static Analysis, Secure Machine Learning, Adversarial Training, Computer Security

F

1 INTRODUCTION

THe significant evolution of network-based services hasallowed users to easily access a comprehensive, wide

multimedia infrastructure that greatly improves the over-all Internet surfing experience. It is common to visualizehigh-quality streams from web pages, to navigate web-sites through complex animations, and to play graphically-advanced games from social networks. Although HTML 5 isnowadays considered the standard de facto to efficiently ren-der multimedia contents, Adobe Small Web Format (SWF,previously known as ShockWave Flash) is still widely used,although many reports predict a fast decline for this technol-ogy. Major streaming websites such as CBS, CNN, Spotify,and HBO still rely on SWF technology [25].

SWF suffered from numerous security problems in thepast two years. According to cvedetails,1 there were 329publicly disclosed vulnerabilities in 2015, and 266 in 2016.That, of course, does not include zero-day vulnerabilitiessuch as the one found in the Hacking Team servers duringthe popular attack perpetrated in 2015 [4]. The HackingTeam incident also provoked a massive media uproar, forc-ing browser vendors to take additional security measuresconcerning Flash player, such as making updates com-pulsory for the user in order to visualize the multimediacontent. However, this is just a weak palliative that doesnot completely solve the whole problem, for at least tworeasons. First, security updates only concern publicly dis-

• D. Maiorca (davide.maiorca@diee.unica.it), B. Biggio (bat-tista.biggio@diee.unica.it), E. M. Chiappe (elena.chiappe@diee.unica.it),and G. Giacinto (giacinto@diee.unica.it) are with the Department ofElectrical and Electronic Engineering, University of Cagliari, Piazzad’Armi, 09123 Cagliari, Italy.

1. https://www.cvedetails.com

closed vulnerabilities; second, not all security issues arerapidly patched. A more secure defense strategy is thereforerequired not only to ensure protection against zero-dayattacks, but also to foresee novel threats.

Signature-based detection of malware, which is stillwidely employed by many free and commercial anti-malware systems, is progressively getting inadequate toaddress the myriad of polymorphic attacks that are gen-erated every second in the wild. This also applies to thedetection of SWF-based malware, whose scripting code(ActionScript) contains heavily obfuscated routines thataim to confuse both automatic and manual analysis. Theadoption of machine learning, combined with static or dy-namic analysis of the embedded scripting code, has becomea compelling alternative to signatures or rule-based sys-tems. With respect to SWF malware, static analysis showedpromising results at detecting malicious samples in the wildby extracting information either from the structure of thefile [27] or from the ActionScript bytecode [34].

Previous work has however not discussed the robustnessof such approaches against evasion attacks, namely, well-crafted manipulations of the input sample at test timeto evade detection. More specifically, research has shownthat machine-learning algorithms are vulnerable to such at-tacks [8], [11] - also recently referred to as adversarial examplesin the context of deep learning [17], [28] - if the attackerpossesses enough information about the system, and if shecan perform enough changes to the extracted features. Suchattacks have been reported also against Android and PDFmalware detectors, but not against learning-based Flashmalware detection tools [8], [15], [18], [26]. Several coun-termeasures have also been proposed to mitigate the impactof such attacks, with the goal of reshaping the decision func-

arX

iv:1

710.

1022

5v1

[cs

.CR

] 2

7 O

ct 2

017

2

tion of the classifier with different techniques [?], [9], [13],[15], [17], [26]. For instance, game-theoretical approachesrequire retraining the classifier on simulated attacks untila game equilibrium between the classifier and the attackeris reached, providing formal guarantees for its existence anduniqueness [?], [13]. In the context of deep neural networks,although the underlying assumptions behind existence anduniqueness of a game equilibrium are not typically satis-fied, the idea of retraining the learning algorithm on theattack samples, referred to as adversarial training, has beenempirically shown to be promising [17].

The main goal of this paper is thus to understandwhether and to which extent systems that statically analyzeSWF files can be robust against adversarial attacks, i.e.,to propose a thoroughly security evaluation procedure forlearning-based SWF malware detectors based on static anal-ysis. To this end, we first propose a representative systemfor Flash malware detection, named FlashBuster. It isa static machine-learning system that employs informationextracted by both the structure and the content of SWFfiles. This allows for a more comprehensive assessmentof the extracted static information, by representing andcombining the content employed by previous state-of-the-art systems. We show that FlashBuster is able to detectthe majority of malware in the wild, thus confirming resultsobtained in previous work. We then evaluate the securityif FlashBuster by simulating evasion attacks with differ-ent levels of the attacker’s knowledge about the targetedsystem [8], [11] (also referred to as white-box and black-box attacks), against an increasing number of modificationsto the input samples. The corresponding security evaluationcurves, depicting how the detection rate decreases againstattack samples that are increasingly manipulated, allow usto comprehensively understand and assess the vulnerabil-ity of FlashBuster under attack. We finally discuss theeffectiveness of adversarial training against such attacks. Tothis end, we re-train FlashBuster on the evasion attacksamples used against it, and surprisingly show that thisstrategy can be ineffective. We argue that this is due to anintrinsic vulnerability of the feature representation, i.e., to thefact that evasion attacks completely mimic the feature valuesof benign data, thus becoming totally indistinguishable for thelearning algorithm. We define this vulnerability in formalterms, and quantitatively evaluate it through the definitionof a specific metric which measures the extent to which theattack samples converge towards benign data.

Our findings highlight a crucial problem that must beconsidered when designing secure machine-learning sys-tems, namely, that of evaluating in advance the vulnerabilityof the given features. Indeed, vulnerable information maycompromise the whole system even if the employed deci-sion function is robust. In this respect, we sketch possibleresearch directions that may lead one to design more securemachine learning-based malware detectors.

This paper is structured as follows. Section 2 providesthe basics to understand the SWF format. Section 3 providesthe fundamentals of the ActionScript bytecode. Section4 describes the architecture of FlashBuster. Section 5describes the threat model and the possible attack scenarios.Section 6 discusses the vulnerabilities that affect learning-based systems, and introduces a quantitative measure of

feature and learning vulnerabilities. Section 7 provides theexperimental evaluation. Section 8 describes the relatedwork in the field. Section 9 discusses and closes the paper.

2 SHOCKWAVE FLASH FILE FORMAT

ShockWave Flash (SWF) is a format that efficiently deliversmultimedia contents, and it is processed by the Adobe Soft-ware such as Adobe Flash Player.2 Because of its compact-ness and adaptability to the most popular web browsers,SWF has been widely used for online applications such asgames, animations, etc.

In this Section, we provide an overview of the SWFformat, by focusing on the analysis of its main components.Moreover, we provide an insight into the ActionScriptVirtual Machine (ASVM), which is used to parse scriptedcontents that might be found in the SWF file.

2.1 SWF BasicsThis Section describes the main blocks of a SWF file, which iscomposed of three basic elements: (1) a header that describesimportant file properties such as the presence of compres-sion, the version of the SWF format, and the number ofvideo frames; (2) a list of tags, i.e., data structures thatestablish and control the operations performed by the readeron the file data; (3) a special tag called End that terminatesthe file.

The SWF format supports two types of tags: definitionand control. Definition tags assign a number called characterID to each object shown by the reader. For example, theDefineFont and DefineSprite tags assign an ID, respec-tively, to a font and a sprite. Such IDs are then placed on adictionary that will be accessed by control tags to establishwhich elements will be visualized on each frame.

Action tags represent special types of control tags whosefunctionalities are triggered by user actions, such as pressinga button, moving the mouse, and so forth. Starting fromSWF 5, such actions were expanded by the introduction of ascripting language called ActionScript. With the releaseof its latest version (3.0), ActionScript code is now com-piled to an entirely new bytecode called ActionScriptBytecode (ABC), and run by the ActionScript VirtualMachine 2 (ASVM 2). More about the SWF file structure canbe found on the official SWF reference [3].

2.2 ActionScript Virtual MachineThis Section describes the essentials of ASVM 2, the newversion of the ActionScript Virtual Machine, released in2006. The introduction of characteristics such as Just In Timecompilation (JIT), namespaces, and packaging, brought tohuge speed improvements compared to the previous ver-sion. The computation in the ASVM 2 is based on the execu-tion of method bodies composed by instructions. Each methodbody runs in a specific context that defines information suchas default parameters.

ASVM 2 runs ABC files in four stages: (i) loading, wherethe ABC file is copied in memory; (ii) linking, where complexdata structures are created; (iii) verification, where all the

2. https://get.adobe.com/en/flashplayer/

3

resolved data structures are verified to be coherent; (iv)execution, where the file is compiled through JIT compilationand executed.

More about ASVM 2 can be found on the official VMreferences [2]. For the purposes of our paper, it is nowimportant to provide a description of the bytecode contents,as FlashBuster relies on analyzing such information toperform detection.

3 ACTIONSCRIPT BYTECODE (ABC)To simplify the comprehension of the static methodologyemployed in this paper, we provide an example of howa simple method written in ActionScript is compiledinto ABC bytecode, by disassembling and decompiling anapplication.3

This method is a constructor of the class PingServersthat performs three operations: (i) A superclassis invoked; (ii) The method addCallback of theExternalInterface class is called to make theActionScript code interact with external containers(e.g., JavaScript, HTML files, and so forth). In particular,this method registers the function pingCallback sothat it could be called by the container with the constantname RESPOND_PINGDATA; (iii) The instance variablespingTimer and pingTimeouts are set to zero. Moreabout ActionScript programming can be found on theofficial ActionScript reference [1].

From the perspective of the ActionScript bytecode,such operations can be summed up as follows: (i) The su-perclass is invoked through the instructions get_local_0,pushscope and constructsuper (lines 12-15). As suchinstructions are not relevant for the scope of this paper,we recommend reading the ActionScript specificationsfor more insights [2]; (ii) The addCallback method isinvoked through the instructions findpropstrict andgetproperty (lines 16-24). Such instructions respectivelylook and fetch the required packages, classes and meth-ods. The resolution of such names is made through datastructures called Names, which are very important for thescope of our paper. We will describe them more in detaillater on; (iii) The fields pingTimer and pingTimeouts areset to zero with the instructions get_local_0, pushbyte,pushnull and initproperty. Again, Name structures areimportant to define which package and class the fieldsbelong to.

From the description above, it is clear that Name struc-tures play a very important role at defining which classesand methods are invoked by the ActionScript code. Wenow provide an insight into such structures.

3.1 NamesNames are data structures composed by one unqualified name(for example, a class name) and one or more namespacesthat typically represent the packages from which classes ormethods are resolved. The Name resolution can occur eitherat compile time or at runtime. Additionally, there mightbe multiple namespaces from which the same unqualified

3. There might be multiple ways to express the disassembled code asActionScript source code.

1 //ActionScript decompiled code2 public function PingServers()3 4 super();5 ExternalInterface.addCallback(Cmd.RESPOND_PINGDATA,this

.pingCallback);6 this.pingTimer = 0;7 this.pingTimeouts = null;8 9

1011 //ActionScript equivalent bytecode12 getlocal_013 pushscope14 getlocal_015 constructsuper 016 findpropstrict Qname(PackageNamespace("flash.external"),"

ExternalInterface")17 getproperty Qname(PackageNamespace("flash.external"),"

ExternalInterface")18 findpropstrict Qname(PackageNamespace("com.xvm.io"),"Cmd")19 getproperty Qname(PackageNamespace("com.xvm.io"),"Cmd")20 getproperty Qname(PackageNamespace(""),"RESPOND_PINGDATA")21 getlocal_022 getproperty Qname(PrivateNamespace("xvm.ping.PingServers:

PingServers"),"pingCallback")23 callproperty Qname(PackageNamespace(""),"addCallback") 224 pop25 getlocal_026 pushbyte 027 initproperty Qname(PrivateNamespace("xvm.ping.PingServers:

PingServers"),"pingTimer")28 getlocal_029 pushnull30 initproperty Qname(PrivateNamespace("xvm.ping.PingServers:

PingServers"),"pingTimeouts")31 returnvoid

Listing 1: An example of how Actionscript code (lines 1-8) is compiled into ABC Bytecode (lines 11-31). Debuginformation have been removed for the sake of brevity.

name can be obtained. The choice of the appropriate Names-pace from a list of candidates typically occurs at runtime.

According to the number of namespaces, Names can besubdivided in two categories:

QName. A data structure composed by one unqualifiedname and one namespace. This is the simplest Name struc-ture that can be found. Both the name and the namespaceare resolved at compile time.

Multiname. A data structure composed by one unqualifiedname and a set of namespaces. The resolution is performedat compile time. Hence, each of the namespaces in the setcan be properly used for the name.

When a name is retrieved at runtime, it is typicallyloaded from the constant pool when it is needed.

3.2 SWF MalwareIn order to better understand the approach proposedin this paper, Listing 2 shows a typical action per-formed by an ActionScript-based malware (MD5:92711a995ecdd0663b608bd664b2ca89), which exploitsthe CVE-2015-3133 (according to VirusTotal) vulner-ability, which allows to remotely execute arbitrary codedue to memory corruption. As we did in the first partof Section 3, we describe the aforementioned actions byshowing both the decompiled ActionScript source andthe disassembled bytecode. The code in the Listing reads anUnsignedByte from the object _loc1_, which is an objectof the class IG (note: this name was originally obfuscated

4

12 //Decompiled ActionScript3 ...4 _loc1_ = new IG();5 _loc1_.endian = Endian.LITTLE_ENDIAN;6 _loc1_.position = 0;7 this.isAS3 = _loc1_.readUnsignedByte() - 1;89 //Disassembled Bytecode

10 ...11 findpropstrict Qname(PackageNamespace(""),"IG")12 constructprop Qname(PackageNamespace(""),"IG") 013 coerce Qname(PackageNamespace("flash.utils"),"ByteArray")14 setlocal_115 getlocal_116 getlex Qname(PackageNamespace("flash.utils"),"Endian")17 getproperty Qname(PackageNamespace(""),"LITTLE_ENDIAN")18 setproperty Qname(PackageNamespace(""),"endian")19 getlocal_120 pushbyte 021 setproperty Qname(PackageNamespace(""),"position")22 getlocal_023 getlocal_124 callproperty Qname(PackageNamespace(""),"readUnsignedByte")

025 decrement

Listing 2: Part of the malicious code contained in the92711a995ecdd0663b608bd664b2ca89 sample. Suchfunction is represented by its decompiled output (lines 4-7) and by its equivalent bytecode output (lines 11-25).

with non ASCII characters). Such class inherits (see thecoerce instruction) from the flash.utils.ByteArraybuilt-in class. The code then performs a subtraction andassigns the output to the variable isAS3. Such value willbe then copied to another array of bytes (we did not re-port this action for space reasons). Note how the readingis performed by following the little endian (by using theflash.utils.Endian) byte order. From the name of thedecompiled variable, we assume that the byte sequencesmight be related to a code that is copied somewhere else.Manipulating information in this way is very commonin Flash-based malware. It is interesting to observe howsystem API methods and classes are essential for the at-tacker to build shellcodes or to perform buffer overflowsor heap spraying attacks. This often happens, as the officialActionScript API allows to easily manage low-level datastructures.

4 FLASHBUSTER ARCHITECTURE

FlashBuster is a static, machine learning-based systemwhose goal is to detect malicious SWF files and to distin-guish them from benign ones. This is achieved by lever-aging information provided by the tag structure and theActionScript bytecode of the file. Our goal was in partic-ular to reproduce a combination of the main characteristicsof previous state-of-the-art systems (see Section 8), whichproved to be effective at detecting SWF malware.

Figure 1 shows the general architecture of the system,which can be divided in three modules:

Parser. This module analyzes the SWF file and extracts infor-mation about its structure and its ActionScript bytecode.

Feature Extractor. This module transforms the informationobtained from the parser in a vector of numbers, whichcharacterizes the whole SWF file.

Preprocessing

Structural Features

Bytecode Features

ClassifierBenign/Malicious[x1,…xn]Tag/Code

Fig. 1: Graphical architecture of FlashBuster.

Classifier. This module decides on the maliciousness ofthe SWF file basing on the feature vector it receives asinput. Such module is a mathematical function that tunes itsparameters by receiving a number of examples taken from aso-called training set. Once its parameters have been set up,the classifier is able to recognize malicious files that havenot been included in the training examples.

In the following, we provide a more detailed descriptionof each component of the system.

4.1 Parser

As previously said, this module performs data preprocess-ing and selects the information that will be further processedby the other modules. FlashBuster leverages a modifiedversion of JPEXS, a powerful, Java-based Flash disassem-bler and decompiler.4 This software is based on RABCDasm,one of the most popular Flash disassemblers, and it addsnew features such as de-obfuscation, file debugging andpreview, etc.

In particular, the parser featured by FlashBuster per-forms the following operations: (i) It performs static de-obfuscation of ActionScript code. This is important, assome of the malicious files might use name obfuscationor other popular techniques to conceal the attacks. (ii) Itextracts the complete SWF structure in terms of tags. (iii)It disassembles the ABC bytecode so that it could be readas a plain-text file. Such operation includes automatic de-obfuscation of the ActionScript code. Both the tag struc-ture and the ABC bytecode are sent to the feature extractormodule for further analysis.

4.2 Feature Extraction

This module is the core of the whole system. It converts theinformation extracted by the parser to a vector of numbersthat will be sent to the classifier. As the information is relatedboth to the structure and the content of the SWF file, wenow provide a detailed description of the features extractedin the two cases.

4.2.1 Structural Features (Tags)

These features are related to the information that can beextracted from the SWF tags, and are crucial to understandwhich objects and actions are executed by the SWF file. Themain idea here is that malware does not contain particularlycomplex multimedia contents, such as video with a lot offrames or audio files. A lot of malware samples simplydisplay images such as rectangles or blank backgrounds. Forthis reason, we extract the following 14 features from the

4. https://www.free-decompiler.com/flash/download/

5

file structure, corresponding to the number of occurrencesof specific SWF tags within the file:

Frames. this feature counts the tag ShowFrame that is usedto display frames.

Shapes. this feature counts the tag DefineShape (in any ofall its four variants), used to define new shapes that will beplot on the screen.

Sounds. this feature checks the presence of sound-related events by counting any of the following tags:DefineSound, SoundStreamHead1, SoundStreamH-ead-2 and SoundStreamBlock.

BinaryData. this feature counts groups of embedded data,represented by the tag DefineBinaryData.

Scripts. this feature counts how many ActionScriptcodes are contained in the file. A SWF file does not neces-sarily require such code to perform its operations, especiallyin benign files (ActionScript has been initially thoughtas an aid to the execution of SWF files). This is doneby analyzing the following tags: DoABC, DoABCDefine,DoInitAction, DoAction.

Fonts. this feature counts font-related objects, by detectingany of the following tags: DefineFont (in all its variants),DefineCompactedFont, DefineFontInfo (in all its vari-ants), DefineFontName.

Sprites. this feature counts the number of sprites by exam-ining the tag DefineSprite.

MorphShapes. this feature counts the number of morphedshapes, (i.e., shapes that might transform into new ones) byexamining the tag DefineMorphShape (and its variants).

Texts. this feature counts text-related objects by checkingany of the following tags: DefineText (along with itsvariants) and DefineEditText.

Images. this feature counts the images contained in thefile by examining any of the following tags (and theirvariants): DefineBits, DefineBitsJPEG, JPEGTablesand DefineBitsLossless.

Videos. this feature counts the number of embeddedvideos by examining the tags DefineVideoStream andVideoFrame.

Buttons. this feature counts the buttons with which theuser can interact with the SWF content. This is done byexamining the tag DefineButton (along with its variants).

Errors. this feature counts the errors made by the parserwhen analzying specific tags. This often happens, for exam-ple, when the SWF file is malformed due to errors in itscompression.

Unknown. this feature counts the tags that do not belong tothe SWF specifications (probably malformed tags).

The reader can find more information about these tagson the official SWF specification [3].

Despite being effective, structural features must be care-fully treated, as benign and malicious files can be similar toeach other in terms of their tag structure. For this reason,structural features alone are not enough to ensure a reliabledetection, and must be integrated with information from thescripted content of the file.

4.2.2 Actionscript Bytecode Features (API calls)As structural features (i.e., tags) might suffer from the limi-tations mentioned in Sect. 4.2.1, we employed an additionalset of features that focus on the content of the scripting codethat might be included in the file. Although it is not strictlynecessary to use ActionScript for benign operations, itsrole is essential to execute attacks. In particular, as shownin Sect. 3.2, the attacker usually needs to resort to systemAPIs to perform memory manipulation or to trigger specificevents. Moreover, APIs can be used to communicate withexternal interfaces or to contact an external URL to automat-ically drop malicious content on the victim’s system.

System APIs belong to the official AdobeActionScript specifications [1]. For this reason, wecreated an additional feature set that checks the presenceor absence of all the classes and methods belonging tosuch specifications. This leads to 4724 new features. Morespecifically, this feature set represents the number of specificSystem methods and classes inside the bytecode. We choseto use only system-based APIs for two reasons: (i) thefeature vector does not include user-defined APIs, so thatthe feature list is independent on the training data that isconsidered for the analysis; (ii) system-based calls are moredifficult to obfuscate, as they are not directly implementedby the user.

With respect to the example described in Sect. 3.2,we therefore consider as features the classes flash.utils.ByteArray and flash.utils.Endian, and themethod readUnsignedByte. On the contrary, we do notconsider the class IG, as it was directly implemented bythe user. The rationale behind counting the occurrencesof system-based methods and classes is that an attackermight systematically use functions such as readByte orwriteByte to manipulate the memory. Alternatively, shemight attempt to repeatedly trigger events or to accessspecific interfaces. Counting the occurrences might alsoincrease the required effort by the attacker to evade theclassification algorithm.

4.3 ClassificationThe features extracted with FlashBuster can be usedwith different classification algorithms. In the experimentalevaluation we describe in Section 7, we tested different clas-sification algorithms. In particular, we focused our attentionon SVM and Random Forests, as these were successfullyemployed in several other malware detection tasks [27], [34].

5 ATTACK MODEL AND SCENARIOS

To assess the security of FlashBuster against adversarialmanipulation of the input data (which can be either per-formed at training time or at test time), we leverage anattack model originally defined in the area of adversarialmachine learning [10], [11]. It builds on the well-knowntaxonomy of Barreno et al. [6], [7], [19] which categorizespotential attacks against machine-learning algorithms alongthree axes: security violation, attack specificity and attack influ-ence. By exploiting this taxonomy, the attack model enablesdefining a number of potential attack scenarios, in terms ofexplicit assumptions on the attacker’s goal, knowledge ofthe system, and capability of manipulating the input data.

6

5.1 Attacker’s Goal

It is defined in terms of two characteristics, i.e., securityviolation and attack specificity.

Security violation. In security engineering, a system can beviolated by compromising its integrity, availability, or privacy.Violating the integrity of FlashBuster amounts to havingmalware samples undetected; its availability is compromisedif it misclassifies benign samples as malware, causing adenial of service to legitimate users; and privacy is violatedif it leaks confidential information about its users.

Attack specificity. The specificity of the attack can be targetedor indiscriminate, based on whether the attacker aims to haveonly specific samples misclassified (e.g., a specific malwaresample to infect a particular device or user), or if anymisclassified sample meets her goal (e.g., if the goal is tolaunch an indiscriminate attack campaign).

We formalize the attacker’s goal here in terms of anobjective function W(A′,θ) ∈ R which evaluates to whichextent the manipulated attack samples A′ meet the at-tacker’s goal.

5.2 Attacker’s Knowledge

The attacker may have different levels of knowledge of thetargeted system [6], [7], [10], [11], [19], [32]. In particular,she may know completely, partially, or do not have anyinformation at all about: (i) the training data D; (ii) thefeature set X , i.e., how input data is mapped onto a vectorof feature values; (iii) the learning algorithm L(D, f), andits decision function f(x), including its (trained) parameters(e.g., feature weights and bias in linear classifiers), if any. Insome applications, the attacker may also exploit feedbackon the classifier’s decisions to improve her knowledge ofthe system, and, more generally, her attack strategy [7], [10],[11], [19].

The attacker’s knowledge can be represented in terms ofa space Θ that encodes knowledge of the data D, the featurespace X , the learning algorithm L(D, f) and its decisionfunction f . In this work, we will simulate both limited- andperfect-knowledge attacks, as detailed below.

5.2.1 Limited-Knowledge (LK) Black-Box AttacksUnder this scenario, the attacker is typically only assumedto know the feature representation X and the learning algo-rithmL, but not the training dataD and the trained classifierf . This is a common assumption under the security-by-design paradigm: the goal is to show that the system maybe reasonably secure even if the attacker knows how itworks but does not know any detail on the specific deployedinstance [7], [8], [10], [11], [15], [19].

In particular, according to the definition proposed byBiggio et al., we distinguish the cases in which either thetraining data or the trained classifier are unknown [23]. Inthe first case, to which we refer as LK attacks with surrogatedata,it is often assumed that the attacker is able to collecta surrogate dataset D and that she can learn a surrogateclassifier f on D to approximate the true f [8], [24]. Notealso that the class labels of D can be modified using feedbackprovided from the targeted classifier f , when available (e.g.,as an online service providing class labels to the input data).

The knowledge-parameter vector can be thus encoded asθLK−SD = (D,X ,L, f).

In the second case, to which we refer to as LK attackswith surrogate learners, we assume that the attacker knowsthe training distribution D, but not the learning model.Hence, she trains a surrogate function on the same train-ing data. Hence, the knowledge-parameter vector can beencoded as θLK−SL = (D,X , L, f).

5.2.2 Perfect-Knowledge (PK) White-Box Attacks

This is the worst-case setting in which also the targetedclassifier is fully known to the attacker, i.e., θ = (D,X ,L, f).Although it is not very likely to happen in practice that theattacker gets to know even the trained classifier’s param-eters, this white-box setting is particularly interesting as itprovides an upper bound on the performance degradationincurred by the system under attack, and can be used asreference to evaluate the effectiveness of the system againstthe other (less pessimistic) attack scenarios.

5.3 Attacker’s Capability

The attacker’s capability of manipulating the input data isdefined in terms of the so-called attack influence and on thebasis of some application-specific constraints.

Attack Influence. This defines whether the attacker can onlymanipulate data at test time (exploratory influence), or if shecan also contaminate the training data (causative influence).This is possible, for instance, if the system is retrainedonline using data collected during operation which can bemanipulated by the attacker [7], [11], [19].

Application-specific constraints. These constraints definehow and to which extent the input data (and its features)can be modified to reach the attacker’s goal, according to thegiven application. In many cases, these constraints can bedirectly encoded in terms of distances in feature space, com-puted between the source malware data and its manipulatedversions [8], [13], [14], [16], [21], [29]. FlashBuster is notan exception to this rule, as we will discuss in the remainderof this section. In general, the attacker’s capability can thusbe represented in terms of a set of possible modificationsΩ(A) performed on the input samples A.

5.4 Attack Strategy

The attack strategy amounts to formalizing the derivationof the attack in terms of an optimization problem [8],[11]. Given the attacker’s goal W(A′,θ), along with aknowledge-parameter vector θ ∈ Θ and a set of manipu-lated attacks A′ ∈ Ω(A), the attack strategy is given as:

A? = arg maxA′∈Ω(A) W(A′;θ) . (1)

Under this formulation, one can characterize different attackscenarios. The two main ones often considered in adver-sarial machine learning are referred to as classifier evasionand poisoning [6]–[8], [10]–[12], [19], [22], [23], [35]. In theremainder of this work we focus on classifier evasion, whilewe refer the reader to [11], [22], [23], [35] for further detailson classifier poisoning.

7

5.5 Evasion AttacksEvasion attacks consist of manipulating malicious samplesat test time to have them misclassified as benign by a trainedclassifier. The attacker’s goal is thus to violate system in-tegrity, either with a targeted or with an indiscriminate attack,depending on whether the attacker is targeting a specificmachine or running an indiscriminate attack campaign.More formally, evasion attacks can be written in terms ofthe following optimization problem:

z? = arg minz′∈Ω(z)

f(Φ(z′)) , (2)

where x′ = Φ(z′) is the feature vector associated to themodified attack sample z′, x = Φ(z) is the feature vectorassociated to the source (unmodified) malware sample z,Φ is the feature extraction function, and f is the surrogateclassifier estimated by the attacker. With respect to Eq. (1),note that here samples can be optimized one at a time, asthey can be independently modified.

As in previous work [8], [11], [15], we first simulate theattack at the feature level, i.e., we directly manipulate thefeature values of malicious samples without constructingthe corresponding real-world samples while running the at-tack. We discuss in Sect. 5.7 how to create the correspondingreal-world evasive malware samples. The above problemcan be thus simplified as:

x∗ = arg minx′ f(x′) (3)s.t. ‖x′ − x‖1 ≤ k , xlb x′ xub , (4)

where we have also made the manipulation constraintsΩ used to attack FlashBuster explicit. In particular, thebox constraint xlb x′ xub (in which the inequalityholds for each element of the vector) bounds the minimumand maximum feature values for the attack sample x′. ForFlashBuster, we will only consider feature injection, i.e.,we will only allow injection of structural and bytecodefeatures within the SWF file to avoid compromising theintrusive functionality of the malware samples. This can besimply accounted for by setting xlb = x. The additional `1distance constraint ‖x′ − x‖1 ≤ k thus sets the maximumnumber k of structural and bytecode features (i.e., tags andAPI calls) that can be injected into the file. The solutionto the above optimization problem amounts to identifyingwhich features should be modified to maximally decreasethe value of the classification function, i.e., to maximize theprobability of evading detection [8], [11]. Clearly, this set offeatures varies depending on the input sample x.

5.6 Evasion Attack AlgorithmIf the objective function (i.e., the decision function of theclassifier) f is not linear, as for kernelized SVMs and randomforests, Problem (3)-(4) corresponds to a nonlinear pro-gramming problem with linear constraints. The solution istherefore typically found at a local minimum of the objectivefunction. Problem (3)-(4) can be solved with standard algo-rithms, but this is not typically very efficient, as such solversdo not exploit specific knowledge about the evasion prob-lem. We thus devise an ad-hoc solver based on exploring adescent direction aligned with the gradient ∇f(x′) usinga bisect line search, similar to that used in our previous

Algorithm 1 Evasion Attack

Input: x, the malicious sample; x(0), the initial location ofthe attack sample; f , the surrogate classifier (Eq. 3); k,the maximum number of injected structural and byte-code features (Eq. 4); xlb and xub, the box constraintbounds (Eq. 4); ε, a small positive constant.

Output: x′, the evasion attack sample.1: i← 02: repeat3: i← i+ 14: t′ = arg mint f(Π(x(i−1) − t∇f(x(i−1))))5: x(i) ← Π(x(i−1) − t′∇f(x(i−1)))6: until |f(x(i))− f(x(i−1))| < ε7: return x(i)

work [26]. Its basic structure is given as Algorithm 1. Tominimize the number of iterations, we explore one feature ata time (starting from the most promising feature, i.e., the oneexhibiting the highest gradient variation in absolute value),leveraging the fact that the solution will be sparse (as theproblem is `1 constrained). We also minimize the number ofgradient and function evaluations to further speed up ourevasion algorithm; e.g., we only re-compute the gradient off(x) when no better point is found on the direction underexploration. Finally, we initialize x(0) twice (first startingfrom x, and then from a benign sample projected onto thefeasible domain), to mitigate the problem of ending up in alocal minima that does not evade detection.5

5.7 Constructing Adversarial Malware Examples

A common problem when performing adversarial attacksagainst machine learning is evaluating whether they can betruly performed in practice. As gradient-descent attacks areperformed at the feature level, the attacker is then supposedto solve the so-called inverse feature-mapping problem, i.e., toreconstruct from the obtained features the sample that canbe deployed against the classifier itself [11], [15], [19].

Most of times, such operation is not easy to perform,not only from a more theoretical standpoint (as discussedin [19]), but also from a more practical perspective. In thespecific case of Flash malware, and of malware in general,generating the corresponding real-world adversarial exam-ples may be complicated, as a single wrong operation cancompromise the intrusive functionality of the embeddedexploitation code [15]. For example, removing one structuralfeature such as one frame or script might totally break theSWF file. This is why we consider only injection of additionalcontent into the SWF file.

Constructing the real-world adversarial example (i.e., themalicious evasive SWF file) is a rather straight-forward pro-cess in our case. In particular, it is possible to inject structuralfeatures by using JPEXS. With its graphical interface, it ispossible to deliberately add control and structural tags. Itis also possible to inject bytecode features by editing the

5. This problem has been first pointed out in [8], where the authorshave introduced a mimicry term to overcome it. Here we just considera different initialization mechanism, which allows us to get rid of thecomplicated mimicry term in the objective function.

8

output disassembled by RABCDasm and re-assembling thefile (this can be also graphically done with JPEXS).

For the purposes of this paper, we created some workingproof-of-concepts, where structural and content-based fea-tures were deliberately and manually added by using theaforementioned tools. However, we plan as future work tomake the creation process automatic.

6 ON THE VULNERABILITY OF FEATURE SPACESAND LEARNING ALGORITHMS

We discuss here an interesting aspect related to the vulner-ability of learning-based systems, first highlighted in [9],[26] and conceptually represented in Fig. 2, which showstwo classifiers on a two-feature space. The classifiers canbe defined as surfaces closed around malicious (left) andbenign (right) samples. The red, blue and green samplesrepresent, respectively, malicious, benign and attack sam-ples. An evasion attack sample is typically misclassified aseither: (i) its feature vector is far enough from those belongingto the rest of known training samples (both malicious andbenign), or (ii) it is indistinguishable from those exhibitedby benign data. In the former case, usually referred to asblind-spot evasion, retraining the classifier on the adversar-ial examples (with adversarial training) should successfullyenable their detection, improving classifier security. Thismeans that the classification error induced by such attackscould have been reduced in advance, by designing a learningalgorithm capable of anticipating this threat; for instance,building a classifier that better encloses benign data, andclassifies as malicious the regions of the feature space wheretraining data is absent or scarce (see, e.g., the classifier inthe right plot of Fig. 2). We thus refer to this vulnerabilityas a vulnerability induced by the learning algorithm (left plotin Fig. 2). In the latter case, instead, retraining the classifierwould be useless, as the whole distribution of the evasionsamples is overlapped with that of benign data in featurespace, i.e., the attack increases the Bayesian (non-reducible)error. We thus refer to this attack as mimicry evasion, and tothe corresponding vulnerability as a vulnerability inducedby the feature representation (right plot in Fig. 2). In fact, if amalware sample can be modified to exhibit the same featurevalues of benign data, it means that the given features areintrinsically weak, and no secure learning algorithm canprevent this issue.

This notion can also be motivated in formal terms,similarly to the risk analysis reported in [9]. From aBayesian perspective, learning algorithms assume an un-derlying (though unknown) distribution p(x, y) govern-ing the generation of benign (y = −1) and malicious(y = +1) data, and aim to minimize the classificationerror E(f) = E(x,y)∼p`(y, f(x)), where E is the expectationoperator, ` is the zero-one loss, and f is the classificationfunction returning the predicted class label (i.e., ±1). Letus denote the optimal classifier achieving the minimum(Bayesian) error on p with f?. It is clear that, if there is noevidence p(x) of (training) data in some regions of the featurespace (usually referred to as blind spots), such regions canbe arbitrarily classified by f? as either benign or maliciouswith no impact on the classification error (the expectationon p will be anyway zero in those regions). This is precisely

blind-spotevasion(learningvulnerability)

mimicryevasion(featurevulnerability)

Fig. 2: Conceptual representation of learning (left) and feature(right) vulnerability. Red, blue and green samples represent,respectively, malicious, benign and attack samples.

the underlying reason behind the vulnerability of learningalgorithms to blind-spot evasion.

Within this setting, evasion attacks can be indeedthought as a manipulation of the input samples x througha function a(x), which essentially introduces a deviationfrom the source distribution p(x, y). By denoting withEa(f) = E(x,y)∼p`(y, f(a(x))) the error of the classifier fon the manipulated samples, with f ′ the optimal (Bayesian)classifier on such manipulated data, we can compute theincrease in the classification error of f? on the manipulateddata as:

Ea(f?)− E(f?) = Ea(f ′)− E(f?)︸ ︷︷ ︸feature vulnerability

+ Ea(f?)− Ea(f ′)︸ ︷︷ ︸learning vulnerability

.

(5)The first term is the increase in Bayesian error before andafter the attack (which characterizes the vulnerability ofthe feature representation), while the second represents theclassification error reducible by retraining on the attacksamples (i.e., the vulnerability of the learning algorithm).

Under this interpretation, we can introduce a metric toquantitatively assess the feature vulnerability. To this end,we first consider the so-called Bhattacharyya Coefficient(BC):

BC =

∫x∈X

√pb(x)pm(x)dx ∈ 0, 1 . (6)

This coefficient essentially evaluates the overlapping be-tween the distributions of benign pb and manipulated attackpm samples over the whole feature space X . If the twodistributions are exactly the same, BC = 1, while if theyare perfectly separated, BC = 0. The convenient aspect ofthis metric is that it has a closed form for several knowndistributions; e.g., in the case of multivariate Gaussian dis-tributions, it is given as BC = exp(−DB), where

DB =1

8(µb − µm)>Σ−1(µb − µm) +

1

2log

detΣ√detΣbdetΣm

,

Σ = 0.5(Σb + Σm), and µb, µm, Σb and Σm are themeans and covariance matrices of benign and attack data,respectively. To assess feature vulnerability, we use thisexpression for BC, and exploit the well-known result thatthe Bayesian error is upper bounded by 1

2BC. Accordingly,we measure the difference between such value computedafter and before the attack, which gives us an (approximate)indication of the increase in the Bayesian error induced by

9

the attack, and thus, a quantitative measure of the featurevulnerability (i.e., of the first term in Eq. 5).

7 EXPERIMENTAL EVALUATION

The experimental evaluation proposed in this paper is di-vided in three parts.

Standard Evaluation. FlashBuster was trained with adataset of randomly chosen malicious and benign SWF files,and it was tested against a number of previously unseenmalicious and benign files. This experiment provides infor-mation on the system general performances in terms of trueand false positives.

Adversarial Evaluation. In this experiment (directly linkedto the previous one), we evaluated the performances ofFlashBuster against adversarial attacks performed ac-cording to a gradient descent strategy (see Section 5.5).

Temporal Evaluation. FlashBuster was trained with adataset of samples that had been first seen before a certainyear, and it was tested against a set of samples that hadbeen released after the same year. This test was performedto ensure the capability of FlashBuster to predict novelvulnerabilities and attacks.

In the following, we describe the dataset we employedfor our experiments, as well as the basic setup of thepreprocessing and feature extractor modules. This setup iscommon to all the evaluations described in this Section.

7.1 Basic SetupDataset. The dataset used for our experiments is composedof 6635 files, 2209 of which are malicious and 4426 arebenign. Notably, every analyzed file (including benign ones)contains ActionScript 3 code. This is to avoid analyz-ing files that do not contain ActionScript code, andthat are therefore most likely benign. The malicious files,as well as part the benign ones, were retrieved from theVirusTotal service.6 Other benign files were retrievedfrom the DigitalCorpora repository.7

Preprocessing. As mentioned in Section 4.1, the originalJPEXS parser was modified to allow a faster analysis ofmultiple SWF files, as well as a better integration withthe other components of FlashBuster. All data relatedto tags and bytecodes are extracted and dumped to files,in order to allow for subsequent analyses by the othermodules of FlashBuster. The extraction time may varyfrom milliseconds to some minutes for very large files.Feature Extraction. As we are counting the occurrence ofeach feature, there can be a considerably high difference invalues between certain features and others. Considering thepossibility of adversarial attacks, this might give the attackermore degrees of freedom to confuse the classifier. For thisreason, we adopted a feature normalization and selectionstrategy that is composed of three stages:

(i) We established an upper limit for the feature valuesin our dataset. For our experiments, we chose 10 as a rea-sonable value that limits the number of injectable featuresin a specific sample (in particular, the maximum amount is

6. https://www.virustotal.com7. http://digitalcorpora.org/corpora/govdocs

given by nf ∗ vmax), where nf is the number of employedfeatures and vmax is the feature maximum value. Withoutlimiting the feature values, it would be possible to generatesamples with anomalous values of specific features, whichwould be rather difficult to be found in practice. Moreover,performing perfect knowledge attacks without limiting thevalues of the features is computationally very expensive.

To confirm that limiting the feature values does not in-fluence classification performances, we repeated our experi-ments with higher upper limits, without noticing significantdifferences in performances.

(ii) We selected the 50 most occurring features in thedataset. Such threshold has been chosen as it was the min-imum value that did not affect classification performancesin our experiments. In particular, we extracted the featureswith the highest score S, given by the following:

S = |p(xi|y = 1)− p(xi|y = −1)| (7)

where xi is the i-feature, whilst 1 and −1 are the labelsrelated to, respectively, malicious and benign file categories.Each feature is taken only once.

(iii) All features were normalized with the popular tf-idfstrategy [5]. This was particularly crucial for SVM classifiers,which perform best with normalized features.Classification. We used the popular machine learning suiteScikit-Learn8, which features the classifiers and the nor-malization strategy we used in our evaluation.Training Procedure and Classifiers. All the performedevaluations (except the temporal one, which was carriedout on a completely different dataset) share the followingelements: (a) The dataset was randomly split by considering50% of it as training set and the remaining 50% as testset. All the classifier parameters were evaluated with a 5-fold cross validation performed on the training set. Werepeated the whole procedure five times, in order to ruleout possible biases related to specific train/test divisions.(b) We performed our tests on three classifiers: (i) RandomForest; (ii) SVM with linear kernel; (iii) SVM with non-linear (RBF) kernel. Additionally, considering the possi-bility of the targeted attacks described in Section 5.5, weretrained the SVM RBF and Random Forest classifiers byadding to the original training set samples generated witha gradient descent strategy. In particular, the training splitswere modified in the following way: for each training split,we generated new attack samples by performing, on 1000randomly chosen malicious training samples, 50, 100 and150 changes. In total, 3000 samples were added to eachoriginal training set split. Notably, adding too many featuresto the samples would have led to the creation of anomaloussamples with unrealistic feature values.

7.2 Standard EvaluationThe standard evaluation was performed by following thecriteria described in Section 7.1. In particular, for each clas-sifier we calculated the average (among its splits) ReceivingOperating Characteristic (ROC). Figure 3 shows the resultsof the evaluation. From this experiment, we see that Ran-dom Forest was the classifier that generally performed best

8. http://scikit-learn.org/stable/

10

0.5 1 2 5 10 20 50 100False Positive Rate (%)

40

60

80

100Tru

e P

osi

tive R

ate

(%

)

RFRF (Adv.)SVM LinearSVM RBFSVM RBF (Adv.)

Fig. 3: Average ROC curves obtained on 5 train-test splits.The test was performed on three classifiers. In particular,we also report the results for the retrained variant of SVMRBF and Random Forests (RF). Vertical bars represent thestandard deviation among the splits.

at detecting malicious SWF files. At 1% of false positives,the classifier was able to detect more than 80% of threats,whilst at 2% the number of detected threats was higher than98%. SVM with RBF kernel exhibited similar performances.Linear models, on the contrary, poorly performed under 2%.

It may be expected that, being the generated attacksmore similar to benign samples, adding them to the trainingset would decrease the performances of the classifiers. How-ever, in our case, such operation only barely affected theexamined classifiers. We interpret this result with the factthat gradient descent attacks create, in this case, malicioussamples that are still considerably different to the originalbenign training distribution. For this reason, the classifiersare still able to correctly discriminate malicious and benigntest samples.

7.3 Adversarial Evaluation

The adversarial evaluation aimed to assess the performanceof the classifiers employed in the previous experiment afterthe gradient descent attacks described in Section 5.5. In thiscase, we evaluated the performances of the classifiers interms of true positives (at 5% false positive rate) for a certainnumber of changes k performed to the feature vector. Ofcourse, the more changes are performed by the algorithm,the more effective the attack is.

It is important to observe that the experiments on SVMsand Random Forests were carried out under different lev-els of knowledge. Notably, Random Forests have a non-differentiable classification function, and for this reasonthe attacker cannot directly perform evasion on the targetmodel. In this case, according to the taxonomy described inSection 5.5, we performed a Limited-Knowledge (LK) attackwith surrogate learners, in which a differentiable surrogatemodel L is employed to craft the adversarial examplesinstead of the target algorithm. To this end, in our ex-periments, we used an SVM with the RBF kernel (whoseparameters were always evaluated by means of a 5-foldcross validation performed on the training set), trained onthe same data D as the target classifier. Conversely, when

the attack is directly performed against the SVM classifier,we simulated a Perfect-Knowledge (PK) attack scenario.

Figure 4 provides the results of the evaluation. Whileall SVM-based classifiers were completely evaded after 100changes, Random Forests could still detect 40% of the at-tacks. This may be however due to the fact that the surrogatemodel (SVM RBF) does not properly approximate the clas-sification function of a Random Forest - i.e., the adversarialexamples crafted against the SVM do not transfer properlyagainst Random Forests. For this reason, we can not statewith certainty that Random Forests are generally more secure(a more powerful white-box attack as that in [20] may enableevading them with higher probability). We leave a moredetailed investigation of this aspect to future work.

Inefficacy of adversarial retraining. Notably, the employedretraining strategy only brings little improvement to therobustness of the classifiers. This can be better explainedby observing the distributions of average feature values forone malicious test set before and after the attack againstSVM RBF, as well as the variation of the related Bhat-tacharyya distance BC among the distributions (describedin Section 6). It is worth noting that such value has beencalculated under specific assumptions: (i) we approximatedour distributions as multivariate Gaussians; (ii) the isotropiccovariance Σ = 1/σ2I is identical for the two classes (beingI the identity matrix).

Figure 5 shows that the BC value increases accordingto two criteria: (i) the increment of the number of changesk; (ii) performing the attack against a retrained classifier.This could further be explained by pointing out the dif-ference between the Bayesian upper bound errors in caseof attack and without the attack (that we define here asmimicry parameter) m = 1

2BCatt − 12BC , where BCatt is

the Bhattacharyya distance calculated after the attack, andBC is the one calculated before the attack. Table 1 showshow this value increases as more changes are performed,and as classifiers are retrained.

TABLE 1: Values of the mimicry parameter m under multi-ple attack scenarios.

Attack mk = 100 (Retrain) .131k = 50 (Retrain) .081k = 100 .076k = 50 .045

This shows that, although the function was retrainedagainst the attacks to reduce the learning vulnerability, thefeature vulnerability related to the employed static featurescannot be reduced by simply retraining the classifier. Inmore detail, the problem here is that none of the employedstatic features is likely to appear in malware much more frequentlythan in benign data, i.e., there is no invariant feature thatcharacterizes malware uniquely from benign data (and thatcan not be removed) [30]. This in turn means that it ispossible to create a malicious sample that is indistinguishablefrom benign ones (even by only injecting content to theinput SWF file) and, thus, additional features are requiredto correctly detect adversarial SWF malware examples.

11

1 10 100k

0.0

0.2

0.4

0.6

0.8

1.0TP@

FP (

5%

)

SVM Linear

SVM RBF

SVM RBF (Adv.)

(a) PK

1 10 100k

0.0

0.2

0.4

0.6

0.8

1.0

TP@

FP (

5%

)

RF

RF (Adv.)

(b) LK

Fig. 4: Detection rate (at false positive rate = 5%) of FlashBuster for SVM (under PK) and Random Forests (under LK)classifiers against gradient descent attacks. Results are averaged on 5 runs. SVM RBF and Random Forest have been alsoretrained with 1000 samples modified with the gradient descent strategy with k = 50, 100 and 150.

0

2

4

6

8

10Benign

0

2

4

6

8

10k=0, BC=0.196

0

2

4

6

8

10k=50, BC=0.286

0

2

4

6

8

10k=100, BC=0.348

0

2

4

6

8

10k=50, BC=0.358

0

2

4

6

8

10k=100, BC=0.458

Fig. 5: Average feature values for one test set in which malicious samples (red) were modified with a gradient descentattack (k is the number of changes) against SVM RBF (purple). The same attacks have been repeated against a retrainedSVM RBF (green). The Bhattacharyya coefficient BC between the malicious distributions and the one related to benign(blue) samples is reported. Note how BC increases when attacking the retrained classifier, meaning that the maliciousdistribution slowly converges towards benign samples.

7.4 Temporal Evaluation

In this evaluation, we aimed to evaluate the capability ofFlashBuster to predict novel and zero-day attacks. To thisend, we trained the system by only using samples whosefirst submission date to the VirusTotal service was before2016 (1422 samples), plus all the benign files in the dataset.The test set was therefore made of those malicious samplesreleased in 2016 (787 samples). We used all the classifiersof the previous experiments. The goal of this analysis wasnot to assess the performances of the system at detectinggeneral or adversarial attacks, but to evaluate the predictivepower of the features we employed against novel attacks. Asthe previous experiments, the parameters of the classifiers

were evaluated with a 5-fold cross validation performed onthe training set.

Table 2 shows the results obtained from this evaluation.It is possible to see that, considering non-retrained models,the linear one performed best at detecting novel samples.In particular, more than 77% of the tested samples weredetected. However, non-linear models also exhibited goodperformances, with only a decrement of 8− 12% in compar-ison to the linear model.

Notably, retrained models perform better than their non-retrained variants. In particular, retrained SVM RBF exhibitsa 8% increment in comparison to the Linear model. Thismeans that generating artificial attacks with the gradient

12

descent strategy makes the classifier more robust againstvariants of known attacks. In particular, the gradient descentattacks perform fine-grained, limited changes to the fea-tures. These variations in the feature values are most likelyto occur when generating new attack variants. This is areasonable result, as the majority of novel malicious attacksare directly derived from existing ones.

TABLE 2: Accuracy performances on five classifiers on a testset composed of data released after 2016. Each classifier hasbeen trained with data released before 2016.

Classifier AccuracySVM RBF (Adv.) .851SVM Linear .779RF (Adv.) .75SVM RBF .695RF .657

8 RELATED WORK

As Flash-based malicious attacks started to considerablygrow in 2015, the number of detection approaches is ratherlimited. FlashDetect [31] is one of the first approachesto the detection of ActionScript 3-based malware. Theauthors instrumented Lightspark, an open source Flashviewer, to perform dynamic analysis of malicious Flash files.From this analysis, the system extracts features such as thenumber of ByteArray-related method calls, the presence ofthe loadBytes method, and so forth. FlashDetect wasemployed inside the Wepawet service, which is sadly notavailable anymore.

Gordon [34] is an approach that resorts to guided-codeexecution to detect malicious SWF files, by statically ana-lyzing in particular their ActionScript bytecode. Morespecifically, the system selects the most suspicious securitypaths from the control flow graph of the code. Such pathshave usually references to security-critical call, such as theones for dynamic code loading. Although not publicly avail-able, proved to be rather effective to detect Flash malware.

Hidost [33] is a static system that only focuses on thestructure of the SWF file. More specifically, it considerssequences of objects belonging to the structure of the SWFfile as features. The system evaluates the most occurringpaths in the training dataset, and extracts features thatare based on the training data. This might be dangerousfrom the perspective of targeted attacks, as a malicioustest file with completely different paths might be able toevade detection. Moreover, the system does not analyze theembedded ActionScript code. In this way, an attackermight simply evade the system by perfectly mimicking thestructure of the file, regardless of the type of code it contains.

Besides scientific works, there are also some off-the-shelf tools that are used to perform obfuscation of SWFfiles (e.g., DoSWF9). Notably, in this paper we did not ana-lyze FlashBuster performances against samples that havebeen obfuscated by such tools. This is a different problemwith respect to the one we analyzed in Section 5.5. In thiscase, obfuscation is performed without having knowledge

9. http://www.doswf.org/

of the target detection system. For this reason, the attackerdoes not have the control of the features that are changed bythe attack. In this paper, we preferred focusing on the worst-case scenario, in which the attacker possesses completeknowledge of the target system.

We consider FlashBuster as a more complete variantof the aforementioned static systems, where informationfrom both the structure and content of the file are extracted.

9 CONCLUSIONS AND FUTURE WORK

In this paper, we proposed a security evaluation of static ma-licious SWF files detectors by introducing FlashBuster,a system that combines information analyzed by previousstate-of-the-art detectors (i.e., file structure and content). Theproposed security evaluation showed an intrinsic vulnera-bility of the static features used by SWF detectors. In par-ticular, by using gradient descent attacks, we demonstratedhow even retraining strategies are not always effective atensuring robustness. More specifically, we measured andshowed how gradient descent attacks make samples moresimilar to their benign counterparts. We plan to improveand solve some of the system limitations in future work:for example, reducing its dependence on JPEXS, whosepossible failures could compromise the whole file analysis.We also plan to perform more experiments on SWF files thatare obfuscated with off-the-shelf tools, in order to evaluatethe resilience of FlashBuster against them.

In general, our claim for future is research is that fo-cusing on improving the classifier decision function canbe effective only if the employed features are intrinsicallyrobust. This means that there should be specific features thatare truly characteristic of malicious behavior and that cannotbe mimicked in benign files. The efforts to guarantee moresecurity should be therefore directed towards a security-oriented design of the features.

REFERENCES

[1] Adobe, “Actionscript language specifications,” http://help.adobe.com/livedocs/specs/actionscript/3/wwhelp/wwhimpl/js/html/wwhelp.htm.

[2] ——, “Actionscript virtual machine 2 overview,”http://wwwimages.adobe.com/content/dam/Adobe/en/devnet/actionscript/articles/avm2overview.pdf.

[3] ——, “Swf file format specifications,” http://wwwimages.adobe.com/content/dam/Adobe/en/devnet/swf/pdf/swf-file-format-spec.pdf.

[4] ArsTechnica, “Hacking teams flash 0-day: Potent enough to infectactual chrome user,” 2015.

[5] R. A. Baeza-Yates and B. Ribeiro-Neto, Modern Information Re-trieval. Boston, MA, USA: Addison-Wesley Longman PublishingCo., Inc., 1999.

[6] M. Barreno, B. Nelson, A. Joseph, and J. Tygar, “The security ofmachine learning,” Machine Learning, vol. 81, pp. 121–148, 2010.

[7] M. Barreno, B. Nelson, R. Sears, A. D. Joseph, and J. D. Tygar, “Canmachine learning be secure?” in ASIACCS. New York, NY, USA:ACM, 2006, pp. 16–25.

[8] B. Biggio, I. Corona, D. Maiorca, B. Nelson, N. Srndic, P. Laskov,G. Giacinto, and F. Roli, “Evasion attacks against machine learningat test time,” in ECML PKDD, Part III, ser. LNCS, H. Blockeel etal., Eds., vol. 8190. Springer Berlin Heidelberg, 2013, pp. 387–402.

[9] B. Biggio, I. Corona, Z.-M. He, P. P. K. Chan, G. Giacinto, D. S. Ye-ung, and F. Roli, “One-and-a-half-class multiple classifier systemsfor secure learning against evasion attacks at test time,” in MultipleClassifier Systems, ser. LNCS, F. Schwenker, F. Roli, and J. Kittler,Eds. Springer Int’l Pub., 2015, vol. 9132, pp. 168–180.

13

[10] B. Biggio, G. Fumera, and F. Roli, “Pattern recognition systemsunder attack: Design issues and research challenges,” Int’l J. Patt.Recogn. Artif. Intell., vol. 28, no. 7, p. 1460002, 2014.

[11] ——, “Security evaluation of pattern classifiers under attack,”IEEE Trans. Knowl. Data Eng., vol. 26, no. 4, pp. 984–996, 2014.

[12] B. Biggio, B. Nelson, and P. Laskov, “Poisoning attacks againstsupport vector machines,” in 29th Int’l Conf. on Machine Learning,J. Langford and J. Pineau, Eds. Omnipress, 2012, pp. 1807–1814.

[13] M. Bruckner, C. Kanzow, and T. Scheffer, “Static prediction gamesfor adversarial learning problems,” J. Mach. Learn. Res., vol. 13, pp.2617–2654, Sept. 2012.

[14] N. Dalvi, P. Domingos, Mausam, S. Sanghai, and D. Verma, “Ad-versarial classification,” in 10th ACM SIGKDD Int’l Conf. Knowl.Disc. & Data Mining, 2004, pp. 99–108.

[15] A. Demontis, M. Melis, B. Biggio, D. Maiorca, D. Arp, K. Rieck,I. Corona, G. Giacinto, and F. Roli, “Yes, machine learning can bemore secure! a case study on android malware detection,” IEEETrans. Dep. Sec. Comp., In press.

[16] A. Globerson and S. T. Roweis, “Nightmare at test time: robustlearning by feature deletion,” in 23rd ICML, W. W. Cohen andA. Moore, Eds., vol. 148. ACM, 2006, pp. 353–360.

[17] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and har-nessing adversarial examples,” in ICLR, 2015.

[18] K. Grosse, N. Papernot, P. Manoharan, M. Backes, and P. D.McDaniel, “Adversarial examples for malware detection,” in ES-ORICS (2), ser. LNCS, vol. 10493. Springer, 2017, pp. 62–79.

[19] L. Huang, A. D. Joseph, B. Nelson, B. Rubinstein, and J. D. Tygar,“Adversarial machine learning,” in AISec, 2011, pp. 43–57.

[20] A. Kantchelian, J. D. Tygar, and A. D. Joseph, “Evasion andhardening of tree ensemble classifiers,” in 33rd ICML, ser. JMLRW&CP, vol. 48. JMLR.org, 2016, pp. 2387–2396.

[21] D. Lowd and C. Meek, “Adversarial learning,” in 11th Int’l Conf.KDD. Chicago, IL, USA: ACM Press, 2005, pp. 641–647.

[22] S. Mei and X. Zhu, “Using machine teaching to identify optimaltraining-set attacks on machine learners,” in 29th AAAI Conf.Artificial Intelligence (AAAI ’15), 2015.

[23] L. Munoz-Gonzalez, B. Biggio, A. Demontis, A. Paudice, V. Won-grassamee, E. C. Lupu, and F. Roli, “Towards poisoning of deeplearning algorithms with back-gradient optimization,” in 10thACM Workshop on Artificial Intelligence and Security, ser. AISec ’17.New York, NY, USA: ACM, 2018.

[24] N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. B. Celik, andA. Swami, “Practical black-box attacks against machine learning,”in ASIA CCS. New York, NY, USA: ACM, 2017, pp. 506–519.

[25] PCWorld, “Flash video is ’on life support,’ but big sites wont letgo,” http://www.pcworld.com/article/3026668/video-players/flash-video-is-on-life-support-but-big-sites-wont-let-go.html,2016.

[26] P. Russu, A. Demontis, B. Biggio, G. Fumera, and F. Roli, “Securekernel machines against evasion attacks,” in AISec. New York,NY, USA: ACM, 2016, pp. 59–69.

[27] N. Srndic and P. Laskov, “Detection of malicious PDF files basedon hierarchical document structure,” in 20th NDSS, 2013.

[28] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan,I. Goodfellow, and R. Fergus, “Intriguing properties of neuralnetworks,” in International Conference on Learning Representations,2014. [Online]. Available: http://arxiv.org/abs/1312.6199

[29] C. H. Teo, A. Globerson, S. Roweis, and A. Smola, “Convex learn-ing with invariances,” in NIPS 20, J. Platt et al., Eds. Cambridge,MA: MIT Press, 2008, pp. 1489–1496.

[30] L. Tong, B. Li, C. Hajaj, and Y. Vorobeychik, “Feature conserva-tion in adversarial classifier evasion: A case study,” CoRR, vol.abs/1708.08327, 2017.

[31] T. Van Overveldt, C. Kruegel, and G. Vigna, “Flashdetect: Action-script 3 malware detection,” in 15th RAID, 2012.

[32] N. Srndic and P. Laskov, “Practical evasion of a learning-basedclassifier: A case study,” in IEEE Symp. S&P, ser. SP ’14. Wash-ington, DC, USA: IEEE CS, 2014, pp. 197–211.

[33] N. Srndic and P. Laskov, “Hidost: A static machine-learning-baseddetector of malicious files,” EURASIP J. Inf. Secur., vol. 2016, no. 1,pp. 45:1–45:20, Dec. 2016.

[34] C. Wressnegger, F. Yamaguchi, D. Arp, and K. Rieck, “Comprehen-sive analysis and detection of flash-based malware,” in DIMVA,2016.

[35] H. Xiao, B. Biggio, G. Brown, G. Fumera, C. Eckert, and F. Roli, “Isfeature selection secure against training data poisoning?” in 32ndICML, F. Bach and D. Blei, Eds., vol. 37, 2015, pp. 1689–1698.

Davide Maiorca (M’16) received the M.Sc. de-gree (Hons.) in Electronic Engineering from theUniversity of Cagliari, Italy, in 2012. He is a Ph.D.Student in Electronic Engineering and ComputerScience at the University of Cagliari, Italy. In2013, he visited the Systems Security group atRuhr-Universitat Bochum, guided by Prof. Dr.Thorsten Holz, and worked on advanced obfus-cation of Android malware. His current researchinterests include adversarial machine learning,malware in documents and Flash applications,

Android malware and mobile fingerprinting. He has been a member ofthe 2016 IEEE Security & Privacy Student Program Committee.

Battista Biggio (SM’17) received the M.Sc.degree (Hons.) in Electronic Engineering andthe Ph.D. degree in Electronic Engineeringand Computer Science from the University ofCagliari, Italy, in 2006 and 2010. Since 2007, hehas been with the Department of Electrical andElectronic Engineering, University of Cagliari,where he is currently an Assistant Professor.In 2011, he visited the University of Tubingen,Germany, and worked on the security of machinelearning to training data poisoning. His research

interests include secure machine learning, multiple classifier systems,kernel methods, biometrics and computer security. Dr. Biggio serves asa reviewer for several international conferences and journals. He is asenior member of the IEEE and member of the IAPR.

Maria Elena Chiappe Maria Elena Chiappe re-ceived her B. Sc. degree in Electronic Engineer-ing with honors from the University of Cagliari,Italy, in 2015. She is studying for her M. Sc.degree in Electronic Engineering, University ofCagliari. Her main research interests includecomputer security, detection of DGA and fast fluxnetworks and machine learning.

Giorgio Giacinto (SM’10) is Associate Profes-sor of Computer Engineering at the Universityof Cagliari, Italy. He obtained the MS degreein Electrical Engineering in 1994, and the Ph.D.degree in Computer Engineering in 1999. Since1995 he joined the PRA Lab of the DIEE, Univer-sity of Cagliari, Italy. His research interests are inthe area of pattern recognition and its applicationto computer security, and image classificationand retrieval. During his career Giorgio Giacintohas published more than 120 papers on inter-

national journals, conferences, and books. He is a senior member ofthe ACM and the IEEE. He has been involved in the scientific coordi-nation of several research projects in the fields of pattern recognitionand computer security, at the local, national and international level.Since 2012, he has been the organizer of the Summer School onComputer Security and Privacy “Building Trust in the Information Age”(http://comsec.diee.unica.it/summer-school).