HECTOR-V: A Heterogeneous CPU Architecture for a SecureRISC-V Execution Environment

Pascal Nasahlpascal.nasahl@iaik.tugraz.at

Graz University of TechnologyGraz, Austria

Robert Schillingrobert.schilling@iaik.tugraz.atGraz University of Technology

Graz, Austria

Mario Wernermario.werner@iaik.tugraz.at

Graz University of TechnologyGraz, Austria

Stefan Mangardstefan.mangard@iaik.tugraz.atGraz University of Technology

Graz, Austria

ABSTRACTTo ensure secure and trustworthy execution of applications in po-tentially insecure environments, vendors frequently embed trustedexecution environments (TEE) into their systems. Applicationsexecuted in this safe, isolated space are protected from adversaries,including a malicious operating system. TEEs are usually buildby integrating protection mechanisms directly into the processoror by using dedicated external secure elements. However, bothof these approaches only cover a narrow threat model resultingin limited security guarantees. Enclaves nested into the applica-tion processor typically provide weak isolation between the secureand non-secure domain, especially when considering side-channela�acks. Although external secure elements do provide strong isola-tion, the slow communication interface to the application processoris exposed to adversaries and restricts the use cases. Independentlyof the used implementation approach, TEEs o�en lack the possi-bility to establish secure communication to external peripherals,and most operating systems executed inside TEEs do not providestate-of-the-art defense strategies, making them vulnerable againstvarious a�acks.

We argue that TEEs, such as Intel SGX or ARM TrustZone, imple-mented on the main application processor, are insecure, especiallywhen considering side-channel a�acks. In this paper, we demon-strate how a heterogeneous multicore architecture can be utilizedto realize a secure TEE design. We directly embed a secure pro-cessor into our HECTOR-V architecture to provide strong isolationbetween the secure and non-secure domain. �e tight coupling ofthe TEE and the application processor enables HECTOR-V to pro-vide mechanisms for establishing secure communication channelsbetween di�erent devices. We further introduce RISC-V Secure Co-Processor (RVSCP), a security-hardened processor tailored for TEEs.To secure applications executed inside the TEE, RVSCP provideshardware enforced control-�ow integrity and rigorously restrictsI/O accesses to certain execution states. RVSCP reduces the trustedcomputing base to a minimum by providing operating system ser-vices directly in hardware.

KEYWORDStrusted execution environment, secure I/O, heterogeneous com-puter architecture, RISC-V

1 INTRODUCTIONWith the growing demand for complex IT applications, such asautonomous driving or smart city infrastructures, so�ware com-plexity increases steadily. �e codebase of the Linux kernel, forexample, increases by 250k lines of code (LOC) each year, reaching27.8M LOC in 2020 [12]. �is is challenging because one can expectroughly 1 to 25 bugs in 1,000 lines of code [41]. While not all ofthese bugs might be exploitable by an a�acker, a growing codebasecomplexity clearly leads to a larger a�ack surface.

One strategy to deal with the growing complexity and meetsecurity goals is to isolate all security critical applications usinga trusted execution environment (TEE) [23]. A TEE establishes asecure execution environment by creating a safe, isolated spaceusing hardware and so�ware primitives [24]. Most TEE threatmodels consider a powerful adversary controlling user space ap-plications, the operating system, or even the hypervisor, trying toin�uence the execution of applications in the trusted environment.To achieve the protection needed for this threat model, TEEs requirestrong isolation between the TEE and the so-called rich executionenvironment (REE).

�e powerful concept of shi�ing security critical applicationsinto a trusted execution environment is already adapted by majorvendors like Intel, ARM, and Apple. One of the most common TEEdesign approaches is to create a virtual secure processor in themain application processor by using hardware extensions. IntelSGX [20] and ARM TrustZone [7] take this approach. Contradictoryto embedding the TEE tightly into the CPU, Google’s Titan [34] andApple’s T2 [3] implements a secure element by externally mountinga dedicated security processor next to the main CPU. However,both TEE design approaches yield di�erent weaknesses. Severalrecent a�acks [13, 14, 17, 28, 29, 37, 49, 54, 66] showed that theisolation of TrustZone and SGX can be bypassed by mounting cacheor transient-based side-channel a�acks. While dedicated secureelements provide strong isolation between REE and TEE, Google’sTitan, e.g., uses a slow SPI connection for communication betweenthe two domains limiting potential use cases. Furthermore, thiso�-chip communication fabric between REE and TEE is physicallyexposed to an a�acker, making it vulnerable against probing a�acks.

Independently of the used TEE design approach, typical TEE im-plementations o�er several weaknesses. First, although a securitybreach within the TEE is fatal, operating systems deployed in the

arX

iv:2

009.

0526

2v1

[cs

.CR

] 1

1 Se

p 20

20

secure environment surprisingly do not o�er state-of-the-art pro-tection mechanisms like ASLR, guard pages, or stack cookies [15].Second, most TEEs do not provide architectural features to establishsecure communication with I/O devices. �e lack of trusted I/Opaths in TEE systems is critical because secrets shared betweenuser and TEE are unprotected.

With the Intel Lake�eld architecture [35] and an AMD patent [42],major vendors recently announced to introduce heterogeneousmulti-core architectures in upcoming processor designs. While theapproach of tightly coupling di�erent processor cores on one chipto balance power and energy e�ciency is already used by ARM’sbig.LITTLE technology [6] in mobile platforms, Intel and AMD areplanning to introduce this concept in forthcoming computer archi-tectures. �is design strategy raises the following research question:RQ1: ”Can the tight coupling of distinct processor cores on a SoC beused to increase the security of trusted execution environments?”

ContributionIn this paper, we introduce HECTOR-V, a design methodology uti-lizing a heterogeneous multi-core architecture to develop securetrusted execution environments. HECTOR-V achieves strong isola-tion between the secure and non-secure domain and provides pro-tection for various side-channel a�acks, such as cache and transient-based a�acks, by using a distinct processor of the heterogeneouscore cluster for the secure environment. �e tight coupling of TEEand REE in the shared SoC infrastructure yields several advantages.In the HECTOR-V architecture, the application processor and TEEare connected through a communication fabric, enabling a high-speed link between the two processors. Additionally, since theTEE is embedded into the SoC, all peripherals integrated into thesystem are also available for the secure environment. To manageperipheral sharing and protect peripherals from unauthorized ac-cesses, HECTOR-V further introduces a secure I/O path concept. �eidenti�er-based strategy deeply nested into the communicationfabric and the peripherals allows a �ne granular protection of thea�ached peripherals. In HECTOR-V, the access permission manage-ment is implemented using a hardware-based security monitor.While the security monitor owner can con�gure a set of accesspermissions, the other parties in the system can request access tocertain peripherals by consulting the security monitor. We furtherextend this mechanism to dynamically grant and deny access tocertain peripherals by introducing the concept of security monitorownership transfer. To enable various use cases, the owner of thesecurity monitor dynamically can transfer the ownership to otherparties.

Furthermore, we introduce RVSCP, a security-hardened RISC-Vprocessor tailored for being used as a TEE. Although the HECTOR-Varchitecture is generic and independent of the used TEE processor,we show that RVSCP further increases the TEE security by utilizingfeatures of HECTOR-V. RVSCP isolates applications within the trustedexecution environment by enforcing the integrity of the control-�ow. We combine the control-�ow information with the secureI/O mechanism to only grant access to certain peripherals whenreaching a prede�ned control-�ow state. To reduce the trustedcomputing base and therefore the a�ack surface to a minimum,

RVSCP implements operating system features, such as multitasking,directly in hardware.

We demonstrate the HECTOR-V concept by introducing a het-erogeneous multi-core architecture for RISC-V. We embedded aRISC-V processor with a control-�ow integrity protection unit intothe HECTOR-V architecture. To verify the functionality of HECTOR-Vand RVSCP design approaches, we use secure boot as a prototypeapplication on our FPGA implementation. In summary, our contri-butions are:HECTOR-V, a design methodology for developing secure TEEs.HECTOR-V utilizes a heterogeneous multi-core architecture to realizea secure trusted execution environment. �e architecture includesa con�gurable security monitor managing access permissions tospeci�c peripherals. �is mechanism allows the system to establishsecure communication channels between peripherals, users, andthe processor cores. �e dynamic transfer of security monitorownership enables the HECTOR-V architecture to realize several usecases, such as secure boot or executing trusted applications.RVSCP, a security-hardened processor integrated into the HECTOR-Varchitecture. RVSCP protects applications within the TEE by using acontrol-�ow integrity protection unit and combines this mechanismwith the peripheral access protection o�ered by HECTOR-V. �esecure processor provides operating system features in hardwareto minimize the trusted computing base.

2 BACKGROUNDTo protect the processing of sensitive data, such as biometric data orcryptographic keys in an untrusted environment, trusted executionenvironments (TEE) are used. �ese secure environments, mostlyused in mobile platforms and cloud computing, comprise a combina-tion of hardware and so�ware features, the trusted computing base(TCB), which needs to be trusted by the system [23]. A TEE guar-antees the secure execution of trusted applications, the trustlets,even when considering a malicious operating system running inthe rich execution environment (REE). �is feature can either berealized with a virtual processor approach, using an external secureprocessor, or embedding a secure processor into the SoC.

2.1 ARM TrustZoneARM TrustZone [7] is a TEE mostly used in mobile platforms.TrustZone protects trustlets executed inside the TEE from mali-cious accesses and tampering approaches, even from the operatingsystem [45]. Due to the massive deployment of ARM cores in An-droid phones, TrustZone is commonly used by vendors, such asSamsung with KNOX [46], to provide secure key storage and pay-ment. �e strong isolation of TrustZone is achieved by partitioningthe system into a secure and non-secure world using hardwareextensions. TrustZone enforces this separation by introducing anon-secure (NS) bit directly into the bus architecture. �is bit in-dicates, in which domain the cache, the TrustZone Address SpaceController (TZASC), and other peripherals are working and pro-tects secure world devices from non-secure world accesses [44].Secure communication between the two domains and the handlingof context switches, interrupts, and exceptions are managed by thesecurity monitor. Since the rigorous segregation between the two

2

security domains is enforced for all SoC components, one physi-cal TrustZone core operates like two virtual cores [63]. To deploymultiple trusted applications in the trusted execution environment,usually, an operating system is mounted in the secure environment.Trusty [27], which is used in Android devices supporting Trust-Zone, consists of a small kernel and provides a communicationinterface to the non-secure world. Trusted applications, such as keystorage and �ngerprint processing, are directly integrated into thekernel by the device vendor. Deploying third-party applications iscurrently not supported by Trusty. Other small footprint operatingsystems deployed in TrustZone assisted TEEs are the open-sourceOP-TEE [25] OS developed by Linaro or �alcomm’s QSEE.Security of TrustZone. �e main idea of a TEE is to reduce thetrusted computing base to a minimum. Kernels deployed in ARMTrustZone comply with this requirement by only providing a smallset of operating system services. Although applications executedin the TEE require strong isolation between the trustlets, operatingsystems like Trusty and OP-TEE fail to provide state-of-the-artprotection mechanisms like ASLR or stack cookies [15]. Due to theinsu�cient protection of applications executed in the TEE, severala�acks, like the �alcomm QSEE a�ack [10] were discovered inthe past. In this particular a�ack, an adversary could gain remotecode execution by exploiting a single bu�er over�ow vulnerabilityin a trustlet. �ese a�acks are critical because they threaten notonly the security of the TEE but also the security of the REE. Since�alcomm’s QEE allows the TEE to modify arbitrary memory, ana�acker easily could obtain control over the Android system exe-cuted in the REE [11]. However, not only the lack of conventionalsecurity concepts in TEE kernels is problematic, but TrustZone alsois vulnerable against various side-channel a�acks. CLKSCREW [51]utilizes the DVFS energy management technique to perform a faulta�ack retrieving secret keys from the TEE or deploying maliciouscode. Although TrustZone separates the cache between the secureand non-secure world, past research demonstrated several cachea�acks [29, 37, 66].

2.2 Intel SGXIntel SGX [20] provides hardware-based isolation for enclaves byintroducing a set of dedicated instructions embedded into the mi-croarchitecture of the CPU. �e threat model covers potentially ma-licious user applications, the operating system, and the hypervisorand protects the enclave content from unauthorized accesses [21].To verify the program loaded into the enclave by untrusted parties,Intel SGX o�ers a so�ware a�estation service. SGX preserves con-�dentiality and integrity of external memory by using transparentmemory encryption [30].Security of SGX. Although SGX provides protection against phys-ical a�ackers targeting the external memory, Intel explicitly ex-cludes side-channel a�acks in their threat model and recommendsdevelopers to prevent such a�acks at enclave level [32]. Past re-search showed that SGX is vulnerable against cache a�acks [13, 28,49], transient a�acks [14, 17, 54], and fault a�acks such as Plun-dervolt [43]. In addition to side-channel a�acks, trustlets in IntelSGX are also vulnerable against classical so�ware a�acks, such asmemory vulnerabilities [19, 36, 57]. Furthermore, SGX does not

natively provide secure I/O interaction; it requires additional tech-niques, such as SGXIO [58], to establish secure communicationwith peripherals.

2.3 Google TitanGoogle Titan [34] is a secure element providing a hardware-basedroot of trust for cloud computing. To mitigate a�acks against theintegrity of the basic �rmware or the operating system, Googleuses Titan to verify the boot process on server platforms [26]. Titaninternally stores signatures of the boot �les and uses cryptographicaccelerators to verify the loaded boot images. In addition, Titan alsoprovides several cryptographic services to the main CPU. �e dedi-cated chip is mounted next to the main application processor andestablishes communication over a slow SPI interconnect. Googlecurrently announced to use the open-source version OpenTitan [39]in the next Titan generations. Similarly, OpenTitan consists of asmall RISC-V processor and several cryptographic hardware accel-erators. A smaller version of the Titan, the Titan-M, is used byGoogle in current Pixel phones.

2.4 Apple Secure Enclave ProcessorApple’s Secure Enclave Processor (SEP) [31], which was introducedwith the iPhone 5s, is a dedicated ARM secure coprocessor directlyembedded into the main SoC. Similar to ARM TrustZone in mobileplatforms, SEP is mainly used to store keys and enable secureprocessing of biometric data [4]. SEP, which runs the L4-basedSEPOS, fully controls several dedicated secure peripherals, such ascryptographic accelerators, and shares other peripherals on the SoCwith the application processor. �e memory controller allows thesecure coprocessor to de�ne isolated memory regions in the mainmemory. To enable communication between the main applicationprocessor and SEP, a mailbox system is used. Since the secureenclave processor also contains a key and signature storage, SEP isalso used to verify the boot process of the application processor.Security of SEP. Although SEP and SEPOS implement basic pro-tection mechanisms like stack cookies, guard pages, and memoryencryption, state-of-the-art defense strategies such as ASLR are stillmissing [40]. In contrast to ARM TrustZone, peripherals are eitherexclusively owned by SEP or are shared, like the power manager orthe PLL clock generator. �is makes SEP possibly vulnerable againsta�acks similar to CLKSCREW [51]. Additionally, the decryptionkeys for some SEP implementations were already leaked [33].

3 THREAT MODELOur threat model considers the common a�ack scenarios on TEEsde�ned by Cerdeira et al. [15]. We are considering an a�ackerdirectly exploiting architectural weaknesses of the TEE and theTEE kernel. Here an adversary uses a combination of bugs in thekernel, �aws in the hardware protection mechanism, and missingstate-of-the-art defense strategies, such as ASLR or guard pages, tocompromise the system. Furthermore, we expect bugs in trustlets,which can be exploited by an a�acker over the communication in-terface between REE and TEE. �e security of applications in REE orTEE also can be threatened by using cache-based or transient-basedside-channel a�acks. Additionally, we are considering a malicioustrustlets explicitly trying to a�ack the SoC. We are extending the

3

Peripheral

SM

Interconnect

Core Cluster

Figure 1: Overall HECTOR-V design.

threat model of common TEEs also to cover a�acks on peripher-als, i.e., illegitimate accesses to secure storage elements, sensitiveperipherals such as a �ngerprint reader or SPI [1], or protected mem-ory regions. In summary, we expect a powerful a�acker having fullcontrol over user applications, the operating system, trustlets, oreven the hypervisor executed on the application processor.

4 DESIGN�is section presents our secure TEE design approach consisting ofthe HECTOR-V architecture and the secure processor RVSCP. We �rstintroduce HECTOR-V consisting of several architectural features,such as trusted I/O paths, a security monitor, and a secure TEEintegration. �en, we introduce RVSCP, a concrete proposal for asecure processor used as a TEE and demonstrate, how RVSCP andHECTOR-V combined, form a secure TEE system. In Section 5, wethen demonstrate a prototype of HECTOR-V and RVSCP integratedinto our RISC-V base platform, the lowRISC chip.

4.1 HECTOR-V Design�e HECTOR-V design proposes architectural features for creatingsecure TEEs. As seen in Figure 1, the TEE in HECTOR-V is realizedby mounting a secure processor directly into the main SoC. �isapproach is similar to ARM’s big.LITTLE technology [6], whereindependent cores are embedded into the chip. However, insteadof using the additional cores for power e�ciency, HECTOR-V usesthe dedicated processor for security purposes. Although placing asecure processor directly into the SoC increases the area overhead,we argue that this approach is feasible. First, we contend thato�ering strong isolation between secure and non-secure domainonly is possible with a dedicated secure processor for the TEE. �isdesign choice completely separates the instruction pipelines andcaches of the cores, resulting in an independent execution �owof REE and TEE applications. Second, services deployed on TEEprocessors, such as applications handling banking information orprocessing passwords, are typically rather small. �is constrainedprocessing requirement allow us to deploy a tiny secure processor.�erefore, when comparing the TEE processor with a state-of-the-art multicore processor, the area overhead is negligible.

While integrating a dedicated secure processor into the SoCinfrastructure is straightforward, a secure and viable TEE systemrequires to establish communication channels between REE andTEE, as well as for peripherals enabling user I/O. In the remainingpart of this section, we introduce our trusted I/O path design uti-lizing a hardware-based security monitor and elaborate how these

concepts allow REE and TEE to share devices on the SoC and realizefeatures, such as secure storage elements.

4.1.1 Trusted I/O Paths. �e trusted I/O path mechanism isthe central element of the HECTOR-V architecture. �is conceptallows HECTOR-V to securely share devices on the SoC, establishsecure communication channels between external users and theTEE or REE, and implement concepts, like secure storage elements.HECTOR-V establishes trusted I/O paths by assigning an uniqueidenti�er to each party of the system, i.e., the processor cores. Whenaccessing a device, like the SD-card or the block RAM (BRAM), theperipheral uses an internal mechanism to verify the identi�er. �isstrategy allows the architecture to enforce access permissions, suchas the �ngerprint reader only can be accessed by the TEE.Identi�er. In HECTOR-V, an identi�er is used to distinguish be-tween legitimate and illegitimate accesses to a peripheral. Whilethis concept is similar to ARM TrustZone [7], TrustZone only usesa 1-bit identi�er to distinguish between accesses from the secureor non-secure world. �e identi�er used by our system consistsof a core identi�er, a process identi�er, and a peripheral identi�er.While the core identi�er permanently and unchangeable is assignedto each processing core, e.g., an identi�er for the application pro-cessor and one identi�er for the secure processor, at design time,the process and peripheral identi�er can be assigned by each entityitself.Interconnect. To e�ciently transport the identi�er from the par-ticipants to the peripherals, we integrate the identi�er directly intothe system interconnect. �e AMBA AXI4 [5] protocol, which isused as interconnect by our lowRISC base platform and many otherSoC designs, allows to embed up to 1024-bits wide user-de�nedsignals into the protocol. By embedding the identi�er into theuser-de�ned signals, which are not used by most AXI devices, theidenti�er is transported without any protocol overhead. Addition-ally, this approach assures that the identi�er is sent along withthe address and data on each AXI request. Since the core iden-ti�er is hardcoded directly into the the interconnect interface ofthe participants, an a�acker can not change this identi�er fromso�ware.Peripherals. HECTOR-V creates a trusted channel between an en-tity, e.g., the application or the secure processor, and a peripheralby enforcing the identi�er-based access check directly in hardwareat the peripheral driver. �e peripherals, e.g., the hardware imple-mentation of the SD-card controller or the BRAM, use the identi�erto �lter illegitimate accesses. �is scheme, which requires the pe-ripherals to be identi�er-aware, is implemented by introducinga lightweight wrapper module for each peripheral instance. Inour architecture, the peripheral wrapper module consists of twocommunication channels: the data channel and the con�gurationchannel. �e data channel interface of the peripheral wrapper isdirectly connected to the AXI4 communication fabric, allowingother parties, e.g., the secure processor, to access the peripheral.By using the con�guration channel, the con�guration party canset or unset the identi�er in the ID �eld. In HECTOR-V, the data andcon�guration channels are physically separated by introducing adedicated, lightweight AXI4-lite [5] communication fabric.

When the identi�er of the AXI4 communication request trans-ported over the communication fabric matches the identi�er stored

4

in the peripheral wrapper ID �eld, the access is permi�ed and therequest is directly forwarded to the actual peripheral driver. Onan ID mismatch, the �rewall mechanism blocks the request and re-turns an exception over the AXI4 channel, which can be processedby the issuer. If the process or peripheral identi�er is set to zero,the peripheral wrapper ignores these ID �elds.

�e peripheral wrapper resides in two states: claimed or un-claimed. While in the unclaimed state all data channel requests areblocked, in the claimed state only the requests with the matchingidenti�er are permi�ed. Furthermore, we di�erentiate betweencon�gurable and non-con�gurable peripheral wrappers. To realizefunctionalities, like secure storage elements, which must not beaccessible by any party except one, non-con�gurable peripheralwrappers store a hardcoded identi�er, consisting of the core andprocess ID, in the ID �eld.

4.1.2 Security Monitor. �e architectural features of identi�er,identi�er transportation, and peripheral �rewalls are the foun-dation for establishing trusted I/O paths on the SoC. To enforcesecurity policies utilizing the trusted paths, we embed a tiny se-curity monitor into the SoC infrastructure. �is hardware-basedsecurity monitor (SM) module is responsible for managing accesspermissions to peripherals. Internally, the SM consists of a tabletracking these access permissions. For each peripheral included inthe system, the security monitor maintains a table entry trackingthe state (claimed or unclaimed) and a list of identi�ers allowed toclaim the device.Security Monitor Owner. �e HECTOR-V architecture introducesthe security monitor owner privilege, which is assigned to a certainparty at design time. �is party solely is responsible for con�guringthe security monitor. More concretely, only the SM owner is allowedto de�ne, which peripheral can be claimed by which participant.�e security monitor stores the identi�er of the current owner andaccepts con�guration commands only from this party.Peripheral Claiming and Releasing. To access a certain periph-eral, the requester �rst needs to claim the peripheral by sending aclaim request to the security monitor. �en, the SM checks, if theperipheral currently is unclaimed and veri�es that the ID of theissuer is in the list of identi�ers permi�ed to access the peripheral.Finally, the security monitor sends the ID of the issuer to the periph-eral wrapper over the AXI4-lite bus and the peripheral �rewall setsthis identi�er in its ID �eld. Now, the identi�er transported in theuser-de�ned signals of each AXI4 request issued by the requestermatches the identi�er in the ID �eld and the requester exclusivelycan access the peripheral. However, if the requester is not allowedto claim the peripheral, i.e., the identi�er of the requester is notin the list of privileged entities, the ID veri�cation fails and thesecurity monitor sends an access denied message back to the issuer.If the requester ID is in the table entry of the peripheral, but theperipheral currently is claimed, the security monitor noti�es theissuer.

HECTOR-V is a cooperative scheme, meaning, the entity currentlyclaiming a peripheral needs to release it a�er using it. To releasea peripheral, the entity sends a dedicated release command to thesecurity monitor. �e security monitor processes this request byclearing the ID �eld of the peripheral wrapper.

Peripheral Access Withdraw. Claiming a peripheral and bind-ing access exclusively to an entity is a powerful concept and estab-lishes a trusted, secure channel between this entity and a peripheral,but it also can be abused. A participant, e.g., the application pro-cessor or the secure processor, in control of an a�acker, couldpermanently occupy one or more peripherals resulting in a denial-of-service (DOS) a�ack. To mitigate such a�acks and manage un-responsive participants, the security monitor has the capability towithdraw access from certain peripherals. A simple approach toimplement this functionality would be to clear the ID �eld in theperipheral wrapper. However, this approach is dangerous becauseit could enable time-of-check to time-of-use (TOCTOU) a�acks.For example, when withdrawing access to a peripheral currentlyprocessing a secret, the SM owner would be able to retrieve theresult of the request.

To securely withdraw access to certain peripherals, HECTOR-Vintroduces a withdrawing mechanism, which can be triggered byany entity in the system. While a withdraw request issued by theSM owner is always granted, a request from other parties �rst needsto be approved by the security monitor. When a withdraw requestis retrieved by the security monitor, the SM simultaneously startsa timer with a prede�ned timeout and noti�es the owner of theperipheral. �e noti�cation of the peripheral owner is realizedby introducing a dedicated interrupt line and a interrupt serviceroutine (ISR) provided by the processors for each peripheral. �eISR implements a cleanup function responsible for clearing secrets,stopping current transactions, and gracefully releasing the corre-sponding peripheral. A�er the timeout is reached in the securitymonitor and the peripheral is not released gracefully by the ISR,the SM forcefully releases the peripheral by clearing the ID �eld inthe �rewall.Security Monitor Ownership Transfer. A signi�cant advan-tage of HECTOR-V, compared to other TEE architectures, is thepossibility to utilize the TEE infrastructure for various use cases.While in TEE systems, like ARM TrustZone, one entity, e.g., thesecure-world, is the exclusive owner of the highest privilege level,HECTOR-V introduces a dynamic ownership transfer mechanism.�e security monitor privilege, which allows the SM owner to con-�gure access privileges and release arbitrary peripherals, can betransferred to any other entity by the SM owner. To initiate a SMownership transfer, the entity sends a request with the identi�erof the new owner to the security monitor. �e security monitoracknowledges this request by se�ing the received identi�er intothe SM owner ID �eld. To obtain a clear state, we recommend thatthe security monitor owner �rst releases all peripherals.

4.2 RVSCP DesignTo build a secure TEE environment, a processor of the heteroge-neous core cluster could be utilized to deploy an operating system.�is operating system then is in charge of assigning each trustlet onthe core an individual process identi�er, annotate data in the cachewith the process ID, and manage accesses to peripherals. However,deploying a fairly complex OS, which increases the TCB and thea�ack surface, is contrary to the usage model of most TEEs. Weargue that trustlets that are commonly deployed in TEEs, such asproviding an interface to a cryptographic accelerator, a key storage,

5

or a �ngerprint sensor, do not extensively make usage of operatingsystem features. �erefore, we reduce the TCB of RVSCP to a mini-mum by providing basic operating system services, such as contextswitches and ID management, directly in hardware. Furthermore,RVSCP utilizes a control-�ow integrity (CFI) unit and the secure I/Oscheme of HECTOR-V to restrict access to sensitive peripherals.

4.2.1 Control-Flow Integrity. To protect a program from a�ackstargeting to alter the control-�ow, CFI schemes are commonlyused [52, 55, 65]. �ese schemes mitigate a�acks like ROP [50] orJOP [16] by ensuring that the control-�ow of the program can notescape the control-�ow graph (CFG) determined at compile time.Enforcing the integrity of the instruction stream can be realizedin di�erent degrees of �neness. While some schemes [2, 8, 18]preserve the correctness of the execution �ow at basic block level,other techniques [22, 60, 61] maintain the integrity of the control-�ow even at instruction granularity.Control-Flow IntegrityUnit. RVSCP utilizes the �ne-grain Sponge-Based Control-Flow Protection (SCFP) [60] scheme to protect trustletswithin the secure processor from a�ack a�empts. �e main idea ofSCFP is to encrypt a program using a sponge-based authenticatedencryption primitive during compile time and decrypt it instructionfor instruction at runtime. Decryption of the individual instruc-tions is realized using a dedicated decryption stage in the processorpipeline. To successfully decrypt an instruction, the pipeline stageneeds to know the key and an internal state. �e SCFP schemeaccumulates information over all previously executed instructionsin this state. If the integrity of the state is violated, the decryptionfails and returns a faulty instruction, which can be detected with acertain probability by the CPU. An a�acker modifying instructions,e.g., using fault injection, or inserting additional instructions, altersthe state and can be detected by SCFP.

4.2.2 Hardware Scheduling. We extend the native SCFP imple-mentation, which allows to execute a bare-metal program on aprocessor, to support multitasking. One approach to enable mul-titasking with CFI protected trustlets could be realized purely inso�ware using an operating system. However, since TEE operat-ing systems, such as OP-TEE [25], do not provide state-of-the-artprotection mechanisms, such as ASLR or guard pages, mountingan operating system would also increase the a�ack surface of theTEE. �erefore, similar to Antikernel [67], RVSCP o�ers hardwarefeatures to run multiple trustlets on the processor and reduce theso�ware TCB to a minimum.Hardware Scheduling Unit. RVSCP introduces a hardware entityproviding minimal OS functionality for trustlets. �is hardwareunit is responsible for performing secure context switches betweenindividual trustlets in hardware. �e round-robin based schedulingmechanism internally maintains a list of trustlets and a�er a certainamount of cycles executed, a context switch is conducted. Whenperforming a context switch, the hardware entity stops the currenttrustlet, stores the architectural state to a secure place, and loadsthe next architectural state of the next trustlet. Additionally, thehardware context switch mechanism also exchanges the decryptionkey used for SCFP. Using an individual key for each trustlet yieldstwo major advantages. First, since the programs are encryptedwith a di�erent key, only the developer with the correct key canaccess the plain program, leading to an IP protection mechanism for

trustlets. Second, using di�erent keys for trustlets enables strongisolation between the applications. A�er the context switch, theexecution is resumed and the next trustlet is executed.

While this hardware scheduling unit allows the processor tobasically consist of several virtual processor cores, only a �xednumber of trustlets can run simultaneously on the physical core.However, since most TEEs are anyway limited in their processingpower and only a well-chosen set of trustlets is usually deployedin TEEs by the vendor, an upper bound of processes is acceptable.Furthermore, we argue that for simple services typically used inmobile platforms, such as a process handling biometric data forunlocking the device or a process handling the secure key storage,no dedicated operating system is needed. Completely omi�ing theoperating system and providing operating system services using atiny hardware unit reduces the so�ware TCB to a minimum andwould even allow to formally verify the simple hardware unit.

4.2.3 Control-Flow Integrity with Secure I/O. �e �ne granularcontrol-�ow integrity unit embedded into the HECTOR-V architec-ture prevents an a�acker from performing control-�ow hijack at-tacks by limiting the control-�ow of the program to only valid pathsthrough its control-�ow graph. However, while the CFI scheme pro-tects the control-�ow by detecting integrity violation of forward-and backward-edges, and thus prevents a�acks such as ROP or JOP,data-oriented a�acks can not be detected by this countermeasure.In such a�acks, an adversary modi�es control- or non-control re-lated data to break the security of the system. By manipulatingcontrol-related data, such as the condition value in an if statement,the a�acker indirectly can in�uence the control-�ow of the pro-gram. Furthermore, an a�acker could leak sensitive data, such aspasswords or cryptographic keys, by manipulating addresses in thesystem. For example, instead of returning the ciphertext over anAPI to the user, an a�acker could modify the address from pointingto the location of the ciphertext in memory to the encryption keystored in a secure storage element instead by exploiting a bu�erover�ow bug. To lower the impact of such a�acks, RVSCP bindsaccess to certain assets to a certain CFI state. More concretely, onlywhen the CFI protected program reaches a prede�ned CFI state, theprogram is permi�ed to access the distinct peripheral. In RVSCP,this strategy is realized by tunneling each interaction request with aperipheral through a dedicated function with a certain state. RVSCPautomatically sets the peripheral identi�er of the processor to thecurrent state. Only if this state matches the ID stored in the periph-eral wrapper, access to the device is granted. If an a�acker calls theperipheral access function outside the valid control-�ow graph, theCFI mechanism detects this violation. Additionally, if the adversarycra�s an address accessing the asset, the state used as an identi�erdoes not match the identi�er of the peripheral and access to thedevice is restricted.

5 IMPLEMENTATIONIn this section, we provide a prototype implementation of theHECTOR-V architecture and the secure RVSCP processor. We �rstintroduce the RISC-V lowRISC chip, which we use as our base plat-form. �en, we extend this platform with the HECTOR-V features.Finally, we present our RVSCP design and demonstrate how thisprocessor is embedded into the SoC infrastructure.

6

SDUART PS2 SPI

lowRISC Chip

DDR3

ControllerBRAM

AXI AXI Crossbar

Rocket Chip

TileLink

Figure 2: lowRISC base platform.

5.1 Base Platform�e prototype implementation is based on the open-sourcelowRISC [38] project. Internally, the lowRISC chip consists of aninstance of the 64-bit RISC-V Rocket chip [9, 56]. �e SoC, capableof running Linux, provides an external o�-chip DDR3 memory andan on-chip BRAM. As shown in Figure 2, peripherals, such as aSD-card controller and a SPI interface are connected over an AXI4communication fabric with the processor core.

5.2 HECTOR-VFigure 3 depicts the prototype implementation of the HECTOR-Varchitecture. �e extended lowRISC base platform consists of theRocket Core application processor ÷© and the secure processorRVSCP integrated into the TEE ï©. By providing an AXI4 masterinterface 1© 2©, REE and TEE gain access to various peripheralsover the shared communication fabric 3©. To di�erentiate betweenREE and TEE, the immutable core identi�er is directly embeddedinto this AXI4 master interface. We further introduce a securitymonitor �©, a memory protection unit (MPU) ©, a reset unit Ê©, anda secure storage element ¤© as part of the HECTOR-V architecture.Interconnect. �e SoC communication infrastructure consists ofa shared AXI4 3© and AXI4-lite C© interconnect. In our architecture,the AXI4 bus is used to enable interaction between the processingcores and the peripherals. We extended the AXI4 bus protocol andthe crossbar to transport the 16-bit user signal along with each busrequest. �is user signal carries the 1-bit core identi�er, the 4-bitprocess identi�er, and the 10-bit peripheral identi�er. While com-munication between peripherals and cores requires a high-speedlink, the con�guration of the security monitor and the peripheralwrappers only consists of small con�guration commands. Hence,we use a lightweight AXI4-lite bus as a con�guration channel. �econ�guration of the security monitor is realized by introducing apoint-to-point communication channel between REE and SM A©and TEE and SM B©. By using a separate AXI4-lite interconnect C©,the con�guration of the peripheral wrappers can only be initiatedby the security monitor. �is strategy ensures that neither the TEEnor the REE directly can manipulate the peripheral �rewalls; con-�guration only can be requested over the security monitor module.Security Monitor. To receive commands from REE and TEE, thesecurity monitor implements two AXI4-lite slave interfaces. �eprotocol used to interact with the security monitor consists of twoprivileged and four unprivileged commands. With the privilegedcon�guration command, the SM owner con�gures the table entriesof the hardware module. �e table includes all peripherals known tothe SM, the current claiming status, and a list of identi�ers allowed

to request access to the device. By using the privileged owner-ship transfer con�guration command, the SM owner can de�ne anew designated SM owner. �e permission to issue a privilegedcon�guration command is checked by the security monitor usingthe SM owner ID �eld stored in the SM module. �e unprivilegedcommands consist of a claiming and release request command al-lowing the issuer to gain access to a peripheral. A status commandcan be used to determine the permission level of the peripheraland if it is currently claimed. To gain access to an already claimedperipheral, the unprivileged withdraw request command can beused. While a withdraw request issued by the SM owner is alwaysgranted, the request of the unprivileged participant �rst needs to beapproved. When the withdraw request is granted, the security mon-itor uses dedicated interrupt lines F© to notify the current owner ofthe peripheral about the withdraw request. For each peripheral, adedicated interrupt line between SM and TEE or REE is introduced.When the request is approved by the security monitor, the con�gu-ration command is forwarded to the peripheral wrapper over theAXI4-lite crossbar C©.Peripheral Wrapper. An AXI4 read or write request issued bythe TEE or REE and transported over the AXI4 interconnect 3©is not directly sent to the peripheral driver. First, a simple logicdeployed in the peripheral wrapper checks if the identi�er storedin the user signal of the AXI4 request matches the identi�er storedin the ID �eld of the wrapper module. When the process ID or theperipheral ID is set to zero, the access control is only conductedwith the core ID, allowing all entities on either the TEE or REE toaccess the peripheral. �en, if the ID transported in the requestmatches the ID stored in the module, the request is forwarded tothe actual peripheral. However, on an ID mismatch, the peripheralwrapper transports the error code SLVERR to the issuer using theRRESP or BRESP AXI4 signal. In addition to the AXI4 slave interface,the peripheral wrappers also implement an AXI4-lite slave interface.�is interface is used by the SM to set or unset the identi�er in theID �eld of the wrapper.Interrupt. �e peripheral wrapper also is responsible for routingthe interrupt line of the peripheral to the current peripheral owner.To realize correct interrupt handling, the peripheral wrapper mod-ules consists of one dedicated interrupt line for each participant(REE and TEE). Based on the identi�er of the current peripheralowner, the interrupt either is routed to the TEE or the REE.So�ware Support. To interact with the security monitor, we pro-vide a Linux kernel module for the application processor. �is ker-nel module allows the user processes to claim, release, withdraw,or query the status of a peripheral. Furthermore, the kernel modulealso provides functionalities to con�gure the security monitor. Tohandle interrupts from the security monitor withdraw mechanism,each peripheral driver is extended with a dedicated cleanup inter-rupt service routine. �is ISR is responsible for clearing any secrets,aborting communication channels with other parties, and releasingthe peripheral gracefully using the release mechanism.

For the TEE, we provide a small library. Similar to the kernelmodule, this library provides basic functionalities to interact withthe security monitorPhysical Memory Protection. Isolating a peripheral by bindingit exclusively to one entity does not work for the shared, externalDDR3 memory. �erefore, the HECTOR-V architecture introduces

7

TEE

IRQRst

SecurityMonitor

Rocket Chip

TileLinkMEM NetworkIRQRst

REE TEEREE TEEREE TEE REE TEE REE TEE REE TEE REE TEE REE TEE TEE

AXI4

AXI4 Crossbar

AXI4-lite

AXI4AXI4-lite

AX

I4-lite A

XI4

AXI4-lite Crossbar

1

3

2

A

B

C

AXI4-liteInterruptReset

AXI4

3x 4x

BRAM

AXI4 Slave Interface

AXI4 Slave Interface

AX

I4-lite

Sla

veSla

ve

ID_

R

Req

ues

t

ID_

C

ID_C

if(ID_R==ID_C)

elseRequest=SLVERR

Request_B

Request_B=Request

Figure 3: �e HECTOR-V architecture.

a memory protection unit (MPU) ©, which is placed directly be-tween the memory controller and the AXI4 bus interface. �e MPUcan be claimed like any other peripheral by the TEE or REE usingthe security monitor and the dedicated AXI4-lite slave interface.�e party currently claiming the MPU is now able to divide thephysical memory into up to 16 regions. �ese regions can then beeither exclusively accessed by one entity or are shared among mul-tiple entities. Each incoming AXI4 read or write access is checkedby the memory protection unit using the identi�er transportedwith the request. With this mechanism, a secure storage place forthe REE and each virtual core of the RVSCP is enabled and sharedcommunication channels between REE and TEE can be established.Reset Unit. �e reset unit Ê© controls the reset lines of the appli-cation and the secure processor. Similar to other peripherals, thisentity can be claimed by each participant in the system. �e ownerof the device can release the reset lines and turn on or o� the otherentity in the system.Secure Storage. To securely store sensitive data, such as crypto-graphic keys, biometric data, or user passwords, HECTOR-V intro-duces a secure storage element ¤©. In contrast to other peripherals,a prede�ned, immutable identi�er consisting of the core ID and theprocess ID is programmed into the ID �eld of the wrapper moduleat design time. �erefore, only the entity with the correspondingidenti�er can access the storage element. In the prototype, each ofthe four virtual cores of RVSCP posses an own secure storage.

5.3 RVSCP�e RVSCP prototype implementation is based on the 32-bit RISC-VREMUS core [47, 60] already o�ering the sponge-based CFI unit.�e REMUS core originally is based on the Ri5CY core [53], whichachieves similar performance as an ARM Cortex-M4 core. We

further extend the core with the RVSCP features and embed theprocessor into the HECTOR-V architecture.TEE Infrastructure. By using the AXI4 master interface 2© con-nected to the AXI4 crossbar 3©, the RVSCP is able to interact withthe peripherals, such as the UART or PS2 controller. Similar to themain application processor, the secure processor implements anAXI4-lite master interface B© to con�gure and transmit peripheralaccess requests to the security monitor.Context Switch. �e hardware scheduler unit is responsible forperforming secure context switches. For the RVSCP prototype, thehardware scheduler maintains a list of four slots enabling fourvirtual cores VC0 . . .VC3 on the RVSCP core. On a context switch,the hardware scheduler saves the current SCFP state and the currentregister �le and loads the state, the decryption key, and the register�le for the next trustlet. To implement the replacement of theregister �le, we added four additional register sets to our processor.Note that the register �le replacement could also be implemented bystoring and loading the content of the registers to memory, e.g., to asecure storage element, to keep the area overhead of the processorlow. To di�erentiate between the four trustlets executed on theRVSCP, each of the four slots gets assigned an individual processidenti�er. While the core ID is identical for all slots, the hardwarescheduler replaces the process ID directly in the AXI4 and AXI4-lite master interface individually for each slot. By using the samecore ID for all four threads, a peripheral could be con�gured to beaccessible by all four trustlets.Decryption Keys. To decrypt the encrypted instruction stream,the SCFP unit needs to know the decryption key for the correspond-ing trustlet. In the prototype implementation, the key is stored ina dedicated control and status register of each virtual core, whichis only accessible from the respective core. To initially load thekey into the secure storage, our prototype trustlet consists of a

8

write_se(addr,data)

*addr = data

...

write_se(0x0,1)

write_se(0x1,2)

main()

Figure 4: Access function for secure peripheral accesses.

small, unencrypted boot code and the actual, encrypted code. �eunencrypted boot code can either generate the key, load the keyfrom the network, or directly from the binary. A�er storing the keyinto the key register, the actual encrypted trustlet starts to execute.Code Storage. Each of the virtual processor cores VC0 . . .VC3 run-ning on the RVSCP processor executes code from an on-chip BRAM.While VC1 . . .VC3 fetch code from a claimable BRAM �©, the �rstvirtual processor core fetches its code from a secure code storageelement �© exclusively and permanently owned by VC0. To utilizeone of the virtual cores at RVSCP as an enclave, the issuer needs tostore the trustlet code on the BRAM of this core. Since the code ofthe �rst virtual core is stored in a secure storage element �©, thiscode cannot be changed by the REE or by the other virtual cores ofRVSCP at any point in time.Control-Flow Integrity with Secure I/O. To implement the con-cept of binding access to peripherals to a certain CFI state, eachAXI4 request leaving the RVSCP is annotated with the current CFIstate. �e processor directly places a compressed form of the cur-rent state into the 10-bit peripheral identi�er �eld of the AXI4 usersignal. To only allow, e.g., write access to the secure element whenreaching a prede�ned state, the trustlet tunnels all write accessesto this peripheral through the function write se. Since decryptingthis function only works, when a certain state SSE is reached, theSCFP scheme automatically patches the state of the callee with thepatch value pa or pb . As seen in Figure 4, the patching mechanismof SCFP ensures that, on each valid call of the access function, stateSSE is reached. By using this state as the peripheral identi�er, ac-cess to a speci�c peripheral only is granted when reaching state SSEthrough the access function. Similar to the setup procedure of thedecryption keys, the boot code of the trustlet claims the peripheralused in the code by sending the identi�er consisting of the coreidenti�er (ID of RVSCP), the process identi�er (ID of the virtualprocessor), and the peripheral identi�er (compressed CFI state SSE )to the security monitor. Now, access to secrets stored in the securestorage element or other sensitive peripherals, like a �ngerprintreader, only is permi�ed when reaching a prede�ned CFI state. Ifa trustlet does not need explicit protection of certain peripherals,the peripheral identi�er in the peripheral claiming request is set tozero.

5.4 Area OverheadTo quantify the area overhead introduced by the additional RISC-Vcore and the HECTOR-V features, we synthesized the lowRISC baseplatform for a Xilinx Kintex-7 series FPGA. Since the HECTOR-Varchitecture is generic, we synthesized the extended lowRISC plat-form with the additional HECTOR-V features and a selection of dif-ferent RISC-V cores used as TEE processor. Table 1 shows the total

hardware overhead of the HECTOR-V architecture consisting of ei-ther the Ri5CY [53], the REMUS [47, 60], or the FRANKENSTEIN [48]core used as secure TEE processor. �e REMUS core, which is basedon the Ri5CY, implements a decryption unit to o�er the Sponge-Based Control-Flow Protection (SCFP) mechanism. FRANKENSTEIN,which is an extended version of the REMUS core, is a 64-bit RISC-Vprocessor including the SCFP scheme and a pointer protectionscheme for secure memory accesses. While the Rocket Core usedin the lowRISC platform is capable of running Linux, the processorcore still is rather small compared to application processors fromvendors like Intel, ARM, and AMD. �erefore, when using a largerprocessor in the HECTOR-V architecture, the relative overhead intro-duced by the additional secure TEE processor is negligible. In Ta-ble 2, we list area number for di�erent components in the HECTOR-Varchitecture using the RI5CY core as secure TEE processor.

6 USE CASES�is section proposes use cases for the secure TEE design consistingof the HECTOR-V architecture and the RVSCP. More concretely, wedemonstrate how the TEE can be used to boot Linux on the mainapplication processor securely. A�er the secure boot use case, weshow how the RVSCP can be utilized to securely execute trustlets.Note that these two scenarios are only a selection of many otheruse cases that can be realized with HECTOR-V.

6.1 Secure BootOn the unmodi�ed lowRISC platform, a zero stage bootloader (ZSBL)permanently stored in the on-chip BRAM �rst loads the Berke-ley boot loader (BBL) from the external SD-card. �en, the ZSBLhands over control to the BBL �rst stage bootloader (FSBL). �eBBL, which is linked against the Linux kernel, fetches the Linuximage and sets up the environment by con�guring the hardware

Table 1: Lookup table (LUT) overhead of the overall archi-tecture consisting of the HECTOR-V features and an additionalsecure processor compared to the lowRISC base project.

Con�guration Area[LUTs]

Area Overhead[%]

lowRISC base platform 55,443 -HECTOR-V with RI5CY 63,648 14.8HECTOR-V with REMUS 67,024 20.89HECTOR-V with FRANKENSTEIN 68,746 23.99

Table 2: Number of lookup tables for di�erent features ofthe HECTOR-V architecture.

Component Area[LUTs]

Area[%]

Rocket Chip 33,341 52.38RI5CY 5,780 9.08Security Monitor 446 0.7Peripheral Wrapper 43 0.07AXI4 Crossbar 3,052 4.79AXI4-lite Crossbar 93 0.14

9

threads (HARTS) and the memory. Finally, the Linux operatingsystem starts. However, loading the FSBL and the Linux imagedirectly from the SD-card to boot up the device is problematic inmany ways. An a�acker with physical access to the device canboot arbitrary code by simply modifying the unprotected boot �lesstored on the SD-card. �is a�ack methodology also can be used inan online a�ack by overwriting the boot �les. To only allow authen-ticated so�ware to boot, vendors frequently embed a secure bootmechanism into their systems. Here, a chain-of-trust is generatedby authenticating each boot �le before executing it.

In our use case implementing secure boot using features of theHECTOR-V architecture, the �rst virtual core VC0 of RVSCP is thedesignated security monitor owner. At design time, the reset unitof the system is con�gured to release the reset line of RVSCP andkeep the REE processor halted when applying power to the SoC.Additionally, the security monitor owned by VC0 is uncon�gured,except for the reset unit which is claimed by the SM owner. Whenapplying power to the device, VC0 starts to execute the ZSBL storedin the secure code storage �©. Since the secure code storage ispermanently and exclusively owned by VC0, the ZSBL is isolatedfrom the other parties and only VC0 can update this code using anupdate mechanism. Due to these strong protection mechanisms,the ZSBL stored in the secure code storage is the system root-of-trust. �e ZSBL code �rst claims the SD-card controller andcon�gures the memory protection unit of the DDR3 memory. �en,the ZSBL fetches the BBL from the SD card and stores it to theexternal memory. Before passing control to the BBL, the ZSBLdetermines the hash value of the loaded BBL image. When thishash value does not match the expected hash value stored in thesecure storage element ¤© of VC0, the boot process is aborted. Again,an update of the expected hash value only can be initiated by the�rst virtual processor core of RVSCP because the secure storageelement exclusively is owned by this party and can not be claimedby any other party at any point in time. If the veri�cation of theBBL was successful, the ZSBL releases the SD-card driver, gives themain application processor access to the DDR3 memory region bycon�guring the MPU, and triggers the reset unit to start the REE.Now, the Rocket Core starts to execute the modi�ed BBL from theexternal memory. �e modi�ed BBL then requests access to theSD-card controller, loads the Linux image from the SD-card, andveri�es the loaded image by comparing the computed hash valuewith the expected hash value. Finally, if the veri�cation processsucceeds, the Linux operating system starts and can claim the �rstperipherals by sending requests to the security monitor. Since eachstage of the boot process now is cryptographically veri�ed, thesystem is in a genuine state. Now, the VC0 passes the SM ownerprivilege to the main application processor (AP). �is change isinitiated by sending the identi�er of the AP to the SM using theownership transfer command. Although the REE is now in fullcontrol of the system, e.g., is able to switch of the secure processorusing the reset unit, secrets stored in the secure storage ¤© and thesecure code storage �© are still protected from AP accesses.

6.2 TrustletsIn this use case, we utilize the virtual processor VC1 of RVSCP toexecute a trustlet. We compile the trustlet with a LLVM-based

toolchain supporting the SCFP scheme and encrypt the programwith the key K . To interact with the outside and to store secrets,the trustlet uses the embedded UART controller and the securestorage element. �e secure storage element is protected frommalicious accesses by tunneling all requests through a dedicatedaccess function and binding the access to it to the CFI state SSE .We de�ne access to the UART controller as uncritical. �ereforethe interaction with this controller is unprotected.

First, the application processor switches o� the RVSCP by utiliz-ing the reset unit. �en, the processor claims a claimable BRAM �©and stores the code, consisting of an unencrypted small boot codeand the encrypted trustlet, to this storage. A�er the code is saved,the AP sets VC1 as owner of the code storage in the security monitor.Furthermore, the MPU is con�gured to provide a memory regionowned by the trustlet to use as RAM and a shared memory regionto establish a communication channel between REE and the trustlet.Finally, the RVSCP is started by releasing the reset line of the secureprocessor and code gets executed. To initialize key K and stateSSE , the boot code of the trustlet writes the decryption key intothe key register of VC1 and claims the secure key storage elementby se�ing the compressed state SSE into the peripheral identi�er�eld. Now, control is passed to the SCFP protected trustlet codeand the decryption stage of the processor decrypts the code usingthe key. To access the UART controller, the trustlet needs to claimthe controller by sending a claim request to the SM. �e securestorage element is accessed by using the dedicated access function.By se�ing the compressed state directly into the peripheral ID �eldof each AXI4 request leaving the processor, access to the securestorage element only is permi�ed when reaching the prede�nedstate SSE . �e communication channel established using the shared,external memory allows the REE and trustlet to exchange informa-tion. When the designated SM owner, the AP, withdraws access tothe UART controller, the claimable BRAM, or the shared memoryregion, the trustlet is noti�ed using an interrupt. �is interrupt ishandled by the ISR implemented on the trustlet. �e ISR clears allsecrets, aborts communication with the UART controller, gracefullyreleases the peripheral, and enters a prede�ned IDLE state.

6.3 Security DiscussionFor the security analysis of our heterogeneous architecture o�eringa secure environment, we consider three a�acker models.Exploitable Trustlet. �e �rst a�ack model considers a trustletcontaining a bug exploitable by an adversary. Similar to recenta�acks [10, 11], the bug, e.g., a bu�er over�ow, could be exploitedover the communication API between REE and TEE. RVSCP protectsagainst this a�acker model by o�ering a CFI unit mitigating againstcontrol-�ow hijacking a�acks on forward- and backward-edges.Furthermore, RVSCP limits the impact of a data-oriented a�ack byrestricting access to sensitive peripherals by combining the CFIunit with the secure I/O concept.Malicious Trustlet �e second a�acker model considers a trustletcontaining explicit code to a�ack the architecture directly. To pre-vent an adversary from performing a DOS a�ack by permanentlyclaiming all peripherals, HECTOR-V o�ers a withdraw mechanism.Since the core, process, and peripheral ID are managed by the hard-ware, a malicious trustlet can not change these IDs in so�ware

10

to gain unauthorized access to peripherals. While the heteroge-neous architecture prevents an a�acker from performing cacheand transient-based a�acks between REE and TEE, RVSCP needsadditional countermeasures, such as �ushing the microarchitec-tural state [64] on a context switch, to also protect against cachea�acks. Due to the simplicity and openness of RVSCP, transient-based a�acks can not be performed by an adversary. RVSCP protectstrustlets within the processor from each other by using a hardwaremechanism to perform context switches, use di�erent encryptionkeys, and use a separate code storage and RAM region for eachtrustlet.Malicious SecurityMonitor Owner. �is a�acker model consid-ers a malicious SM owner targeting the other entities. �e conceptof se�ing the ID immutable into the ID �eld of critical elementsprevents the SM owner from accessing secrets, such as keys, hashvalues, or the secure boot code. In HECTOR-V, withdrawing accessto speci�c peripherals can only be initiated by using the dedicatedwithdrawing command. Since the withdrawing mechanism noti�esthe current owner of the peripheral about the incoming withdrawprocedure, sensitive data can be �rst cleared. To prevent the SMowner to switch o� a di�erent entity in the system and then accesssecrets stored in the peripherals, the reset unit could, similar to thewithdrawing mechanism, notify the entity about the incoming reset.Except for rese�ing participants and withdrawing peripherals, theexecution of trustlets can not be interrupted by the SM owner.

7 RELATEDWORKConcurrent developed with the HECTOR-V architecture, SiFive re-cently introduced WorldGuard [62]. In WorldGuard, each core getsassigned a world ID and each process on the core can be anno-tated with a process ID. Similar to HECTOR-V, the ID is transportedusing the interconnect and requests from participants are �lteredby peripherals, the memory, and the caches. However, both archi-tectures di�er in various design choices. First, HECTOR-V uses ahardware-based security monitor which only can be con�gured byone party. In contrast to WorldGuard, the security monitor own-ership can be dynamically transferred to other parties allowing�exible use cases. Additionally, the security monitor allows eachparticipant in HECTOR-V to request access to certain peripheralsand request access to already claimed peripherals using a withdrawrequest. We further propose a concrete secure processor design uti-lizing features of the heterogeneous architecture to create a trustedexecution environment. Moreover, we comprehensively describethe hardware-so�ware interaction and demonstrate features ofHECTOR-V by introducing several use case scenarios.

8 CONCLUSION AND FUTUREWORKIn this paper, we proposed HECTOR-V, a secure TEE design strategyusing a heterogeneous CPU architecture. Our design establishessecure paths between peripherals and the cores by tagging eachparty with an identi�er. �e peripherals enforce access permissionsby checking the identi�er, which is transported along with eachbus request. To con�gure these access permissions, we integrate ahardware-based security monitor into the architecture. �e secu-rity monitor, which exclusively can set permissions, is owned bya con�guration party. By allowing to transfer this ownership to

other parties, HECTOR-V allows �exible permission management. Incontrast to similar design approaches, we provide a noti�er-basedmechanism to withdraw access to certain peripherals securely. Wefurther introduce RVSCP, a security-hardened CPU design tailoredfor our architecture. RVSCP combines a �ne-granular control-�owintegrity scheme with the secure I/O concept of HECTOR-V to re-strict access to assets. To complete our TEE design, we introducesecure data and code storage elements, a reset unit, and a memoryprotection unit. We examine the features of our architecture in asecure boot and enclave scenario.Future Work. To o�er a feature commonly used by TEEs, ourplatform can be extended with a remote a�estation mechanism.�is feature could be integrated into RVSCP by utilizing the SCFPscheme [59]. Furthermore, the security monitor can be extended toalso support �ne-granular access permissions, such as read or writeonly. Additionally, as already discussed in Section 6.3, RVSCP needsto be extended to mitigate against cache-based a�acks by either�ushing the cache or integrating ID checks in the cache. Althoughthe identi�er-based system guarantees that peripherals only can beaccessed by certain parties, communication with external devicesa�ached are not authenticated. �e HECTOR-V architecture may beextended with a mechanism to authenticate externally a�acheddevices to ensure that, e.g., the TEE only is communicating withtrusted devices.

ACKNOWLEDGMENTS�is project has received funding from the European ResearchCouncil (ERC) under the European Union’s Horizon 2020 researchand innovation programme (grant agreement No 681402) and bythe Austrian Research Promotion Agency (FFG) via the competencecenter Know-Center (grant number 844595), which is funded in thecontext of COMET - Competence Centers for Excellent Technolo-gies by BMVIT, BMWFW, and Styria.

REFERENCES[1] 2018. CVE-2016-10423.[2] Martın Abadi, Mihai Budiu, Ulfar Erlingsson, and Jay Liga�i. 2009. Control-�ow

integrity principles, implementations, and applications. ACM Trans. Inf. Syst.Secur. 13 (2009). h�ps://doi.org/10.1145/1609956.1609960

[3] Apple. 2020. About the Apple T2 Security Chip. Accessed: 2020-07-10.[4] Apple. 2020. Product security certi�cations, validations, and guidance for SEP:

Secure Key Store. Accessed: 2020-07-11.[5] ARM. 2019. AMBA AXI and ACE Protocol Speci�cation. arm.com (2019).[6] ARM. 2020. Processing Architecture for Power E�ciency and Performance. Ac-

cessed: 2020-07-07.[7] Architecure ARM. 2009. Security technology building a secure system using

trustzone technology (white paper). ARM Limited (2009).[8] Divya Arora, Srivaths Ravi, Anand Raghunathan, and Niraj K. Jha. 2006.

Hardware-Assisted Run-Time Monitoring for Secure Program Execution onEmbedded Processors. IEEE Trans. Very Large Scale Integr. Syst. 14 (2006).h�ps://doi.org/10.1109/TVLSI.2006.887799

[9] Krste Asanovic, Rimas Avizienis, Jonathan Bachrach, Sco� Beamer, David Bian-colin, Christopher Celio, Henry Cook, Daniel Dabbelt, John Hauser, AdamIzraelevitz, Sagar Karandikar, Ben Keller, Donggyu Kim, John Koenig, YunsupLee, Eric Love, Martin Maas, Albert Magyar, Howard Mao, Miquel Moreto, AlbertOu, David A Pa�erson, Brian Richards, Colin Schmidt, Stephen Twigg, Huy Vo,and Andrew Waterman. 2016. �e Rocket Chip Generator. EECS Department,University of California, Berkeley, Technical Report (2016).

[10] Gal Beniamini. 2016. QSEE privilege escalation vulnerability and exploit (CVE-2015-6639). Accessed: 2020-05-13.

[11] Gal Beniamini. 2016. War of the Worlds - Hijacking the Linux Kernel from QSEE.Accessed: 2020-05-13.

[12] Swapnil Bhartiya. 2020. Linux in 2020: 27.8 million lines of code in the kernel, 1.3million in systemd. Accessed: 2020-07-07.

11

[13] Ferdinand Brasser, Urs Muller, Alexandra Dmitrienko, Kari Kostiainen, SrdjanCapkun, and Ahmad-Reza Sadeghi. 2017. So�ware Grand Exposure: SGX CacheA�acks Are Practical. In Workshop on O�ensive Technologies – WOOT.

[14] Jo Van Bulck, Marina Minkin, O�r Weisse, Daniel Genkin, Baris Kasikci, FrankPiessens, Mark Silberstein, �omas F. Wenisch, Yuval Yarom, and Raoul Strackx.2018. Foreshadow: Extracting the Keys to the Intel SGX Kingdom with TransientOut-of-Order Execution. In USENIX Security Symposium.

[15] David Cerdeira, Nuno Santos, Pedro Fonseca, and Sandro Pinto. 2020. SoK:Understanding the Prevailing Security Vulnerabilities in TrustZone-assisted TEESystems. In Proceedings of the IEEE Symposium on Security and Privacy (S&P),San Francisco, CA, USA.

[16] Stephen Checkoway, Lucas Davi, Alexandra Dmitrienko, Ahmad-Reza Sadeghi,Hovav Shacham, and Marcel Winandy. 2010. Return-oriented programmingwithout returns. In Conference on Computer and Communications Security – CCS.h�ps://doi.org/10.1145/1866307.1866370

[17] Guoxing Chen, Sanchuan Chen, Yuan Xiao, Yinqian Zhang, Zhiqiang Lin, andTen-Hwang Lai. 2020. SgxPectre: Stealing Intel Secrets From SGX Enclaves viaSpeculative Execution. IEEE Secur. Priv. 18 (2020). h�ps://doi.org/10.1109/MSEC.2019.2963021

[18] Nick Christoulakis, George Christou, Elias Athanasopoulos, and Sotiris Ioanni-dis. 2016. HCFI: Hardware-enforced Control-Flow Integrity. In Conference onData and Application Security and Privacy – CODASPY. h�ps://doi.org/10.1145/2857705.2857722

[19] Tobias Cloosters, Michael Rodler, and Lucas Davi. 2020. TeeRex: Discovery andExploitation of Memory Corruption Vulnerabilities in {SGX} Enclaves. In 29th{USENIX} Security Symposium ({USENIX} Security 20).

[20] Intel Corporation. 2019. Intel 64 and IA-32 Architectures So�ware Developer’sManual. (2019).

[21] Victor Costan and Srinivas Devadas. 2016. Intel SGX Explained. ePrint 2016/86(2016).

[22] Ruan de Clercq, Johannes Gotzfried, David Ubler, Pieter Maene, and IngridVerbauwhede. 2017. SOFIA: So�ware and control �ow integrity architecture.Comput. Secur. 68 (2017). h�ps://doi.org/10.1016/j.cose.2017.03.013

[23] Jan-Erik Ekberg, Kari Kostiainen, and N. Asokan. 2014. �e Untapped Potentialof Trusted Execution Environments on Mobile Devices. IEEE Secur. Priv. 12 (2014).h�ps://doi.org/10.1109/MSP.2014.38

[24] Jan-Erik Ekberg, Kari Kostiainen, and N Asokan. 2014. �e untapped potentialof trusted execution environments on mobile devices. IEEE Security & Privacy12 (2014).

[25] Trusted Firmware. 2020. OP-TEE. Accessed: 2020-05-12.[26] Google. 2020. Titan in depth: Security in plaintext. Accessed: 2020-07-07.[27] Google. 2020. Trusty TEE. Accessed: 2020-06-07.[28] Johannes Gotzfried, Moritz Eckert, Sebastian Schinzel, and Tilo Muller. 2017.

Cache A�acks on Intel SGX. In Proceedings of the 10th European Workshopon Systems Security, EUROSEC 2017, Belgrade, Serbia, April 23, 2017. h�ps://doi.org/10.1145/3065913.3065915

[29] Roberto Guanciale, Hamed Nemati, Christoph Baumann, and Mads Dam. 2016.Cache Storage Channels: Alias-Driven A�acks and Veri�ed Countermeasures.In IEEE Symposium on Security and Privacy – S&P. h�ps://doi.org/10.1109/SP.2016.11

[30] Shay Gueron. 2016. A Memory Encryption Engine Suitable for General PurposeProcessors. ePrint 2016/204 (2016).

[31] Apple Inc. 2020. Security enclave processor for a system on a chip. Accessed:2020-06-07.

[32] Intel. 2017. Intel SGX and Side-Channels. Accessed: 2020-05-23.[33] �e iPhone Wiki. 2020. Greensburg 14G60 (iPhone6,1). Accessed: 2020-07-07.[34] Sco� Johnson, Dominic Rizzo, Parthasarathy Ranganathan, Jon McCune, and

Richard Ho. 2018. Titan: enabling a transparent silicon root of trust for Cloud.In Hot Chips: A Symposium on High Performance Chips.

[35] Sanjeev Khushu and Wilfred Gomes. 2019. Lake�eld: Hybrid cores in 3D Package.In 2019 IEEE Hot Chips 31 Symposium (HCS), Cupertino, CA, USA, August 18-20,2019. h�ps://doi.org/10.1109/HOTCHIPS.2019.8875641

[36] Jae-Hyuk Lee, Jin Soo Jang, Yeongjin Jang, Nohyun Kwak, Yeseul Choi, ChanghoChoi, Taesoo Kim, Marcus Peinado, and Brent ByungHoon Kang. 2017. Hackingin Darkness: Return-oriented Programming against Secure Enclaves. In USENIXSecurity Symposium.

[37] Moritz Lipp, Daniel Gruss, Raphael Spreitzer, Clementine Maurice, and StefanMangard. 2016. ARMageddon: Cache A�acks on Mobile Devices. In USENIXSecurity Symposium.

[38] LowRISC. 2019. lowRISC Chip. Accessed: 2020-05-07.[39] lowRISC. 2020. OpenTitan Open source silicon root of trust (RoT). Accessed:

2020-06-22.[40] Tarjei Mandt, Mathew Solnik, and David Wang. 2016. Demystifying the secure

enclave processor. Black Hat Las Vegas (2016).[41] Steve McConnell. 2004. Code complete. Pearson Education.[42] Elliot H. MednickEdward McLellan. 2020. Instruction subset implementation for

low power operation. US10698472B2 (2020).

[43] Kit Murdock, David Oswald, Flavio D Garcia, Jo Van Bulck, Daniel Gruss, andFrank Piessens. 2020. Plundervolt: So�ware-based fault injection a�acks againstIntel SGX. In 2020 IEEE Symposium on Security and Privacy (SP).

[44] Bernard Ngabonziza, Daniel Martin, Anna Bailey, Haehyun Cho, and SarahMartin. 2016. TrustZone Explained: Architectural Features and Use Cases. In 2ndIEEE International Conference on Collaboration and Internet Computing, CIC 2016,Pi�sburgh, PA, USA, November 1-3, 2016. h�ps://doi.org/10.1109/CIC.2016.065

[45] Sandro Pinto and Nuno Santos. 2019. Demystifying Arm TrustZone: A Compre-hensive Survey. ACM Comput. Surv. 51 (2019). h�ps://doi.org/10.1145/3291047

[46] Samsung. 2020. KNOX and ARM®TrustZone®. Accessed: 2020-05-07.[47] David Scha�enrath. 2016. Fault-A�ack Secure Processor Design. Graz University

of Technology (2016).[48] Robert Schilling, Mario Werner, Pascal Nasahl, and Stefan Mangard. 2018. Point-

ing in the Right Direction - Securing Memory Accesses in a Faulty World. InAnnual Computer Security Applications Conference – ACSAC. h�ps://doi.org/10.1145/3274694.3274728

[49] Michael Schwarz, Samuel Weiser, Daniel Gruss, Clementine Maurice, and StefanMangard. 2017. Malware Guard Extension: Using SGX to Conceal Cache A�acks.In Detection of Intrusions and Malware & Vulnerability Assessment – DIMVA.h�ps://doi.org/10.1007/978-3-319-60876-1 1

[50] Hovav Shacham. 2007. �e geometry of innocent �esh on the bone: return-into-libc without function calls (on the x86). In Conference on Computer andCommunications Security – CCS. h�ps://doi.org/10.1145/1315245.1315313

[51] Adrian Tang, Simha Sethumadhavan, and Salvatore J. Stolfo. 2017. CLKSCREW:Exposing the Perils of Security-Oblivious Energy Management. In USENIX Secu-rity Symposium.

[52] Caroline Tice, Tom Roeder, Peter Collingbourne, Stephen Checkoway, UlfarErlingsson, Luis Lozano, and Geo� Pike. 2014. Enforcing Forward-Edge Control-Flow Integrity in GCC & LLVM. In USENIX Security Symposium.

[53] Andreas Traber, Florian Zaruba, Sven Stucki, Antonio Pullini, Germain Haugou,Eric Flamand, Frank K Gurkaynak, and Luca Benini. 2016. PULPino: A smallsingle-core RISC-V SoC. In 3rd RISCV Workshop.

[54] Jo Van Bulck, Daniel Moghimi, Michael Schwarz, Moritz Lipp, Marina Minkin,Daniel Genkin, Yuval Yarom, Berk Sunar, Daniel Gruss, and Frank Piessens.2020. LVI: Hijacking transient execution through microarchitectural load valueinjection. In 41th IEEE Symposium on Security and Privacy (S&P’20).

[55] Zhi Wang and Xuxian Jiang. 2010. HyperSafe: A Lightweight Approach toProvide Lifetime Hypervisor Control-Flow Integrity. In IEEE Symposium onSecurity and Privacy – S&P. h�ps://doi.org/10.1109/SP.2010.30

[56] Andrew Waterman, Yunsup Lee, David A. Pa�erson, and Krste Asanovic. 2011.�e RISC-V Instruction Set Manual, Volume I: Base User-Level ISA. TechnicalReport. EECS Department, University of California, Berkeley.

[57] Nico Weichbrodt, Anil Kurmus, Peter R. Pietzuch, and Rudiger Kapitza. 2016.AsyncShock: Exploiting Synchronisation Bugs in Intel SGX Enclaves. In EuropeanSymposium on Research in Computer Security – ESORICS. h�ps://doi.org/10.1007/978-3-319-45744-4 22

[58] Samuel Weiser and Mario Werner. 2017. SGXIO: Generic Trusted I/O Path forIntel SGX. In Conference on Data and Application Security and Privacy – CODASPY.h�ps://doi.org/10.1145/3029806.3029822

[59] Mario Werner. 2020. System Architectures and Techniques for E�cient, Secure,and Trusted Code Execution. Accessed: 2020-08-13.

[60] Mario Werner, �omas Unterluggauer, David Scha�enrath, and Stefan Mangard.2018. Sponge-Based Control-Flow Protection for IoT Devices. In European Sym-posium on Security and Privacy – EuroS&P. h�ps://doi.org/10.1109/EuroSP.2018.00023

[61] Mario Werner, Erich Wenger, and Stefan Mangard. 2015. Protecting the ControlFlow of Embedded Processors against Fault A�acks. In Smart Card Research andAdvanced Applications – CARDIS. h�ps://doi.org/10.1007/978-3-319-31271-2 10

[62] Bob Wheeler. 2019. SIFIVE SECURES RISC-V. Microprocessor report (2019).[63] Johannes Winter. 2008. Trusted computing building blocks for embedded linux-

based ARM trustzone platforms. In Conference on Computer and CommunicationsSecurity – CCS. h�ps://doi.org/10.1145/1456455.1456460

[64] Nils Wisto�, Moritz Schneider, Frank K. Gurkaynak, Luca Benini, and GernotHeiser. 2020. Prevention of Microarchitectural Covert Channels on an Open-Source 64-bit RISC-V Core. arXiv abs/2005.02193 (2020).

[65] Mingwei Zhang and R. Sekar. 2013. Control Flow Integrity for COTS Binaries.In USENIX Security Symposium.

[66] Ning Zhang, He Sun, Kun Sun, Wenjing Lou, and Yiwei �omas Hou. 2016.CacheKit: Evading Memory Introspection Using Cache Incoherence. In EuropeanSymposium on Security and Privacy – EuroS&P. h�ps://doi.org/10.1109/EuroSP.2016.34

[67] Andrew D. Zonenberg and Bulent Yener. 2016. Antikernel: A Decentral-ized Secure Hardware-So�ware Operating System Architecture. In Crypto-graphic Hardware and Embedded Systems – CHES. h�ps://doi.org/10.1007/978-3-662-53140-2 12

12