The Magic of Intel’s SGX
A Tutorial on Programming a Secure Enclave
This article and the demo are joint work with Daniel Eggelmann and Sascha Greve under supervision of Prof. Dr. Sebastian Gajek.
TL;TR
This article aims at:
- introducing the new programming paradigm of secure enclaves enabled by Intel’ SGX technology
- giving a tutorial how to create a “hello world” secure enclave
LINK TO THE DEMO REPO
“There’s no silver bullet solution with cybersecurity, a layered approach is the only viable defence.”
— James Scott, Institute for Critical Infrastructure Technology
While the danger from inside the internet gets bigger and bigger, attacks on uncritical as well as on critical systems get more common, its daily business in the present world. As long as you or your company are interested in long term safety.
If your boss doesn’t want to spend a few dollars for IT-Security with the reason that you can simply unplug the ethernet cable from the critical systems in the company, you’re fine. It’s the ultimate way on the lowest layer to make your system secure. If that’s not the case, feel free to use this guide on why and how to use Intel’s Security Guard Extension for your programming projects to ensure security from the hardware layer on. You’re welcome.
Intel introduced the Security Guard Extensions (SGX) in the 3rd quarter of 2015 together with the start of the new Skylake processor series. The focus of SGX is to protect sensitive data against untrusted user, even on already compromised systems, with the help of hard implemented security and crypto mechanism inside the CPU. New instructions and memory access changes making it possible for the software developer to create encrypted enclaves containing sensible data from banking apps or key wallets while retaining confidentiality and integrity. These enclaves are only accessible from inside itself and plain text is only visible while it is processed inside the CPU, keeping the stored information safe, even from privileged software like VMM’s, BIOS or OS’.
Okay, but what was the essence of the informative but non-meaningful paragraph before? This blog entry will answer following points:
● Why is my sensible data safe from unauthorized access and modification?
● How is my sensible data kept confidential and integrity protected?
● Where are possible gateways to break the SGX protection?
And further:
● How can I build my own SGX protected applications?
● What do I need for programming with SGX?
● What does the application user need for using SGX / Is the app user capable of using the SGX functionality I built into my app?
Let’s bone up the theory
About apps and enclaves
The security goals of SGX are to protect the confidentiality and integrity of its enclaves. This means untrusted parties like applications or system software should not be able to gain information from the enclave or manipulate the enclaves data and code. We start with information of the data structure and the general architecture of SGX, which lead to the desired safety.
In the SGX model an app is divided into a trusted and an untrusted part. The untrusted part is the one communicating with the rest of the system as well as creating the enclave, which is considered as the safe part of the app. Inside the enclave the sensible data is stored. Fixed starting positions inside the enclave ensure the correct execution inside the enclave. To enter this enclaves, the CPU has to be in enclave mode, which acts like a switch into the safe world.
Further enclave instances are isolated from each other, so they can only interact through their untrusted part. A specific enclave can only be addressed by its trusted app part. This makes it even impossible for malicious programs to get into an enclave, when the CPU is in enclave mode, but the access command comes from an (for the enclave) untrusted memory address.
The enclave bunker
Enclaves are stored in the Enclave Page Cache (EPC). Following figure shows the specific SGX structures, including data used to security check and manage the enclave entry points. These are, besides the SIGSTRUCT and the Version Page Array, enclave related elements. Each enclave has its own SECS, one or more TCS’ and corresponding SSA’s. The EPC in general has it’s SIGSTRUCT and VA Page.
Further the Enclave Page Cache Map (EPCM) is used to manage the security attributes of the Enclave Pages. The EPCM contains further information for each page listed in the EPC. The figure below lists the data acquired for each page.
The EPC is located inside the Processor Reserved Memory (PRM) inside the DRAM, the EPCM is a look-up table inside the CPU with enclave related data.
Since we store our data outside the CPU in the PRM, we must ensure its safeness by encrypting it with the help of the Memory Encryption Engine (MEE). This way the data in the PRM is nothing more than noise to other parties, only the MEE is capable to real time decrypt the data inside the CPU. Therefore, the CPU is the only place in the system capable of reading the clear data stored inside the enclaves.
The CPU as our doorman
As we have seen before, the only participant privileged to enter the EPC, respectively the PRM, is the CPU. The method of deciding whether the processor can enter this secure area depends on the mode, the CPU is running at. SGX implements an Enclave Mode, which the CPU must enter to get specific memory access semantics.
This mode gets enabled, once internal enclave code wants to run inside its specific enclave area in the EPC. To sum up, there are three requirements an access must fulfil to get protected memory access: the processor runs in enclave mode, the requested page is part of the same enclave the access has been requested and the page access must be through the right specific virtual address.
If a running enclave is stopped or interrupted, any context information like registers are removed from the CPU, therefore by interrupting the enclave an attacker cannot gain any in-formation from the enclave. But before this happens, related registers are evacuated into their explicit enclave, where they are saved in an encrypted state in the EPC in the PRM on the DRAM.
Blue line: Enclave 1 can access every page inside its own enclave. Additionally, it can access non PRM memory. That means, that only our enclave can run commands inside itself. Further the enclave can run commands outside its safe space trough the untrusted side of its app. Access from the PRM into the enclave is only possible through defined call functionalities for entering an enclave trough the untrusted app side, there is no other way entering an enclave.
Red line: If code inside a page from e.g. enclave n wants to enter a page of enclave 1, it gets redirected to a nonexistent memory. This is backed by the virtual memory address check of the MMU. If an access is tried from a virtual address outside the explicit enclaves’ virtual address, it gets either redirected to nonexistent memory or it throws a signal fault. This behaviour prevents attackers entering from the safe PRM area to neighbouring enclaves, just because of the point, that the CPU is running in enclave mode. Entering from the non PRM area is generally blocked by the circumstance, that the CPU isn’t running in enclave mode.
More detailed information for SGX structures available @ quarkslab blog entry about Intel SGX and good paperwork @ Intel, Innovative Instructions and Software Model for Isolated Execution
Prepare your development environment
To develop applications with Intel SGX following software components are required:
● Visual Studio
● Intel SGX Platform Software (This is needed to execute SGX applications)
SGX applications can be developed and debugged without SGX hardware. A simulator is shipped with the SGX SDK.
Let’s say hello to the world
The link to the example’s github repo:
Design
Applications developed for SGX need to incorporate special considerations. SGX applications are divided into two parts. The trusted part of an SGX application is executed within the enclave, while the untrusted part is executed outside. All parts of the application dealing with sensitive data need to be placed within the trusted part.
Between the trusted and untrusted part an interface is implemented. This interface consists of e-calls and o-calls. E-calls are functions within the enclave called from untrusted code, meanwhile o-calls are functions of the untrusted code called from within the enclave.
These interfaces are defined with a special file format, called the EDL-file. Interfaces are defined by their signature. The signature is assigned to the trusted part of the application if it is an interface of the enclave. Otherwise it is assigned to the untrusted part.
Special considerations need to be taken for buffers or pointers passed as parameters. Within the squared brackets in front of a parameter the direction and the size of a buffer are defined. On the trusted site enclave interface buffers with the direction “in” are read from the normal application memory to the enclave memory, while the direction “out” results in data being passed from enclave memory to the normal application. Untrusted interface directions are opposite to their trusted equivalents.
Implementation
The first step of the implementation is to set up two projects in Visual Studio. An C++-application containing the untrusted code and a seperate project for the enclave. The Intel SGX SDK provides a wizard for Visual Studio that sets up the enclave project correctly.
After creating the projects, the EDL file needs to be filled with the interfaces. These interfaces need to be implemented within the enclave and inside the normal application.
From within the application SGX needs to be enabled with:
Further the enclave must be created, with signed library compiled from the enclaves code:
Now the enclave can be used by calling any of the enclaves e-calls.
The usage of e-calls differs from calling normal methods. Two additional parameters have to be provided, first the id of the enclave (eid) and a pointer to a variable that will hold the result of the e-call. The actual return value of the e-call will only indicate, whether the e-call itself was successful, which does not necessarily indicate the successful execution.
To invoke an o-call from within the enclave, the corresponding method has to be defined in our untrusted application, after that it can be called like a normal function.
Example
As an example, an application to sign messages with ECDSA (Elliptic Curve Digital Signature Algorithm) will be developed. All cryptographic functionality like key generation, signing and verification will be handled within the enclave. Private keys will only be stored within the enclave or sealed and stored on the filesystem. That means untrusted code don’t has access to it. All cryptographic functionality will be implemented with the help of SGXs cryptographic API.
Interfaces
The enclave provides five interfaces, named: esv_init, esv_sealkeys, esv_sign, esv_verify and esv_close. With two interfaces from the untrusted application esv_read_data and esv_write_data, the enclave is able to read and write files.
Init
Esv_init initializes everything necessary to use the signing and verifying interfaces. It receives a optional filename for loading sealed keys from filesystem.
First the context for the elliptic curve cryptography is retrieved. This context has to be used with all calls to cryptographic methods for elliptic curves.
If a file name was provided (sealed_data_file), the file is loaded with the o-call esv_read_data. Then the data is copied into the memory of the enclave, unsealing cannot work if the provided buffer is located outside of the enclave. The size of the unsealed data is determined with sgx_get_encrypt_txt_len, which result is used to reserve a buffer for the unsealed data. After the buffer is allocated the data is unsealed and written into the destined buffer.
If no sealed data is provided, a new key pair (public, private key) is generated with sgx_ecc256_create_key_pair.
Either way the public and private key are stored in global variables of the enclave.
Signing
Esv_sign receives as parameters a message to sign. The message is passed on to sgx_ecdsa_sign, which signs the message and saves the signature in the provided signature buffer. The signature is saved to a file with the filename consisting of the message and the extension “.sig”. It should be noted that strings passed as parameters to o- and e-calls cannot be altered in size, otherwise the method will crash when returning. Therefore the filename is created with snprintf and saved to a new string. The o-call esv_write_data is used to write to the file. For the sake of simplicity we use only strings in our example but you could of course also sign other forms of data.
Verification
Esv_verify takes a message and its signature. The signature for the message is then verified.
The method sgx_ecdsa_verify executes the signature verification, its result is returned by the ecall. If the signature is valid, the result will be zero, otherwise unequal to zero.
Key-Sealing
Esv_seal_keys saves the current public and private key stored in the enclave to a file. We use a data structure for that, called “esv_sealed_data_t”. The filename is provided as parameter. To avoid third parties from outside the enclave to access the keys, they are sealed with SGXs data sealing feature.
First the size of the sealed data is calculated with sgx_calc_sealed_data_size. Then a corresponding buffer is allocated. With sgx_seal_data the data is encrypted and saved to the allocated buffer. If the sealing was successful the data is saved to a file, with the o-call esv_write_data.
If the application ends you can use the method sgx_destroy_enclave to destroy an enclave and release its associated resources and invalidates the enclave ID or handle.
Is my SGX app really safe?
Further we will discuss specific questions related to SGX safety aspects. No defense layer is unbreakable, even so latest security technologies aren’t insuperable. Examples of how to break SGX should be found with the help of your favourite search engine in no time. This blog entry doesn’t aim for this information due to the fact, that new technologies cause new exploits on a base — this post could barely sufficient cover new exploits — therefore only a few more or less open safety depending questions are mentioned at the end. We will focus on the safe and sound line of defense.
Who guarantess, that no attacker infiltrates my enclave already during initialization?
SGX treats the system software as untrusted and potentially malicious. For this reason the measurement of enclaves is used. It allows the application creating the enclave to ensure the setup was done correct. This includes the commands used as well as the data and code setup in the enclave. This prevents the system software or privileged attackers from manipulating the enclaves code and data during the setup, for example by loading other pages then requested by the application or manipulating the pages before they are loaded into the enclave.
Why is my sensible data safe from unauthorized access and modification?
SGX decides inside the CPU, who is trusted to work inside an enclave. This way a powerful, hardware-based security gate is implemented, and no vulnerable software is used for this decision-making process.
The CPU manages inside its Memory Management Unit (MMU), whether an access to a specific area of its PRM is allowed or not. See section 4 for that process.
Inside each enclave one or more Thread Control Structures (TCS) contain fixed entry points for enclave entering, making it impossible to enter through illegal gateways. A safe programming style inside the enclave will make it impossible for attackers to enter the enclave or to investigate possible in- and outputs of the enclave by simple testing.
Further the EPCM contains the authorized virtual address and the enclave owner, which is necessary for entering a specific enclave. If an access from a non-trusted virtual address occurs, it will be redirected to non-existent memory or a signal fault will be throwed. This EPCM is located inside the CPU, and only the CPU is capable to access it in realtime
Last but not least - every page inside the enclave has different permission like read-only. This builds an extra barrier for attackers, to harm enclaves.
How is my sensible data kept confidential and integrity protected?
Each enclave contains the SECS, where its specific hash for integrity checks is stored. This hash is stored in the enclave, which is stored inside the EPC, which gets real time decrypted through the MEE inside the CPU in enclave mode. The SGX unit contains an fixed secret key, generated at the manufacturing process of the chip, which should be even unknown to Intel (you have to trust Intel at that point) for decrypting inside the MEE. The decryption is only possible with this secret key, which is inaccessible from outside the SGX unit.
Where are possible gateways to break the SGX protection on the software/hardware side?
Data inside the enclave is stored safe, unless an unprivileged party gets inside it. Since enclaves get created at the beginning of an app start from the untrusted part of the app, a gateway for attackers could lay here.
Further the SGX on a specific system is only safe, if the secret hardware-based key inside the Memory Encryption Engine is unknown to everybody else as the SGX module itself.
The Processor Reserved Memory gets initiated at the very beginning of starting the system from the BIOS. SGX is only stated safe, if the PRM is inaccessible for other parts of the system besides the CPU. Hardware attacks trough physical ways would be needed for this example.
A malicious enclave can attack software through cache attacks to steal secret information of enclaves. A proof of concept is published in the paper Malware Guard Extension: Using SGX to Conceal Cache Attacks.
Speculative Executions are a plus for the performance but allows to read decrypted enclave data when inside the CPU e.g. in the cache illegally by other software.
Further the given Intel security guidelines must be followed to prevent leakages caused by bad code.
Conclusion
Productive applications need to take special design considerations to make use of SGX. The applications code base needs to be divided into trusted and untrusted components. In addition, developers need to provide alternatives implementations for devices without SGX hardware.
The general implementation of SGX is fairly simply, the software requirements are limited, as well as the necessary steps needed for setup. The programming interface is well documented and easy to use.
Debugging applications can be sometimes problematic. Error codes or messages output by the api interfaces are often not clear, which leads to having to solve problems by trial and error. For example changing sizes of SGX allocated buffers can result in unexpected crashes or sealed data has to be copied into the enclaves memory before it can be decrypted.
But if implemented right, SGX offers additional protection, with a fairly low amount of effort needed to put into development.
