Modified:
Published:
In this post, we are going to discuss the latest Intel SGX technology from Intel. Intel SGX is from the suite of TEE solutions that has technologies from various vendors such as ARM TrustZone from ARM, XX from XX, and much more. These solutions enable a capability in the latest computer chips that have been missing until now. It allows the hardware of the system to guarantee the CFI property of an execution. CFI stands for control-flow integrity and is used to determine the security of an executing program.
A TEE is an essential framework in today’s setting, where the computing is moving into the remote data centers. A third party manages these data centers with physical and root-level access to the machines. This presents a unique security challenge for customers. How can a user be sure that their code and data are safe on these remote machines? Also, how to guarantee a tamper-free execution of the code. We will now discuss a TEE solution in the context of Intel SGX, also referred to as just SGX. SGX solutions cryptographically bind the execution context of a binary to a hardware secret. This binding ensures that the code is running on a real-machine, and not on a virtual machine. Furthermore, the hardware checks the signature of a binary after loading it into the memory with a pre-computed signature of a known valid application. Any mismatch in this suggests tampering of the code and the execution halts.
Execution protection
Apart from this, SGX also protects a binary during the execution. It does so by marking a portion of the DRAM as secure during the boot process. All the content in this region of the memory is encrypted and integrity verified by a hardware component called MEE or memory encryption engine. MEE is a part of the SGX suite, and transparently handles the integrity checks and encryption-decryption in this secure memory. This secure memory is called PRM, or process reserved memory. The size of this PRM can be 32MB, 64MB, or 128MB. We will be using the 128MB setting for all our experiments and explanation in this paper. The SGX hardware manages the PRM and uses a part of this to store metadata. The SGX reserves 32MB in the PRM to store the metadata. The rest of the memory, 92MB, is used to store user data and is called EPC or enclave page cache.
SGX Components
Intel SGX is a set of 17 new instructions, as shown in Table XX. Along with this, it constitutes of an MEE that is responsible for transparent encryption-decryption of the data. MEE also ensures the integrity and the freshness of the data. The MEE detects any tampering with the data and is capable of shutting the system down in such cases. It also constitutes specific registers. The provisioning and verification system uses these registers during code execution.
Enclave
Intel SGX assigns an enclave to each code executing within it. SGX hardware manages all the metadata associated with the enclave.
Structures
SGX also contains 13 new data structures, out of which the enclave management uses 8 of them, page management 3 of them, and resource management 2 of them. Table XX shows the data structures and their usage. Page information (PAGEINFO): An EPC page uses a pageinfo structure to point inside the EPC. This structure contains pointers to SECINFO and SECS structures. Security information (SECINFO): The metadata related to an EPC page, such as access control information along with its type (SECS, TCS, REG, VA), is stored in this structure. SXG enclave control structure (SECS): Each enclave is assigned a SECS structure that stores the metadata related to an enclave. SECS stores the hash and code size. Thread control structure TCS: Each enclave is assigned with a TCS, which indicates an execution point in the code. As Intel SGX supports multi-threading, it can have multiple TCS structures associated with it. Save state area SSA: Each TCS is associated with at-least an SSA structure that stores the state of the thread during the execution. Stack and Heap: Each enclave has its stack and heap. PCMD paging crypto metadata: This is used to track the metadata of an evicted page. This structure points to a SECINFO and has a MAC. VA Version array: This array stores the version number of the pages evicted from the EPC. It has 512 slots of 8 bytes to store the version number. Hence, it can save 512 pages. Assuming a page size of 4KB, the total size comes to 2MB. This space is not enough; therefore, it must also use something else.
Page swappings.
As noted earlier, the size of the EPC is limited to 92 MB of memory. This space is not enough for most of the workloads nowadays. Hence, SGX must provision for swapping of pages in and out of the EPC to the non-secure memory. As mentioned in SGX explained, and the VAULT paper (which probably took it from SGX explained) is that SGX maintains a Merkle tree for the pages that are evicted from the EPC. This tree of hashes for the evicted pages is called an eviction tree.
- Where is this eviction tree stored?
- When is it updated?
- When is it used?
- What is the overhead of doing so?
- In what cases it may cause significant overhead?
EPCM
Diving a little deeper into the workings of the Intel SGX points out a security hole due to the caching effects of the CPU. As already discussed, SGX assumes that the CPU is secure, and hence the data can reside in plain text format in the caches. The TLB will contain the virtual to physical address mapping for these data points. Due to the multiprocess nature of the kernel, an enclave will context switched many times. Leaving the TLB entries as it creates a security vulnerability, as the OS can manipulate the page table entries of a non-secure application and read the data from the cache. To prevent this kind of attack, SGX flushes all the entries during a context switch. Furthermore, as the page table is under OS control, a malicious OS can manipulate the page table entries during a TLB miss to send false data. To prevent this, SGX maintains an array of mapping, called enclave page cache map or EPCM. It maps the physical address of a page to its virtual address and its SECS structure. After the TLB resolution, EPCM ensures the correctness of the address resolution. This address validation is done only for the address range in the EPC pointed out by ELRANGE (Enclave Linear Address Range).
Overheads
In this section, we are going to explore the overheads that might creep in the performance of executing a binary when using Intel SGX. The significant overhead comes from accessing the data in the DRAM. Within DRAM, it matters whether the data is in the EPC region or the un-secure region. If the data is in the caches, then it just takes 2–3 cycles to access the data. If there is a miss in the cache, then the data is brought in from the EPC. In this case, the MEE transparently handles the decryption and the validation of the data. This step takes around 200 cycles. However, it might be the case that the data is not present in the EPC, and has to be bought in from the un-secure region of the memory. This step is the most expensive one, as the data is first decrypted and verified by the CPU, followed by the MEE operations. The whole procedure takes around 40K CPU cycles. Furthermore, it might be the case that the data has is in the swap disk. In that case, this whole step takes 20,000 additional cycles [https://www.nvmexpress.org/wp-content/uploads/NVM-Express-Optimized-Interface-for-PCI-Express-SSDs-SF13_SSDS004_100.pdf Slide 16].
Enclave provisioning
