| Context Management Library |
| ************************** |
| |
| This document provides an overview of the Context Management library implementation |
| in Trusted Firmware-A (TF-A). It enumerates and describes the APIs implemented |
| and their accessibility from other components at EL3. |
| |
| Overview |
| ======== |
| |
| Arm TrustZone architecture facilitates hardware-enforced isolation between |
| software running in various security states (Secure/Non-Secure/Realm). |
| The general-purpose registers, most of the system registers and vector registers |
| are not banked per world. When moving between the security states it is the |
| responsibility of the secure monitor software (BL31(AArch64) / BL32(Aarch32)) |
| in TF-A, not the hardware, to save and restore register state. |
| Refer to `Trustzone for AArch64`_ for more details. |
| |
| EL3 Runtime Firmware, also termed as secure monitor firmware, is integrated |
| with a context management library to handle the context of the CPU, managing the |
| saving and restoring of register states across the worlds. |
| |
| TF-A Context |
| ============ |
| |
| In TF-A, the context is represented as a data structure used by the EL3 firmware |
| to preserve the state of the CPU at the next lower exception level (EL) in a given |
| security state and save enough EL3 metadata to be able to return to that exception |
| level and security state. The memory for the context data structures are allocated |
| in BSS section of EL3 firmware. |
| |
| In a trusted system at any instance, a given CPU could be executing in one of the |
| security states (Non-Secure, Secure, Realm). Each world must have its |
| configuration of system registers independent of other security states to access |
| and execute any of the architectural features. |
| |
| If the CPU switches across security states (for example: from Non-secure to Secure |
| or vice versa), the register contents, especially the ones that are not banked |
| (EL2/EL1, vector, general-purpose registers), will be overwritten, as the software |
| running in either state has the privileges to access them. Additionally, some of |
| the architectural features enabled in the former security state will be unconditionally |
| accessible in the latter security state as well. This can be a major concern when |
| dealing with security-specific bits, as they need to be explicitly enabled or |
| disabled in each state to prevent data leakage across the worlds. |
| |
| In general, an ideal trusted system should have Secure world-specific configurations |
| that are not influenced by Normal World operations. Therefore, for each CPU, we |
| need to maintain world-specific context to ensure that register entries from one |
| world do not leak or impact the execution of the CPU in other worlds. |
| This will help ensure the integrity and security of the system, preventing any |
| unauthorized access or data corruption between the different security states. |
| |
| Design |
| ====== |
| |
| The Context Management library in TF-A is designed to cover all the requirements |
| for maintaining world-specific context essential for a trusted system. |
| This includes implementing CPU context initialization and management routines, |
| as well as other helper APIs that are required by dispatcher components in EL3 |
| firmware, which are collectively referred to as CPU Context Management. |
| The APIs and their usecases are listed in detail under the :ref:`Library APIs` |
| section. |
| |
| Originally, the Context Management library in TF-A was designed to cater for a |
| two-world system, comprising of Non-Secure and Secure Worlds. In this case, the |
| EL3 Firmware is assumed to be running in Secure World. |
| With introduction of Realm Management Extension (RME), from Armv9.2 a system |
| can have four distinct worlds (Non-Secure, Secure, Realm, Root). |
| RME isolates EL3 from all other Security states and moves it into its own security |
| state called root. EL3 firmware now runs at Root World and thereby is |
| trusted from software in Non-secure, Secure, and Realm states. |
| Refer to `Security States with RME`_ for more details. |
| |
| Key principles followed in designing the context management library : |
| |
| 1. **EL3 should only initialize immediate used lower EL** |
| |
| Context Management library running at EL3 should only initialize and monitor the |
| immediate used lower EL. This implies that, when S-EL2 is present in the system, |
| EL3 should initialise and monitor S-EL2 registers only. S-EL1 registers should |
| not be the concern of EL3 while S-EL2 is in place. In systems where S-EL2 is |
| absent, S-EL1 registers should be initialised from EL3. |
| |
| 2. **Decentralized model for context management** |
| |
| Each world (Non-Secure, Secure, and Realm) should have their separate component |
| in EL3 responsible for their respective world context management. |
| Both the Secure and Realm world have associated dispatcher components in EL3 |
| firmware to allow management of the respective worlds. For the Non-Secure world, |
| PSCI Library (BL31)/context management library provides routines to help |
| initialize the Non-Secure world context. |
| |
| 3. **Flexibility for Dispatchers to select desired feature set to save and restore** |
| |
| Each feature is supported with a helper function ``is_feature_supported(void)``, |
| to detect its presence at runtime. This helps dispatchers to select the desired |
| feature set, and thereby save and restore the configuration associated with them. |
| |
| 4. **Dynamic discovery of Feature enablement by EL3** |
| |
| TF-A supports three states for feature enablement at EL3, to make them available |
| for lower exception levels. |
| |
| .. code:: c |
| |
| #define FEAT_STATE_DISABLED 0 |
| #define FEAT_STATE_ENABLED 1 |
| #define FEAT_STATE_CHECK 2 |
| |
| A pattern is established for feature enablement behavior. |
| Each feature must support the 3 possible values with rigid semantics. |
| |
| - **FEAT_STATE_DISABLED** - all code relating to this feature is always skipped. |
| Firmware is unaware of this feature. |
| |
| - **FEAT_STATE_ALWAYS** - all code relating to this feature is always executed. |
| Firmware expects this feature to be present in hardware. |
| |
| - **FEAT_STATE_CHECK** - same as ``FEAT_STATE_ALWAYS`` except that the feature's |
| existence will be checked at runtime. Default on dynamic platforms (example: FVP). |
| |
| .. note:: |
| ``FEAT_RAS`` is an exception here, as it impacts the execution of EL3 and |
| it is essential to know its presence at compile time. Refer to ``ENABLE_FEAT`` |
| macro under :ref:`Build Options` section for more details. |
| |
| Code Structure |
| ============== |
| |
| `lib/el3_runtime/(aarch32/aarch64)`_ - Context library code directory. |
| |
| Source Files |
| ~~~~~~~~~~~~ |
| |
| #. ``context_mgmt.c`` : consists of core functions that setup, save and restore |
| context for different security states alongside high level feature enablement |
| APIs for individual worlds. |
| |
| #. ``cpu_data_array.c`` : contains per_cpu_data structure instantiation. |
| |
| #. ``context.S`` : consists of functions that save and restore some of the context |
| structure members in assembly code. |
| |
| #. ``cpu_data.S`` : consists of helper functions to initialise per_cpu_data pointers. |
| |
| #. ``el3_common_macros.S`` : consists of macros to facilitate actions to be performed |
| during cold and warmboot and el3 registers initialisation in assembly code. |
| |
| Header Files |
| ~~~~~~~~~~~~ |
| |
| #. ``context_mgmt.h`` : contains the public interface to Context Management Library. |
| |
| #. ``context.h`` : contains the helper macros and definitions for context entries. |
| |
| #. ``cpu_data.h`` : contains the public interface to Per CPU data structure. |
| |
| #. ``context_debug.h`` : contains public interface to report context memory |
| utilisation across the security states. |
| |
| #. ``context_el2.h`` : internal header consisting of helper macros to access EL2 |
| context entries. Used by ``context.h``. |
| |
| Apart from these files, we have some context related source files under ``BL1`` |
| and ``BL31`` directory. ``bl1_context_mgmt.c`` ``bl31_context_mgmt.c`` |
| |
| Bootloader Images utilizing Context Management Library |
| ====================================================== |
| |
| +-------------------------------------------+-----------------------------+ |
| | Bootloader | Context Management Library | |
| +-------------------------------------------+-----------------------------+ |
| | BL1 | Yes | |
| +-------------------------------------------+-----------------------------+ |
| | BL2 | No | |
| +-------------------------------------------+-----------------------------+ |
| | BL31 (Aarch64- EL3runtime firmware) | Yes | |
| +-------------------------------------------+-----------------------------+ |
| | BL32 (Aarch32- EL3runtime firmware) | Yes | |
| +-------------------------------------------+-----------------------------+ |
| |
| CPU Data Structure |
| ================== |
| For a given system, depending on the CPU count, the platform statically |
| allocates memory for the CPU data structure. |
| |
| .. code:: c |
| |
| /* The per_cpu_ptr_cache_t space allocation */ |
| cpu_data_t percpu_data[PLATFORM_CORE_COUNT]; |
| |
| This CPU data structure has a member element with an array of pointers to hold |
| the Non-Secure, Realm and Secure security state context structures as listed below. |
| |
| .. code:: c |
| |
| typedef struct cpu_data { |
| #ifdef __aarch64__ |
| void *cpu_context[CPU_DATA_CONTEXT_NUM]; |
| #endif |
| |
| .... |
| .... |
| |
| }cpu_data_t; |
| |
| |CPU Data Structure| |
| |
| At runtime, ``cpu_context[CPU_DATA_CONTEXT_NUM]`` array will be intitialised with |
| the Secure, Non-Secure and Realm context structure addresses to ensure proper |
| handling of the register state. |
| See :ref:`Library APIs` section for more details. |
| |
| CPU Context and Memory allocation |
| ================================= |
| |
| CPU Context |
| ~~~~~~~~~~~ |
| The members of the context structure used by the EL3 firmware to preserve the |
| state of CPU across exception levels for a given security state are listed below. |
| |
| .. code:: c |
| |
| typedef struct cpu_context { |
| gp_regs_t gpregs_ctx; |
| el3_state_t el3state_ctx; |
| el1_sysregs_t el1_sysregs_ctx; |
| |
| #if CTX_INCLUDE_EL2_REGS |
| el2_sysregs_t el2_sysregs_ctx; |
| #endif |
| |
| #if CTX_INCLUDE_FPREGS |
| fp_regs_t fpregs_ctx; |
| #endif |
| |
| cve_2018_3639_t cve_2018_3639_ctx; |
| #if CTX_INCLUDE_PAUTH_REGS |
| pauth_t pauth_ctx; |
| #endif |
| |
| #if CTX_INCLUDE_MPAM_REGS |
| mpam_t mpam_ctx; |
| #endif |
| |
| } cpu_context_t; |
| |
| Context Memory Allocation |
| ~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| CPUs maintain their context per world. The individual context memory allocation |
| for each CPU per world is allocated by the world-specific dispatcher components |
| at compile time as shown below. |
| |
| |Context memory allocation| |
| |
| NS-Context Memory |
| ~~~~~~~~~~~~~~~~~ |
| It's important to note that the Normal world doesn't possess the dispatcher |
| component found in the Secure and Realm worlds. Instead, the PSCI library at EL3 |
| handles memory allocation for ``Non-Secure`` world context for all CPUs. |
| |
| .. code:: c |
| |
| static cpu_context_t psci_ns_context[PLATFORM_CORE_COUNT]; |
| |
| Secure-Context Memory |
| ~~~~~~~~~~~~~~~~~~~~~ |
| Secure World dispatcher (such as SPMD) at EL3 allocates the memory for ``Secure`` |
| world context of all CPUs. |
| |
| .. code:: c |
| |
| static spmd_spm_core_context_t spm_core_context[PLATFORM_CORE_COUNT]; |
| |
| Realm-Context Memory |
| ~~~~~~~~~~~~~~~~~~~~ |
| Realm World dispatcher (RMMD) at EL3 allocates the memory for ``Realm`` world |
| context of all CPUs. |
| |
| .. code:: c |
| |
| rmmd_rmm_context_t rmm_context[PLATFORM_CORE_COUNT]; |
| |
| To summarize, the world-specific context structures are synchronized with |
| per-CPU data structures, which means that each CPU will have an array of pointers |
| to individual worlds. The figure below illustrates the same. |
| |
| |CPU Context Memory Configuration| |
| |
| Context Setup/Initialization |
| ============================ |
| |
| The CPU has been assigned context structures for every security state, which include |
| Non-Secure, Secure and Realm. It is crucial to initialize each of these structures |
| during the bootup of every CPU before they enter any security state for the |
| first time. This section explains the specifics of how the initialization of |
| every CPU context takes place during both cold and warm boot paths. |
| |
| Context Setup during Cold boot |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| The cold boot path is mainly executed by the primary CPU, other than essential |
| CPU initialization executed by all CPUs. After executing BL1 and BL2, the Primary |
| CPU jumps to the BL31 image for runtime services initialization. |
| During this process, the per_cpu_data structure gets initialized with statically |
| allocated world-specific context memory. |
| |
| Later in the cold boot sequence, the BL31 image at EL3 checks for the presence |
| of a Secure world image at S-EL2. If detected, it invokes the secure context |
| initialization sequence under SPMD. Additionally, based on RME enablement, |
| the Realm context gets initialized from the RMMD at EL3. Finally, before exiting |
| to the normal world, the Non-Secure context gets initialized via the context |
| management library. At this stage, all Primary CPU contexts are initialized |
| and the CPU exits EL3 to enter the Normal world. |
| |
| |Context Init ColdBoot| |
| |
| .. note:: |
| The figure above illustrates a scenario on FVP for one of the build |
| configurations with TFTF component at NS-EL2. |
| |
| Context Setup during Warmboot |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| During a warm boot sequence, the primary CPU is responsible for powering on the |
| secondary CPUs. Refer to :ref:`CPU Reset` and :ref:`Firmware Design` sections for |
| more details on the warm boot. |
| |
| |Context Init WarmBoot| |
| |
| The primary CPU initializes the Non-Secure context for the secondary CPU while |
| restoring re-entry information for the Non-Secure world. |
| It initialises via ``cm_init_context_by_index(target_idx, ep )``. |
| |
| ``psci_warmboot_entrypoint()`` is the warm boot entrypoint procedure. |
| During the warm bootup process, secondary CPUs have their secure context |
| initialized through SPMD at EL3. Upon successful SP initialization, the SPD |
| power management operations become shared with the PSCI library. During this |
| process, the SPMD duly registers its handlers with the PSCI library. |
| |
| .. code:: c |
| |
| file: psci_common.c |
| const spd_pm_ops_t *psci_spd_pm; |
| |
| file: spmd_pm.c |
| const spd_pm_ops_t spmd_pm = { |
| .svc_on_finish = spmd_cpu_on_finish_handler, |
| .svc_off = spmd_cpu_off_handler |
| } |
| |
| Secondary CPUs during their bootup in the ``psci_cpu_on_finish()`` routine get |
| their secure context initialised via the registered SPMD handler |
| ``spmd_cpu_on_finish_handler()`` at EL3. |
| The figure above illustrates the same with reference of Primary CPU running at |
| NS-EL2. |
| |
| .. _Library APIs: |
| |
| Library APIs |
| ============ |
| |
| The public APIs and types can be found in ``include/lib/el3_runtime/context_management.h`` |
| and this section is intended to provide additional details and clarifications. |
| |
| Context Initialization for Individual Worlds |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| The library implements high level APIs for the CPUs in setting up their individual |
| context for each world (Non-Secure, Secure and Realm). |
| |
| .. c:function:: static void setup_context_common(cpu_context_t *ctx, const entry_point_info_t *ep); |
| |
| This function is responsible for the general context initialization that applies |
| to all worlds. It will be invoked first, before calling the individual |
| world-specific context setup APIs. |
| |
| .. c:function:: static void setup_ns_context(cpu_context_t *ctx, const struct entry_point_info *ep); |
| .. c:function:: static void setup_realm_context(cpu_context_t *ctx, const struct entry_point_info *ep); |
| .. c:function:: static void setup_secure_context(cpu_context_t *ctx, const struct entry_point_info *ep); |
| |
| Depending on the security state that the CPU needs to enter, the respective |
| world-specific context setup handlers listed above will be invoked once per-CPU |
| to set up the context for their execution. |
| |
| .. c:function:: void cm_manage_extensions_el3(void) |
| |
| This function initializes all EL3 registers whose values do not change during the |
| lifetime of EL3 runtime firmware. It is invoked from each CPU via the cold boot |
| path ``bl31_main()`` and in the WarmBoot entry path ``void psci_warmboot_entrypoint()``. |
| |
| Runtime Save and Restore of Registers |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| EL1 Registers |
| ------------- |
| |
| .. c:function:: void cm_el1_sysregs_context_save(uint32_t security_state); |
| .. c:function:: void cm_el1_sysregs_context_restore(uint32_t security_state); |
| |
| These functions are utilized by the world-specific dispatcher components running |
| at EL3 to facilitate the saving and restoration of the EL1 system registers |
| during a world switch. |
| |
| EL2 Registers |
| ------------- |
| |
| .. c:function:: void cm_el2_sysregs_context_save(uint32_t security_state); |
| .. c:function:: void cm_el2_sysregs_context_restore(uint32_t security_state); |
| |
| These functions are utilized by the world-specific dispatcher components running |
| at EL3 to facilitate the saving and restoration of the EL2 system registers |
| during a world switch. |
| |
| Pauth Registers |
| --------------- |
| |
| Pointer Authentication feature is enabled by default for Non-Secure world and |
| disabled for Secure and Realm worlds. In this case, we don't need to explicitly |
| save and restore the Pauth registers during world switch. |
| However, ``CTX_INCLUDE_PAUTH_REGS`` flag is explicitly used to enable Pauth for |
| lower exception levels of Secure and Realm worlds. In this scenario, we save the |
| general purpose and Pauth registers while we enter EL3 from lower ELs via |
| ``prepare_el3_entry`` and restore them back while we exit EL3 to lower ELs |
| via ``el3_exit``. |
| |
| .. code:: c |
| |
| .macro save_gp_pmcr_pauth_regs |
| func restore_gp_pmcr_pauth_regs |
| |
| Feature Enablement for Individual Worlds |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| .. c:function:: static void manage_extensions_nonsecure(cpu_context_t *ctx); |
| .. c:function:: static void manage_extensions_secure(cpu_context_t *ctx); |
| .. c:function:: static void manage_extensions_realm(cpu_context_t *ctx) |
| |
| Functions that allow the enabling and disabling of architectural features for |
| each security state. These functions are invoked from the top-level setup APIs |
| during context initialization. |
| |
| Further, a pattern is established for feature enablement code (AArch64). |
| Each feature implements following APIs as applicable: |
| Note: (``xxx`` is the name of the feature in the APIs) |
| |
| - ``is_feat_xxx_supported()`` and ``is_feat_xxx_present()`` - mandatory for all features. |
| |
| - ``xxx_enable(cpu_context * )`` and ``xxx_disable(cpu_context * )`` - optional |
| functions to enable the feature for the passed context only. To be called in |
| the respective world's setup_context to select behaviour. |
| |
| - ``xxx_init_el3()`` - optional function to enable the feature in-place in any EL3 |
| registers that are never context switched. The values they write must never |
| change, otherwise the functions mentioned in previous point should be used. |
| Invoked from ``cm_manage_extensions_el3()``. |
| |
| - ``xxx_init_el2_unused()`` - optional function to enable the feature in-place |
| in any EL2 registers that are necessary for execution in EL1 with no EL2 present. |
| |
| The above mentioned rules, followed for ``FEAT_SME`` is shown below: |
| |
| .. code:: c |
| |
| void sme_enable(cpu_context_t *context); |
| void sme_init_el3(void); |
| void sme_init_el2_unused(void); |
| void sme_disable(cpu_context_t *context); |
| |
| Per-world Context |
| ================= |
| |
| Apart from the CPU context structure, we have another structure to manage some |
| of the EL3 system registers whose values are identical across all the CPUs |
| referred to as ``per_world_context_t``. |
| The Per-world context structure is intended for managing EL3 system registers with |
| identical values across all CPUs, requiring only a singular context entry for each |
| individual world. This structure operates independently of the CPU context |
| structure and is intended to manage specific EL3 registers. |
| |
| .. code-block:: c |
| |
| typedef struct per_world_context { |
| uint64_t ctx_cptr_el3; |
| uint64_t ctx_zcr_el3; |
| uint64_t ctx_mpam3_el3; |
| } per_world_context_t; |
| |
| These functions facilitate the activation of architectural extensions that possess |
| identical values across all cores for the individual Non-secure, Secure, and |
| Realm worlds. |
| |
| *Copyright (c) 2024, Arm Limited and Contributors. All rights reserved.* |
| |
| .. |Context Memory Allocation| image:: ../resources/diagrams/context_memory_allocation.png |
| .. |CPU Context Memory Configuration| image:: ../resources/diagrams/cpu_data_config_context_memory.png |
| .. |CPU Data Structure| image:: ../resources/diagrams/percpu-data-struct.png |
| .. |Context Init ColdBoot| image:: ../resources/diagrams/context_init_coldboot.png |
| .. |Context Init WarmBoot| image:: ../resources/diagrams/context_init_warmboot.png |
| .. _Trustzone for AArch64: https://developer.arm.com/documentation/102418/0101/TrustZone-in-the-processor/Switching-between-Security-states |
| .. _Security States with RME: https://developer.arm.com/documentation/den0126/0100/Security-states |
| .. _lib/el3_runtime/(aarch32/aarch64): https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/lib/el3_runtime |