docs(spm): add design documentation
Add documentation how to build EL3 SPMC,
briefly describes all FF-A interfaces,
SP boot flow, SP Manifest, Power Management,
Boot Info Protocol, Runtime model and state
transition and Interrupt Handling.
Signed-off-by: Shruti Gupta <shruti.gupta@arm.com>
Change-Id: I630df1d50a4621b344a09e462563eacc90109de4
diff --git a/docs/components/el3-spmc.rst b/docs/components/el3-spmc.rst
new file mode 100644
index 0000000..1a2d427
--- /dev/null
+++ b/docs/components/el3-spmc.rst
@@ -0,0 +1,597 @@
+EL3 Secure Partition Manager
+****************************
+
+.. contents::
+
+Foreword
+========
+
+This document describes the design of the EL3 SPMC based on the FF-A specification.
+EL3 SPMC provides reference FF-A compliant implementation without S-EL2 virtualization support,
+to help adopt and migrate to FF-A early.
+EL3 SPMC implementation in TF-A:
+
+- Manages a single S-EL1 Secure Partition
+- Provides a standard protocol for communication and memory sharing between FF-A endpoints.
+- Provides support for EL3 Logical Partitions to support easy migration from EL3 to S-EL1.
+
+Sample reference stack
+======================
+
+The following diagram illustrates a possible configuration when the
+FEAT_SEL2 architecture extension is not implemented, showing the SPMD
+and SPMC at EL3, one S-EL1 secure partition, with an optional
+Hypervisor:
+
+.. image:: ../resources/diagrams/ff-a-spm-at-el3.png
+
+TF-A build options
+==================
+
+This section explains the TF-A build options involved in building
+an FF-A based SPM where the SPMD and SPMC are located at EL3:
+
+- **SPD=spmd**: this option selects the SPMD component to relay the FF-A
+ protocol from NWd to SWd back and forth. It is not possible to
+ enable another Secure Payload Dispatcher when this option is chosen.
+- **SPMC_AT_EL3**: this option adjusts the SPMC exception level to being
+ at EL3.
+- **ARM_SPMC_MANIFEST_DTS**: this option specifies a manifest file
+ providing SP description. It is required when
+ ``SPMC_AT_EL3`` is enabled, the secure partitions are loaded
+ by BL2 on behalf of the SPMC.
+
+Notes:
+
+- BL32 option is re-purposed to specify the S-EL1 TEE or SP image.
+ BL32 option can be omitted if using TF-A Test Secure Payload as SP.
+- BL33 option can specify the TFTF binary or a normal world loader
+ such as U-Boot or the UEFI framework payload.
+
+Sample TF-A build command line when the SPMC is located at EL3:
+
+.. code:: shell
+
+ make \
+ CROSS_COMPILE=aarch64-none-elf- \
+ SPD=spmd \
+ SPMD_SPM_AT_SEL2=0 \
+ SPMC_AT_EL3=1 \
+ BL32=<path-to-tee-binary> (opt for TSP) \
+ BL33=<path-to-bl33-binary> \
+ PLAT=fvp \
+ all fip
+
+FVP model invocation
+====================
+
+Sample FVP command line invocation:
+
+.. code:: shell
+
+ <path-to-fvp-model>/FVP_Base_RevC-2xAEMvA -C pctl.startup=0.0.0.0 \
+ -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 \
+ -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin \
+ -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin \
+ -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log \
+ -C bp.pl011_uart2.out_file=fvp-uart2.log -C bp.vis.disable_visualisation=1
+
+
+Platform Guide
+==============
+
+- Platform Hooks See - `[4]`_
+
+ - plat_spmc_shmem_begin
+ - plat_spmc_shmem_reclaim
+
+SPMC provides platform hooks related to memory management interfaces.
+These hooks can be used for platform specific implementations like
+for managing access control, programming TZ Controller or MPUs.
+These hooks are called by SPMC before the initial share request completes,
+and after the final reclaim has been completed.
+
+- Datastore
+
+ - plat_spmc_shmem_datastore_get
+
+ EL3 SPMC uses datastore for tracking memory transaction descriptors.
+ On FVP platform datastore is allocated from TZC DRAM section.
+ Other platforms need to allocate a similar secure memory region
+ to be used as shared memory datastore.
+
+ The accessor function is used during SPMC initialization to obtain
+ address and size of the datastore.
+ SPMC will also zero out the provided memory region.
+
+- Platform Defines See - `[5]`_
+
+ - SECURE_PARTITION_COUNT
+ Number of Secure Partitions supported: must be 1.
+
+ - NS_PARTITION_COUNT
+ Number of NWd Partitions supported.
+
+ - MAX_EL3_LP_DESCS_COUNT
+ Number of Logical Partitions supported.
+
+Logical Secure Partition (LSP)
+==============================
+
+- The SPMC provides support for statically allocated EL3 Logical Secure Partitions
+ as per FF-A v1.1 specification.
+- The DECLARE_LOGICAL_PARTITION macro can be used to add a LSP.
+- For reference implementation See - `[2]`_
+
+.. image:: ../resources/diagrams/ff-a-lsp-at-el3.png
+
+SPMC boot
+=========
+
+The SPMD and SPMC are built into the BL31 image along with TF-A's runtime components.
+BL2 loads the BL31 image as a part of (secure) boot process.
+
+The SPMC manifest is loaded by BL2 as the ``TOS_FW_CONFIG`` image `[9]`_.
+
+BL2 passes the SPMC manifest address to BL31 through a register.
+
+At boot time, the SPMD in BL31 runs from the primary core, initializes the core
+contexts and launches the SPMC passing the following information through
+registers:
+
+- X0 holds the SPMC manifest blob address.
+- X4 holds the currently running core linear id.
+
+Parsing SP partition manifests
+------------------------------
+
+SPMC consumes the SP manifest, as defined in `[7]`_.
+SP manifest fields align with Hafnium SP manifest for easy porting.
+
+.. code:: shell
+
+ compatible = "arm,ffa-manifest-1.0";
+
+ ffa-version = <0x00010001>; /* 31:16 - Major, 15:0 - Minor */
+ id = <0x8001>;
+ uuid = <0x6b43b460 0x74a24b78 0xade24502 0x40682886>;
+ messaging-method = <0x3>; /* Direct Messaging Only */
+ exception-level = <0x2>; /* S-EL1 */
+ execution-state = <0>;
+ execution-ctx-count = <8>;
+ gp-register-num = <0>;
+ power-management-messages = <0x7>;
+
+
+Passing boot data to the SP
+---------------------------
+
+In `[1]`_ , the section "Boot information protocol" defines a method for passing
+data to the SPs at boot time. It specifies the format for the boot information
+descriptor and boot information header structures, which describe the data to be
+exchanged between SPMC and SP.
+The specification also defines the types of data that can be passed.
+The aggregate of both the boot info structures and the data itself is designated
+the boot information blob, and is passed to a Partition as a contiguous memory
+region.
+
+Currently, the SPM implementation supports the FDT type which is used to pass the
+partition's DTB manifest.
+
+The region for the boot information blob is statically allocated (4K) by SPMC.
+BLOB contains Boot Info Header, followed by SP Manifest contents.
+
+The configuration of the boot protocol is done in the SP manifest. As defined by
+the specification, the manifest field 'gp-register-num' configures the GP register
+which shall be used to pass the address to the partitions boot information blob when
+booting the partition.
+
+Supported interfaces
+====================
+
+The following interfaces are exposed to SPs only:
+
+- ``FFA_MSG_WAIT``
+- ``FFA_MEM_RETRIEVE_REQ``
+- ``FFA_MEM_RETRIEVE_RESP``
+- ``FFA_MEM_RELINQUISH``
+- ``FFA_SECONDARY_EP_REGISTER``
+
+The following interfaces are exposed to both NS Client and SPs:
+
+- ``FFA_VERSION``
+- ``FFA_FEATURES``
+- ``FFA_RX_RELEASE``
+- ``FFA_RXTX_MAP``
+- ``FFA_RXTX_UNMAP``
+- ``FFA_PARTITION_INFO_GET``
+- ``FFA_ID_GET``
+- ``FFA_MSG_SEND_DIRECT_REQ``
+- ``FFA_MSG_SEND_DIRECT_RESP``
+- ``FFA_MEM_FRAG_TX``
+- ``FFA_SPM_ID_GET``
+
+The following additional interfaces are forwarded from SPMD to support NS Client:
+
+- ``FFA_RUN``
+- ``FFA_MEM_LEND``
+- ``FFA_MEM_SHARE``
+- ``FFA_MEM_FRAG_RX``
+- ``FFA_MEM_RECLAIM``
+
+
+FFA_VERSION
+-----------
+
+``FFA_VERSION`` requires a *requested_version* parameter from the caller.
+SPMD forwards call to SPMC, the SPMC returns its own implemented version.
+SPMC asserts SP and SPMC are at same FF-A Version.
+
+FFA_FEATURES
+------------
+
+FF-A features supported by the SPMC may be discovered by secure partitions at
+boot (that is prior to NWd is booted) or run-time.
+
+The SPMC calling FFA_FEATURES at secure physical FF-A instance always get
+FFA_SUCCESS from the SPMD.
+
+The request made by an Hypervisor or OS kernel is forwarded to the SPMC and
+the response relayed back to the NWd.
+
+
+FFA_RXTX_MAP
+------------
+
+FFA_RXTX_UNMAP
+--------------
+
+When invoked from a secure partition FFA_RXTX_MAP maps the provided send and
+receive buffers described by their PAs to the EL3 translation regime
+as secure buffers in the MMU descriptors.
+
+When invoked from the Hypervisor or OS kernel, the buffers are mapped into the
+SPMC EL3 translation regime and marked as NS buffers in the MMU
+descriptors.
+
+The FFA_RXTX_UNMAP unmaps the RX/TX pair from the translation regime of the
+caller, either it being the Hypervisor or OS kernel, as well as a secure
+partition.
+
+FFA_PARTITION_INFO_GET
+----------------------
+
+Partition info get call can originate:
+
+- from SP to SPMC
+- from Hypervisor or OS kernel to SPMC. The request is relayed by the SPMD.
+
+The format (v1.0 or v1.1) of the populated data structure returned is based upon the
+FFA version of the calling entity.
+
+EL3 SPMC also supports returning only the count of partitions deployed.
+
+All LSPs and SP are discoverable from FFA_PARTITION_INFO_GET call made by
+either SP or NWd entities.
+
+FFA_ID_GET
+----------
+
+The FF-A ID space is split into a non-secure space and secure space:
+
+- FF-A ID with bit 15 clear relates to VMs.
+- FF-A ID with bit 15 set related to SPs or LSPs.
+- FF-A IDs 0, 0xffff, 0x8000 are assigned respectively to the Hypervisor
+ (or OS Kernel if Hyp is absent), SPMD and SPMC.
+
+This convention helps the SPM to determine the origin and destination worlds in
+an FF-A ABI invocation. In particular the SPM shall filter unauthorized
+transactions in its world switch routine. It must not be permitted for a VM to
+use a secure FF-A ID as origin world by spoofing:
+
+- A VM-to-SP direct request/response shall set the origin world to be non-secure
+ (FF-A ID bit 15 clear) and destination world to be secure (FF-A ID bit 15
+ set).
+- Similarly, an SP-to-LSP direct request/response shall set the FF-A ID bit 15
+ for both origin and destination IDs.
+
+An incoming direct message request arriving at SPMD from NWd is forwarded to
+SPMC without a specific check. The SPMC is resumed through eret and "knows" the
+message is coming from normal world in this specific code path. Thus the origin
+endpoint ID must be checked by SPMC for being a normal world ID.
+
+An SP sending a direct message request must have bit 15 set in its origin
+endpoint ID and this can be checked by the SPMC when the SP invokes the ABI.
+
+The SPMC shall reject the direct message if the claimed world in origin endpoint
+ID is not consistent:
+
+- It is either forwarded by SPMD and thus origin endpoint ID must be a "normal
+ world ID",
+- or initiated by an SP and thus origin endpoint ID must be a "secure world ID".
+
+
+FFA_MSG_SEND_DIRECT_REQ
+-----------------------
+
+FFA_MSG_SEND_DIRECT_RESP
+------------------------
+
+This is a mandatory interface for secure partitions participating in direct request
+and responses with the following rules:
+
+- An SP can send a direct request to LSP.
+- An LSP can send a direct response to SP.
+- An SP cannot send a direct request to an Hypervisor or OS kernel.
+- An Hypervisor or OS kernel can send a direct request to an SP or LSP.
+- An SP and LSP can send a direct response to an Hypervisor or OS kernel.
+- SPMD can send direct request to SPMC.
+
+FFA_SPM_ID_GET
+--------------
+
+Returns the FF-A ID allocated to an SPM component which can be one of SPMD
+or SPMC.
+
+At initialization, the SPMC queries the SPMD for the SPMC ID, using the
+FFA_ID_GET interface, and records it. The SPMC can also query the SPMD ID using
+the FFA_SPM_ID_GET interface at the secure physical FF-A instance.
+
+Secure partitions call this interface at the virtual FF-A instance, to which
+the SPMC returns the SPMC ID.
+
+The Hypervisor or OS kernel can issue the FFA_SPM_ID_GET call handled by the
+SPMD, which returns the SPMC ID.
+
+FFA_ID_GET
+----------
+
+Returns the FF-A ID of the calling endpoint.
+
+FFA_MEM_SHARE
+-------------
+
+FFA_MEM_LEND
+------------
+
+- If SP is borrower in the memory transaction, these calls are forwarded to SPMC.
+ SPMC performs Relayer responsibilities, caches the memory descriptors in the datastore,
+ and allocates FF-A memory handle.
+- If format of descriptor was v1.0, SPMC converts the descriptor to v1.1 before caching.
+ In case of fragmented sharing, conversion of memory descriptors happens after last
+ fragment has been received.
+- Multiple borrowers (including NWd endpoint) and fragmented memory sharing are supported.
+
+FFA_MEM_RETRIEVE_REQ
+--------------------
+
+FFA_MEM_RETRIEVE_RESP
+---------------------
+
+- Memory retrieve is supported only from SP.
+- SPMC fetches the cached memory descriptor from the datastore,
+- Performs Relayer responsiilities and sends FFA_MEM_RETRIEVE_RESP back to SP.
+- If descriptor size is more than RX buffer size, SPMC will send the descriptor in fragments.
+- SPMC will set NS Bit to 1 in memory descriptor response.
+
+FFA_MEM_FRAG_RX
+---------------
+
+FFA_MEM_FRAG_TX
+---------------
+
+FFA_MEM_FRAG_RX is to be used by:
+
+- SP if FFA_MEM_RETRIEVE_RESP returned descriptor with fragment length less than total length.
+- or by SPMC if FFA_MEM_SHARE/FFA_MEM_LEND is called with fragment length less than total length.
+
+SPMC validates handle and Endpoint ID and returns response with FFA_MEM_FRAG_TX.
+
+FFA_SECONDARY_EP_REGISTER
+-------------------------
+
+When the SPMC boots, secure partition is initialized on its primary
+Execution Context.
+
+The FFA_SECONDARY_EP_REGISTER interface is to be used by a secure partition
+from its first execution context, to provide the entry point address for
+secondary execution contexts.
+
+A secondary EC is first resumed either upon invocation of PSCI_CPU_ON from
+the NWd or by invocation of FFA_RUN.
+
+Power management
+================
+
+In platforms with or without secure virtualization:
+
+- The NWd owns the platform PM policy.
+- The Hypervisor or OS kernel is the component initiating PSCI service calls.
+- The EL3 PSCI library is in charge of the PM coordination and control
+ (eventually writing to platform registers).
+- While coordinating PM events, the PSCI library calls backs into the Secure
+ Payload Dispatcher for events the latter has statically registered to.
+
+When using the SPMD as a Secure Payload Dispatcher:
+
+- A power management event is relayed through the SPD hook to the SPMC.
+- In the current implementation CPU_ON (svc_on_finish), CPU_OFF
+ (svc_off), CPU_SUSPEND (svc_suspend) and CPU_SUSPEND_RESUME (svc_suspend_finish)
+ hooks are registered.
+
+Secure partitions scheduling
+============================
+
+The FF-A specification `[1]`_ provides two ways to relinquinsh CPU time to
+secure partitions. For this a VM (Hypervisor or OS kernel), or SP invokes one of:
+
+- the FFA_MSG_SEND_DIRECT_REQ interface.
+- the FFA_RUN interface.
+
+Additionally a secure interrupt can pre-empt the normal world execution and give
+CPU cycles by transitioning to EL3.
+
+Partition Runtime State and Model
+=================================
+
+EL3 SPMC implements Partition runtime states are described in v1.1 FF-A specification `[1]`_
+
+An SP can be in one of the following state:
+
+- RT_STATE_WAITING
+- RT_STATE_RUNNING
+- RT_STATE_PREEMPTED
+- RT_STATE_BLOCKED
+
+An SP will transition to one of the following runtime model when not in waiting state:
+
+- RT_MODEL_DIR_REQ
+- RT_MODEL_RUN
+- RT_MODEL_INIT
+- RT_MODEL_INTR
+
+Platform topology
+=================
+
+SPMC only supports a single Pinned MP S-EL1 SP. The *execution-ctx-count*
+SP manifest field should match the number of physical PE.
+
+Interrupt handling
+==================
+
+Secure Interrupt handling
+-------------------------
+
+- SPMC is capable of forwarding Secure interrupt to S-EL1 SP
+ which has preempted the normal world.
+- Interrupt is forwarded to SP using FFA_INTERRUPT interface.
+- Interrupt Number is not passed, S-EL1 SP can access the GIC registers directly.
+- Upon completion of Interrupt handling SP is expected to return to
+ SPMC using FFA_MSG_WAIT interface.
+- SPMC returns to normal world after interrupt handling is completed.
+
+In the scenario when secure interrupt occurs while the secure partition is running,
+the SPMC is not involved and the handling is implementation defined in the TOS.
+
+Non-Secure Interrupt handling
+-----------------------------
+
+The 'managed exit' scenario is the responsibility of the TOS and the SPMC is not involved.
+
+Test Secure Payload (TSP)
+=========================
+
+- TSP provides reference implementation of FF-A programming model.
+- TSP has the following support:
+
+ - SP initialization on all CPUs.
+ - Consuming Power Messages including CPU_ON, CPU_OFF, CPU_SUSPEND, CPU_SUSPEND_RESUME.
+ - Event Loop to receive Direct Requests.
+ - Sending Direct Response.
+ - Memory Sharing helper library.
+ - Ability to handle secure interrupt (timer).
+
+TSP Tests in CI
+---------------
+
+- TSP Tests are exercised in the TF-A CI using prebuilt FF-A Linux Test driver in NWd.
+- Expected output:
+
+.. code:: shell
+
+ #ioctl 255
+ Test: Echo Message to SP.
+ Status: Completed Test Case: 1
+ Test Executed Successfully
+
+ Test: Message Relay vis SP to EL3 LSP.
+ Status: Completed Test Case: 2
+ Test Executed Successfully
+
+ Test: Memory Send.
+ Verified 1 constituents successfully
+ Status: Completed Test Case: 3
+ Test Executed Successfully
+
+ Test: Memory Send in Fragments.
+ Verified 256 constituents successfully
+ Status: Completed Test Case: 4
+ Test Executed Successfully
+
+ Test: Memory Lend.
+ Verified 1 constituents successfully
+ Status: Completed Test Case: 5
+ Test Executed Successfully
+
+ Test: Memory Lend in Fragments.
+ Verified 256 constituents successfully
+ Status: Completed Test Case: 6
+ Test Executed Successfully
+
+ Test: Memory Send with Multiple Endpoints.
+ random: fast init done
+ Verified 256 constituents successfully
+ Status: Completed Test Case: 7
+ Test Executed Successfully
+
+ Test: Memory Lend with Multiple Endpoints.
+ Verified 256 constituents successfully
+ Status: Completed Test Case: 8
+ Test Executed Successfully
+
+ Test: Ensure Duplicate Memory Send Requests are Rejected.
+ Status: Completed Test Case: 9
+ Test Executed Successfully
+
+ Test: Ensure Duplicate Memory Lend Requests are Rejected.
+ Status: Completed Test Case: 10
+ Test Executed Successfully
+
+ 0 Tests Failed
+
+ Exiting Test Application - Total Failures: 0
+
+
+References
+==========
+
+.. _[1]:
+
+[1] `Arm Firmware Framework for Arm A-profile <https://developer.arm.com/docs/den0077/latest>`__
+
+.. _[2]:
+
+[2] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/fvp_el3_spmc_logical_sp.c
+
+.. _[3]:
+
+[3] `Trusted Boot Board Requirements
+Client <https://developer.arm.com/documentation/den0006/d/>`__
+
+.. _[4]:
+
+[4] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/fvp_el3_spmc.c
+
+.. _[5]:
+
+[5] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/include/platform_def.h
+
+.. _[6]:
+
+[6] https://trustedfirmware-a.readthedocs.io/en/latest/components/ffa-manifest-binding.html
+
+.. _[7]:
+
+[7] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/fdts/fvp_tsp_sp_manifest.dts
+
+.. _[8]:
+
+[8] https://lists.trustedfirmware.org/archives/list/tf-a@lists.trustedfirmware.org/thread/CFQFGU6H2D5GZYMUYGTGUSXIU3OYZP6U/
+
+.. _[9]:
+
+[9] https://trustedfirmware-a.readthedocs.io/en/latest/design/firmware-design.html#dynamic-configuration-during-cold-boot
+
+--------------
+
+*Copyright (c) 2020-2022, Arm Limited and Contributors. All rights reserved.*