doc/board/ti/k3.rst

.. SPDX-License-Identifier: GPL-2.0+ OR BSD-3-Clause
.. sectionauthor:: Bryan Brattlof <bb@ti.com>

K3 Generation
=============

Summary
-------

Texas Instrument's K3 family of SoCs utilize a heterogeneous multicore
and highly integrated device architecture targeted to maximize
performance and power efficiency for a wide range of industrial,
automotive and other broad market segments.

Typically the processing cores and the peripherals for these devices are
partitioned into three functional domains to provide ultra-low power
modes as well as accommodating application and industrial safety systems
on the same SoC.  These functional domains are typically called the:

* Wakeup (WKUP) domain
* Micro-controller (MCU) domain
* Main domain

For a more detailed view of what peripherals are attached to each
domain, consult the device specific documentation.

K3 Based SoCs
-------------

.. toctree::
   :maxdepth: 1

   am62x_sk
   am65x_evm
   j7200_evm
   j721e_evm

Boot Flow Overview
------------------

For all K3 SoCs the first core started will be inside the Security
Management Subsystem (SMS) which will secure the device and start a core
in the wakeup domain to run the ROM code. ROM will then initialize the
boot media needed to load the binaries packaged inside `tiboot3.bin`,
including a 32bit U-Boot SPL, (called the wakup SPL) that ROM will jump
to after it has finished loading everything into internal SRAM.

.. code-block:: text

   |  WKUP Domain
    ROM -> WKUP SPL ->

The wakeup SPL, running on a wakeup domain core, will initialize DDR and
any peripherals needed load the larger binaries inside the `tispl.bin`
into DDR.  Once loaded the wakeup SPL will start one of the 'big'
application cores inside the main domain to initialize the main domain,
starting with Trusted Firmware-A (TF-A), before moving on to start
OP-TEE and the main domain's U-Boot SPL.

.. code-block:: text

   |  WKUP Domain   | Main Domain ->
    ROM -> WKUP SPL -> TF-A -> OP-TEE -> Main SPL

The main domain's SPL, running on a 64bit application core, has
virtually unlimited space (billions of bytes now that DDR is working) to
initialize even more peripherals needed to load in the `u-boot.img`
which loads more firmware into the micro-controller & wakeup domains and
finally prepare the main domain to run Linux.

.. code-block:: text

   |  WKUP Domain   | Main Domain ->
    ROM -> WKUP SPL -> TF-A -> OP-TEE -> Main SPL -> UBoot -> Linux

This is the typical boot flow for all K3 based SoCs, however this flow
offers quite a lot in the terms of flexibility, especially on High
Security (HS) SoCs.

Boot Flow Variations
^^^^^^^^^^^^^^^^^^^^

All K3 SoCs will generally use the above boot flow with two main
differences depending on the capabilities of the boot ROM and the number
of cores inside the device. These differences split the bootflow into
essentially 4 unique but very similar flows:

* Split binary with a combined firmware: (eg: AM65)
* Combined binary with a combined firmware: (eg: AM64)
* Split binary with a split firmware: (eg: J721E)
* Combined binary with a split firmware: (eg: AM62)

For devices that utilize the split binary approach, ROM is not capable
of loading the firmware into the SoC requiring the wakeup domain's
U-Boot SPL to load the firmware.

Devices with a split firmware will have two firmwares loaded into the
device at different times during the bootup process. TI's Foundational
Security (TIFS), needed to operate the Security Management Subsystem,
will either be loaded by ROM or the WKUP U-Boot SPL, then once the
wakeup U-Boot SPL has completed, the second Device Management (DM)
firmware can be loaded on the now free core in the wakeup domain.

For more information on the bootup process of your SoC, consult the
device specific boot flow documentation.

Software Sources
----------------

All scripts and code needed to build the `tiboot3.bin`, `tispl.bin` and
`u-boot.img` for all K3 SoCs can be located at the following places
online

.. k3_rst_include_start_boot_sources

* **Das U-Boot**

  | **source:** https://source.denx.de/u-boot/u-boot.git
  | **branch:** master

* **Trusted Firmware-A (TF-A)**

  | **source:** https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/
  | **branch:** master

* **Open Portable Trusted Execution Environment (OP-TEE)**

  | **source:** https://github.com/OP-TEE/optee_os.git
  | **branch:** master

* **TI Firmware (TIFS, DM, DSMC)**

  | **source:** https://git.ti.com/git/processor-firmware/ti-linux-firmware.git
  | **branch:** ti-linux-firmware

.. k3_rst_include_end_boot_sources

Build Procedure
---------------

Depending on the specifics of your device, you will need three or more
binaries to boot your SoC.

* `tiboot3.bin` (bootloader for the wakeup domain)
* `tispl.bin` (bootloader for the main domain)
* `u-boot.img`

During the bootup process, both the 32bit wakeup domain and the 64bit
main domains will be involved. This means everything inside the
`tiboot3.bin` running in the wakeup domain will need to be compiled for
32bit cores and most binaries in the `tispl.bin` will need to be
compiled for 64bit main domain CPU cores.

All of that to say you will need both a 32bit and 64bit cross compiler
(assuming you're using an x86 desktop)

.. k3_rst_include_start_common_env_vars_desc
.. list-table:: Generic environment variables
   :widths: 25 25 50
   :header-rows: 1

   * - S/w Component
     - Env Variable
     - Description
   * - All Software
     - CC32
     - Cross compiler for ARMv7 (ARM 32bit), typically arm-linux-gnueabihf-
   * - All Software
     - CC64
     - Cross compiler for ARMv8 (ARM 64bit), typically aarch64-linux-gnu-
   * - All Software
     - LNX_FW_PATH
     - Path to TI Linux firmware repository
   * - All Software
     - TFA_PATH
     - Path to source of Trusted Firmware-A
   * - All Software
     - OPTEE_PATH
     - Path to source of OP-TEE
.. k3_rst_include_end_common_env_vars_desc

.. k3_rst_include_start_common_env_vars_defn
.. code-block:: bash

 $ export CC32=arm-linux-gnueabihf-
 $ export CC64=aarch64-linux-gnu-
 $ export LNX_FW_PATH=path/to/ti-linux-firmware
 $ export TFA_PATH=path/to/trusted-firmware-a
 $ export OPTEE_PATH=path/to/optee_os
.. k3_rst_include_end_common_env_vars_defn

We will also need some common environment variables set up for the various
other build sources. we shall use the following, in the build descriptions below:

.. k3_rst_include_start_board_env_vars_desc
.. list-table:: Board specific environment variables
   :widths: 25 25 50
   :header-rows: 1

   * - S/w Component
     - Env Variable
     - Description
   * - U-Boot
     - UBOOT_CFG_CORTEXR
     - Defconfig for Cortex-R (Boot processor).
   * - U-Boot
     - UBOOT_CFG_CORTEXA
     - Defconfig for Cortex-A (MPU processor).
   * - Trusted Firmware-A
     - TFA_BOARD
     - Platform name used for building TF-A for Cortex-A Processor.
   * - Trusted Firmware-A
     - TFA_EXTRA_ARGS
     - Any extra arguments used for building TF-A.
   * - OP-TEE
     - OPTEE_PLATFORM
     - Platform name used for building OP-TEE for Cortex-A Processor.
   * - OP-TEE
     - OPTEE_EXTRA_ARGS
     - Any extra arguments used for building OP-TEE.
.. k3_rst_include_end_board_env_vars_desc

Building tiboot3.bin
^^^^^^^^^^^^^^^^^^^^^

1. To generate the U-Boot SPL for the wakeup domain, use the following
   commands, substituting :code:`{SOC}` for the name of your device (eg:
   am62x) to package the various firmware and the wakeup UBoot SPL into
   the final `tiboot3.bin` binary. (or the `sysfw.itb` if your device
   uses the split binary flow)

.. k3_rst_include_start_build_steps_spl_r5
.. code-block:: bash

 $ # inside u-boot source
 $ make $UBOOT_CFG_CORTEXR
 $ make CROSS_COMPILE=$CC32 BINMAN_INDIRS=$LNX_FW_PATH
.. k3_rst_include_end_build_steps_spl_r5

At this point you should have all the needed binaries to boot the wakeup
domain of your K3 SoC.

**Combined Binary Boot Flow** (eg: am62x, am64x, ... )

   `tiboot3-{SOC}-{gp/hs-fs/hs}.bin`

**Split Binary Boot Flow** (eg: j721e, am65x)

   | `tiboot3-{SOC}-{gp/hs-fs/hs}.bin`
   | `sysfw-{SOC}-{gp/hs-fs/hs}-evm.itb`

.. note ::

   It's important to rename the generated `tiboot3.bin` and `sysfw.itb`
   to match exactly `tiboot3.bin` and `sysfw.itb` as ROM and the wakeup
   UBoot SPL will only look for and load the files with these names.

Building tispl.bin
^^^^^^^^^^^^^^^^^^^

The `tispl.bin` is a standard fitImage combining the firmware need for
the main domain to function properly as well as Device Management (DM)
firmware if your device using a split firmware.

2. We will first need TF-A, as it's the first thing to run on the 'big'
   application cores on the main domain.

.. k3_rst_include_start_build_steps_tfa
.. code-block:: bash

 $ # inside trusted-firmware-a source
 $ make CROSS_COMPILE=$CC64 ARCH=aarch64 PLAT=k3 SPD=opteed $TFA_EXTRA_ARGS \
        TARGET_BOARD=$TFA_BOARD
.. k3_rst_include_end_build_steps_tfa

Typically all `j7*` devices will use `TARGET_BOARD=generic` or `TARGET_BOARD
=j784s4` (if it is a J784S4 device), while typical Sitara (`am6*`) devices
use the `lite` option.

3. The Open Portable Trusted Execution Environment (OP-TEE) is designed
   to run as a companion to a non-secure Linux kernel for Cortex-A cores
   using the TrustZone technology built into the core.

.. k3_rst_include_start_build_steps_optee
.. code-block:: bash

 $ # inside optee_os source
 $ make CROSS_COMPILE=$CC32 CROSS_COMPILE64=$CC64 CFG_ARM64_core=y $OPTEE_EXTRA_ARGS \
         PLATFORM=$OPTEE_PLATFORM
.. k3_rst_include_end_build_steps_optee

4. Finally, after TF-A has initialized the main domain and OP-TEE has
   finished, we can jump back into U-Boot again, this time running on a
   64bit core in the main domain.

.. k3_rst_include_start_build_steps_uboot
.. code-block:: bash

 $ # inside u-boot source
 $ make $UBOOT_CFG_CORTEXA
 $ make CROSS_COMPILE=$CC64 BINMAN_INDIRS=$LNX_FW_PATH \
        BL31=$TFA_PATH/build/k3/$TFA_BOARD/release/bl31.bin \
        TEE=$OPTEE_PATH/out/arm-plat-k3/core/tee-raw.bin
.. k3_rst_include_end_build_steps_uboot

At this point you should have every binary needed initialize both the
wakeup and main domain and to boot to the U-Boot prompt

**Main Domain Bootloader**

   | `tispl.bin` for HS devices or `tispl.bin_unsigned` for GP devices
   | `u-boot.img` for HS devices or `u-boot.img_unsigned` for GP devices

Fit Signature Signing
---------------------

K3 Platforms have fit signature signing enabled by default on their primary
platforms. Here we'll take an example for creating fit image for J721e platform
and the same can be extended to other platforms

1. Describing FIT source

  .. code-block:: bash

    /dts-v1/;

    / {
            description = "Kernel fitImage for j721e-hs-evm";
            #address-cells = <1>;

            images {
                    kernel-1 {
                            description = "Linux kernel";
                            data = /incbin/("Image");
                            type = "kernel";
                            arch = "arm64";
                            os = "linux";
                            compression = "none";
                            load = <0x80080000>;
                            entry = <0x80080000>;
                            hash-1 {
                                    algo = "sha512";
                            };

                    };
                    fdt-ti_k3-j721e-common-proc-board.dtb {
                            description = "Flattened Device Tree blob";
                            data = /incbin/("k3-j721e-common-proc-board.dtb");
                            type = "flat_dt";
                            arch = "arm64";
                            compression = "none";
                            load = <0x83000000>;
                            hash-1 {
                                    algo = "sha512";
                            };

                    };
            };

            configurations {
                    default = "conf-ti_k3-j721e-common-proc-board.dtb";
                    conf-ti_k3-j721e-common-proc-board.dtb {
                            description = "Linux kernel, FDT blob";
                            fdt = "fdt-ti_k3-j721e-common-proc-board.dtb";
                            kernel = "kernel-1";
                            signature-1 {
                                    algo = "sha512,rsa4096";
                                    key-name-hint = "custMpk";
                                    sign-images = "kernel", "fdt";
                            };
                    };
            };
    };

  You would require to change the '/incbin/' lines to point to the respective
  files in your local machine and the key-name-hint also needs to be changed
  if you are using some other key other than the TI dummy key that we are
  using for this example.

2. Compile U-boot for the respective board

.. include::  k3.rst
    :start-after: .. k3_rst_include_start_build_steps_uboot
    :end-before: .. k3_rst_include_end_build_steps_uboot

.. note::

    The changes only affect a72 binaries so the example just builds that

3. Sign the fit image and embed the dtb in uboot

  Now once the build is done, you'll have a dtb for your board that you'll
  be passing to mkimage for signing the fitImage and embedding the key in
  the u-boot dtb.

  .. code-block:: bash

    mkimage -r -f fitImage.its -k $UBOOT_PATH/board/ti/keys -K
    $UBOOT_PATH/build/a72/dts/dt.dtb

  For signing a secondary platform, pass the -K parameter to that DTB

  .. code-block:: bash

    mkimage -f fitImage.its -k $UBOOT_PATH/board/ti/keys -K
    $UBOOT_PATH/build/a72/arch/arm/dts/k3-j721e-sk.dtb

  .. note::

    If changing `CONFIG_DEFAULT_DEVICE_TREE` to the secondary platform,
    binman changes would also be required so that correct dtb gets packaged.

    .. code-block:: bash

      diff --git a/arch/arm/dts/k3-j721e-binman.dtsi b/arch/arm/dts/k3-j721e-binman.dtsi
      index 673be646b1e3..752fa805fe8d 100644
      --- a/arch/arm/dts/k3-j721e-binman.dtsi
      +++ b/arch/arm/dts/k3-j721e-binman.dtsi
      @@ -299,8 +299,8 @@
       #define SPL_J721E_SK_DTB "spl/dts/k3-j721e-sk.dtb"

       #define UBOOT_NODTB "u-boot-nodtb.bin"
      -#define J721E_EVM_DTB "u-boot.dtb"
      -#define J721E_SK_DTB "arch/arm/dts/k3-j721e-sk.dtb"
      +#define J721E_EVM_DTB "arch/arm/dts/k3-j721e-common-proc-board.dtb"
      +#define J721E_SK_DTB "u-boot.dtb"

5. Rebuilt u-boot

   This is required so that the modified dtb gets updated in u-boot.img

.. include::  k3.rst
    :start-after: .. k3_rst_include_start_build_steps_uboot
    :end-before: .. k3_rst_include_end_build_steps_uboot

6. (Optional) Enabled FIT_SIGNATURE_ENFORCED

   By default u-boot will boot up the fit image without any authentication as
   such if the public key is not embedded properly, to check if the public key
   nodes are proper you can enable FIT_SIGNATURE_ENFORCED that would not rely
   on the dtb for anything else then the signature node for checking the fit
   image, rest other things will be enforced such as the property of
   required-keys. This is not an extensive check so do manual checks also

   This is by default enabled for devices with TI_SECURE_DEVICE enabled.

.. note::

   The devices now also have distroboot enabled so if the fit image doesn't
   work then the fallback to normal distroboot will be there on hs devices,
   this will need to be explicitly disabled by changing the boot_targets.

Saving environment
------------------

SAVEENV is disabled by default and for the new flow uses Uenv.txt as the default
way for saving the environments. This has been done as Uenv.txt is more granular
then the saveenv command and can be used across various bootmodes too.

**Writing to MMC/EMMC**

.. code-block::

  => env export -t $loadaddr <list of variables>
  => fatwrite mmc ${mmcdev} ${loadaddr} ${bootenvfile} ${filesize}

**Reading from MMC/EMMC**

By default run envboot will read it from the MMC/EMMC partition ( based on
mmcdev) and set the environments.

If manually needs to be done then the environment can be read from the
filesystem and then imported

.. code-block::

  => fatload mmc ${mmcdev} ${loadaddr} ${bootenvfile}
  => env import -t ${loadaddr} ${filesize}