fix(gicv3): workaround for NVIDIA erratum T241-FABRIC-4
The purpose of this patch is to address the T241 erratum T241-FABRIC-4,
which causes unexpected behavior in the GIC when multiple transactions
are received simultaneously from different sources. This hardware issue
impacts NVIDIA server platforms that use more than two T241 chips
interconnected. Each chip has support for 320 {E}SPIs.
This issue occurs when multiple packets from different GICs are
incorrectly interleaved at the target chip. The erratum text below
specifies exactly what can cause multiple transfer packets susceptible
to interleaving and GIC state corruption. GIC state corruption can
lead to a range of problems, including kernel panics, and unexpected
behavior.
Erratum documentation:
https://developer.nvidia.com/docs/t241-fabric-4/nvidia-t241-fabric-4-errata.pdf
The workaround is to ensure that MMIO accesses target the GIC on the
socket that holds the data, for example SPI ranges owned by the socket’s
GIC. This ensures that the GIC will not utilize the inter-socket AXI
Stream interface for servicing these GIC MMIO accesses.
This patch updates the functions that use the GICD_In{E} registers to
ensure that the accesses are directed to the chip that owns the SPI,
instead of using the global alias.
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>
Change-Id: I04e33ba64eb306bd5fdabb56e63cbe273d8cd632
diff --git a/include/drivers/arm/gic600_multichip.h b/include/drivers/arm/gic600_multichip.h
index bda406b..978d735 100644
--- a/include/drivers/arm/gic600_multichip.h
+++ b/include/drivers/arm/gic600_multichip.h
@@ -1,5 +1,6 @@
/*
* Copyright (c) 2019, ARM Limited. All rights reserved.
+ * Copyright (c) 2023, NVIDIA Corporation. All rights reserved.
*
* SPDX-License-Identifier: BSD-3-Clause
*/
@@ -15,8 +16,11 @@
*/
#define GIC600_MAX_MULTICHIP 16
-/* SPI IDs array consist of min and max ids */
-#define GIC600_SPI_IDS_SIZE 2
+typedef struct multichip_spi_ids_desc {
+ uintptr_t gicd_base;
+ uint32_t spi_id_min;
+ uint32_t spi_id_max;
+} multichip_spi_ids_desc_t;
/*******************************************************************************
* GIC-600 multichip data structure describes platform specific attributes
@@ -37,19 +41,23 @@
* The 'chip_addrs' field contains array of chip addresses. These addresses are
* implementation specific values.
*
- * The 'spi_ids' field contains array of minimum and maximum SPI interrupt ids
- * that each chip owns. Note that SPI interrupt ids can range from 32 to 960 and
- * it should be group of 32 (i.e., SPI minimum and (SPI maximum + 1) should be
- * a multiple of 32). If a chip doesn't own any SPI interrupts a value of {0, 0}
- * should be passed.
+ * The 'multichip_spi_ids_desc_t' field contains array of descriptors used to
+ * provide minimum and maximum SPI interrupt ids that each chip owns and the
+ * corresponding chip base address. Note that SPI interrupt ids can range from
+ * 32 to 960 and it should be group of 32 (i.e., SPI minimum and (SPI maximum +
+ * 1) should be a multiple of 32). If a chip doesn't own any SPI interrupts a
+ * value of {0, 0, 0} should be passed.
******************************************************************************/
struct gic600_multichip_data {
uintptr_t rt_owner_base;
unsigned int rt_owner;
unsigned int chip_count;
uint64_t chip_addrs[GIC600_MAX_MULTICHIP];
- unsigned int spi_ids[GIC600_MAX_MULTICHIP][GIC600_SPI_IDS_SIZE];
+ multichip_spi_ids_desc_t spi_ids[GIC600_MAX_MULTICHIP];
};
+uintptr_t gic600_multichip_gicd_base_for_spi(uint32_t spi_id);
void gic600_multichip_init(struct gic600_multichip_data *multichip_data);
+bool gic600_multichip_is_initialized(void);
+
#endif /* GIC600_MULTICHIP_H */