blob: bab1086203c07ea0b678d142272967d43ae95965 [file] [log] [blame]
Paul Beesley236d2462019-03-05 17:19:37 +00001PSCI Performance Measurements on Arm Juno Development Platform
2==============================================================
3
Joel Hutton9e605632019-02-25 15:18:56 +00004This document summarises the findings of performance measurements of key
John Tsichritzis63801cd2019-07-05 14:22:12 +01005operations in the Trusted Firmware-A Power State Coordination Interface (PSCI)
6implementation, using the in-built Performance Measurement Framework (PMF) and
7runtime instrumentation timestamps.
Joel Hutton9e605632019-02-25 15:18:56 +00008
9Method
10------
11
12We used the `Juno R1 platform`_ for these tests, which has 4 x Cortex-A53 and 2
13x Cortex-A57 clusters running at the following frequencies:
14
15+-----------------+--------------------+
16| Domain | Frequency (MHz) |
17+=================+====================+
18| Cortex-A57 | 900 (nominal) |
19+-----------------+--------------------+
20| Cortex-A53 | 650 (underdrive) |
21+-----------------+--------------------+
22| AXI subsystem | 533 |
23+-----------------+--------------------+
24
25Juno supports CPU, cluster and system power down states, corresponding to power
26levels 0, 1 and 2 respectively. It does not support any retention states.
27
Harrison Mutai21cb9652023-05-17 13:09:16 +010028Given that runtime instrumentation using PMF is invasive, there is a small
29(unquantified) overhead on the results. PMF uses the generic counter for
30timestamps, which runs at 50MHz on Juno.
Joel Hutton9e605632019-02-25 15:18:56 +000031
Harrison Mutai21cb9652023-05-17 13:09:16 +010032The following source trees and binaries were used:
Joel Hutton9e605632019-02-25 15:18:56 +000033
Harrison Mutai21cb9652023-05-17 13:09:16 +010034- TF-A [`v2.9-rc0`_]
35- TFTF [`v2.9-rc0`_]
Joel Hutton9e605632019-02-25 15:18:56 +000036
Thaddeus Serna8709cc92023-08-14 13:28:59 -050037Please see the Runtime Instrumentation :ref:`Testing Methodology
38<Runtime Instrumentation Methodology>`
39page for more details.
Joel Hutton9e605632019-02-25 15:18:56 +000040
Harrison Mutai21cb9652023-05-17 13:09:16 +010041Procedure
42---------
Joel Hutton9e605632019-02-25 15:18:56 +000043
Harrison Mutai21cb9652023-05-17 13:09:16 +010044#. Build TFTF with runtime instrumentation enabled:
Joel Hutton9e605632019-02-25 15:18:56 +000045
Harrison Mutai21cb9652023-05-17 13:09:16 +010046 .. code:: shell
Joel Hutton9e605632019-02-25 15:18:56 +000047
Harrison Mutai21cb9652023-05-17 13:09:16 +010048 make CROSS_COMPILE=aarch64-none-elf- PLAT=juno \
49 TESTS=runtime-instrumentation all
Joel Hutton9e605632019-02-25 15:18:56 +000050
Harrison Mutai21cb9652023-05-17 13:09:16 +010051#. Fetch Juno's SCP binary from TF-A's archive:
Joel Hutton9e605632019-02-25 15:18:56 +000052
Harrison Mutai21cb9652023-05-17 13:09:16 +010053 .. code:: shell
Joel Hutton9e605632019-02-25 15:18:56 +000054
Harrison Mutai21cb9652023-05-17 13:09:16 +010055 curl --fail --connect-timeout 5 --retry 5 -sLS -o scp_bl2.bin \
56 https://downloads.trustedfirmware.org/tf-a/css_scp_2.12.0/juno/release/juno-bl2.bin
Joel Hutton9e605632019-02-25 15:18:56 +000057
Harrison Mutai21cb9652023-05-17 13:09:16 +010058#. Build TF-A with the following build options:
Joel Hutton9e605632019-02-25 15:18:56 +000059
Harrison Mutai21cb9652023-05-17 13:09:16 +010060 .. code:: shell
Joel Hutton9e605632019-02-25 15:18:56 +000061
Harrison Mutai21cb9652023-05-17 13:09:16 +010062 make CROSS_COMPILE=aarch64-none-elf- PLAT=juno \
63 BL33="/path/to/tftf.bin" SCP_BL2="scp_bl2.bin" \
64 ENABLE_RUNTIME_INSTRUMENTATION=1 fiptool all fip
65
66#. Load the following images onto the development board: ``fip.bin``,
67 ``scp_bl2.bin``.
68
69Results
70-------
71
72``CPU_SUSPEND`` to deepest power level
73~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
74
75.. table:: ``CPU_SUSPEND`` latencies s) to deepest power level in
Harrison Mutai90688452023-11-10 17:35:33 +000076 parallel (v2.9)
Harrison Mutai21cb9652023-05-17 13:09:16 +010077
Harrison Mutai90688452023-11-10 17:35:33 +000078 +---------+------+-----------+--------+-------------+
79 | Cluster | Core | Powerdown | Wakeup | Cache Flush |
80 +---------+------+-----------+--------+-------------+
81 | 0 | 0 | 104.58 | 241.20 | 5.26 |
82 +---------+------+-----------+--------+-------------+
83 | 0 | 1 | 384.24 | 22.50 | 138.76 |
84 +---------+------+-----------+--------+-------------+
85 | 1 | 0 | 244.56 | 22.18 | 5.16 |
86 +---------+------+-----------+--------+-------------+
87 | 1 | 1 | 670.56 | 18.58 | 4.44 |
88 +---------+------+-----------+--------+-------------+
89 | 1 | 2 | 809.36 | 269.28 | 4.44 |
90 +---------+------+-----------+--------+-------------+
91 | 1 | 3 | 984.96 | 219.70 | 79.62 |
92 +---------+------+-----------+--------+-------------+
Harrison Mutai21cb9652023-05-17 13:09:16 +010093
94.. table:: ``CPU_SUSPEND`` latencies s) to deepest power level in
Harrison Mutai90688452023-11-10 17:35:33 +000095 parallel (v2.10)
Harrison Mutai21cb9652023-05-17 13:09:16 +010096
Harrison Mutai90688452023-11-10 17:35:33 +000097 +---------+------+-------------------+--------+-------------+
98 | Cluster | Core | Powerdown | Wakeup | Cache Flush |
99 +---------+------+-------------------+--------+-------------+
100 | 0 | 0 | 242.66 (+132.03%) | 245.1 | 5.4 |
101 +---------+------+-------------------+--------+-------------+
102 | 0 | 1 | 522.08 (+35.87%) | 26.24 | 138.32 |
103 +---------+------+-------------------+--------+-------------+
104 | 1 | 0 | 104.36 (-57.33%) | 27.1 | 5.32 |
105 +---------+------+-------------------+--------+-------------+
106 | 1 | 1 | 382.56 (-42.95%) | 23.34 | 4.42 |
107 +---------+------+-------------------+--------+-------------+
108 | 1 | 2 | 807.74 | 271.54 | 4.64 |
109 +---------+------+-------------------+--------+-------------+
110 | 1 | 3 | 981.36 | 221.8 | 79.48 |
111 +---------+------+-------------------+--------+-------------+
112
113.. table:: ``CPU_SUSPEND`` latencies s) to deepest power level in
114 serial (v2.9)
115
116 +---------+------+-----------+--------+-------------+
117 | Cluster | Core | Powerdown | Wakeup | Cache Flush |
118 +---------+------+-----------+--------+-------------+
119 | 0 | 0 | 236.56 | 23.24 | 138.18 |
120 +---------+------+-----------+--------+-------------+
121 | 0 | 1 | 236.86 | 23.28 | 138.10 |
122 +---------+------+-----------+--------+-------------+
123 | 1 | 0 | 281.04 | 22.80 | 77.24 |
124 +---------+------+-----------+--------+-------------+
125 | 1 | 1 | 100.28 | 18.52 | 4.54 |
126 +---------+------+-----------+--------+-------------+
127 | 1 | 2 | 100.12 | 18.78 | 4.50 |
128 +---------+------+-----------+--------+-------------+
129 | 1 | 3 | 100.36 | 18.94 | 4.44 |
130 +---------+------+-----------+--------+-------------+
131
132.. table:: ``CPU_SUSPEND`` latencies s) to deepest power level in
133 serial (v2.10)
134
135 +---------+------+-----------+--------+-------------+
136 | Cluster | Core | Powerdown | Wakeup | Cache Flush |
137 +---------+------+-----------+--------+-------------+
138 | 0 | 0 | 236.84 | 27.1 | 138.36 |
139 +---------+------+-----------+--------+-------------+
140 | 0 | 1 | 236.96 | 27.1 | 138.32 |
141 +---------+------+-----------+--------+-------------+
142 | 1 | 0 | 280.06 | 26.94 | 77.5 |
143 +---------+------+-----------+--------+-------------+
144 | 1 | 1 | 100.76 | 23.42 | 4.36 |
145 +---------+------+-----------+--------+-------------+
146 | 1 | 2 | 100.02 | 23.42 | 4.44 |
147 +---------+------+-----------+--------+-------------+
148 | 1 | 3 | 100.08 | 23.2 | 4.4 |
149 +---------+------+-----------+--------+-------------+
Harrison Mutai21cb9652023-05-17 13:09:16 +0100150
151``CPU_SUSPEND`` to power level 0
152~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Joel Hutton9e605632019-02-25 15:18:56 +0000153
Harrison Mutai21cb9652023-05-17 13:09:16 +0100154.. table:: ``CPU_SUSPEND`` latencies s) to power level 0 in
Harrison Mutai90688452023-11-10 17:35:33 +0000155 parallel (v2.9)
Harrison Mutai21cb9652023-05-17 13:09:16 +0100156
Harrison Mutai90688452023-11-10 17:35:33 +0000157 +---------+------+-----------+--------+-------------+
158 | Cluster | Core | Powerdown | Wakeup | Cache Flush |
159 +---------+------+-----------+--------+-------------+
160 | 0 | 0 | 662.34 | 15.22 | 8.08 |
161 +---------+------+-----------+--------+-------------+
162 | 0 | 1 | 802.00 | 15.50 | 8.16 |
163 +---------+------+-----------+--------+-------------+
164 | 1 | 0 | 385.22 | 15.74 | 7.88 |
165 +---------+------+-----------+--------+-------------+
166 | 1 | 1 | 106.16 | 16.06 | 7.44 |
167 +---------+------+-----------+--------+-------------+
168 | 1 | 2 | 524.38 | 15.64 | 7.34 |
169 +---------+------+-----------+--------+-------------+
170 | 1 | 3 | 246.00 | 15.78 | 7.72 |
171 +---------+------+-----------+--------+-------------+
Harrison Mutai21cb9652023-05-17 13:09:16 +0100172
Harrison Mutai90688452023-11-10 17:35:33 +0000173.. table:: ``CPU_SUSPEND`` latencies s) to power level 0 in
174 parallel (v2.10)
175
176 +---------+------+-------------------+--------+-------------+
177 | Cluster | Core | Powerdown | Wakeup | Cache Flush |
178 +---------+------+-------------------+--------+-------------+
179 | 0 | 0 | 801.04 | 18.66 | 8.22 |
180 +---------+------+-------------------+--------+-------------+
181 | 0 | 1 | 661.28 | 19.08 | 7.88 |
182 +---------+------+-------------------+--------+-------------+
183 | 1 | 0 | 105.9 (-72.51%) | 20.3 | 7.58 |
184 +---------+------+-------------------+--------+-------------+
185 | 1 | 1 | 383.58 (+261.32%) | 20.4 | 7.42 |
186 +---------+------+-------------------+--------+-------------+
187 | 1 | 2 | 523.52 | 20.1 | 7.74 |
188 +---------+------+-------------------+--------+-------------+
189 | 1 | 3 | 244.5 | 20.16 | 7.56 |
190 +---------+------+-------------------+--------+-------------+
Harrison Mutai21cb9652023-05-17 13:09:16 +0100191
Harrison Mutai90688452023-11-10 17:35:33 +0000192.. table:: ``CPU_SUSPEND`` latencies s) to power level 0 in serial (v2.9)
193
194 +---------+------+-----------+--------+-------------+
195 | Cluster | Core | Powerdown | Wakeup | Cache Flush |
196 +---------+------+-----------+--------+-------------+
197 | 0 | 0 | 99.80 | 15.94 | 5.42 |
198 +---------+------+-----------+--------+-------------+
199 | 0 | 1 | 99.76 | 15.80 | 5.24 |
200 +---------+------+-----------+--------+-------------+
201 | 1 | 0 | 278.26 | 16.16 | 4.58 |
202 +---------+------+-----------+--------+-------------+
203 | 1 | 1 | 96.88 | 16.00 | 4.52 |
204 +---------+------+-----------+--------+-------------+
205 | 1 | 2 | 96.80 | 16.12 | 4.54 |
206 +---------+------+-----------+--------+-------------+
207 | 1 | 3 | 96.88 | 16.12 | 4.54 |
208 +---------+------+-----------+--------+-------------+
209
210.. table:: ``CPU_SUSPEND`` latencies s) to power level 0 in serial (v2.10)
211
212 +---------+------+-----------+--------+-------------+
213 | Cluster | Core | Powerdown | Wakeup | Cache Flush |
214 +---------+------+-----------+--------+-------------+
215 | 0 | 0 | 99.84 | 18.86 | 5.54 |
216 +---------+------+-----------+--------+-------------+
217 | 0 | 1 | 100.2 | 18.82 | 5.66 |
218 +---------+------+-----------+--------+-------------+
219 | 1 | 0 | 278.12 | 20.56 | 4.48 |
220 +---------+------+-----------+--------+-------------+
221 | 1 | 1 | 96.68 | 20.62 | 4.3 |
222 +---------+------+-----------+--------+-------------+
223 | 1 | 2 | 96.94 | 20.14 | 4.42 |
224 +---------+------+-----------+--------+-------------+
225 | 1 | 3 | 96.68 | 20.46 | 4.32 |
226 +---------+------+-----------+--------+-------------+
Harrison Mutai21cb9652023-05-17 13:09:16 +0100227
228``CPU_OFF`` on all non-lead CPUs
229~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
230
231``CPU_OFF`` on all non-lead CPUs in sequence then, ``CPU_SUSPEND`` on the lead
232core to the deepest power level.
233
Harrison Mutai90688452023-11-10 17:35:33 +0000234.. table:: ``CPU_OFF`` latencies s) on all non-lead CPUs (v2.9)
Harrison Mutai21cb9652023-05-17 13:09:16 +0100235
Harrison Mutai90688452023-11-10 17:35:33 +0000236 +---------+------+-----------+--------+-------------+
237 | Cluster | Core | Powerdown | Wakeup | Cache Flush |
238 +---------+------+-----------+--------+-------------+
239 | 0 | 0 | 235.76 | 26.14 | 137.80 |
240 +---------+------+-----------+--------+-------------+
241 | 0 | 1 | 235.40 | 25.72 | 137.62 |
242 +---------+------+-----------+--------+-------------+
243 | 1 | 0 | 174.70 | 22.40 | 77.26 |
244 +---------+------+-----------+--------+-------------+
245 | 1 | 1 | 100.92 | 24.04 | 4.52 |
246 +---------+------+-----------+--------+-------------+
247 | 1 | 2 | 100.68 | 22.44 | 4.36 |
248 +---------+------+-----------+--------+-------------+
249 | 1 | 3 | 101.36 | 22.70 | 4.52 |
250 +---------+------+-----------+--------+-------------+
251
252.. table:: ``CPU_OFF`` latencies s) on all non-lead CPUs (v2.10)
253
254 +---------------------------------------------------+
255 | test_rt_instr_cpu_off_serial (latest) |
256 +---------+------+-----------+--------+-------------+
257 | Cluster | Core | Powerdown | Wakeup | Cache Flush |
258 +---------+------+-----------+--------+-------------+
259 | 0 | 0 | 236.04 | 30.02 | 137.9 |
260 +---------+------+-----------+--------+-------------+
261 | 0 | 1 | 235.38 | 29.7 | 137.72 |
262 +---------+------+-----------+--------+-------------+
263 | 1 | 0 | 175.18 | 26.96 | 77.26 |
264 +---------+------+-----------+--------+-------------+
265 | 1 | 1 | 100.56 | 28.34 | 4.32 |
266 +---------+------+-----------+--------+-------------+
267 | 1 | 2 | 100.38 | 26.82 | 4.3 |
268 +---------+------+-----------+--------+-------------+
269 | 1 | 3 | 100.86 | 26.98 | 4.42 |
270 +---------+------+-----------+--------+-------------+
Harrison Mutai21cb9652023-05-17 13:09:16 +0100271
272``CPU_VERSION`` in parallel
273~~~~~~~~~~~~~~~~~~~~~~~~~~~
274
Harrison Mutai90688452023-11-10 17:35:33 +0000275.. table:: ``CPU_VERSION`` latency s) in parallel on all cores (2.9)
276
277 +-------------+--------+-------------+
278 | Cluster | Core | Latency |
279 +-------------+--------+-------------+
280 | 0 | 0 | 1.48 |
281 +-------------+--------+-------------+
282 | 0 | 1 | 1.04 |
283 +-------------+--------+-------------+
284 | 1 | 0 | 0.56 |
285 +-------------+--------+-------------+
286 | 1 | 1 | 0.92 |
287 +-------------+--------+-------------+
288 | 1 | 2 | 0.96 |
289 +-------------+--------+-------------+
290 | 1 | 3 | 0.96 |
291 +-------------+--------+-------------+
292
293.. table:: ``CPU_VERSION`` latency s) in parallel on all cores (2.10)
Harrison Mutai21cb9652023-05-17 13:09:16 +0100294
Harrison Mutai90688452023-11-10 17:35:33 +0000295 +-------------+--------+----------------------+
296 | Cluster | Core | Latency |
297 +-------------+--------+----------------------+
298 | 0 | 0 | 1.1 (-25.68%) |
299 +-------------+--------+----------------------+
300 | 0 | 1 | 1.06 |
301 +-------------+--------+----------------------+
302 | 1 | 0 | 0.58 |
303 +-------------+--------+----------------------+
304 | 1 | 1 | 0.88 |
305 +-------------+--------+----------------------+
306 | 1 | 2 | 0.92 |
307 +-------------+--------+----------------------+
308 | 1 | 3 | 0.9 |
309 +-------------+--------+----------------------+
Harrison Mutai21cb9652023-05-17 13:09:16 +0100310
311Annotated Historic Results
312--------------------------
313
314The following results are based on the upstream `TF master as of 31/01/2017`_.
315TF-A was built using the same build instructions as detailed in the procedure
316above.
317
318In the results below, CPUs 0-3 refer to CPUs in the little cluster (A53) and
319CPUs 4-5 refer to CPUs in the big cluster (A57). In all cases CPU 4 is the lead
320CPU.
321
322``PSCI_ENTRY`` corresponds to the powerdown latency, ``PSCI_EXIT`` the wakeup latency, and
323``CFLUSH_OVERHEAD`` the latency of the cache flush operation.
Joel Hutton9e605632019-02-25 15:18:56 +0000324
325``CPU_SUSPEND`` to deepest power level on all CPUs in parallel
326~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
327
328+-------+---------------------+--------------------+--------------------------+
329| CPU | ``PSCI_ENTRY`` (us) | ``PSCI_EXIT`` (us) | ``CFLUSH_OVERHEAD`` (us) |
330+=======+=====================+====================+==========================+
331| 0 | 27 | 20 | 5 |
332+-------+---------------------+--------------------+--------------------------+
333| 1 | 114 | 86 | 5 |
334+-------+---------------------+--------------------+--------------------------+
335| 2 | 202 | 58 | 5 |
336+-------+---------------------+--------------------+--------------------------+
337| 3 | 375 | 29 | 94 |
338+-------+---------------------+--------------------+--------------------------+
339| 4 | 20 | 22 | 6 |
340+-------+---------------------+--------------------+--------------------------+
341| 5 | 290 | 18 | 206 |
342+-------+---------------------+--------------------+--------------------------+
343
344A large variance in ``PSCI_ENTRY`` and ``PSCI_EXIT`` times across CPUs is
345observed due to TF PSCI lock contention. In the worst case, CPU 3 has to wait
346for the 3 other CPUs in the cluster (0-2) to complete ``PSCI_ENTRY`` and release
347the lock before proceeding.
348
349The ``CFLUSH_OVERHEAD`` times for CPUs 3 and 5 are higher because they are the
350last CPUs in their respective clusters to power down, therefore both the L1 and
351L2 caches are flushed.
352
353The ``CFLUSH_OVERHEAD`` time for CPU 5 is a lot larger than that for CPU 3
354because the L2 cache size for the big cluster is lot larger (2MB) compared to
355the little cluster (1MB).
356
357``CPU_SUSPEND`` to power level 0 on all CPUs in parallel
358~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
359
360+-------+---------------------+--------------------+--------------------------+
361| CPU | ``PSCI_ENTRY`` (us) | ``PSCI_EXIT`` (us) | ``CFLUSH_OVERHEAD`` (us) |
362+=======+=====================+====================+==========================+
363| 0 | 116 | 14 | 8 |
364+-------+---------------------+--------------------+--------------------------+
365| 1 | 204 | 14 | 8 |
366+-------+---------------------+--------------------+--------------------------+
367| 2 | 287 | 13 | 8 |
368+-------+---------------------+--------------------+--------------------------+
369| 3 | 376 | 13 | 9 |
370+-------+---------------------+--------------------+--------------------------+
371| 4 | 29 | 15 | 7 |
372+-------+---------------------+--------------------+--------------------------+
373| 5 | 21 | 15 | 8 |
374+-------+---------------------+--------------------+--------------------------+
375
376There is no lock contention in TF generic code at power level 0 but the large
377variance in ``PSCI_ENTRY`` times across CPUs is due to lock contention in Juno
378platform code. The platform lock is used to mediate access to a single SCP
379communication channel. This is compounded by the SCP firmware waiting for each
380AP CPU to enter WFI before making the channel available to other CPUs, which
381effectively serializes the SCP power down commands from all CPUs.
382
383On platforms with a more efficient CPU power down mechanism, it should be
384possible to make the ``PSCI_ENTRY`` times smaller and consistent.
385
386The ``PSCI_EXIT`` times are consistent across all CPUs because TF does not
387require locks at power level 0.
388
389The ``CFLUSH_OVERHEAD`` times for all CPUs are small and consistent since only
390the cache associated with power level 0 is flushed (L1).
391
392``CPU_SUSPEND`` to deepest power level on all CPUs in sequence
393~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
394
395+-------+---------------------+--------------------+--------------------------+
396| CPU | ``PSCI_ENTRY`` (us) | ``PSCI_EXIT`` (us) | ``CFLUSH_OVERHEAD`` (us) |
397+=======+=====================+====================+==========================+
398| 0 | 114 | 20 | 94 |
399+-------+---------------------+--------------------+--------------------------+
400| 1 | 114 | 20 | 94 |
401+-------+---------------------+--------------------+--------------------------+
402| 2 | 114 | 20 | 94 |
403+-------+---------------------+--------------------+--------------------------+
404| 3 | 114 | 20 | 94 |
405+-------+---------------------+--------------------+--------------------------+
406| 4 | 195 | 22 | 180 |
407+-------+---------------------+--------------------+--------------------------+
408| 5 | 21 | 17 | 6 |
409+-------+---------------------+--------------------+--------------------------+
410
Paul Beesleyf2ec7142019-10-04 16:17:46 +0000411The ``CFLUSH_OVERHEAD`` times for lead CPU 4 and all CPUs in the non-lead cluster
Joel Hutton9e605632019-02-25 15:18:56 +0000412are large because all other CPUs in the cluster are powered down during the
413test. The ``CPU_SUSPEND`` call powers down to the cluster level, requiring a
414flush of both L1 and L2 caches.
415
416The ``CFLUSH_OVERHEAD`` time for CPU 4 is a lot larger than those for the little
417CPUs because the L2 cache size for the big cluster is lot larger (2MB) compared
418to the little cluster (1MB).
419
420The ``PSCI_ENTRY`` and ``CFLUSH_OVERHEAD`` times for CPU 5 are low because lead
421CPU 4 continues to run while CPU 5 is suspended. Hence CPU 5 only powers down to
422level 0, which only requires L1 cache flush.
423
424``CPU_SUSPEND`` to power level 0 on all CPUs in sequence
425~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
426
427+-------+---------------------+--------------------+--------------------------+
428| CPU | ``PSCI_ENTRY`` (us) | ``PSCI_EXIT`` (us) | ``CFLUSH_OVERHEAD`` (us) |
429+=======+=====================+====================+==========================+
430| 0 | 22 | 14 | 5 |
431+-------+---------------------+--------------------+--------------------------+
432| 1 | 22 | 14 | 5 |
433+-------+---------------------+--------------------+--------------------------+
434| 2 | 21 | 14 | 5 |
435+-------+---------------------+--------------------+--------------------------+
436| 3 | 22 | 14 | 5 |
437+-------+---------------------+--------------------+--------------------------+
438| 4 | 17 | 14 | 6 |
439+-------+---------------------+--------------------+--------------------------+
440| 5 | 18 | 15 | 6 |
441+-------+---------------------+--------------------+--------------------------+
442
443Here the times are small and consistent since there is no contention and it is
444only necessary to flush the cache to power level 0 (L1). This is the best case
445scenario.
446
447The ``PSCI_ENTRY`` times for CPUs in the big cluster are slightly smaller than
448for the CPUs in little cluster due to greater CPU performance.
449
450The ``PSCI_EXIT`` times are generally lower than in the last test because the
451cluster remains powered on throughout the test and there is less code to execute
452on power on (for example, no need to enter CCI coherency)
453
454``CPU_OFF`` on all non-lead CPUs in sequence then ``CPU_SUSPEND`` on lead CPU to deepest power level
455~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
456
457The test sequence here is as follows:
458
4591. Call ``CPU_ON`` and ``CPU_OFF`` on each non-lead CPU in sequence.
460
4612. Program wake up timer and suspend the lead CPU to the deepest power level.
462
4633. Call ``CPU_ON`` on non-lead CPU to get the timestamps from each CPU.
464
465+-------+---------------------+--------------------+--------------------------+
466| CPU | ``PSCI_ENTRY`` (us) | ``PSCI_EXIT`` (us) | ``CFLUSH_OVERHEAD`` (us) |
467+=======+=====================+====================+==========================+
468| 0 | 110 | 28 | 93 |
469+-------+---------------------+--------------------+--------------------------+
470| 1 | 110 | 28 | 93 |
471+-------+---------------------+--------------------+--------------------------+
472| 2 | 110 | 28 | 93 |
473+-------+---------------------+--------------------+--------------------------+
474| 3 | 111 | 28 | 93 |
475+-------+---------------------+--------------------+--------------------------+
476| 4 | 195 | 22 | 181 |
477+-------+---------------------+--------------------+--------------------------+
478| 5 | 20 | 23 | 6 |
479+-------+---------------------+--------------------+--------------------------+
480
481The ``CFLUSH_OVERHEAD`` times for all little CPUs are large because all other
482CPUs in that cluster are powerered down during the test. The ``CPU_OFF`` call
483powers down to the cluster level, requiring a flush of both L1 and L2 caches.
484
485The ``PSCI_ENTRY`` and ``CFLUSH_OVERHEAD`` times for CPU 5 are small because
486lead CPU 4 is running and CPU 5 only powers down to level 0, which only requires
487an L1 cache flush.
488
489The ``CFLUSH_OVERHEAD`` time for CPU 4 is a lot larger than those for the little
490CPUs because the L2 cache size for the big cluster is lot larger (2MB) compared
491to the little cluster (1MB).
492
493The ``PSCI_EXIT`` times for CPUs in the big cluster are slightly smaller than
494for CPUs in the little cluster due to greater CPU performance. These times
495generally are greater than the ``PSCI_EXIT`` times in the ``CPU_SUSPEND`` tests
496because there is more code to execute in the "on finisher" compared to the
497"suspend finisher" (for example, GIC redistributor register programming).
498
499``PSCI_VERSION`` on all CPUs in parallel
500~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
501
502Since very little code is associated with ``PSCI_VERSION``, this test
503approximates the round trip latency for handling a fast SMC at EL3 in TF.
504
505+-------+-------------------+
506| CPU | TOTAL TIME (ns) |
507+=======+===================+
508| 0 | 3020 |
509+-------+-------------------+
510| 1 | 2940 |
511+-------+-------------------+
512| 2 | 2980 |
513+-------+-------------------+
514| 3 | 3060 |
515+-------+-------------------+
516| 4 | 520 |
517+-------+-------------------+
518| 5 | 720 |
519+-------+-------------------+
520
521The times for the big CPUs are less than the little CPUs due to greater CPU
522performance.
523
524We suspect the time for lead CPU 4 is shorter than CPU 5 due to subtle cache
525effects, given that these measurements are at the nano-second level.
526
John Tsichritzis63801cd2019-07-05 14:22:12 +0100527--------------
528
Harrison Mutai341740c2023-02-13 18:30:04 +0000529*Copyright (c) 2019-2023, Arm Limited and Contributors. All rights reserved.*
John Tsichritzis63801cd2019-07-05 14:22:12 +0100530
Harrison Mutai341740c2023-02-13 18:30:04 +0000531.. _Juno R1 platform: https://developer.arm.com/documentation/100122/latest/
Joel Hutton0f79fb12019-02-26 16:23:54 +0000532.. _TF master as of 31/01/2017: https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/?id=c38b36d
Harrison Mutai21cb9652023-05-17 13:09:16 +0100533.. _v2.9-rc0: https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/?h=v2.9-rc0