Patrick Georgi (pgeorgi@google.com) just uploaded a new patch set to gerrit, which you can find at http://review.coreboot.org/9527
-gerrit
commit f5049653a29d39421c0509378974899338f62722 Author: Aaron Durbin adurbin@chromium.org Date: Thu Nov 20 22:17:54 2014 -0600
arm64: ensure secondary CPU's stack tops are not in the cache
Secondary CPUs were intermittently not coming online as expected. Upon investigation it was found that a cache line needed to be invalidated that corresponded to the top of the stack for the failing CPU.
Currently the secondary CPUs come online with caching disabled. However, the code paths are using C and thus the stack it is assigned. The MMU is enabled in C after it's pushed its return path onto the stack that went directly to ram. When the cache line corresponding to its stack is valid in the cache it will hit once the MMU is enabled. That hit will have invalid data w.r.t. the return addresses pushed directly into ram.
This is not the best solution as the only way to guarantee we don't hit such a situation is to tightly manage resource usage up until the point of MMU enablement. That can be done in a followup patch.
BUG=chrome-os-partner:33962 BRANCH=None TEST=On ryu where secondary CPUs weren't coming online consistently, they now come up.
Change-Id: I03237656da180d1f74df3a8e00029ba8d778bca8 Signed-off-by: Patrick Georgi pgeorgi@chromium.org Original-Commit-Id: 06ab6afc996cf92c45d4cd6850e31167c2946a95 Original-Signed-off-by: Aaron Durbin adurbin@chromium.org Original-Change-Id: I32de749ea48c19e23442e6dc5678c5369ac3b2b6 Original-Reviewed-on: https://chromium-review.googlesource.com/231219 Original-Reviewed-by: Furquan Shaikh furquan@chromium.org Original-Tested-by: Furquan Shaikh furquan@chromium.org --- src/arch/arm64/cpu_ramstage.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
diff --git a/src/arch/arm64/cpu_ramstage.c b/src/arch/arm64/cpu_ramstage.c index ce81f93..ec1ac0f 100644 --- a/src/arch/arm64/cpu_ramstage.c +++ b/src/arch/arm64/cpu_ramstage.c @@ -19,6 +19,7 @@
#include <stdint.h> #include <stdlib.h> +#include <arch/cache.h> #include <arch/lib_helpers.h> #include <cpu/cpu.h> #include <console/console.h> @@ -155,6 +156,13 @@ static void init_cpu_info(struct bus *bus) cpu_mark_online(cpu_info()); }
+static void invalidate_cpu_stack_top(unsigned int id) +{ + const size_t size = 128; + char *stack = cpu_get_stack(id); + dcache_invalidate_by_mva(stack - size, size); +} + void arch_initialize_cpus(device_t cluster, struct cpu_control_ops *cntrl_ops) { size_t max_cpus; @@ -208,6 +216,10 @@ void arch_initialize_cpus(device_t cluster, struct cpu_control_ops *cntrl_ops) if (!cpu_online(ci)) { /* Start the CPU. */ printk(BIOS_DEBUG, "Starting CPU%x\n", ci->id); + + /* Ensure CPU's top of stack is not in the cache. */ + invalidate_cpu_stack_top(ci->id); + if (cntrl_ops->start_cpu(ci->id, entry)) { printk(BIOS_ERR, "Failed to start CPU%x\n", ci->id);