Dear microcode experts, please give us some feedback on how to do microcode updates properly in coreboot.
While fixing a crash initializing Intel SGX on HT enabled CPUs I noticed that coreboot doesn't care for HT at all when doing MSR initialisation. I proposed the following patch [1], but it turns out that there are different opinions about updating microcode in parallel.
In total there are four ways and only one can be correct: 1. Up to now coreboot updated all threads in parallel without a lock. That seems to follow Intel SDM (Order Number: 253668-070US) Chapter 8.7.11 2. According to Intel SDM (Order Number: 253668-070US) Chapter 9.11.6.3 microcode updates must only be done on one thread for each core, as long as the sibling thread doesn't do an update at time. 3. The kernel patch [2] states that the sibling thread must be idle while updating microcode on one core. 4. The "Bios Writers Guide" (Document Number 504790 / 550049) states that microcode must be done on all threads, but synchronized between sibling threads to only update one at a time.
Which one is the correct way and how do you know? @Intel Can you update the SDM to clarify?
Thank you for your time.
1: https://review.coreboot.org/c/coreboot/+/35739/ 2: https://lore.kernel.org/patchwork/patch/889353/
Regards, Patrick Rudolph
9elements GmbH, Kortumstraße 19-21, 44787 Bochum, Germany Email: patrick.rudolph@9elements.com Phone: +49 234 / 68 94 188
Sitz der Gesellschaft: Bochum Handelsregister: Amtsgericht Bochum, HRB 13207 Geschäftsführung: Eray Basar, Sebastian Deutsch
On 08.10.19 09:50, Patrick Rudolph wrote:
- The "Bios Writers Guide" (Document Number 504790 / 550049) states
that microcode must be done on all threads, but synchronized between sibling threads to only update one at a time.
A minor correction: BWGs from Ivy Bridge on (including 550049) simply state that it can be done in parallel (and dropped the synchronization clause).
Nico
Hi Patrick,
On Tue, Oct 08, 2019 at 09:50:09AM +0200, Patrick Rudolph wrote:
I proposed the following patch [1], but it turns out that there are different opinions about updating microcode in parallel.
See the functions microcode_reload_late() and __reload_late() here:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch...
This is Ashok's latest proposal which seems to work so far. :)
In total there are four ways and only one can be correct:
- Up to now coreboot updated all threads in parallel without a lock.
That seems to follow Intel SDM (Order Number: 253668-070US) Chapter 8.7.11
Yes, you don't need to grab a lock but ...
- According to Intel SDM (Order Number: 253668-070US) Chapter
9.11.6.3 microcode updates must only be done on one thread for each core, as long as the sibling thread doesn't do an update at time.
... yes, you need to update on only one thread of a HT core.
- The kernel patch [2] states that the sibling thread must be idle
while updating microcode on one core.
Yes, that's why we're doing the stop_machine() thing which IPIs all threads into __reload_late() and we hold them there or let them update, depending on whether they're HT0 or not. The second call to apply_microcode_local() simply updates the ucode revision number for the thread calling it and exits early.
- The "Bios Writers Guide" (Document Number 504790 / 550049) states
that microcode must be done on all threads, but synchronized between sibling threads to only update one at a time.
I think that's wrong, if the above code is correct.
But this is what I've gathered from recent discussions. I'll let Intel people correct me if I've missed or misrepresented something.
HTH.
We very strongly suggest that you idle the sibbling, and depending on your cpu model, you might need some other tweaks. It's best in general for the system to be as quiet as possible.
Hi
On Tue, Oct 08, 2019 at 06:47:07AM -0700, Van De Ven, Arjan wrote:
We very strongly suggest that you idle the sibbling, and depending on your cpu model, you might need some other tweaks. It's best in general for the system to be as quiet as possible.
Since you say this is coreboot i'm assuming this is preboot before you bring up the OS.
If the sequence of bringup is that you bringup each logical cpu one at a time much like how Linux early boot is, then you don't have to worry. Since you would expect teh CPU is just waiting to boot in Wais-for-SIPI state.
You can see linux earlyboot simply checks if the version on the cpu is older than the one packaged in initrd, then we would apply the newer microcode.
Once the first thread in the core loads the microcode the thread-sibling would automatically
Boris provided excellant references to the Linux code for late-loading where the cpus are already doing work in the OS like scheduling, handling interrupts etc. Then those rules apply.
Cheers, Ashok
-
Borislav Petkov -
Nico Huber -
Patrick Rudolph -
Raj, Ashok -
Van De Ven, Arjan