[coreboot] Out of MTRRs

Mon Mar 19 18:22:26 CET 2018

On Mon, Mar 19, 2018 at 10:55 AM, Jay Talbott
<JayTalbott at sysproconsulting.com> wrote:
> See below…
> - Jay
>

[Mon Mar 19 09:41:59.840 2018] MTRR Range: Start=79fff000 End=7a000000
(Size 1000)
[Mon Mar 19 09:41:59.840 2018] MTRR Range: Start=7a000000 End=7a800000
(Size 800000)
[Mon Mar 19 09:41:59.840 2018] MTRR Range: Start=7a800000 End=7ac00000
(Size 400000)
[Mon Mar 19 09:41:59.840 2018] MTRR Range: Start=7ac00000 End=7ae00000
(Size 200000)
[Mon Mar 19 09:41:59.840 2018] MTRR Range: Start=7ae00000 End=7af00000
(Size 100000)
[Mon Mar 19 09:41:59.840 2018] MTRR Range: Start=7af00000 End=7af80000
(Size 80000)
[Mon Mar 19 09:41:59.840 2018] MTRR Range: Start=7af80000 End=7afc0000
(Size 40000)
[Mon Mar 19 09:41:59.840 2018] MTRR Range: Start=7afc0000 End=7afe0000
(Size 20000)
[Mon Mar 19 09:41:59.840 2018] MTRR Range: Start=7afe0000 End=7aff0000
(Size 10000)
[Mon Mar 19 09:41:59.840 2018] MTRR Range: Start=7aff0000 End=7aff8000
(Size 8000)
[Mon Mar 19 09:41:59.840 2018] No more variable MTRRs: 10
[Mon Mar 19 09:41:59.840 2018] No more variable MTRRs: 10
[Mon Mar 19 09:41:59.840 2018] No more variable MTRRs: 10

This is from postcar loading. I don't understand why the code is is
adding that many MTRR entries during postcar loading. The only that
should be added is the ranges needed to load and run ramstage.  The
postcar MTRR code *does not* combine entries. It just determines the
number of MTRRs to meet what is sent in. It seems 0x79fff000--7aff8000
is being requested. That's a pretty bad request given the current
constraints. Do you know why that was requested? And why it can't be
more naturally aligned?

Below is the final MTRR calculation in ramstage:

[Mon Mar 19 09:42:04.473 2018] MTRR: Physical address space:
[Mon Mar 19 09:42:04.473 2018] 0x0000000000000000 - 0x00000000000a0000
size 0x000a0000 type 6
[Mon Mar 19 09:42:04.473 2018] 0x00000000000a0000 - 0x00000000000c0000
size 0x00020000 type 0
[Mon Mar 19 09:42:04.473 2018] 0x00000000000c0000 - 0x000000007b800000
size 0x7b740000 type 6
[Mon Mar 19 09:42:04.473 2018] 0x000000007b800000 - 0x00000000c0000000
size 0x44800000 type 0
[Mon Mar 19 09:42:04.473 2018] 0x00000000c0000000 - 0x00000000d0000000
size 0x10000000 type 1
[Mon Mar 19 09:42:04.473 2018] 0x00000000d0000000 - 0x0000000100000000
size 0x30000000 type 0
[Mon Mar 19 09:42:04.473 2018] 0x0000000100000000 - 0x000000047f000000
size 0x37f000000 type 6
[Mon Mar 19 09:42:04.473 2018] MTRR: Fixed MSR 0x250 0x0606060606060606
[Mon Mar 19 09:42:04.473 2018] MTRR: Fixed MSR 0x258 0x0606060606060606
[Mon Mar 19 09:42:04.473 2018] MTRR: Fixed MSR 0x259 0x0000000000000000
[Mon Mar 19 09:42:04.473 2018] MTRR: Fixed MSR 0x268 0x0606060606060606
[Mon Mar 19 09:42:04.473 2018] MTRR: Fixed MSR 0x269 0x0606060606060606
[Mon Mar 19 09:42:04.473 2018] MTRR: Fixed MSR 0x26a 0x0606060606060606
[Mon Mar 19 09:42:04.473 2018] MTRR: Fixed MSR 0x26b 0x0606060606060606
[Mon Mar 19 09:42:04.480 2018] MTRR: Fixed MSR 0x26c 0x0606060606060606
[Mon Mar 19 09:42:04.480 2018] MTRR: Fixed MSR 0x26d 0x0606060606060606
[Mon Mar 19 09:42:04.480 2018] MTRR: Fixed MSR 0x26e 0x0606060606060606
[Mon Mar 19 09:42:04.480 2018] MTRR: Fixed MSR 0x26f 0x0606060606060606
[Mon Mar 19 09:42:04.480 2018] call enable_fixed_mtrr()
[Mon Mar 19 09:42:04.480 2018] CPU physical address size: 39 bits
[Mon Mar 19 09:42:04.480 2018] MTRR: default type WB/UC MTRR counts: 6/11.
[Mon Mar 19 09:42:04.480 2018] MTRR: WB selected as default type.
[Mon Mar 19 09:42:04.480 2018] MTRR: 0 base 0x000000007b800000 mask
0x0000007fff800000 type 0
[Mon Mar 19 09:42:04.480 2018] MTRR: 1 base 0x000000007c000000 mask
0x0000007ffc000000 type 0
[Mon Mar 19 09:42:04.480 2018] MTRR: 2 base 0x0000000080000000 mask
0x0000007fc0000000 type 0
[Mon Mar 19 09:42:04.480 2018] MTRR: 3 base 0x00000000c0000000 mask
0x0000007ff0000000 type 1
[Mon Mar 19 09:42:04.480 2018] MTRR: 4 base 0x00000000d0000000 mask
0x0000007ff0000000 type 0
[Mon Mar 19 09:42:04.480 2018] MTRR: 5 base 0x00000000e0000000 mask
0x0000007fe0000000 type 0
[Mon Mar 19 09:42:04.480 2018] MTRR: Fixed MSR 0x250 0x0606060606060606
[Mon Mar 19 09:42:04.480 2018] MTRR: Fixed MSR 0x258 0x0606060606060606
[Mon Mar 19 09:42:04.480 2018] MTRR: Fixed MSR 0x259 0x0000000000000000
[Mon Mar 19 09:42:04.480 2018] MTRR: Fixed MSR 0x268 0x0606060606060606
[Mon Mar 19 09:42:04.480 2018] MTRR: Fixed MSR 0x269 0x0606060606060606
[Mon Mar 19 09:42:04.480 2018] MTRR: Fixed MSR 0x26a 0x0606060606060606
[Mon Mar 19 09:42:04.480 2018] MTRR: Fixed MSR 0x26b 0x0606060606060606
[Mon Mar 19 09:42:04.486 2018] MTRR: Fixed MSR 0x26c 0x0606060606060606
[Mon Mar 19 09:42:04.486 2018] MTRR: Fixed MSR 0x26d 0x0606060606060606
[Mon Mar 19 09:42:04.486 2018] MTRR: Fixed MSR 0x26e 0x0606060606060606
[Mon Mar 19 09:42:04.486 2018] MTRR: Fixed MSR 0x26f 0x0606060606060606
[Mon Mar 19 09:42:04.486 2018] MTRR: TEMPORARY Physical address space:
[Mon Mar 19 09:42:04.486 2018] call enable_fixed_mtrr()
[Mon Mar 19 09:42:04.486 2018] 0x0000000000000000 - 0x00000000000a0000
size 0x000a0000 type 6
[Mon Mar 19 09:42:04.486 2018] 0x00000000000a0000 - 0x00000000000c0000
size 0x00020000 type 0
[Mon Mar 19 09:42:04.486 2018] 0x00000000000c0000 - 0x000000007b800000
size 0x7b740000 type 6
[Mon Mar 19 09:42:04.486 2018] 0x000000007b800000 - 0x00000000ffe00000
size 0x84600000 type 0
[Mon Mar 19 09:42:04.486 2018] 0x00000000ffe00000 - 0x0000000100000000
size 0x00200000 type 5
[Mon Mar 19 09:42:04.486 2018] 0x0000000100000000 - 0x000000047f000000
size 0x37f000000 type 6
[Mon Mar 19 09:42:04.486 2018] CPU physical address size: 39 bits
[Mon Mar 19 09:42:04.486 2018] MTRR: Removing WRCOMB type. WB/UC MTRR
counts: 13/11 > 8.
[Mon Mar 19 09:42:04.486 2018] MTRR: Fixed MSR 0x250 0x0606060606060606
[Mon Mar 19 09:42:04.486 2018] MTRR: Fixed MSR 0x250 0x0606060606060606
[Mon Mar 19 09:42:04.486 2018] MTRR: Fixed MSR 0x258 0x0606060606060606
[Mon Mar 19 09:42:04.486 2018] MTRR: Fixed MSR 0x259 0x0000000000000000
[Mon Mar 19 09:42:04.486 2018] MTRR: Fixed MSR 0x268 0x0606060606060606
[Mon Mar 19 09:42:04.859 2018] MTRR: Fixed MSR 0x269 0x0606060606060606
[Mon Mar 19 09:42:04.859 2018] MTRR: Fixed MSR 0x26a 0x0606060606060606
[Mon Mar 19 09:42:04.859 2018] MTRR: Fixed MSR 0x26b 0x0606060606060606
[Mon Mar 19 09:42:04.859 2018] MTRR: Fixed MSR 0x26c 0x0606060606060606
[Mon Mar 19 09:42:04.859 2018] MTRR: Fixed MSR 0x26d 0x0606060606060606
[Mon Mar 19 09:42:04.859 2018] MTRR: Fixed MSR 0x26e 0x0606060606060606
[Mon Mar 19 09:42:04.859 2018] MTRR: Fixed MSR 0x26f 0x0606060606060606
[Mon Mar 19 09:42:04.859 2018] MTRR: Fixed MSR 0x258 0x0606060606060606
[Mon Mar 19 09:42:04.859 2018] call enable_fixed_mtrr()
[Mon Mar 19 09:42:04.859 2018] MTRR: Fixed MSR 0x259 0x0000000000000000
[Mon Mar 19 09:42:04.859 2018] MTRR: Fixed MSR 0x268 0x0606060606060606
[Mon Mar 19 09:42:04.859 2018] MTRR: Fixed MSR 0x269 0x0606060606060606
[Mon Mar 19 09:42:04.859 2018] MTRR: Fixed MSR 0x26a 0x0606060606060606
[Mon Mar 19 09:42:04.859 2018] MTRR: Fixed MSR 0x26b 0x0606060606060606
[Mon Mar 19 09:42:04.859 2018] MTRR: Fixed MSR 0x26c 0x0606060606060606
[Mon Mar 19 09:42:04.859 2018] MTRR: Fixed MSR 0x26d 0x0606060606060606
[Mon Mar 19 09:42:04.859 2018] MTRR: Fixed MSR 0x26e 0x0606060606060606
[Mon Mar 19 09:42:04.859 2018] MTRR: Fixed MSR 0x26f 0x0606060606060606
[Mon Mar 19 09:42:04.859 2018] CPU physical address size: 39 bits
[Mon Mar 19 09:42:04.859 2018] call enable_fixed_mtrr()
[Mon Mar 19 09:42:04.859 2018] MTRR: default type WB/UC MTRR counts: 13/11.
[Mon Mar 19 09:42:04.859 2018] MTRR: UC selected as default type.
[Mon Mar 19 09:42:04.859 2018] CPU physical address size: 39 bits
[Mon Mar 19 09:42:04.859 2018] MTRR: 0 base 0x0000000000000000 mask
0x0000007fc0000000 type 6
[Mon Mar 19 09:42:04.859 2018] MTRR: 1 base 0x0000000040000000 mask
0x0000007fe0000000 type 6
[Mon Mar 19 09:42:04.865 2018] MTRR: 2 base 0x0000000060000000 mask
0x0000007ff0000000 type 6
[Mon Mar 19 09:42:04.865 2018] MTRR: 3 base 0x0000000070000000 mask
0x0000007ff8000000 type 6
[Mon Mar 19 09:42:04.865 2018] MTRR: 4 base 0x0000000078000000 mask
0x0000007ffc000000 type 6
[Mon Mar 19 09:42:04.865 2018] MTRR: 5 base 0x000000007b800000 mask
0x0000007fff800000 type 0
[Mon Mar 19 09:42:04.865 2018] MTRR: 6 base 0x00000000ffe00000 mask
0x0000007fffe00000 type 5
[Mon Mar 19 09:42:04.865 2018] MTRR: 7 base 0x0000000100000000 mask
0x0000007f00000000 type 6
[Mon Mar 19 09:42:04.865 2018] Taking a reserved OS MTRR.
[Mon Mar 19 09:42:04.865 2018] MTRR: 8 base 0x0000000200000000 mask
0x0000007e00000000 type 6
[Mon Mar 19 09:42:04.865 2018] Taking a reserved OS MTRR.
[Mon Mar 19 09:42:04.865 2018] MTRR: 9 base 0x0000000400000000 mask
0x0000007f80000000 type 6
[Mon Mar 19 09:42:04.865 2018] Taking a reserved OS MTRR.
[Mon Mar 19 09:42:04.865 2018] ERROR: Not enough MTRRs available! MTRR
indexis 10 with 10 MTTRs in total.
[Mon Mar 19 09:42:04.865 2018] Not enough MTRRs: 11 vs 10
[Mon Mar 19 09:42:04.865 2018] Unable to insert temporary MTRR range:
0x00000000ffe00000 - 0x0000000100000000 size 0x00200000 type 5
[Mon Mar 19 09:42:04.865 2018]
[Mon Mar 19 09:42:04.865 2018] MTRR check
[Mon Mar 19 09:42:04.865 2018] Fixed MTRRs   : Enabled
[Mon Mar 19 09:42:04.865 2018] Variable MTRRs: Enabled

Looks like the temporary mapping is causing all the spam. You could do
a couple fo things: don't try calling mtrr_use_temp_range().
Additionally you could set your settings differently such that it
reduces the MTRR pressure; something like requesting different io hole
sizes, etc.

0x0000000100000000 - 0x000000047f000000 size 0x37f000000 type 6

That one hurts a lot. Count the bits set in the size. That's 9 entries
right there.

Nico has some patches outstanding that could possibly better optimize
this condition. https://review.coreboot.org/#/c/coreboot/+/21915/ and
2 more on top.