Tested and working on hardware :D
-Corey
On Tue, Jan 6, 2009 at 4:52 AM, Corey Osgood corey.osgood@gmail.com wrote:
Tested and working on hardware :D
Great job!
As I read your patch I wondered if there's a good reason to disable caching. Can we just take out the #ifdefs? Or maybe make it an #ifndef DONT_CACHE_AND_RUN_REAL_SLOW_ON_C7? :)
Acked-by: Myles Watson mylesgw@gmail.com
Thanks, Myles
On Tue, Jan 6, 2009 at 7:39 AM, Myles Watson mylesgw@gmail.com wrote:
On Tue, Jan 6, 2009 at 4:52 AM, Corey Osgood corey.osgood@gmail.com wrote:
Tested and working on hardware :D
Great job!
As I read your patch I wondered if there's a good reason to disable caching. Can we just take out the #ifdefs? Or maybe make it an #ifndef DONT_CACHE_AND_RUN_REAL_SLOW_ON_C7? :)
Acked-by: Myles Watson mylesgw@gmail.com
Thanks, Myles
I think that we need to examine this for all the CPUs. In a legacy BIOS all the runtime code will be in memory, so caching the ROM is only need prior to memory setup. That is not the case for coreboot because we have the XIP printk and LAR code. We should use one MTRR for caching the ROM. So MTRR for ROM caching is setup by the CPU specific CAR code and then later in normal MTRR setup we add an MTRR that will support XIP for the rest of POST and payload loading. There are two minor drawbacks to doing this.
1. You use an MTRR that could be used by the OS. No big deal, there are more than enough MTRRs for every confguration I can think of.
2. You might use cache space for ROM code that will fetch slowly and might only run once. This is no longer a problem with large L2 and L3 caches. Even on Geode it should be ok. Also, the ROM shouldn't be accessed once the OS loads.
The last thing is if the ROM should be cached read-only or write-through. If set RO, flashrom will need to be MTRR aware and set it to WT when writing ROMs.
Marc
On Tue, Jan 6, 2009 at 12:31 PM, Marc Jones marcj303@gmail.com wrote:
On Tue, Jan 6, 2009 at 7:39 AM, Myles Watson mylesgw@gmail.com wrote:
On Tue, Jan 6, 2009 at 4:52 AM, Corey Osgood corey.osgood@gmail.com
wrote:
Tested and working on hardware :D
Great job!
As I read your patch I wondered if there's a good reason to disable caching. Can we just take out the #ifdefs? Or maybe make it an #ifndef DONT_CACHE_AND_RUN_REAL_SLOW_ON_C7? :)
Acked-by: Myles Watson mylesgw@gmail.com
Thanks, Myles
Ooops, this never got committed, figured that out very quickly last night. r1164
I think that we need to examine this for all the CPUs. In a legacy BIOS all the runtime code will be in memory, so caching the ROM is only need prior to memory setup. That is not the case for coreboot because we have the XIP printk and LAR code. We should use one MTRR for caching the ROM. So MTRR for ROM caching is setup by the CPU specific CAR code and then later in normal MTRR setup we add an MTRR that will support XIP for the rest of POST and payload loading. There are two minor drawbacks to doing this.
- You use an MTRR that could be used by the OS. No big deal, there
are more than enough MTRRs for every confguration I can think of.
- You might use cache space for ROM code that will fetch slowly and
might only run once. This is no longer a problem with large L2 and L3 caches. Even on Geode it should be ok. Also, the ROM shouldn't be accessed once the OS loads.
The last thing is if the ROM should be cached read-only or write-through. If set RO, flashrom will need to be MTRR aware and set it to WT when writing ROMs.
Currently, at least with C7 CAR and I think some others, CAR uses the first variable MTRR, MTRR 0, and the ROM is cached at MTRR 1 by stage 0. Once the MTRR setup code runs, the ROM cache is killed off, and things slow down. I see a few solutions:
1. Give ROM caching priority, MTRR code first sets up a WB MTRR 0 to cache the ROM, then clears out MTRR 1-8, and sets up MTRRs for RAM. This way the ROM is never not cached, and the code should be easier to write. 2. Give memory priority, MTRR code clears all variable MTRRs, sets up memory, and then does the ROM cache as the very last thing. If memory eats up all the MTRRs, oh well, the rest of the boot will be slow. Any calls to XIP functions between the time MTRRs are cleared and the time the ROM MTRR is set up will be slow. 3. Give memory priority, but reserve the last MTRR for the ROM cache. At that point you might as well go for 1.
Thanks, Corey
On Tue, Apr 14, 2009 at 9:56 AM, Corey Osgood corey.osgood@gmail.com wrote:
On Tue, Jan 6, 2009 at 12:31 PM, Marc Jones marcj303@gmail.com wrote:
On Tue, Jan 6, 2009 at 7:39 AM, Myles Watson mylesgw@gmail.com wrote:
On Tue, Jan 6, 2009 at 4:52 AM, Corey Osgood corey.osgood@gmail.com wrote:
Tested and working on hardware :D
Great job!
As I read your patch I wondered if there's a good reason to disable caching. Can we just take out the #ifdefs? Or maybe make it an #ifndef DONT_CACHE_AND_RUN_REAL_SLOW_ON_C7? :)
Acked-by: Myles Watson mylesgw@gmail.com
Thanks, Myles
Ooops, this never got committed, figured that out very quickly last night. r1164
I think that we need to examine this for all the CPUs. In a legacy BIOS all the runtime code will be in memory, so caching the ROM is only need prior to memory setup. That is not the case for coreboot because we have the XIP printk and LAR code. We should use one MTRR for caching the ROM. So MTRR for ROM caching is setup by the CPU specific CAR code and then later in normal MTRR setup we add an MTRR that will support XIP for the rest of POST and payload loading. There are two minor drawbacks to doing this.
- You use an MTRR that could be used by the OS. No big deal, there
are more than enough MTRRs for every confguration I can think of.
- You might use cache space for ROM code that will fetch slowly and
might only run once. This is no longer a problem with large L2 and L3 caches. Even on Geode it should be ok. Also, the ROM shouldn't be accessed once the OS loads.
The last thing is if the ROM should be cached read-only or write-through. If set RO, flashrom will need to be MTRR aware and set it to WT when writing ROMs.
Currently, at least with C7 CAR and I think some others, CAR uses the first variable MTRR, MTRR 0, and the ROM is cached at MTRR 1 by stage 0. Once the MTRR setup code runs, the ROM cache is killed off, and things slow down. I see a few solutions:
- Give ROM caching priority, MTRR code first sets up a WB MTRR 0 to cache
the ROM, then clears out MTRR 1-8, and sets up MTRRs for RAM. This way the ROM is never not cached, and the code should be easier to write. 2. Give memory priority, MTRR code clears all variable MTRRs, sets up memory, and then does the ROM cache as the very last thing. If memory eats up all the MTRRs, oh well, the rest of the boot will be slow. Any calls to XIP functions between the time MTRRs are cleared and the time the ROM MTRR is set up will be slow. 3. Give memory priority, but reserve the last MTRR for the ROM cache. At that point you might as well go for 1.
I think that you have to do #2 to be "safe" but there are enough MTRRs that you shouldn't have a problem. As you pointed out the real problem is between the end of CAR and you are still executing out of the ROM. Copying data out of the ROM doesn't help that much since the device is slow and narrow (you can't fetch way at a time like real memory). In V2 MTRRs get setup 3 times. 1. CAR MTRR setup 2. End CAR basic memory MTRR setup 3. MTRR setup which happens after device setup near the end.
In v2 step 2 is very basic generic setup and could be improved. From step 2 to 3 you want the ROM cached since it copies and decompress from the ROM. I don't think that the ROM is left cachable at this point for K8/FAM10 and needs to be fixed.
At step 3 the devices have been setup and MTRRs need to be setup for the devices (UMA video memory). The only ROM access after this point is to copy out the payload. Caching is a minimal help there and not a big deal if you are out of MTRRs.
The last thing is that you might want to save few MTRRs for the OS to use. I think that in v2 it is handled in step 3.
In v3 we only do step 1 and 2 for K8/FAM10 but step 2 is more advanced than in v2. It sets the MTRRs, fixed and variable and caches the ROM. A step 3 needs to be added but then the late init needs to be AP CPU aware and we didn't go down that route yet.....
I hope that was clear enough to help...
Marc
-
Corey Osgood -
Marc Jones -
Myles Watson