Re: [coreboot] Are multiple GPUs with lots of memory (ex: 8GB+8GB) supported in coreboot?
So .. what's the solution?..
Why this statement: ".. It's not a straight-forward solution to simply push all prefetchable BARs above 4GiB (that would be NICs and GFX too)..." ?
From what I'm seeing this feature is not only usefull, it is essential.. (if we were to pretend always supporting real high-performance HW like powerfull GPUs..). As Taiidan said, it's really a shame that nobody continued working on this..
What can be done?.. Maybe some of us would be able/willing to pick-up this task, but as this implies modifying the resource allocator which seems like a critical element of coreboot.. (if I'm not mistaken..), someone with experience should (at least) give the entry points..
My 2 satoshis, Florentin
----- Mail d'origine ----- De: Kyösti Mälkki kyosti.malkki@gmail.com À: Taiidan@gmx.com Cc: Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net, Arthur Heymans arthur@aheymans.xyz, coreboot coreboot@coreboot.org Envoyé: Sun, 11 Feb 2018 22:47:05 +0100 (CET) Objet: Re: [coreboot] Are multiple GPUs with lots of memory (ex: 8GB+8GB) supported in coreboot?
On Sun, Feb 11, 2018 at 11:04 PM, Taiidan@gmx.com Taiidan@gmx.com wrote:
On 02/11/2018 03:32 PM, Carl-Daniel Hailfinger wrote:
On 10.02.2018 19:57, Arthur Heymans wrote:
"Taiidan@gmx.com" Taiidan@gmx.com writes:
I would like to add this information to the wiki so I am wondering if anyone has successfully used for instance dual 8GB graphics cards with coreboot.
I don't think that this gpu memory is mapped into the linear memory space. Such a GPU will typically have a PCI memory resource BAR that is (only) 256M large (could be 512M or more these days).
I am not sure if this would be an issue due to coreboot only having 32bit MMIO space.
It depends a bit on how things are configured but with a fairly common 2G mmio space below 4G I think it is unlikely for 2 external GPU's to be an issue.
It would of course be nice to have 64bit BAR support but that would require substantial changes in the allocator and more importantly a lot of time spend in a sane and thoughtful design...
I think Myles Watson (not 100% sure) worked on an improved resource allocator with 64 bit resource support for v3. I do not know if it ever was merged in v4, or if that was instead a port of the v2 allocator to v3, in which case this would be mostly what we have now. The mails about this are mostly from 2009.
Thanks for the info everyone :]
That is a damn shame it didn't get merged, it would be quite useful to make coreboot a better server platform - it is needed for many nics, sr-iov, etc.
Hi
Some of that work did get merged, bust mostly hacked in for certain AMD parts only. After some years, that code did not even build anymore as it was behind some #ifdef and Kconfig options, and no board had it enabled by default. 64 bit PCI BARs (as configuration registers) are supported, but no MMIO resources are pushed over 4GiB boundary as stated above.
You need some classification of what resources must reside in 32bit space. That is, all the resources payload may need to use. It's not a straight-forward solution to simply push all prefetchable BARs above 4GiB (that would be NICs and GFX too). Any off-board, prefetchable 64bit BARs that are large (say over 64MiB) in size might a viable starting point, where 32bit space would not run out of room. I have had a look at this before, but without funding did not pursue for submitting or completing it.
There was also work maybe two years ago, reusing some of the design choices from UEFI. I think the developer never returned with answers or change requests for the changeset. AFAIR it relied on identinfying whether a device has an oprom associated with it, so I did not find that work mergeable.
Kyösti
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
My experience is that the hard part isn't getting the allocator fixed, it's getting the fixes upstream since they would affect practically every single board in the tree. Before even working on this, you would probably need to make sure you can get community test coverage for e.g.
50% of the boards in the tree. That's why I gave up on even pushing
for this support back when, and why every single attempt that I'm aware of has bitrotted into oblivion.
For my part, I can test on AMD non-AGESA hardware, but you'd have to line up the Intel side and probably AMD AGESA testing to even have a shot at adding this support.
Just my experience, take it with a grain of salt. :-)
On 02/11/2018 06:02 PM, echelon@free.fr wrote:
So .. what's the solution?..
Why this statement: ".. It's not a straight-forward solution to simply push all prefetchable BARs above 4GiB (that would be NICs and GFX too)..." ?
From what I'm seeing this feature is not only usefull, it is essential.. (if we were to pretend always supporting real high-performance HW like powerfull GPUs..). As Taiidan said, it's really a shame that nobody continued working on this.. What can be done?.. Maybe some of us would be able/willing to pick-up this task, but as this implies modifying the resource allocator which seems like a critical element of coreboot.. (if I'm not mistaken..), someone with experience should (at least) give the entry points..
My 2 satoshis, Florentin
----- Mail d'origine ----- De: Kyösti Mälkki kyosti.malkki@gmail.com À: Taiidan@gmx.com Cc: Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net, Arthur Heymans arthur@aheymans.xyz, coreboot coreboot@coreboot.org Envoyé: Sun, 11 Feb 2018 22:47:05 +0100 (CET) Objet: Re: [coreboot] Are multiple GPUs with lots of memory (ex: 8GB+8GB) supported in coreboot?
On Sun, Feb 11, 2018 at 11:04 PM, Taiidan@gmx.com Taiidan@gmx.com wrote:
On 02/11/2018 03:32 PM, Carl-Daniel Hailfinger wrote:
On 10.02.2018 19:57, Arthur Heymans wrote:
"Taiidan@gmx.com" Taiidan@gmx.com writes:
I would like to add this information to the wiki so I am wondering if anyone has successfully used for instance dual 8GB graphics cards with coreboot.
I don't think that this gpu memory is mapped into the linear memory space. Such a GPU will typically have a PCI memory resource BAR that is (only) 256M large (could be 512M or more these days).
I am not sure if this would be an issue due to coreboot only having 32bit MMIO space.
It depends a bit on how things are configured but with a fairly common 2G mmio space below 4G I think it is unlikely for 2 external GPU's to be an issue.
It would of course be nice to have 64bit BAR support but that would require substantial changes in the allocator and more importantly a lot of time spend in a sane and thoughtful design...
I think Myles Watson (not 100% sure) worked on an improved resource allocator with 64 bit resource support for v3. I do not know if it ever was merged in v4, or if that was instead a port of the v2 allocator to v3, in which case this would be mostly what we have now. The mails about this are mostly from 2009.
Thanks for the info everyone :]
That is a damn shame it didn't get merged, it would be quite useful to make coreboot a better server platform - it is needed for many nics, sr-iov, etc.
Hi
Some of that work did get merged, bust mostly hacked in for certain AMD parts only. After some years, that code did not even build anymore as it was behind some #ifdef and Kconfig options, and no board had it enabled by default. 64 bit PCI BARs (as configuration registers) are supported, but no MMIO resources are pushed over 4GiB boundary as stated above.
You need some classification of what resources must reside in 32bit space. That is, all the resources payload may need to use. It's not a straight-forward solution to simply push all prefetchable BARs above 4GiB (that would be NICs and GFX too). Any off-board, prefetchable 64bit BARs that are large (say over 64MiB) in size might a viable starting point, where 32bit space would not run out of room. I have had a look at this before, but without funding did not pursue for submitting or completing it.
There was also work maybe two years ago, reusing some of the design choices from UEFI. I think the developer never returned with answers or change requests for the changeset. AFAIR it relied on identinfying whether a device has an oprom associated with it, so I did not find that work mergeable.
Kyösti
- -- Timothy Pearson Raptor Engineering +1 (415) 727-8645 (direct line) +1 (512) 690-0200 (switchboard) https://www.raptorengineering.com
-
echelon＠free.fr -
Timothy Pearson