Li-Ta Lo ollie@lanl.gov writes:
I think there is some problem in the resource allocation code so it can not handle devices with 2 prefmem.
Maybe. I know there was some kind of problem on the opteron, early on but I forget what. The code for all resource types is the same so if we can handle two resources on a bus of the same type it should not be quite as simple as that.
Is the code just picking the "largest" resource ? I am really confused by the code in devices.c
No.
The code should be quite simple but it is recursive and highly abstracted which makes it hard to follow.
The high level overview is the code works in two passes. The first pass is to determine the size of the resource window needed. The second pass is to determine the actual resource assignments.
There are many was to assign resources on a bus. After some experiences with tight memory situations I implemented a near optimal solution. The solution is optimal if all of your resources are a power of 2 in size.
Basically the code is a loop. For each iteration the code finds the largest unassigned resource. Then the resource constraints of that resource are considered and padding between the previous resources and the current resources are inserted if necessary. Then we get into the next iteration.
The reason this is optimal if all of your resources are a power of two in size is because if your previous resource is a larger or equal power of two no padding will be needed for the current resource.
The situation Yhlu has is below and it is weird. The resources are assigned properly but when the are clumped together into a range register on the bus that value is incorrect. Which is very weird.
YhLu has:
ASSIGN RESOURCES, bus 3 PCI: 03:03.0 10 <- [0xe0000000 - 0xefffffff] prefmem PCI: 03:03.0 18 <- [0xf0000000 - 0xf3ffffff] prefmem
And then on bus 2:
PCI: 01:02.0 24 <- [0xe0000000 - 0xffffffff] bus 3 prefmem
The bus 3 prefmem should be PCI: 01:02.0 24 <- [0xe0000000 - 0xf3ffffff] bus 3 prefmem
So it looks like stuck bits or something.
Ollie can you get a boot log from Orange? Unless they are different types of infiniband adapters things should be fairly comparable.
Eric