I've yanked all of the sdram initialization from my s2895 so that I can use my HT device as RAM. I've set up the BARs in the device and in the Opteron so that I should have access to 0-2GB. It hangs here:
testx = 5a5a5a5a Copying data from cache to RAM -- switching to use RAM as stack...
Somehow the memory accesses aren't reaching my device. I added some debugging, and this line before the debugging:
set_var_mtrr(0, 0x08000000, 0x10000000, MTRR_TYPE_WRBACK);
Here's the output:
xd1000_ram_init: ram_dev=4000000 id=554e1172 xd1000_BARs xd1000_ram_init: testing "RAM" val = ffffffff After loop p=10180020 *p=ffffffff val(@0x08180020)=8189920 *p(08180020) = 8189920 *p(08980020) = ffffffff *p(09180020) = ffffffff *p(09980020) = ffffffff *p(0a180020) = ffffffff *p(0a980020) = ffffffff *p(0b180020) = ffffffff *p(0b980020) = ffffffff *p(0c180020) = ffffffff *p(0c980020) = ffffffff *p(0d180020) = ffffffff *p(0d980020) = ffffffff *p(0e180020) = ffffffff *p(0e980020) = ffffffff *p(0f180020) = ffffffff *p(0f980020) = ffffffff v_esp=000cffe8
It looks like one value sticks but not the rest. Here's the debugging code:
printk_debug("%s: testing "RAM"", __func__);
// Try to force the values out of the cache. *((volatile u32 *)0x08180000) = 0x5a5a5a50; *((volatile u32 *)0x08180008) = 0x5a5a5a51; *((volatile u32 *)0x08180010) = 0x5a5a5a52; *((volatile u32 *)0x08180018) = 0x5a5a5a53; *((volatile u32 *)0x08180020) = 0x5a5a5a54; *((volatile u32 *)0x08180028) = 0x5a5a5a55; *((volatile u32 *)0x08180030) = 0x5a5a5a56; *((volatile u32 *)0x08180038) = 0x5a5a5a57; *((volatile u32 *)0x08180040) = 0x5a5a5a58; *((volatile u32 *)0x09180020) = 0x5a5a5a59; *((volatile u32 *)0x0A180020) = 0x5a5a5a5a; *((volatile u32 *)0x0B180020) = 0x5a5a5a5b; *((volatile u32 *)0x0C180020) = 0x5a5a5a5c; *((volatile u32 *)0x0D180020) = 0x5a5a5a5d; *((volatile u32 *)0x0E180020) = 0x5a5a5a5e; *((volatile u32 *)0x0F180020) = 0x5a5a5a5f; // Check value.
val = *((volatile u32 *) 0x08180020); printk_debug(" val = %x\n", val);
// Write to various locations. for (p = (volatile u32 *) 0x08180020; p < ((volatile u32 *) 0x10000000); p+=0x200000) *p = (u32) p + 0x00009900; val = *((volatile u32 *) 0x08180020); printk_debug("After loop p=%p *p=%x val(@0x08180020)=%x\n", p, *p, val);
//Read them back for (p = (volatile u32 *) 0x08180020; p < ((volatile u32 *) 0x10000000); p+=0x200000) { val = *p; printk_debug(" *p(%p) = %lx\n", p, val); }
So I'm going through all the code I disabled trying to figure out what else the Opteron needs to send memory references onto the bus.
Any suggestions?
Thanks, Myles
Myles Watson wrote:
I've yanked all of the sdram initialization from my s2895 so that I can use my HT device as RAM. I've set up the BARs in the device and in the Opteron so that I should have access to 0-2GB. It hangs here:
testx = 5a5a5a5a Copying data from cache to RAM -- switching to use RAM as stack...Somehow the memory accesses aren't reaching my device. I added some debugging, and this line before the debugging:
set_var_mtrr(0, 0x08000000, 0x10000000, MTRR_TYPE_WRBACK);
I think you need to have setup correctly the resource routing registers. Also maybe it is more like a PCI space? If so then you need to setup the MTRR and ask to send the accesses to PCI space and not to ram.
Rudolf
Here's the output:
xd1000_ram_init: ram_dev=4000000 id=554e1172 xd1000_BARs xd1000_ram_init: testing "RAM" val = ffffffff After loop p=10180020 *p=ffffffff val(@0x08180020)=8189920 *p(08180020) = 8189920 *p(08980020) = ffffffff *p(09180020) = ffffffff *p(09980020) = ffffffff *p(0a180020) = ffffffff *p(0a980020) = ffffffff *p(0b180020) = ffffffff *p(0b980020) = ffffffff *p(0c180020) = ffffffff *p(0c980020) = ffffffff *p(0d180020) = ffffffff *p(0d980020) = ffffffff *p(0e180020) = ffffffff *p(0e980020) = ffffffff *p(0f180020) = ffffffff *p(0f980020) = ffffffff v_esp=000cffe8
It looks like one value sticks but not the rest. Here's the debugging code:
printk_debug("%s: testing \"RAM\"", __func__);// Try to force the values out of the cache. *((volatile u32 *)0x08180000) = 0x5a5a5a50; *((volatile u32 *)0x08180008) = 0x5a5a5a51; *((volatile u32 *)0x08180010) = 0x5a5a5a52; *((volatile u32 *)0x08180018) = 0x5a5a5a53; *((volatile u32 *)0x08180020) = 0x5a5a5a54; *((volatile u32 *)0x08180028) = 0x5a5a5a55; *((volatile u32 *)0x08180030) = 0x5a5a5a56; *((volatile u32 *)0x08180038) = 0x5a5a5a57; *((volatile u32 *)0x08180040) = 0x5a5a5a58; *((volatile u32 *)0x09180020) = 0x5a5a5a59; *((volatile u32 *)0x0A180020) = 0x5a5a5a5a; *((volatile u32 *)0x0B180020) = 0x5a5a5a5b; *((volatile u32 *)0x0C180020) = 0x5a5a5a5c; *((volatile u32 *)0x0D180020) = 0x5a5a5a5d; *((volatile u32 *)0x0E180020) = 0x5a5a5a5e; *((volatile u32 *)0x0F180020) = 0x5a5a5a5f; // Check value.
val = *((volatile u32 *) 0x08180020); printk_debug(" val = %x\n", val);// Write to various locations. for (p = (volatile u32 *) 0x08180020; p < ((volatile u32 *) 0x10000000); p+=0x200000) *p = (u32) p + 0x00009900; val = *((volatile u32 *) 0x08180020); printk_debug("After loop p=%p *p=%x val(@0x08180020)=%x\n", p, *p, val);
//Read them back for (p = (volatile u32 *) 0x08180020; p < ((volatile u32 *) 0x10000000); p+=0x200000) { val = *p; printk_debug(" *p(%p) = %lx\n", p, val); }
So I'm going through all the code I disabled trying to figure out what else the Opteron needs to send memory references onto the bus.
Any suggestions?
Thanks, Myles
Somehow the memory accesses aren't reaching my device. I added some debugging, and this line before the debugging:
set_var_mtrr(0, 0x08000000, 0x10000000, MTRR_TYPE_WRBACK);
I think you need to have setup correctly the resource routing registers.
I thought I had this right, but I'm double checking.
Also maybe it is more like a PCI space? If so then you need to setup the MTRR and ask to send the accesses to PCI space and not to ram.
I want to use it like RAM, not PCI space. I'll look more into the definitions of the MTRRs to see what else is there.
Thanks, Myles
On 09.06.2009 18:48 Uhr, Myles Watson wrote:
I've yanked all of the sdram initialization from my s2895 so that I can use my HT device as RAM. I've set up the BARs in the device and in the Opteron so that I should have access to 0-2GB. It hangs here:
testx = 5a5a5a5a Copying data from cache to RAM -- switching to use RAM as stack...Somehow the memory accesses aren't reaching my device. I added some debugging, and this line before the debugging:
set_var_mtrr(0, 0x08000000, 0x10000000, MTRR_TYPE_WRBACK);
Hm.. doesn't the base address need at least an alignment of size?
Have you tried disabling caching completely for the area?
// Try to force the values out of the cache.
If you are trying to avoid the accesses from being cached, they need to cross a cache line.
On the core duo a cache line is 0x40 bytes, so I used an offset of 0x80 for a dual channel interleaving system between memory accesses in order to make sure that I read from the same DIMM and still cross the cache line border.
*((volatile u32 *)0x08180000) = 0x5a5a5a50; *((volatile u32 *)0x08180008) = 0x5a5a5a51; *((volatile u32 *)0x08180010) = 0x5a5a5a52; *((volatile u32 *)0x08180018) = 0x5a5a5a53; *((volatile u32 *)0x08180020) = 0x5a5a5a54; *((volatile u32 *)0x08180028) = 0x5a5a5a55; *((volatile u32 *)0x08180030) = 0x5a5a5a56; *((volatile u32 *)0x08180038) = 0x5a5a5a57; *((volatile u32 *)0x08180040) = 0x5a5a5a58;
The above accesses would imho not cross a cache line, unless a cache line on opteron is as small as 8 byte, which I find hard to believe.
Stefan
set_var_mtrr(0, 0x08000000, 0x10000000, MTRR_TYPE_WRBACK);
Hm.. doesn't the base address need at least an alignment of size?
Yes. Thanks.
Have you tried disabling caching completely for the area?
Not here, I did before. I was hoping that caching would avoid the alignment issues I've been seeing.
// Try to force the values out of the cache.
If you are trying to avoid the accesses from being cached, they need to cross a cache line.
I wasn't trying to cross a cache line. I was trying to write to the same cache line enough times that it would force it to write it back.
The above accesses would imho not cross a cache line, unless a cache line on opteron is as small as 8 byte, which I find hard to believe.
You're right.
Thanks, Myles
Myles,
Maybe leave the MTRR off until you get it working. I think that you can leave it in CAR mode until you have tested that the memory is working.
I assume you are very failiar with the FAM10 BKDG 2.6.4.1.1 DRAM and MMIO Memory Space.
Can you successfully access the memory as MMIO, F1x[BC:80] before trying F1x[1, 0][7C:40] DRAM address setup? I am not sure what will happen with a DRAM access that tries to go out a non-coherent link.
Marc
Myles,
Maybe leave the MTRR off until you get it working. I think that you can leave it in CAR mode until you have tested that the memory is working.
I'm trying that now.
I assume you are very failiar with the FAM10 BKDG 2.6.4.1.1 DRAM and MMIO Memory Space.
I've been reading it, but it seems like you can never be familiar enough with data sheets and other docs :)
Can you successfully access the memory as MMIO, F1x[BC:80] before trying F1x[1, 0][7C:40] DRAM address setup? I am not sure what will happen with a DRAM access that tries to go out a non-coherent link.
I was hoping that the DRAM address registers are all disabled before RAM init. I'll look again to make sure.
Thanks, Myles
-
Marc Jones -
Myles Watson -
Rudolf Marek -
Stefan Reinauer