Hi,
I have now reached a state in my 440BX code where the RAM init _could_ work (maybe). The current code I have is mostly a very ugly hack and I won't post any patches until I cleaned up the mess. Also, I'm currently hard-coding some stuff from known-good 'lspci -xxx 0:0.0' values, so that's not really useful for anybody with a different configuration, anyway.
Here's what I'm able to get over serial right now:
[...] Testing DRAM : 10000000-02000000 DRAM fill: 10000000-02000000 10000000 DRAM filled DRAM verify: 10000000-02000000 10000000 DRAM range verified. Done. Copying LinuxBIOS to ram. Jumping to LinuxBIOS.
I'm basically using this code in auto.c (based on bitworks/ims stuff):
sdram_initialize(sizeof(cpu)/sizeof(cpu[0]), cpu); enable_mainboard_devices(); ram_check(0x10000000, 0x2000000);
Is that the canonical way to test whether RAM init works? The above serial output _looks_ fine, but nothing happens after the "Jumping to LinuxBIOS." anymore (but that may have other reasons).
I tried using different payloads (filo, memtest) but with no success. What else do I have to configure for a payload to be able to start? I added uses CONFIG_CONSOLE_VGA uses CONFIG_PCI_ROM_RUN default CONFIG_CONSOLE_VGA=1 default CONFIG_PCI_ROM_RUN=1 to Options.lb in the hope to get VGA output, but no luck. Also, I tried adding some stuff to Config.lb; still no luck. I guess this needs some more work later, but I don't want to fix all of that right now.
What's the absolute minimum config I need to get _something_ to boot, so that I can verify that RAM init works?
Can someone provide a known-good memtest payload with serial support I could use? Is that supposed to work if the ram_check() succeeds?
Uwe.
I'm basically using this code in auto.c (based on bitworks/ims stuff):
sdram_initialize(sizeof(cpu)/sizeof(cpu[0]), cpu); enable_mainboard_devices(); ram_check(0x10000000, 0x2000000);
Is that the canonical way to test whether RAM init works?
The "canonical" way is to attach a memory bus analyser to your board. Failing that, you can run some stress tests (like, boot Linux and do some disk+network load on it, or similar). If everything works, your memory works, eh? :-)
What's the absolute minimum config I need to get _something_ to boot, so that I can verify that RAM init works?
You can dump out the PCI config space of the memory controller after its init -- so you can check if that looks good (against the data sheet). After that, try a "Hello, World!" payload. Take little steps at a time :-)
Segher
On Sun, Nov 26, 2006 at 07:02:51PM +0100, Segher Boessenkool wrote:
Is that the canonical way to test whether RAM init works?
The "canonical" way is to attach a memory bus analyser to your board. Failing that, you can run some stress tests (like, boot Linux and do some disk+network load on it, or similar). If everything works, your memory works, eh? :-)
Sure, if I could boot Linux or memtest my problems would be solved. But currently I haven't yet managed to do so. The problem now is, that I don't know whether booting memtest fails because I need to setup more of the mainboard-specific stuff or because my RAM init doesn't work yet.
What's the absolute minimum config I need to get _something_ to boot, so that I can verify that RAM init works?
You can dump out the PCI config space of the memory controller after its init -- so you can check if that looks good (against the data sheet).
I think it does, I specifically hard-coded the values so that they match the output I got from a running Linux using the factory BIOS (and as far as I can see that's fine compared with what the datasheet says, too).
lspci -xxx 0:0.0 on Linux system: 00: 86 80 90 71 06 00 10 22 02 00 00 06 00 40 00 00 10: 08 00 00 d8 00 00 00 00 00 00 00 00 00 00 00 00 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 30: 00 00 00 00 a0 00 00 00 00 00 00 00 00 00 00 00 40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 50: 0c 8a 00 ff 00 00 00 09 00 10 11 01 00 00 00 00 60: 00 00 00 00 00 00 08 08 00 00 00 00 00 2a 00 00 70: 20 1f 0a 38 00 10 10 01 23 ff 10 38 00 00 00 00 80: 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 90: 98 cc 00 00 04 61 00 00 00 05 00 00 00 00 00 00 a0: 02 00 10 00 03 02 00 1f 00 00 00 00 00 00 00 00 b0: 80 20 00 00 30 00 00 00 00 00 9f 03 20 10 00 00 c0: 00 00 00 00 ff ff ff ff 18 0c ff ff 61 00 00 00 d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 e0: 4c ad ff bb 8a 3e 00 80 2c d3 f7 cf 9d 3e 00 00 f0: 40 01 00 00 00 f8 00 60 20 0f 00 00 00 00 00 00
Northbridge dump from LB via serial: 00: 86 80 90 71 06 00 10 22 02 00 00 06 00 40 00 00 10: 08 00 00 d0 00 00 00 00 00 00 00 00 00 00 00 00 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 30: 00 00 00 00 a0 00 00 00 00 00 00 00 00 00 00 00 40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 50: 0c 8a 00 ff 00 00 00 09 00 10 11 01 00 00 00 00 60: 00 00 00 00 00 00 08 08 00 00 00 00 00 2a 00 00 70: 20 1f 0a 38 00 10 10 01 23 ff 10 38 00 00 00 00 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 90: 80 00 00 00 04 61 00 00 00 05 00 00 00 00 00 00 a0: 02 00 10 00 03 02 00 1f 00 00 00 00 00 00 00 00 b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 c0: 00 00 00 00 ff ff ff ff 18 0c 00 00 00 00 00 00 d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f0: 00 00 00 00 00 f8 00 00 20 0f 00 00 00 00 00 00
There _are_ differences, but nothing related to the RAM config registers as far as I can see.
Uwe.
Is that the canonical way to test whether RAM init works?
The "canonical" way is to attach a memory bus analyser to your board. Failing that, you can run some stress tests (like, boot Linux and do some disk+network load on it, or similar). If everything works, your memory works, eh? :-)
Sure, if I could boot Linux or memtest my problems would be solved. But currently I haven't yet managed to do so. The problem now is, that I don't know whether booting memtest fails because I need to setup more of the mainboard-specific stuff or because my RAM init doesn't work yet.
That's why you should try doing one (tiny) step at a time, and add lots of debugging/tracing statements. Figure out exactly what works and what doesn't :-)
There _are_ differences, but nothing related to the RAM config registers as far as I can see.
I'll have a look at it too, second pair of eyes...
Segher
There _are_ differences, but nothing related to the RAM config registers as far as I can see.
I'll have a look at it too, second pair of eyes...
Okay... So, is this your DIMM?
100MHz SDRAM, 64MB, in last (4th) slot. 8-bit devices. 9 row bits, 14 column bits, 4 banks. CL=3, RCD=3, RP=3 (all in clock cycles).
You did forget a few registers though -- the three bytes at offset 0xca, and the 16 bytes at offset 0xe0. Be careful to set the enable bit (bit 7 in 0xe7) last.
Also, you probably didn't program the MRS on the memory. You should do that with the register at 0x76. The correct sequence is:
- precharge all banks, wait tRP - refresh, wait tRC (do this step 8 times) - write to MRS, wait 2 memory cycles
(this system is slow enough that all those "waits" can be NOPs, fwiw).
The bits in MRS you need are: 6..4: CAS latency (CL), 3 in your case (so 011) [the datasheet is wrong here]; 3: burst type, only interleave (1) is supported on this chip; 2..0: burst length, 010=4 is the only thing supported on i440BX; So, that makes 0111010 i.e. 0x3a.
...and then you can use the memory.
To figure out what host addresses you should use to get onto the DRAM address pins A13..A0, look at the tables in chapter 4.3. I think shifting to the left by 15 bits would work in your case, so accessing address 0x1d0000. This should be done per DIMM rank, but you only have one anyway :-)
Well there you go, experiment a bit and you'll get it right. You're lucky this memory controller is so ancient (and very simple even then).
Have fun,
Segher
On Mon, Nov 27, 2006 at 09:05:12AM +0100, Segher Boessenkool wrote:
Okay... So, is this your DIMM?
100MHz SDRAM, 64MB, in last (4th) slot.
Yep. There are only 3 slots on the board, but the SPD dump says it's the last one (number 3, i.e., the 4th), too.
You did forget a few registers though -- the three bytes at offset 0xca, and the 16 bytes at offset 0xe0.
Yeah, I previously ignored those as the code from v1 didn't set them AFAIK. I set them now, though. http://tracker.linuxbios.org/trac/LinuxBIOS/browser/trunk/LinuxBIOSv1/src/no...
Be careful to set the enable bit (bit 7 in 0xe7) last.
Yep. As the last step in sdram_set_registers() or as the last step in RAM init in general?
Also, you probably didn't program the MRS on the memory.
Yes, I'm quite sure I messed that up. In general, in do_ram_command() I have to set DRAMC bits 7-5 to the respective command and then read32() from some memory location for the change to "take effect", right? If so, which memory location is that? Does it depend on some input data?
I still don't really understand this part of the RAM init, so the code might be very, very wrong...
You should do that with the register at 0x76. The correct sequence is:
- precharge all banks, wait tRP
- refresh, wait tRC (do this step 8 times)
- write to MRS, wait 2 memory cycles
I think I got this part right mostly.
To figure out what host addresses you should use to get onto the DRAM address pins A13..A0, look at the tables in chapter 4.3. I think shifting to the left by 15 bits would work in your case, so accessing address 0x1d0000. This should be done per DIMM rank, but you only have one anyway :-)
Hm, so just read32(0x1d0000) and that's it? I tried that but it failed. (but there are a ton of other errors I might still have in the code)
Attached is my stripped-down raminit.c which does the bare minimum hardcoding of the values and nothing more. It doesn't work, and after quite a lot of testing I can't seem to find the problem(s).
Shifting by 15 causes a hang in ram_check() so I tried 23 (looking at the v1 code). Again, I don't know what I'm doing here ;-)
Btw, I'm now running ram_check(0x1000, 0x4000000); which should check all of the 64MB of RAM, right?
Thanks, Uwe.
Be careful to set the enable bit (bit 7 in 0xe7) last.
Yep. As the last step in sdram_set_registers() or as the last step in RAM init in general?
The last step of setting the registers e0..ef.
Also, you probably didn't program the MRS on the memory.
Yes, I'm quite sure I messed that up.
Heh, nothing will work then ;-)
In general, in do_ram_command() I have to set DRAMC bits 7-5 to the respective command and then read32 () from some memory location for the change to "take effect", right?
Write to that location, instead. It's not that you make "a change take effect" -- it directly sends a command on the memory bus, instead.
If so, which memory location is that? Does it depend on some input data?
0x1d0 in the rank, typically. There is some tiny documentation in the intel sheet; the idea is to send the correct pattern on the DRAM address pins (use the selected map to figure out what pattern that is).
I still don't really understand this part of the RAM init, so the code might be very, very wrong...
SDRAM requires an initialisation sequence to work properly. One of the steps of that init sequence is programming the MRS register, so the DRAM knows what CAS latency (CL) is in use, what burst length (BL), etc.
You should do that with the register at 0x76. The correct sequence is:
- precharge all banks, wait tRP
- refresh, wait tRC (do this step 8 times)
- write to MRS, wait 2 memory cycles
I think I got this part right mostly.
How about you show the code, much easier to diagnose that way :-)
Attached is my stripped-down raminit.c which does the bare minimum hardcoding of the values and nothing more.
:-)
I'll have a look at it later.
It doesn't work, and after quite a lot of testing I can't seem to find the problem(s).
Shifting by 15 causes a hang in ram_check() so I tried 23 (looking at the v1 code). Again, I don't know what I'm doing here ;-)
I'm not too sure either, but I'll see what I can find.
Btw, I'm now running ram_check(0x1000, 0x4000000); which should check all of the 64MB of RAM, right?
Well except for the low 4kB, heh ;-)
Segher
Did you check the digitallogic board in V1 that uses 440bx? It used to work ...
ron
Is there a equivalent of a snapshot repository for v1 (like http://snapshots.linuxbios.org is for v2) ?
_________________________________________________________________ Geef jouw Hotmail kleur met Windows Live Mail! Stap nu over! http://imagine-windowslive.com/mail/launch/default.aspx?Locale=nl-nl
* Idwer Vollering idwer_v@hotmail.com [061201 18:48]:
Is there a equivalent of a snapshot repository for v1 (like http://snapshots.linuxbios.org is for v2) ?
Not really. Since v1 is not actively developed anymore, it is just sitting in the svn repository.
You can download the latest snapshot via viewvc though:
http://www.linuxbios.org/viewvc/trunk/LinuxBIOSv1.tar.gz?view=tar
Regards, Stefan
On Fri, Dec 01, 2006 at 09:14:39AM -0700, ron minnich wrote:
Did you check the digitallogic board in V1 that uses 440bx? It used to work ...
I looked at the v1 440bx raminit code, as well as a bunch of v2 chipsets for pointers, but I can't seem to find what exactly the problem is.
Uwe.
Also, you probably didn't program the MRS on the memory.
Yes, I'm quite sure I messed that up. In general, in do_ram_command () I have to set DRAMC bits 7-5 to the respective command and then read32 () from some memory location for the change to "take effect", right? If so, which memory location is that? Does it depend on some input data?
I still don't really understand this part of the RAM init, so the code might be very, very wrong...
It looks reasonable. You want to shift by 3 though, not 23, not 15, so you read from 0x1d0 for writing to the MRS.
It seems the i440 might want the address bits inverted on the high banks in some configurations, btw; so if it won't work, you can try 0x1d0 ^ 0xff8 or so.
From the comments it seems you used the DDR-II spec, not the SDRAM spec; but the code looks okay.
Segher
On Sat, Dec 02, 2006 at 09:30:58AM +0100, Segher Boessenkool wrote:
It looks reasonable. You want to shift by 3 though, not 23, not 15, so you read from 0x1d0 for writing to the MRS.
OK, a few questions:
* Do I read32() from somewhere for _every_ RAM command or only for MRS?
* The v1 code seems to read from the highest RAM address for each DRB register. In my case: Get contents of DRB6 (0x8), shift left by 23 as the DRB registers store multiples of 8 MB, so I get my 64 MB. Correct?
Now; do I read32() from * (8 << 23) * (8 << 23) + 0x1d0 * 0x0 + 0x1d0 * 0x0 + 0x1d0 AND (8 << 23) + 0x1d0 * 0x0 * ???
Do I read from x for most commands but from (x + 0x1d0) for MRS? Or should I read from (x + 0x1d0) for all commands?
I've tried a lot of variations here, but nothing worked. Maybe some other parts of the code are still broken?
It seems the i440 might want the address bits inverted on the high banks in some configurations, btw; so if it won't work, you can try 0x1d0 ^ 0xff8 or so.
Doesn't seem to work either.
Attached my latest code and a minicom log...
Uwe.
It looks reasonable. You want to shift by 3 though, not 23, not 15, so you read from 0x1d0 for writing to the MRS.
OK, a few questions:
- Do I read32() from somewhere for _every_ RAM command or only for
MRS?
For every command. For commands other than "load MRS", you use address 0.
- The v1 code seems to read from the highest RAM address for each DRB register.
You need to do this init sequence for every rank of memory. Using each DRB gives you a lot of duplicates, which can be harmless; it can also give you 0 as the top-of-rank address (if the first ranks are empty) which is bad.
In my case: Get contents of DRB6 (0x8), shift left by 23 as the DRB registers store multiples of 8 MB, so I get my 64 MB. Correct?
Yes. So you need to address somewhere between the previous rank top, and this 64MB. Using the _previous_ rank top as-is might be best. Example:
rank 0 empty DRB=0 don't init rank 1 0..64MB-1 DRB=8 init @ 0 (+0x1d0) rank 2 64MB..128MB-1 DRB=16 init @ 64MB (+0x1d0) rank 3 empty DRB=16 don't init
Now; do I read32() from * (8 << 23) * (8 << 23) + 0x1d0 * 0x0 + 0x1d0 * 0x0 + 0x1d0 AND (8 << 23) + 0x1d0 * 0x0 * ???
You init exactly the memory that's there; nothing more, nothing less.
A useful trick is to temporarily set the DRB registers such that the one rank under consideration is mapped to address 0, all the rest disabled; only later set the full "real" DRB map. This might or might not make the code simpler, your choice.
Do I read from x for most commands but from (x + 0x1d0) for MRS? Or should I read from (x + 0x1d0) for all commands?
x for most commands. The 0x1d0 actually is the data sent to the MRS reg on the memory; it is sent over the address lines.
I've tried a lot of variations here, but nothing worked. Maybe some other parts of the code are still broken?
Could be, hard to tell.
Attached my latest code and a minicom log...
I'll see if I see anything.
Segher
On 12/2/06, Segher Boessenkool segher@kernel.crashing.org wrote:
- The v1 code seems to read from the highest RAM address for each DRB register.
You need to do this init sequence for every rank of memory. Using each DRB gives you a lot of duplicates, which can be harmless; it can also give you 0 as the top-of-rank address (if the first ranks are empty) which is bad.
no, not really duplicates, I think. You set up the DRBs for max size, you do all the DRAM setup, you size the actual RAM and reprogram the DRBs. This will work even if there is air in the socket. It ensures that a single DRB does not hit the same DRAM bank.
ron
You need to do this init sequence for every rank of memory. Using each DRB gives you a lot of duplicates, which can be harmless; it can also give you 0 as the top-of-rank address (if the first ranks are empty) which is bad.
no, not really duplicates, I think. You set up the DRBs for max size, you do all the DRAM setup, you size the actual RAM and reprogram the DRBs. This will work even if there is air in the socket. It ensures that a single DRB does not hit the same DRAM bank.
Yes, if you set up the DRBs so that no DRAM ranks are overlapping, you can program them all separately. This is the kind of thing people do if they cannot read the SPDs or just don't want to for some reason (many fuctory BIOSes do).
At some later stage you'll have to program the DRBs properly; and you cannot change the config while the DRAM controller is active (even the i440 "specification update" warns against this) -- the kind of CPUs attached to this chip won't do any prefetch or similar, so you actually *can* make this work, but in principle it's a bad idea to change settings on a running memory controller.
If you use the SPDs properly (or at all!), you really don't have to fall back to tricks like this.
Segher
On 12/3/06, Segher Boessenkool segher@kernel.crashing.org wrote:
Yes, if you set up the DRBs so that no DRAM ranks are overlapping, you can program them all separately. This is the kind of thing people do if they cannot read the SPDs or just don't want to for some reason (many fuctory BIOSes do).
yes, this was very common on lots of V1 boards, even in SPD cases.
The reason was that some dram setup could be done in assembly, but doing DRB in C was simpler.
At some later stage you'll have to program the DRBs properly; and
ah well some things you can. DRB is one of them.
Of course, given an OS that can handle non-contiguous ram, there would never be a need to anything BUT size the DRB to max size. But that was not the case in 1999.
If you use the SPDs properly (or at all!), you really don't have to fall back to tricks like this.
certain tricks are ok if they simplify assembly. This was one of them in some cases IIRC.
ron
Of course, given an OS that can handle non-contiguous ram, there would never be a need to anything BUT size the DRB to max size. But that was not the case in 1999.
There's no _need_ sure, but the result is pretty nasty: say, a DRB is set up for 128MB, but the DIMM it covers is only 64MB; then any access to the "high half" of the DRB aliases to the lower addresses. Not a problem /an sich/, but it makes certain problems hard to debug.
If you use the SPDs properly (or at all!), you really don't have to fall back to tricks like this.
certain tricks are ok if they simplify assembly. This was one of them in some cases IIRC.
Yeah, I know what you're saying. But if you don't use assembler code, please let's not do too many tricks (you can take this argument and apply it to my code too, promise :-) )
Segher
On 12/3/06, Segher Boessenkool segher@kernel.crashing.org wrote:
Of course, given an OS that can handle non-contiguous ram, there would never be a need to anything BUT size the DRB to max size. But that was not the case in 1999.
There's no _need_ sure, but the result is pretty nasty: say, a DRB is set up for 128MB, but the DIMM it covers is only 64MB; then any access to the "high half" of the DRB aliases to the lower addresses. Not a problem /an sich/, but it makes certain problems hard to debug.
no, because the e820 tables or whatever would set up a set of regions of memory. OS would never access the memory that does not exist.
ron
* Uwe Hermann uwe@hermann-uwe.de [061126 22:03]:
On Sun, Nov 26, 2006 at 07:02:51PM +0100, Segher Boessenkool wrote:
Is that the canonical way to test whether RAM init works?
I suggest you compare the copied image with the original. CONFIG_COMPRESS=0 should make this easier..
src/cpu/x86/car/copy_and_run.c or src/arch/i386/init/crt0.S.lb depending on whether you use CAR or not.
If you use CAR, you need to make sure it is disabled before you do the jump.
Sure, if I could boot Linux or memtest my problems would be solved. But currently I haven't yet managed to do so. The problem now is, that I don't know whether booting memtest fails because I need to setup more of the mainboard-specific stuff or because my RAM init doesn't work yet.
Your problem is a lot earlier. You are not even executing linuxbios_ram yet. This is long before it even looks at the payload.
There _are_ differences, but nothing related to the RAM config registers as far as I can see.
Its probably safe to try poking those values hardcoded into the north bridge.
Hint: Try testing the lower memory:
ram_check(0x10000000, 0x2000000); is going for 256-512 Megs.
The LinuxBIOS image is copied to _RAMBASE which normally defaults to 0x4000 (@16k). So you might have 2 dimms a 256M and while the upper one works, the lower one doesnt.
In the end the ram_check() function does not reliably test your memory. It will not detect whether your memory works, but it will detect whether it fails ;-)
Also note: the order in which your registers are set may play a role. For example there might be a register with a DRAM_ENABLE bit which must be set last, after all other DRAM related bits are set, or the values set later on might be ignored.
Grouping the register value hardcode pokes might be a good idea, to find out which registers have to be set in which context...
Stefan
Hint: Try testing the lower memory:
ram_check(0x10000000, 0x2000000); is going for 256-512 Megs.
Ah, there's the (or a) problem. There's a zero missing. No memory was tested at all.
The LinuxBIOS image is copied to _RAMBASE which normally defaults to 0x4000 (@16k). So you might have 2 dimms a 256M and while the upper one works, the lower one doesnt.
Yeah.
In the end the ram_check() function does not reliably test your memory. It will not detect whether your memory works, but it will detect whether it fails ;-)
And not always, either.
Also note: the order in which your registers are set may play a role. For example there might be a register with a DRAM_ENABLE bit which must be set last, after all other DRAM related bits are set, or the values set later on might be ignored.
Yes. And you often have to wait for certain bits to flip before you write other bits, etc.
Segher
-
Idwer Vollering -
ron minnich -
Segher Boessenkool -
Stefan Reinauer -
Uwe Hermann