OpenBIOS April 2012

openbios@openbios.org

16 participants
40 discussions

Re: [OpenBIOS] [PATCH 0/8] RFC: OFMEM cleanup/memory reduction patch
by Mark Cave-Ayland 14 Apr '12

14 Apr '12

On 10/04/12 18:41, Blue Swirl wrote: > Yes, but again, image size savings are not very interesting. Savings > in image loading time is easily lost during decompression. Well, the use case I'm considering at the moment is that if you build OpenBIOS SPARC32 with -O0 -g and then try to load it into QEMU, QEMU reports that the image is too large. So I'm guessing this is a limitation of the memory layout of the machine, rather than the space occupied by the binary. >> Secondly, if we're copying the data to an OFMEM-allocated area then why >> don't we just compress it at build time and then decompress it during the >> copy using DEFLATE (see my previous email)? Then we can further reduce the >> dictionary payload size from ~180K to around ~37K, although as you rightly >> point out there could be a small delay on startup - given the small size >> involved (and the fact we can lock the TLB entry like we do with the Forth >> machine), I don't think the penalty will be too bad. > > But only the decompressed result is what matters wrt memory usage, > isn't it? After Forth has started, there shouldn't be any difference > in RAM use, except that the decompressor code takes more space. Again for the use case above, this would not be a problem if we were to decompress and relocate the dictionary into RAM. >> I'm fairly confident I can come up with an experimental patch for this - >> would people mind if we added zlib as a build-time dependency, and puff.c to >> the OpenBIOS codebase? At the very least, if the decompression appears too >> expensive the first stage on its own would still be a good idea. > > For maximum compression, bzImage (still remember those?) style > approach could be used for whole image. I can't say I've ever looked at that. But if you're happy to at least consider a patch, then I'd like to invest some time looking at this. ATB, Mark.

2 1

Re: [OpenBIOS] Regression in booting HelenOS/sparc64
by Artyom Tarasenko 14 Apr '12

14 Apr '12

git checkout cb437e48 On 4/14/12, Artyom Tarasenko <atar4qemu(a)gmail.com> wrote: > Sorry for the top-posting, that's the only option Gmail offers for my > smartphone. > > I used the commit before the one to which there was a repairing commit > with comment "we lost OR'ing" from Blue and Jan. > > HTH, > Artyom > > > > > On 4/13/12, Mark Cave-Ayland <mark.cave-ayland(a)ilande.co.uk> wrote: >> On 12/04/12 18:54, Artyom Tarasenko wrote: >> >>> As of today qemu git/master and OpenBIOS/trunk can't boot >>> HelenOS-0.4.3-sparc64-ultra.iso . >>> >>> $ sparc64-softmmu/qemu-system-sparc64 -m 512 -boot d -cdrom >>> /mnt/terra/projects/sparc-images/HelenOS-0.4.3-sparc64-ultra.iso -bios >>> openbios-sparc64-trunk >>> Unassigned mem access to 000001ff00700000 from 000000000041c00c >>> ^^^Booting stops here >>> >>> It looks like the regression is coming form the OpenBIOS, because >>> older qemu versions can boot HelenOS with older OpenBIOS, but not with >>> trunk. Can't check other way around because current qemu doesn't start >>> with older OpenBIOS at all. >> >> Hmmm it's going to take a while to bisect something like this :/ Do you >> at least have a last known good combination (QEMU git commit/OpenBIOS >> SVN revision) to start working from? >> >> >> ATB, >> >> Mark. >> >> -- >> OpenBIOS http://openbios.org/ >> Mailinglist: http://lists.openbios.org/mailman/listinfo >> Free your System - May the Forth be with you >> > > > -- > Regards, > Artyom Tarasenko > > solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu > -- Regards, Artyom Tarasenko solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu

1 0

Re: [OpenBIOS] Regression in booting HelenOS/sparc64
by Mark Cave-Ayland 14 Apr '12

14 Apr '12

On 12/04/12 18:54, Artyom Tarasenko wrote: > As of today qemu git/master and OpenBIOS/trunk can't boot > HelenOS-0.4.3-sparc64-ultra.iso . > > $ sparc64-softmmu/qemu-system-sparc64 -m 512 -boot d -cdrom > /mnt/terra/projects/sparc-images/HelenOS-0.4.3-sparc64-ultra.iso -bios > openbios-sparc64-trunk > Unassigned mem access to 000001ff00700000 from 000000000041c00c > ^^^Booting stops here > > It looks like the regression is coming form the OpenBIOS, because > older qemu versions can boot HelenOS with older OpenBIOS, but not with > trunk. Can't check other way around because current qemu doesn't start > with older OpenBIOS at all. Hmmm it's going to take a while to bisect something like this :/ Do you at least have a last known good combination (QEMU git commit/OpenBIOS SVN revision) to start working from? ATB, Mark.

2 1

OT: Brooktree IC used in cg3 graphics framebuffer?
by Mark Cave-Ayland 13 Apr '12

13 Apr '12

Hi all, A slightly off-topic question, however there may be some people on the list who will know the answer to this: what was the number of the Brooktree IC used on the cg3 framebuffer? According to various sources, it looks like the BT458 was used on the cg6 - is this the same as the cg3? ATB, Mark.

5 8

Re: [OpenBIOS] [PATCH 1/8] Rename ofmem_arch_early_map_pages() to ofmem_arch_map_pages().
by Mark Cave-Ayland 13 Apr '12

13 Apr '12

On 10/04/12 18:45, Igor Kovalenko wrote: >> Well, it's more interesting than that. For SPARC, when you map the memory it >> is first added to the translation_t linked list. The TLB miss handlers >> simply iterate over this list in order to find the mapping before >> continuing. Therefore when creating a mapping, since the translation_t list >> already contains the entry created by ofmem_map_page_range(), all the >> ofmem_arch_map_pages() function has to do is update any existing TLB >> entries. >> > > The "early" variant was intended to perform one-time injection of > mapped pages that were acquired before c source code is entered so it > was not safe to call common mapping routine yet. I was afraid to > implement mapping code in assembly. I think I see this now. Okay - I think it would be possible to either merge this into the existing OFMEM code or refactor it to make use of the existing functions. It looks like I definitely need to produce a v2 of this patch, so I won't commit it so far and submit a revised version shortly. Many thanks, Mark.

1 0

Regression in booting HelenOS/sparc64
by Artyom Tarasenko 12 Apr '12

12 Apr '12

As of today qemu git/master and OpenBIOS/trunk can't boot HelenOS-0.4.3-sparc64-ultra.iso . $ sparc64-softmmu/qemu-system-sparc64 -m 512 -boot d -cdrom /mnt/terra/projects/sparc-images/HelenOS-0.4.3-sparc64-ultra.iso -bios openbios-sparc64-trunk Unassigned mem access to 000001ff00700000 from 000000000041c00c ^^^Booting stops here It looks like the regression is coming form the OpenBIOS, because older qemu versions can boot HelenOS with older OpenBIOS, but not with trunk. Can't check other way around because current qemu doesn't start with older OpenBIOS at all. -- Regards, Artyom Tarasenko solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu

1 0

Re: [OpenBIOS] [PATCH 0/8] RFC: OFMEM cleanup/memory reduction patch
by Tarl Neustaedter 10 Apr '12

10 Apr '12

On 2012-Apr-10 14:53 , Artyom Tarasenko wrote: > The problem is currently OpenBIOS won't work on a bare metal, since > it's tightly coupled with qemu. So, PROM size at this point is a > purely academical question. Gotcha. I thought there was some ability to run on bare metal, in which case prom limitations might have had meaning. > So, basically, if OpenBIOS has to support FCode "dropin" images, it > must support the compression type (s?) used in OBP? Only if you want to "borrow" dropins from an existing Openboot image. It's straight "/usr/bin/compress" used to compress them. I mentioned it only to let you know that we did have to resort to such methods to fit our proms :-)

1 0

Re: [OpenBIOS] [PATCH 0/8] RFC: OFMEM cleanup/memory reduction patch
by Tarl Neustaedter 10 Apr '12

10 Apr '12

On 2012-Apr-10 13:41 , Blue Swirl wrote: > Yes, but again, image size savings are not very interesting. Savings > in image loading time is easily lost during decompression. > Why do you think image size saving aren't interesting? Certainly in the past couple of decades of Openboot development, we've several times been severely squeezed by prom sizes - in some cases, we had to rip out functionality to be able to fit in our PROM. For what it's worth, Openboot currently compresses the "dropin" images (FCode standalone images for devices) before assembling them into the PROM image, and decompresses those when they are brought in with find-dropin.

3 2

[PATCH 0/8] RFC: OFMEM cleanup/memory reduction patch
by Mark Cave-Ayland 10 Apr '12

10 Apr '12

This patchset aims to tidy up various parts of OFMEM, with the ultimate aim of reducing the image sizes. The first 4 patches perform the reorganisation, while the remainder of the patches ensure that video memory and the Forth machine memory are no longer contained within the image. Note: the resulting SPARC64 image is now approximately half of its original size - BUT because we require 512K alignment for various parts of the image because of the 512K MMU page size, our ability to claim this space back directly is limited. Any further ideas on this would be welcomed. Mark Cave-Ayland (8): Rename ofmem_arch_early_map_pages() to ofmem_arch_map_pages(). SPARC32: Move ofmem_arch_map_pages() into ofmem_sparc32.c with all the other architecture-specific code. SPARC64: Refactor tte-data code in preparation for moving architecture-specific code to ofmem_sparc64.c. SPARC64: Move ofmem_arch_map_pages()/ofmem_arch_unmap_pages() into ofmem_sparc64.c with all the other architecture-specific code. Remove architecture-specific routines from ofmem.h. SPARC64: Remove video initialisation hack. SPARC64: Swap Forth machine memory allocation over to OFMEM memory pool. SPARC32: Swap Forth machine memory allocation over to OFMEM memory pool. openbios-devel/arch/ppc/ofmem.c | 2 +- openbios-devel/arch/ppc/qemu/ofmem.c | 6 +- openbios-devel/arch/sparc32/lib.c | 95 +--------- openbios-devel/arch/sparc32/ofmem_sparc32.c | 82 ++++++++- openbios-devel/arch/sparc32/openbios.c | 27 ++- openbios-devel/arch/sparc64/ldscript | 6 - openbios-devel/arch/sparc64/lib.c | 210 ++------------------ openbios-devel/arch/sparc64/ofmem_sparc64.c | 176 ++++++++++++++++- openbios-devel/arch/sparc64/ofmem_sparc64.h | 32 --- openbios-devel/arch/sparc64/openbios.c | 30 ++- openbios-devel/drivers/iommu.c | 4 +- openbios-devel/drivers/vga_vbe.c | 9 +- .../include/arch/sparc32/ofmem_sparc32.h | 28 +++ openbios-devel/include/arch/sparc64/io.h | 2 +- .../include/arch/sparc64/ofmem_sparc64.h | 50 +++++ openbios-devel/include/libopenbios/ofmem.h | 22 +-- openbios-devel/libopenbios/ofmem_common.c | 6 +- openbios-devel/packages/video.c | 4 +- 18 files changed, 405 insertions(+), 386 deletions(-) delete mode 100644 openbios-devel/arch/sparc64/ofmem_sparc64.h create mode 100644 openbios-devel/include/arch/sparc32/ofmem_sparc32.h create mode 100644 openbios-devel/include/arch/sparc64/ofmem_sparc64.h -- 1.7.2.5

4 12

Re: [OpenBIOS] [PATCH 0/8] RFC: OFMEM cleanup/memory reduction patch
by Mark Cave-Ayland 10 Apr '12

10 Apr '12

On 10/04/12 17:43, Blue Swirl wrote: >> Then we can dramatically reduce the dictionary usage within the file too. I >> wonder if we could incorporate a simple decompressor based upon something >> like pdlzip (http://lzip.nongnu.org/pdlzip.html) to perform LZMA >> decompression/relocation on the dictionary early within the startup process? > > ROM image size is not very interesting, but CPU cycles wasted at > startup are. I think run time memory needs would not be changed by > compression, the dictionary needs to be uncompressed to memory anyway > before it can be used. > > But we could avoid copying the dictionary when copying ROM to RAM > initially. Then the dictionary could be copied to OFMEM allocated area > before Forth start. Some large arrays (FSYS_BUF, dstack, rstack etc.) > could also be allocated before use. Would this help? Yes - perhaps I haven't explained this particularly clearly. I'm thinking of a 2 stage strategy - the first stage is to copy the dictionary to an OFMEM-allocated area on startup. This will save space because at the moment the entire dictionary (including space for all the definitions used by various bootloaders) is within the OpenBIOS image, which is currently 512K in the case of SPARC64. Hence we would only need ~180K within the image rather than 512K for this. Secondly, if we're copying the data to an OFMEM-allocated area then why don't we just compress it at build time and then decompress it during the copy using DEFLATE (see my previous email)? Then we can further reduce the dictionary payload size from ~180K to around ~37K, although as you rightly point out there could be a small delay on startup - given the small size involved (and the fact we can lock the TLB entry like we do with the Forth machine), I don't think the penalty will be too bad. I'm fairly confident I can come up with an experimental patch for this - would people mind if we added zlib as a build-time dependency, and puff.c to the OpenBIOS codebase? At the very least, if the decompression appears too expensive the first stage on its own would still be a good idea. ATB, Mark.

2 1