On 10/04/12 18:41, Blue Swirl wrote:
> Yes, but again, image size savings are not very interesting. Savings
> in image loading time is easily lost during decompression.
Well, the use case I'm considering at the moment is that if you build
OpenBIOS SPARC32 with -O0 -g and then try to load it into QEMU, QEMU
reports that the image is too large. So I'm guessing this is a
limitation of the memory layout of the machine, rather than the space
occupied by the binary.
>> Secondly, if we're copying the data to an OFMEM-allocated area then why
>> don't we just compress it at build time and then decompress it during the
>> copy using DEFLATE (see my previous email)? Then we can further reduce the
>> dictionary payload size from ~180K to around ~37K, although as you rightly
>> point out there could be a small delay on startup - given the small size
>> involved (and the fact we can lock the TLB entry like we do with the Forth
>> machine), I don't think the penalty will be too bad.
>
> But only the decompressed result is what matters wrt memory usage,
> isn't it? After Forth has started, there shouldn't be any difference
> in RAM use, except that the decompressor code takes more space.
Again for the use case above, this would not be a problem if we were to
decompress and relocate the dictionary into RAM.
>> I'm fairly confident I can come up with an experimental patch for this -
>> would people mind if we added zlib as a build-time dependency, and puff.c to
>> the OpenBIOS codebase? At the very least, if the decompression appears too
>> expensive the first stage on its own would still be a good idea.
>
> For maximum compression, bzImage (still remember those?) style
> approach could be used for whole image.
I can't say I've ever looked at that. But if you're happy to at least
consider a patch, then I'd like to invest some time looking at this.
ATB,
Mark.
git checkout cb437e48
On 4/14/12, Artyom Tarasenko <atar4qemu(a)gmail.com> wrote:
> Sorry for the top-posting, that's the only option Gmail offers for my
> smartphone.
>
> I used the commit before the one to which there was a repairing commit
> with comment "we lost OR'ing" from Blue and Jan.
>
> HTH,
> Artyom
>
>
>
>
> On 4/13/12, Mark Cave-Ayland <mark.cave-ayland(a)ilande.co.uk> wrote:
>> On 12/04/12 18:54, Artyom Tarasenko wrote:
>>
>>> As of today qemu git/master and OpenBIOS/trunk can't boot
>>> HelenOS-0.4.3-sparc64-ultra.iso .
>>>
>>> $ sparc64-softmmu/qemu-system-sparc64 -m 512 -boot d -cdrom
>>> /mnt/terra/projects/sparc-images/HelenOS-0.4.3-sparc64-ultra.iso -bios
>>> openbios-sparc64-trunk
>>> Unassigned mem access to 000001ff00700000 from 000000000041c00c
>>> ^^^Booting stops here
>>>
>>> It looks like the regression is coming form the OpenBIOS, because
>>> older qemu versions can boot HelenOS with older OpenBIOS, but not with
>>> trunk. Can't check other way around because current qemu doesn't start
>>> with older OpenBIOS at all.
>>
>> Hmmm it's going to take a while to bisect something like this :/ Do you
>> at least have a last known good combination (QEMU git commit/OpenBIOS
>> SVN revision) to start working from?
>>
>>
>> ATB,
>>
>> Mark.
>>
>> --
>> OpenBIOS http://openbios.org/
>> Mailinglist: http://lists.openbios.org/mailman/listinfo
>> Free your System - May the Forth be with you
>>
>
>
> --
> Regards,
> Artyom Tarasenko
>
> solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu
>
--
Regards,
Artyom Tarasenko
solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu
On 12/04/12 18:54, Artyom Tarasenko wrote:
> As of today qemu git/master and OpenBIOS/trunk can't boot
> HelenOS-0.4.3-sparc64-ultra.iso .
>
> $ sparc64-softmmu/qemu-system-sparc64 -m 512 -boot d -cdrom
> /mnt/terra/projects/sparc-images/HelenOS-0.4.3-sparc64-ultra.iso -bios
> openbios-sparc64-trunk
> Unassigned mem access to 000001ff00700000 from 000000000041c00c
> ^^^Booting stops here
>
> It looks like the regression is coming form the OpenBIOS, because
> older qemu versions can boot HelenOS with older OpenBIOS, but not with
> trunk. Can't check other way around because current qemu doesn't start
> with older OpenBIOS at all.
Hmmm it's going to take a while to bisect something like this :/ Do you
at least have a last known good combination (QEMU git commit/OpenBIOS
SVN revision) to start working from?
ATB,
Mark.
Hi all,
A slightly off-topic question, however there may be some people on the
list who will know the answer to this: what was the number of the
Brooktree IC used on the cg3 framebuffer? According to various sources,
it looks like the BT458 was used on the cg6 - is this the same as the cg3?
ATB,
Mark.
On 10/04/12 18:45, Igor Kovalenko wrote:
>> Well, it's more interesting than that. For SPARC, when you map the memory it
>> is first added to the translation_t linked list. The TLB miss handlers
>> simply iterate over this list in order to find the mapping before
>> continuing. Therefore when creating a mapping, since the translation_t list
>> already contains the entry created by ofmem_map_page_range(), all the
>> ofmem_arch_map_pages() function has to do is update any existing TLB
>> entries.
>>
>
> The "early" variant was intended to perform one-time injection of
> mapped pages that were acquired before c source code is entered so it
> was not safe to call common mapping routine yet. I was afraid to
> implement mapping code in assembly.
I think I see this now. Okay - I think it would be possible to either
merge this into the existing OFMEM code or refactor it to make use of
the existing functions.
It looks like I definitely need to produce a v2 of this patch, so I
won't commit it so far and submit a revised version shortly.
Many thanks,
Mark.
As of today qemu git/master and OpenBIOS/trunk can't boot
HelenOS-0.4.3-sparc64-ultra.iso .
$ sparc64-softmmu/qemu-system-sparc64 -m 512 -boot d -cdrom
/mnt/terra/projects/sparc-images/HelenOS-0.4.3-sparc64-ultra.iso -bios
openbios-sparc64-trunk
Unassigned mem access to 000001ff00700000 from 000000000041c00c
^^^Booting stops here
It looks like the regression is coming form the OpenBIOS, because
older qemu versions can boot HelenOS with older OpenBIOS, but not with
trunk. Can't check other way around because current qemu doesn't start
with older OpenBIOS at all.
--
Regards,
Artyom Tarasenko
solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu
On 2012-Apr-10 14:53 , Artyom Tarasenko wrote:
> The problem is currently OpenBIOS won't work on a bare metal, since
> it's tightly coupled with qemu. So, PROM size at this point is a
> purely academical question.
Gotcha. I thought there was some ability to run on bare metal, in which
case prom limitations might have had meaning.
> So, basically, if OpenBIOS has to support FCode "dropin" images, it
> must support the compression type (s?) used in OBP?
Only if you want to "borrow" dropins from an existing Openboot image.
It's straight "/usr/bin/compress" used to compress them. I mentioned it
only to let you know that we did have to resort to such methods to fit
our proms :-)
On 2012-Apr-10 13:41 , Blue Swirl wrote:
> Yes, but again, image size savings are not very interesting. Savings
> in image loading time is easily lost during decompression.
>
Why do you think image size saving aren't interesting? Certainly in the
past couple of decades of Openboot development, we've several times been
severely squeezed by prom sizes - in some cases, we had to rip out
functionality to be able to fit in our PROM.
For what it's worth, Openboot currently compresses the "dropin" images
(FCode standalone images for devices) before assembling them into the
PROM image, and decompresses those when they are brought in with
find-dropin.
This patchset aims to tidy up various parts of OFMEM, with the ultimate aim
of reducing the image sizes. The first 4 patches perform the reorganisation,
while the remainder of the patches ensure that video memory and the Forth
machine memory are no longer contained within the image.
Note: the resulting SPARC64 image is now approximately half of its original
size - BUT because we require 512K alignment for various parts of the image
because of the 512K MMU page size, our ability to claim this space back
directly is limited. Any further ideas on this would be welcomed.
Mark Cave-Ayland (8):
Rename ofmem_arch_early_map_pages() to ofmem_arch_map_pages().
SPARC32: Move ofmem_arch_map_pages() into ofmem_sparc32.c with all
the other architecture-specific code.
SPARC64: Refactor tte-data code in preparation for moving
architecture-specific code to ofmem_sparc64.c.
SPARC64: Move ofmem_arch_map_pages()/ofmem_arch_unmap_pages() into
ofmem_sparc64.c with all the other architecture-specific code.
Remove architecture-specific routines from ofmem.h.
SPARC64: Remove video initialisation hack.
SPARC64: Swap Forth machine memory allocation over to OFMEM memory
pool.
SPARC32: Swap Forth machine memory allocation over to OFMEM memory
pool.
openbios-devel/arch/ppc/ofmem.c | 2 +-
openbios-devel/arch/ppc/qemu/ofmem.c | 6 +-
openbios-devel/arch/sparc32/lib.c | 95 +---------
openbios-devel/arch/sparc32/ofmem_sparc32.c | 82 ++++++++-
openbios-devel/arch/sparc32/openbios.c | 27 ++-
openbios-devel/arch/sparc64/ldscript | 6 -
openbios-devel/arch/sparc64/lib.c | 210 ++------------------
openbios-devel/arch/sparc64/ofmem_sparc64.c | 176 ++++++++++++++++-
openbios-devel/arch/sparc64/ofmem_sparc64.h | 32 ---
openbios-devel/arch/sparc64/openbios.c | 30 ++-
openbios-devel/drivers/iommu.c | 4 +-
openbios-devel/drivers/vga_vbe.c | 9 +-
.../include/arch/sparc32/ofmem_sparc32.h | 28 +++
openbios-devel/include/arch/sparc64/io.h | 2 +-
.../include/arch/sparc64/ofmem_sparc64.h | 50 +++++
openbios-devel/include/libopenbios/ofmem.h | 22 +--
openbios-devel/libopenbios/ofmem_common.c | 6 +-
openbios-devel/packages/video.c | 4 +-
18 files changed, 405 insertions(+), 386 deletions(-)
delete mode 100644 openbios-devel/arch/sparc64/ofmem_sparc64.h
create mode 100644 openbios-devel/include/arch/sparc32/ofmem_sparc32.h
create mode 100644 openbios-devel/include/arch/sparc64/ofmem_sparc64.h
--
1.7.2.5
On 10/04/12 17:43, Blue Swirl wrote:
>> Then we can dramatically reduce the dictionary usage within the file too. I
>> wonder if we could incorporate a simple decompressor based upon something
>> like pdlzip (http://lzip.nongnu.org/pdlzip.html) to perform LZMA
>> decompression/relocation on the dictionary early within the startup process?
>
> ROM image size is not very interesting, but CPU cycles wasted at
> startup are. I think run time memory needs would not be changed by
> compression, the dictionary needs to be uncompressed to memory anyway
> before it can be used.
>
> But we could avoid copying the dictionary when copying ROM to RAM
> initially. Then the dictionary could be copied to OFMEM allocated area
> before Forth start. Some large arrays (FSYS_BUF, dstack, rstack etc.)
> could also be allocated before use. Would this help?
Yes - perhaps I haven't explained this particularly clearly. I'm
thinking of a 2 stage strategy - the first stage is to copy the
dictionary to an OFMEM-allocated area on startup. This will save space
because at the moment the entire dictionary (including space for all the
definitions used by various bootloaders) is within the OpenBIOS image,
which is currently 512K in the case of SPARC64. Hence we would only need
~180K within the image rather than 512K for this.
Secondly, if we're copying the data to an OFMEM-allocated area then why
don't we just compress it at build time and then decompress it during
the copy using DEFLATE (see my previous email)? Then we can further
reduce the dictionary payload size from ~180K to around ~37K, although
as you rightly point out there could be a small delay on startup - given
the small size involved (and the fact we can lock the TLB entry like we
do with the Forth machine), I don't think the penalty will be too bad.
I'm fairly confident I can come up with an experimental patch for this -
would people mind if we added zlib as a build-time dependency, and
puff.c to the OpenBIOS codebase? At the very least, if the decompression
appears too expensive the first stage on its own would still be a good idea.
ATB,
Mark.