mingw build problem with seabios/tools/kconfig

[Scott Duplichan]

Hello,

I am trying to overcome problems building seabios/tools/kconfig from
the windows/mingw environment. Here are the failures:

1) Link step for tools/kconfig/conf fails to find regcomp/regexec/regfree.
   In the mingw environment these functions are available in libregex, 
   but libregex is not included in the standard library search. I have
   not found a way to add '-lregex' to the link command line:
   $(Q)$(HOSTCC) $(HOSTLDFLAGS) -o $@ $(addprefix $(obj)/,$($(@F)-objs)) $(HOST_LOADLIBES) $(HOSTLOADLIBES_$(@F))
   HOSTLDFLAGS occurs too early in the file list for this use.
    	
2) Link step for tools/kconfig/conf fails to find function uname().
   I can come up with a library to provide this function but need
   help passing it to the linker, same as with item 1.

3) Function mkdir() takes only one argument in the mingw environment,
   yet takes 2 arguments otherwise. I have not found a good way to 
   overcome this difference.

Thanks,
Scott

[Peter Stuge]

Scott Duplichan wrote:
> I have not found a way to add '-lregex' to the link command line:
> $(Q)$(HOSTCC) $(HOSTLDFLAGS) -o $@ $(addprefix $(obj)/,$($(@F)-objs)) $(HOST_LOADLIBES) $(HOSTLOADLIBES_$(@F))
> HOSTLDFLAGS occurs too early in the file list for this use.

I'd say the above command line is incorrect. HOSTLDFLAGS should come
right before HOST_LOADLIBES.
> Function mkdir() takes only one argument in the mingw environment,
> yet takes 2 arguments otherwise. I have not found a good way to 
> overcome this difference.

#ifdef WIN32
    mkdir(path);
#else
    mkdir(path, mask);
#endif

//Peter

[Scott Duplichan]

Peter Stuge wrote:

]Scott Duplichan wrote:
]> I have not found a way to add '-lregex' to the link command line:
]> $(Q)$(HOSTCC) $(HOSTLDFLAGS) -o $@ $(addprefix $(obj)/,$($(@F)-objs)) $(HOST_LOADLIBES) ]$(HOSTLOADLIBES_$(@F))
]> HOSTLDFLAGS occurs too early in the file list for this use.
]
]I'd say the above command line is incorrect. HOSTLDFLAGS should come
]right before HOST_LOADLIBES.

I suppose so, at least for the use of adding libraries.

]> 3) Function mkdir() takes only one argument in the mingw environment,
]>    yet takes 2 arguments otherwise. I have not found a good way to 
]>    overcome this difference.
]
]#ifdef WIN32
]	mkdir(path);
]#else
]	mkdir(path, mask);
]#endif

This method works. I will repost this message on the seabios list
if no mingw change/fix can be found to get this build working.

]
]//Peter

Thanks,
Scott

[Patrick Georgi]

Am 13.03.2011 04:47, schrieb Scott Duplichan:
> Hello,
>
> I am trying to overcome problems buildingseabios/tools/kconfigfrom
>
> the windows/mingw environment.Hereare the failures:
>
> 1)Link step fortools/kconfig/conffails to findregcomp/regexec/regfree.
>
> In the mingw environment these functions are available in libregex,
>
> but libregexis notincluded inthe standard library search. I have
>
> not found away to add'-lregex'to the link command line:

The main issue is that libregex isn't available reliably on mingw.
On coreboot, I simply added a libregex implementation to the tree and
added it to the build.

In general, maybe just copy over coreboot's kconfig and look for the
changes (there are some you'd have to revert)?

Patrick

[Scott Duplichan]

Patrick Georgi wrote:

]Am 13.03.2011 04:47, schrieb Scott Duplichan:
]> Hello,
]> 
]> I am trying to overcome problems buildingseabios/tools/kconfigfrom
]> the windows/mingw environment.Hereare the failures:
]> 1)Link step fortools/kconfig/conffails to findregcomp/regexec/regfree.
]>   In the mingw environment these functions are available in libregex,
]>   but libregexis notincluded inthe standard library search. I have
]>   not found away to add'-lregex'to the link command line:

]The main issue is that libregex isn't available reliably on mingw.
]On coreboot, I simply added a libregex implementation to the tree and
]added it to the build.

I thought about this method too. I could just add the needed objects
to a library that is already included in the default search.

]In general, maybe just copy over coreboot's kconfig and look for the
]changes (there are some you'd have to revert)?

This is essentially what I am doing now. To run abuild, I change
the default payload from seabios to none. I am hoping to find a
solution that works with no modification to the coreboot trunk
at all.

]Patrick

Here is another problem with using mingw to build coreboot. 
Building projects that include the new AMD reference code
(ASRock E350, AMD Persimmon, AMD Inagua) fail with the message,
"Couldn't commit memory for cygwin heap",
http://gitorious.org/msys/msys/blobs/MSYS-1_0_14/src/winsup/cygwin/cygheap.c...
Restarting the build works around the problem. I found cygheap.cc
uses a very simple malloc/free scheme where the memory originates
from Windows function VirtualAlloc, yet is never returned by calling
VirtualFree. In addition, VirtualAlloc allocates one 4096 byte page
minimum, which is wasting a lot for allocations of only a few bytes.
For now, I just expanded CYGHEAPSIZE a bit. A better fix may be 
needed in the future.

Thanks,
Scott

[Patrick Georgi]

Am 13.03.2011 20:25, schrieb Scott Duplichan:
> ]In general, maybe just copy over coreboot's kconfig and look for the
> ]changes (there are some you'd have to revert)?
>
> This is essentially what I am doing now. To run abuild, I change
> the default payload from seabios to none. I am hoping to find a
> solution that works with no modification to the coreboot trunk
> at all.

I was thinking about changing seabios' kconfig the same way we changed
coreboot's kconfig.
No changes to coreboot necessary.
> Here is another problem with using mingw to build coreboot. 
> Building projects that include the new AMD reference code
> (ASRock E350, AMD Persimmon, AMD Inagua) fail with the message,
> "Couldn't commit memory for cygwin heap",
> http://gitorious.org/msys/msys/blobs/MSYS-1_0_14/src/winsup/cygwin/cygheap.c...
> Restarting the build works around the problem. I found cygheap.cc
> uses a very simple malloc/free scheme where the memory originates
> from Windows function VirtualAlloc, yet is never returned by calling
> VirtualFree. In addition, VirtualAlloc allocates one 4096 byte page
> minimum, which is wasting a lot for allocations of only a few bytes.
> For now, I just expanded CYGHEAPSIZE a bit. A better fix may be 
> needed in the future.

I'm not quite sure - are you using mingw or cygwin?

Patrick

[Scott Duplichan]

Patrick Georgi wrote:

] I'm not quite sure - are you using mingw or cygwin?
] Patrick

I am using mingw. For my OS (Windows Server 2003 x64), cygwin
has a severe performance problem. Build time is at least 4
times that of mingw. Building is really painful when I use
cygwin.

The cygwin version of cygheap.cc seems more refined than the
mingw version of that file, though the cygwin version also
lacks a call to VirtualFree.

Thanks,
Scott

[Kevin O'Connor]

On Sat, Mar 12, 2011 at 09:47:56PM -0600, Scott Duplichan wrote:
> Hello,
>
> I am trying to overcome problems building seabios/tools/kconfig from
> the windows/mingw environment. Here are the failures:
> Link step for tools/kconfig/conf fails to find regcomp/regexec/regfree.
> In the mingw environment these functions are available in libregex, 
> but libregex is not included in the standard library search. I have
> not found a way to add '-lregex' to the link command line:
> $(Q)$(HOSTCC) $(HOSTLDFLAGS) -o $@ $(addprefix $(obj)/,$($(@F)-objs)) $(HOST_LOADLIBES) $(HOSTLOADLIBES_$(@F))
> HOSTLDFLAGS occurs too early in the file list for this use.

Can you add it to HOST_LOADLIBES?
> Link step for tools/kconfig/conf fails to find function uname().
> I can come up with a library to provide this function but need
> help passing it to the linker, same as with item 1.
>
> Function mkdir() takes only one argument in the mingw environment,
> yet takes 2 arguments otherwise. I have not found a good way to 
> overcome this difference.

What about making a file "mingw_fixups.h" and changing the HOSTCFLAGS
to add "-include mingw_fixups.h".  The header can then do something
like:

#define uname(arg1) memset(arg1, 0, sizeof(*arg1))
#define mkdir(arg1,arg2) mkdir(arg1)

-Kevin

[Scott Duplichan]

Kevin O'Connor wrote:

]On Sat, Mar 12, 2011 at 09:47:56PM -0600, Scott Duplichan wrote:
]> Hello,
]> 
]> I am trying to overcome problems building seabios/tools/kconfig from
]> the windows/mingw environment. Here are the failures:
]> 
]> 1) Link step for tools/kconfig/conf fails to find regcomp/regexec/regfree.
]>    In the mingw environment these functions are available in libregex, 
]>    but libregex is not included in the standard library search. I have
]>    not found a way to add '-lregex' to the link command line:
]>    $(Q)$(HOSTCC) $(HOSTLDFLAGS) -o $@ $(addprefix $(obj)/,$($(@F)-objs)) $(HOST_LOADLIBES) ]$(HOSTLOADLIBES_$(@F))
]>    HOSTLDFLAGS occurs too early in the file list for this use.
]
]Can you add it to HOST_LOADLIBES?

When added to HOST_LOADLIBES before starting the build, the value
is cleared by coreboot/util/kconfig/makefile line 103,

100: # Use recursively expanded variables so we do not call gcc unless
101: # we really need to do so. (Do not call gcc as part of make mrproper)
102: HOST_EXTRACFLAGS = $(shell $(CONFIG_SHELL) $(check-lxdialog) -ccflags)
103: HOST_LOADLIBES   = $(shell $(CONFIG_SHELL) $(check-lxdialog) -ldflags $(HOSTCC))

]> 2) Link step for tools/kconfig/conf fails to find function uname().
]>    I can come up with a library to provide this function but need
]>    help passing it to the linker, same as with item 1.
]> 
]> 3) Function mkdir() takes only one argument in the mingw environment,
]>    yet takes 2 arguments otherwise. I have not found a good way to 
]>    overcome this difference.
]
]What about making a file "mingw_fixups.h" and changing the HOSTCFLAGS
]to add "-include mingw_fixups.h".  The header can then do something
]like:
]
]#define uname(arg1) memset(arg1, 0, sizeof(*arg1))
]#define mkdir(arg1,arg2) mkdir(arg1)

This idea is workable. I Used:
#define _SYS_UTSNAME_H
struct utsname{char release[20];};
#define uname(arg1) memset(arg1, 0, sizeof(*arg1))
#define mkdir(arg1,...) mkdir(arg1)

The '#define _SYS_UTSNAME_H' blocks utsname.h inclusion. This is needed
to prevent a conflict between the macro definition and the function
prototype. A minimal utsname struct definition must be added back. The
mkdir macro is changed to variadic type to prevent a compile fail.

I bypassed the libregex problem for the moment and ran with the above
change to see what other problems remain. A seabios/tools/kconfig/confdata.c
call to rename(newname, dirname) fails. It appears the destination file must
not exist for the win32 version of this function to succeed. Here is how the
how the problem was overcome for coreboot:
http://tracker.coreboot.org/trac/coreboot/changeset/4952
When I add the UNLINK_IF_NECESSARY part of this change, the rename succeeds.
Could seabios adopt this change? For simplicity, the neighboring definition
#define mkdir(x,y) mkdir(x) could be considered too.

Thanks,
Scott

]
]-Kevin

[Kevin O'Connor]

On Sun, Mar 13, 2011 at 09:55:55PM -0500, Scott Duplichan wrote:
> Kevin O'Connor wrote:
> ]What about making a file "mingw_fixups.h" and changing the HOSTCFLAGS
> ]to add "-include mingw_fixups.h".  The header can then do something
> ]like:
> ]
> ]#define uname(arg1) memset(arg1, 0, sizeof(*arg1))
> ]#define mkdir(arg1,arg2) mkdir(arg1)
> I bypassed the libregex problem for the moment and ran with the above
> change to see what other problems remain. A seabios/tools/kconfig/confdata.c
> call to rename(newname, dirname) fails. It appears the destination file must
> not exist for the win32 version of this function to succeed.

Can you fix this up using the same "mingw_fixups.h" trick?  That is,
something like:

static inline void myrename() { ...}

#define rename(...) myrename(...)

If you're having a conflict with a system header, you should be able
to include the header in mingw_fixups prior to the #define.
> Here is how the
> how the problem was overcome for coreboot:
> http://tracker.coreboot.org/trac/coreboot/changeset/4952
> When I add the UNLINK_IF_NECESSARY part of this change, the rename succeeds.
> Could seabios adopt this change?

It's possible, but I'd prefer to keep the source code close to the
Linux version - it makes merges later on easier.

-Kevin

[Stefan Reinauer]

On 3/13/11 8:30 PM, Kevin O'Connor wrote:
> > Here is how the
> > how the problem was overcome for coreboot:
> > http://tracker.coreboot.org/trac/coreboot/changeset/4952
> > When I add the UNLINK_IF_NECESSARY part of this change, the rename succeeds.
> > Could seabios adopt this change?
>
> It's possible, but I'd prefer to keep the source code close to the
> Linux version - it makes merges later on easier.

Then maybe we should send our patches upstream?

[Kevin O'Connor]

On Sun, Mar 13, 2011 at 08:49:17PM -0700, Stefan Reinauer wrote:
> On 3/13/11 8:30 PM, Kevin O'Connor wrote:
> > > Here is how the
> > > how the problem was overcome for coreboot:
> > > http://tracker.coreboot.org/trac/coreboot/changeset/4952
> > > When I add the UNLINK_IF_NECESSARY part of this change, the rename succeeds.
> > > Could seabios adopt this change?
> >
> > It's possible, but I'd prefer to keep the source code close to the
> > Linux version - it makes merges later on easier.
>
> Then maybe we should send our patches upstream?

Yes - getting the patches into the Linux kbuild source would be best.

-Kevin

[Scott Duplichan]

Kevin O'Connor wrote:

]On Sun, Mar 13, 2011 at 08:49:17PM -0700, Stefan Reinauer wrote:
]> On 3/13/11 8:30 PM, Kevin O'Connor wrote:
]> >>Here is how the
]> >>how the problem was overcome for coreboot:
]> >>http://tracker.coreboot.org/trac/coreboot/changeset/4952
]> >>When I add the UNLINK_IF_NECESSARY part of this change, the rename succeeds.
]> >>Could seabios adopt this change?
]> >It's possible, but I'd prefer to keep the source code close to the
]> >Linux version - it makes merges later on easier.
]> 
]> Then maybe we should send our patches upstream?
]
]Yes - getting the patches into the Linux kbuild source would be best.
]
]-Kevin

I might not be the best one for explaining the benefit of Windows
support to the linux crowd. So to get past this problem, I
modified my mingw to accomodate the new seabios kconfig tool build.

To make the mkdir function compatible with both environments, I
changed its prototype to include a variable argument list after
the first argument. To fix the rename function, I substituted the
gnulib version of this function. To get past the default lib search
differences, I added libregex and libws2_32 to libmingw32. To 
solve the missing uname problem, I added the gnulib version to
libmingw32. The result is here:
http://notabs.org/coreboot/windows-build.htm

Now I can get through abuild, almost. I still get an occasional 
git failure (see attached). Not sure what causes it, but debugging
that is for another day.

Thanks,
Scott