restarting build,was: firefox3 failed attempt, final summary

[jimis]

Hello all,

during the last months I've been trying on my spare time to port firefox3 to Irix. I started with a vanilla firefox3 source package, without using any other patches, and tried to compile using latest nekoware gcc. Unfortunately I have failed, but have reached to a point which I should share. The package compiles cleanly, but segfaults when running.

Here are the steps to compile firefox 3.0.1 on Irix using gcc:

1. Change configure to get the gcc version correctly, otherwise unrelated things break while running configure. Related bug: 449671. Also change configure so that -rpath-link is converted to -rpath, the proper ld option for Irix. Change configure to comment out -Wall in CFLAGS and CXXFLAGS in the -irix6 part because warnings stop the compilation.Related patch.
2. Apply jsgc.c patch because MAP_ANONYMOUS is not implemented on Irix.
3. Apply the xptcall refactoring patch for Irix. Related bug: 443234.
4. Apply xpcom_glue_standalone_Makefile.in.diff so that the gtkEmbed related stuff is built. Bugs related: 444500, 298044.
5. Patch widget/src/gtk2/nsWindow.cpp because Irix doesn't have XFree86 includes. Related bug: 449695.
6. Patch toolkit/xre/nsNativeAppSupportUnix.cpp to replace non ANSI C functions setenv() and unsetenv() with their glib counterparts. Related bug: 449702.
7. Patch gfx/src/thebes/nsThebesDeviceContext.cpp to replace round() with NS_round(). Irix doesn't provide round() but rint() for rounding. Related bug: 449718.
8. Fix the various Makefiles by applying Makefiles-for-Irix.patch. It fixes issues like gp_overflow(5) for the linker, removes some hardsetted -mips3 flags, fixes order of libraries on the ld command etc...
9. Enforce my CFLAGS for NSS too that doesn't respect them, by applying nss-enforce-cflags.diff.

My .mozconfig file is the following:

Code: Select all

PERL=/usr/nekoware/bin/perl

CFLAGS='-UR4000 -mips4'; export CFLAGS
CXXFLAGS=-mips4; export CXXFLAGS

mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../obj1-@CONFIG_GUESS@
ac_add_options --enable-application=browser
mk_add_options MOZ_CO_PROJECT=browser

ac_add_options --disable-dbus
ac_add_options --disable-javaxpcom

ac_add_options --disable-optimize

ac_add_options --with-system-jpeg
ac_add_options --with-system-zlib
ac_add_options --with-system-bz2
ac_add_options --enable-system-cairo

ac_add_options --disable-gnomevfs
ac_add_options --disable-gnomeui
ac_add_options --disable-plugins
ac_add_options --disable-accessibility
ac_add_options --disable-inspector-apis
ac_add_options --disable-pref-extensions
ac_add_options --disable-extensions
ac_add_options --disable-installer
ac_add_options --disable-updater
ac_add_options --disable-parental-controls

ac_add_options --enable-debug

Various notes:

• The package compiles but doesn't run. Many tests pass but others fail miserably (some xpcshell tests), and no debugger can help me. dbx 7.3.7 segfaults badly, gdb 6.3 reports a corrupted stack and gdb 6.8 hangs. Similar misbehavior when debugging the core files. I ended up debugging the debugger but had no luck... That's the primary reason for my failure.
• To compile --with-system-png, the neko_libpng should be patched to support APNG.
• I was using latest versions for cairo,glib,gtk,libpixman,pango. They all compiled easily under gcc, with the following small quirks in case you want to build them:
  
  • libpixman needed a small patch, but should now be fixed upstream.
  • glib also needed a small patch, but this fix I think has landed to trunk, not to the stable branch.
  • I compiled cairo succesfully with --enable-glitz! This option exists also for firefox3. Imagine how nice it would be to have that beast accelerated with OpenGL...
• TODO: compile with other compilers. Latest gcc or SGI cc. Find a fast system for the job, my miserable O2 was too slow, needed a full day for an unoptimized build.

That is all for now! I wish you better luck. Thanks to all the people who helped me one way or the other (joerg, alver on the IRC, nekonoko for the package updates). I will be out of town for a long time, so don't be insulted if it takes some time to answer.

[porter]

I strongly recommend compilation with maximum warnings on anything except the most trivial projects, for instance how can configure correctly determine the APIs if you don't use the strongest warnings?

With gcc I use

Code: Select all

-Wall -Werror

and with MIPSpro I use

Code: Select all

-fullwarn -diag_error 3322,3604,1515,1196,1164,1498,1035,3584

I find IRIX a pretty standard Motif based UNIX, if it configures and compiles on Solaris and AIX there should not be major problems on IRIX unless people are doing silly things.

Eg, assuming little-endian, 32 bit and GCC count as "silly things".

[squeen]

This is a seriously good effort. Thanks for moving this port forward.

[josehill]

This is a seriously good effort. Thanks for moving this port forward.

Yes - thanks a lot for sharing your notes, jimis.

[dexter1]

Hello all,

After two months of developing a patch, i have finally made some decent progress in compiling firefox 3.0.10 with mipspro 7.4.4 on my 6.5.28m O200 rig.
In short, i can either make a shared firefox-bin binary with libxul.so or a static-linked one. Pity only that both will bomb out with the infamous Bus Error. This means that we're close, but that there are some serious byte alignment errors concerning some classes/structs.

This means we have to wade through all the tests and find and fix broken bits of code, in which i can definitely use some assistance. Thanks to Jimis we have the most problematic code fixed: the xptcall assembly part.

The main difference with GCC is that MIPSPro is extremely strict in preallocating space for classes in combination with templates. If you try to forward-declare a class by using its empty definition: class nsIFile; and then use any kind of constructor/destructor on this class, say f.i. nsIcontainer(nsRefPtr<nsIFile>){} MIPSPro will bomb out complaining about incomplete types or "no instance of nsRefPtr<blabla> matches * blabla" because it wants to know the complete compile size of the class in advance.
Initially trying other C++ frontends doesn't seem to ressolve the issue: i tried with the default 7.4.4 frontend, the -Zf,_245 frontend from 7.4.3 and the -Zf,_330 experimental frontend.
The fix is however surprisingly simple: in most cases commenting out the empty declaration and inserting the relevant header file with the nsIFile definition will fix the problem.

The only real problem is when classes are forward declared in the same header as their definition. In that case you have to completely turn the file upside down and juggle with some constructors/destructors.
On other times these headers are created using idl files, so this means that you have to patch them after a compile error. Still coding om some idl files to fix this problem, but fortunately this only occurs once.

Anyway, watch this space, as i will publish the patch file and my .mozconfig tomorrow when i'm at my work, so everybody can start chipping in for help.

Yours,

Dex

[kramlq]

In short, i can either make a shared firefox-bin binary with libxul.so or a static-linked one. Pity only that both will bomb out with the infamous Bus Error. This means that we're close, but that there are some serious byte alignment errors concerning some classes/structs.

You may know about this already, but perhaps 'sysmips(MIPS_FIXADE, 1, 0, 0)' could be used as a temporary hack initially so you can progress with the port. It enables an exception handler for alignment exceptions that will fix them on the fly (with the downside that each unaligned access now costs many extra instructions).

[bplaa.yai]

In short, i can either make a shared firefox-bin binary with libxul.so or a static-linked one.

I think having a separate libxul.so, in a different / installed by default subsystem is the best way, as it would allow libxul dependant applications (I'm working on songbird lately) to have a dependency on this subsystem. On my dev system libxul.so is huge (51MB), and I guess it makes sense to have it as a shared component...

[dexter1]

Re Kramiq: i didn't to be honest i'll have to find main {} somewhere and plaster this subroutine call at the beginning. Let's see what this does.

Re Bplaa.yai: a separate libxul.so would be nice for space concerns, however, expect that libxul.so will be relatively closely tied to the firefox version, so i don't think it would be wise to introduce a separate libxul.so and yet another dependency version headache. Let's get it running first and packaged.

Okay, here are my changes:

First, get firefox 3.0.10 from ftp://ftp.mozilla.org/pub/mozilla.org/firefox/releases/3.0.10/source/firefox-3.0.10-source.tar.bz2
Then, make sure you have the basic gtk2 environment and build environment installed: neko_glib, neko_gtk, neko_glitz, neko_zlib, neko_idl, neko_idn, neko_freetype2, neko_zip, neko_python, neko_perl, neko_pango, neko_libxrender, neko_libxml2, neko_libtool, neko_make, neko_tar, neko_iconv, neko_expat, neko_cairo, neko_png, neko_atk
That should cover most of it.
Unpack the tarball and you get a directory called mozilla/

Now get the patches:
irix.patch which is a complete rollup of mine and Jimis' patches, excluding some gcc-ism's
idl.patch which is a pregenerated header file fix during a make crash mid-compile. UglyMutha, but does the trick.
.mozconfig which has all the environment plus basic mozilla configure flags.

Before patching the source with irix.patch, look at three absolute path header file includes, they start with "/work/everdij/firefox/mozilla", please change those where your mozilla build environment lives. This i need to fix in some makefiles.

Okay, put .mozconfig in mozilla/ directory, patch the source with patch -p0 < irix.patch and in that directory start the build with gmake -f client.mk build >& log & It will autostart ./configure before the actual source compile.

On my dual R12K@270MHz 1024MB O200 rig the build takes about 3 hours and 50 minutes, after it will bomb out at 3 errors in "nsAnnotationService.cpp". Apply the idl.patch with patch -p0 < idl.patch and continue the make with gmake -f client.mk build >& log2 &
After another 50 minutes the firefox binary is built and installed in "mozilla/dist/bin/firefox-bin". If you want to run the binary directly, go to "mozilla/dist/bin" and type ./run-mozilla.sh ./firefox and be greeted with the bus error. Lazy people can just set the shared directory linker env variable with setenv LD_LIBRARY_PATH mozilla/dist/bin and type firefox directly (also very handy for the numerous test binaries beginning with "Test*"

A ldd of the binary file is:

Code: Select all

mech003 /work/everdij/firefox/mozilla/dist/bin> setenv LD_LIBRARY_PATH .
mech003 /work/everdij/firefox/mozilla/dist/bin> ldd firefox-bin
        libpthread.so  =>        /usr/lib32/libpthread.so
        libxul.so  =>    ./libxul.so
        libmozjs.so  =>  ./libmozjs.so
        libxpcom.so  =>  ./libxpcom.so
        libplds4.so  =>  ./libplds4.so
        libplc4.so  =>   ./libplc4.so
        libnspr4.so  =>  ./libnspr4.so
        libdl.so  =>     /usr/lib32/libdl.so
        libgtk-x11-2.0.so.1  =>  /usr/nekoware/lib/libgtk-x11-2.0.so.1
        libatk-1.0.so.1  =>      /usr/nekoware/lib/libatk-1.0.so.1
        libgdk-x11-2.0.so.1  =>  /usr/nekoware/lib/libgdk-x11-2.0.so.1
        libgdk_pixbuf-2.0.so.1  =>       /usr/nekoware/lib/libgdk_pixbuf-2.0.so.1
        libpangocairo-1.0.so.1  =>       /usr/nekoware/lib/libpangocairo-1.0.so.1
        libpango-1.0.so.1  =>    /usr/nekoware/lib/libpango-1.0.so.1
        libcairo.so.3  =>        /usr/nekoware/lib/libcairo.so.3
        libgmodule-2.0.so.1  =>  /usr/nekoware/lib/libgmodule-2.0.so.1
        libfreetype.so.7  =>     /usr/nekoware/lib/libfreetype.so.7
        libfontconfig.so.2  =>   /usr/nekoware/lib/libfontconfig.so.2
        libglitz.so.2  =>        /usr/nekoware/lib/libglitz.so.2
        libpng12.so.0  =>        /usr/nekoware/lib/libpng12.so.0
        libz.so  =>      /usr/nekoware/lib/libz.so
        libm.so  =>      /usr/lib32/libm.so
        libXrender.so.1  =>      /usr/nekoware/lib/libXrender.so.1
        libX11.so.1  =>  /usr/lib32/libX11.so.1
        libgobject-2.0.so.1  =>  /usr/nekoware/lib/libgobject-2.0.so.1
        libglib-2.0.so.1  =>     /usr/nekoware/lib/libglib-2.0.so.1
        libintl.so.4  =>         /usr/nekoware/lib/libintl.so.4
        libsocket.so  =>         /usr/lib32/libsocket.so
        libCsup.so  =>   /usr/lib32/libCsup.so
        libC.so.2  =>    /usr/lib32/libC.so.2
        libCio.so.1  =>  /usr/lib32/libCio.so.1
        libc.so.1  =>    /usr/lib32/libc.so.1
        libsqlite3.so  =>        ./libsqlite3.so
        libsmime3.so  =>         ./libsmime3.so
        libssl3.so  =>   ./libssl3.so
        libnss3.so  =>   ./libnss3.so
        libnssutil3.so  =>       ./libnssutil3.so
        libsoftokn3.so  =>       ./libsoftokn3.so
        libpangoft2-1.0.so.1  =>         /usr/nekoware/lib/libpangoft2-1.0.so.1
        libXft.so.2  =>  /usr/nekoware/lib/libXft.so.2
        libXt.so  =>     /usr/lib32/libXt.so
        libgthread-2.0.so.1  =>  /usr/nekoware/lib/libgthread-2.0.so.1
        libXext.so  =>   /usr/lib32/libXext.so
        libfastm.so  =>  /usr/lib32/libfastm.so
        libpixman.so.1  =>       /usr/nekoware/lib/libpixman.so.1
        libexpat.so.1  =>        /usr/nekoware/lib/libexpat.so.1
        libiconv.so.3  =>        /usr/nekoware/lib/libiconv.so.3
        libgen.so  =>    /usr/lib32/libgen.so   delay-load
mech003 /work/everdij/firefox/mozilla/dist/bin>

So now you know. Good luck to the people that want to help me with this. If you want to debug the binary just --enable-debug and --disable optimization in .mozconfig. Sometimes it helps to debug with a static binary (200mb!!!) , so here's my static .mozconfig

I will just continue undeterred with this port, stubborn bastard as i am. Cheerio!

Dex

[deBug]

I will just continue undeterred with this port, stubborn bastard as i am. Cheerio!

Dex

Good luck.
Let us know if we mere mortals (non developers) can help out in any way.

//Harry

[nekonoko]

I gave the static variant a shot this evening. Went pretty smoothly for the most part, but died near the end with:

Code: Select all

ld32: INFO    171: Multigot invoked. Gp relative region broken up into 4 separate regions.
ld32: ERROR   33 : Unresolved text symbol "JSAutoTempValueRooter::operator delete(void*,unsigned int)" -- 1st referenced by ../../staticlib/components/libxpconnect.a(XPCSafeJSObjectWrapper.o).
        Use linker option -v to see when and which objects, archives and dsos are loaded. 
ld32: INFO    152: Output file removed because of error.

I'll try a dynamic build tomorrow; for now I'm heading off to see an early morning showing of Star Trek.

[dexter1]

I gave the static variant a shot this evening. Went pretty smoothly for the most part, but died near the end with:

Code: Select all

ld32: INFO    171: Multigot invoked. Gp relative region broken up into 4 separate regions.
ld32: ERROR   33 : Unresolved text symbol "JSAutoTempValueRooter::operator delete(void*,unsigned int)" -- 1st referenced by ../../staticlib/components/libxpconnect.a(XPCSafeJSObjectWrapper.o).
        Use linker option -v to see when and which objects, archives and dsos are loaded. 
ld32: INFO    152: Output file removed because of error.

I'll try a dynamic build tomorrow; for now I'm heading off to see an early morning showing of Star Trek.

Ah, my bad. You can't mix libxul with a static build but you also can't enable pref-extensions without libxul, so the .mozconfig should have been

Code: Select all

CC=c99
CXX=CC
CFLAGS='-O2 -n32 -INLINE -woff 1174'
CPPFLAGS='-I/usr/nekoware/include'
LDFLAGS='-L/usr/nekoware/lib'
LIBS='-lm'
CXXFLAGS='-O2 -Zf,_245 -n32 -INLINE -woff 1110,1171,1201,1355,1460,3201,3303,3625,3649'
PERL=/usr/nekoware/bin/perl
BUILD_OFFICIAL=1
MOZILLA_OFFICIAL=1

ac_add_options --enable-application=browser
mk_add_options MOZ_CO_PROJECT=browser

ac_add_options --disable-dbus
ac_add_options --disable-javaxpcom

ac_add_options --disable-gnomevfs
ac_add_options --disable-gnomeui
ac_add_options --disable-plugins
ac_add_options --disable-accessibility
ac_add_options --disable-inspector-apis
ac_add_options --disable-installer
ac_add_options --disable-updater
ac_add_options --disable-parental-controls
ac_add_options --disable-cairo

ac_add_options --prefix=/usr/nekoware
ac_add_options --without-pthread
ac_add_options --disable-optimize
ac_add_options --enable-debug
ac_add_options --enable-static
ac_add_options --disable-shared
ac_add_options --disable-tests
ac_add_options --disable-freetype2
ac_add_options --disable-accessibility

ac_add_options --disable-libxul
ac_add_options --disable-pref-extensions
ac_add_options --disable-extensions


I just tested this .mozconfig build file and it compiles cleanly. I'll upload it shortly.

[nekonoko]

I appreciate your looking into it, but it still fails with the same error for me; I must have something fundamentally wrong with my build environment (at least as far as Firefox 3 goes). I'm at a loss as to what it could be. Thinking it may be parallel gmake related, I tried both with and without 'mk_add_options MOZ_MAKE_FLAGS="-j5"' (the difference in build times is astounding) but it made no difference either way.

[dexter1]

I appreciate your looking into it, but it still fails with the same error for me; I must have something fundamentally wrong with my build environment (at least as far as Firefox 3 goes). I'm at a loss as to what it could be. Thinking it may be parallel gmake related, I tried both with and without 'mk_add_options MOZ_MAKE_FLAGS="-j5"' (the difference in build times is astounding) but it made no difference either way.

I would appreciate if someone could come over here and kick me hard in various places. I found out that i forgot a patch:

Code: Select all

--- js/src/jscntxt.h   Tue Jul  1 19:22:52 2008
+++ js/src/jscntxt.h   Mon May  4 12:30:30 2009
@@ -820,7 +820,7 @@
     }
 
   private:
-#ifndef AIX
+#if !(defined(AIX) || defined(IRIX))
     static void *operator new(size_t);
     static void operator delete(void *, size_t);
 #endif


Mea culpa, sorry sorry. I blame the ridiculous length of the patch, it's so easy to miss a quick patch.

I'll do some more digging in the coming days. Sofar i found out that there is an ns_ASSERTION being triggered in nsXPCOMStrings.cpp where for example nsStringContainer_base (12 bytes) is smaller than their content nsString (16 bytes). Aha, this can cause the alignment errors. Unfortunately, i failed in implementing sysmips() in mozilla/browser/app/nsBrowserApp.cpp, i'm probably overlooking something.

Thanks for pointing me where the makeflag '-j5' should go. Yet another brainfart

[nekonoko]

Mea culpa, sorry sorry. I blame the ridiculous length of the patch, it's so easy to miss a quick patch

No problem at all - was just a bit disappointed that it was looking like my build system wasn't up to the task. With that final patch I was able to compile successfully and am now sitting at the 'Bus error' upon attempting a run

Thanks for pointing me where the makeflag '-j5' should go. Yet another brainfart

That should drop your build times significantly - really handy while debugging. For example, with -j5 it takes my quad 700MHz Tezro about 40 minutes to build Firefox-3. I expect that quad O200 you just unearthed will be able to finish a complete build within a couple hours.

[dexter1]

Hi all,

Still trodding along, now doing patches for 3.0.11. Unfortunately i am encountering health problems again, as my neurologist instructed me to stop medication and symptoms are reappearing, so this is going to progress slowly.

Good news is that the two stage compile is gone. Everything can be compiled using a single patch and in one go, that took me some time. i also had a look at 3.5 and it already complained about Pango not being version 0.14 or higher, so this probably means that 3.5 is still not going to happen soon, though it looks like the code has been extensively rewritten (and hopefully cleaned up)

The only big problems remaining are the assert errors about stringcontainers. i've already fixed a va_copy oops and am now progressing with cvd through the binary (threads, aarrgghh)

yours headachy,
dex