Nekochan Net

Official Chat Channel: #nekochan // irc.nekochan.net
It is currently Tue Oct 21, 2014 2:36 pm

All times are UTC - 8 hours [ DST ]


Forum rules


Any posts concerning pirated software or offering to buy/sell/trade commercial software are subject to removal.



Post new topic Reply to topic  [ 11 posts ] 
Author Message
Unread postPosted: Sat Jun 05, 2004 2:13 am 
Offline
Moderator
Moderator
User avatar

Joined: Thu Feb 20, 2003 7:57 am
Posts: 2062
Location: Voorburg, The Netherlands
Hi all,

I've opened an SGI Case concerning the fact that most of our Nekoware Beta crashes on IRIX machines below 6.5.21m. Neko and i have found out that it's a libc issue, and last night i have made a full document available for SGI support stating the following:
--------------------
Code:
Dear SGI support engineer,

I have placed binaries, source code patches and complete tardists of openssh-3.8.1p1 openssl-0.9.7d and zlib-1.2.1 into this directory to assist you in examining SGI case #2537486: "Applications built on 6.5.21m fails to run on 6.5.20m"

Actually the case should be written as: "Applications built on >=6.5.21m fails to run on <=6.5.20m." I have verified e.g. that builds on 6.5.23m coredump on 6.5.19m as well. The problem is indifferent for MIPSPro 7.4.1 versus 7.3.1.3m and also for "-mips3" versus "-mips4". I will reveal that there's been a major change to the libc.so.1 library in 6.5.21m, which causes this coredump. Let me explain later in detail, but let me first give you an overview of what i have uploaded:


1) Files

This file:
- README_2537486.txt

The versions file from my O200:
- versions.txt

The original Nekoware (www.nekochan.net) tardists made on IRIX 6.5.22/MIPSPro 7.4.1(different people and different machines):
- neko_openssh-3.8.1p1.tardist
- neko_openssl-0.9.7d.tardist
- neko_zlib-1.2.1.tardist

The source code tarballs and relnotes (txt), which include patchfiles and how to build the packages are already included in the tardists (opt subsystem), but are included here for completeness:
- openssh-3.8.1p1.tar.gz
- openssl-0.9.7d.tar.gz
- zlib-1.2.1.tar.gz
- openssh.txt
- openssl.txt
- neko_zlib.txt

Also included are the bare libraries libcrypto.so.0.9.7 , libz.so.1.2.1 and the executable scp build on the build-system below:
- libcrypto.so.0.9.7
- libz.so.1.2.1
- scp


2) Hardware/Software

The build was done on a quad O200/IRIX6.5.23m/MIPSPro7.3.1.3m:

mech003 /work/everdij/test/usr/nekoware/src> hinv -v
[SNIP]

mech003 /work/everdij/test/usr/nekoware/src> uname -aR
IRIX64 mech003 6.5 6.5.23m 01080747 IP27

mech003 /work/everdij/test/usr/nekoware/src> cc -version
MIPSpro Compilers: Version 7.3.1.3m


3) Build

- Please read the neko_zlib.txt and openssl.txt and openssh.txt to familiarize with the build process of the two libraries openssl and zlib and the openssh program suite. You need the freeware gmake program! Note the -mips4. You can change it to -mips3 but it doesn't change the results. Make sure you build on a system with irix OS >=6.5.21m and MIPSPro 7.3.1.3 or 7.4.1

- First make zlib and install it. Read neko_zlib.txt and apply the patch. Openssl needs this library to build. Make sure the following environment variables are set:

CC=cc
CFLAGS=-O3 -mips4 -n32 -I/usr/nekoware/include
LDFLAGS=-mips4 -rpath /usr/nekoware/lib
CPPFLAGS=-I/usr/nekoware/include
LD_LIBRARY_PATH=/usr/nekoware/lib

- Then build openssl libraries and install them as well. Read openssl.txt and apply the patch.

- At last build openssh. Read openssh.txt. You do not need to install the openssh set of programs.

you should have in /usr/nekoware/lib :

libcrypto.a         libssl.a            libz.a              libz.so.1.2.1
libcrypto.so        libssl.so           libz.so             pkgconfig
libcrypto.so.0      libssl.so.0         libz.so.1
libcrypto.so.0.9.7  libssl.so.0.9.7     libz.so.1.2

and in src/openssh-3.8.1p1 ( among a lot of other programs )

.
scp
.


4) Test

- Check the library links of scp:
- ldd ./scp:
        libcrypto.so.0.9.7  =>   /usr/nekoware/lib/libcrypto.so.0.9.7
        libz.so  =>      /usr/nekoware/lib/libz.so   
        libgen.so  =>    /usr/lib32/libgen.so   
        libc.so.1  =>    /usr/lib32/libc.so.1   
- Run it:
- ./scp
usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file]
           [-l limit] [-o ssh_option] [-P port] [-S program]
           [[user@]host1:]file1 [...] [[user@]host2:]file2


As a testmachine i've chosen an Indy running at 6.5.19m/MIPSPro 7.3.1.3m:

mech036 /tmp/tmp> hinv -v
[SNIP]

mech036 /tmp/tmp> uname -aR
IRIX mech036 6.5 6.5.19m 01100303 IP22

mech036 /tmp/tmp> cc -version
MIPSpro Compilers: Version 7.3.1.3m


Now transfer the following files you've built to the /tmp directory on a testmachine with 6.5.20m or 6.5.19m. :

- /usr/nekoware/lib/libcrypto.so.0.9.7
- /usr/nekoware/lib/libz.so.1.2.1
- src/openssh-3.8.1p1/scp

- do 'ln -s libz.so.1.2.1 libz.so' and 'chmod a+x scp'
- then ldd ./scp and run ./scp

- ldd ./scp:
        libcrypto.so.0.9.7  =>   /tmp/libcrypto.so.0.9.7   
        libz.so  =>      /tmp/libz.so       
        libgen.so  =>    /usr/lib32/libgen.so   
        libc.so.1  =>    /usr/lib32/libc.so.1   

- ./scp
Trace/BPT/RangeErr/DivZero/Ovflow trap

The crash (Sometimes it coredumps, sometimes not.)


Now copy the file /usr/lib32/libc.so.1 from a 6.5.21m or 6.5.22m machine to /tmp on the testmachine. ldd and rerun ./scp
- ldd ./scp
        libcrypto.so.0.9.7  =>   /tmp/libcrypto.so.0.9.7   
        libz.so  =>      /tmp/libz.so       
        libgen.so  =>    /usr/lib32/libgen.so   
        libc.so.1  =>    /tmp/libc.so.1     

- ./scp
usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file]
           [-l limit] [-o ssh_option] [-P port] [-S program]
           [[user@]host1:]file1 [...] [[user@]host2:]file2

Exactly what the program should have done...


So hopefully i've shown to you that an openssh build on IRIX machines with 6.5.21m 6.5.22m or 6.5.23m causes the resulting binaries to be so dependent on the libc.so.1 version, that older libc.so.1 versions crashes the binary. My choice for scp as program is rather arbitrary. I could also have chosen the program ssh, but the results are the same for that program as well.

Some programs do not show this odd behaviour. We compiled and ran the latest bzip2 without problems on machines with older IRIX OS'ses.

I hope this is a bit clear. Please contact me at F.Everdij@citg.tudelft.nl if you need more info.

Yours Sincerely,

Frank Everdij

--------------------

So now it's in their hands. If anyone can confirm this with a smaller test-suite, i'm all ears! Anything to clear up this mess before 6.5.21m is out would be a real livesaver...


Last edited by dexter1 on Mon Jun 07, 2004 12:45 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject:
Unread postPosted: Mon Jun 07, 2004 9:21 am 
Offline
Moderator
Moderator
User avatar

Joined: Thu Feb 20, 2003 7:57 am
Posts: 2062
Location: Voorburg, The Netherlands
Update:

I have emailed a couple of times with SGI support, and we have identified the problem. They have kindly asked me not to publish any information about this yet, so i can't tell you more. But atleast the problem is being looked at!

Fingers X


Top
 Profile  
 
 Post subject:
Unread postPosted: Mon Jun 07, 2004 9:30 am 
Offline
User avatar

Joined: Fri Mar 14, 2003 6:22 am
Posts: 3102
Location: living in a linux-blunderland
i'm not suprised, blender fell apart when we got to 6.5.21 :(


Top
 Profile  
 
 Post subject:
Unread postPosted: Mon Jun 07, 2004 10:57 am 
Offline
Moderator
Moderator
User avatar

Joined: Thu Feb 20, 2003 7:57 am
Posts: 2062
Location: Voorburg, The Netherlands
Well, i've had a response from one of the upper echelons of SGI management. The basic story is that it's strlcpy and strlcat again. These are defined and included in libc in 6.5.21m, but not in 6.5.20m.

This means that every package needing strlcpy and strlcat and is built on 6.5.21m and above will not work on 6.5.20m and below. I threw in some paragraphs about software compatibility mandate, which support did see some issue, but the prevailing consensus was that strlcpy and strlcat are NEW functions which should not belong in older irixes anyway. Simply recompile without strlcpy and strlcat.

AWW MAN! :cry:

Oh, and no ETA yet on 6.5.21m for free supportfolio yet, i have asked this and hopefully i get a response.

AWW MAN! :cry:

All is not lost however. I remember a neat trick to add functionality back into old libc's. Will try this right now.

fingers X-ed again


Top
 Profile  
 
 Post subject:
Unread postPosted: Mon Jun 07, 2004 12:44 pm 
Offline
Moderator
Moderator
User avatar

Joined: Thu Feb 20, 2003 7:57 am
Posts: 2062
Location: Voorburg, The Netherlands
Huuuhh! We should do more fingers crossing!

I think i have a working fix for the inability of running Nekoware Beta on 6.5.20m and below. This is it:

Code:
/*
 * strlcat and strlcpy backport to <6.5.21m
 * Inspired by __libc_attr fix by Walter Roberson
 * Arranged by Frank Everdij <F.Everdij@citg.tudelft.nl>
 *
 * To use, simply:
 *  cc -o /usr/lib32/libstrlc.so -shared libstrlc.c
 * or any place you desire, and
 *  setenv _RLD_LIST DEFAULT:/usr/lib32/libstrlc.so
 * On some systems _RLDN32_LIST or _RLD64_LIST is necessary
 *
 */

/* OPENBSD ORIGINAL: lib/libc/string/strlcpy.c */

/*   $OpenBSD: strlcpy.c,v 1.8 2003/06/17 21:56:24 millert Exp $   */

/*
 * Copyright (c) 1998 Todd C. Miller <Todd.Miller@courtesan.com>
 *
 * Permission to use, copy, modify, and distribute this software for any
 * purpose with or without fee is hereby granted, provided that the above
 * copyright notice and this permission notice appear in all copies.
 *
 * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
 * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
 * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
 * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
 * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
 * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
 * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 */

#include <sys/types.h>
#include <string.h>

/*
 * Copy src to string dst of size siz.  At most siz-1 characters
 * will be copied.  Always NUL terminates (unless siz == 0).
 * Returns strlen(src); if retval >= siz, truncation occurred.
 */
size_t
strlcpy(char *dst, const char *src, size_t siz)
{
   register char *d = dst;
   register const char *s = src;
   register size_t n = siz;

   /* Copy as many bytes as will fit */
   if (n != 0 && --n != 0) {
      do {
         if ((*d++ = *s++) == 0)
            break;
      } while (--n != 0);
   }

   /* Not enough room in dst, add NUL and traverse rest of src */
   if (n == 0) {
      if (siz != 0)
         *d = '\0';      /* NUL-terminate dst */
      while (*s++)
         ;
   }

   return(s - src - 1);   /* count does not include NUL */
}

/* OPENBSD ORIGINAL: lib/libc/string/strlcat.c */

/*   $OpenBSD: strlcat.c,v 1.11 2003/06/17 21:56:24 millert Exp $   */

/*
 * Copyright (c) 1998 Todd C. Miller <Todd.Miller@courtesan.com>
 *
 * Permission to use, copy, modify, and distribute this software for any
 * purpose with or without fee is hereby granted, provided that the above
 * copyright notice and this permission notice appear in all copies.
 *
 * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
 * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
 * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
 * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
 * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
 * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
 * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 */

/*
 * Appends src to string dst of size siz (unlike strncat, siz is the
 * full size of dst, not space left).  At most siz-1 characters
 * will be copied.  Always NUL terminates (unless siz <= strlen(dst)).
 * Returns strlen(src) + MIN(siz, strlen(initial dst)).
 * If retval >= siz, truncation occurred.
 */
size_t
strlcat(char *dst, const char *src, size_t siz)
{
   register char *d = dst;
   register const char *s = src;
   register size_t n = siz;
   size_t dlen;

   /* Find the end of dst and adjust bytes left but don't go past end */
   while (n-- != 0 && *d != '\0')
      d++;
   dlen = d - dst;
   n = siz - dlen;

   if (n == 0)
      return(dlen + strlen(s));
   while (*s != '\0') {
      if (n != 1) {
         *d++ = *s;
         n--;
      }
      s++;
   }
   *d = '\0';

   return(dlen + (s - src));   /* count does not include NUL */
}


So it's just creating a shared library libstrlc.so, put it somewhere, and add the library to the linker list:

- cc -o /usr/nekoware/lib/libstrlc.so -shared libstrlc.c
- setenv _RLD_LIST DEFAULT:/usr/nekoware/lib/libstrlc.so
- run your nekoware beta stuff

... but /usr/lib32 path as stated in the code should work fine as well.
I've put a binary version here: http://www.mechanics.citg.tudelft.nl/~e ... ibstrlc.so for people without cc.

So far i have been able to use sftp ssh and scp from nekoware beta on an 6.5.20m system and they all work fine. Machines <6.5.19m will not be able to use openssl libs because /dev/random is missing.

I would invite everybody with 6.5.20m to try out my patch. If successful i will clean it up a bit and upload it to nekochan's ftp server.

<aliG>Booyakasha!!</aliG>


Top
 Profile  
 
 Post subject:
Unread postPosted: Mon Jun 07, 2004 1:26 pm 
Offline
Moderator
Moderator
User avatar

Joined: Fri May 09, 2003 6:10 am
Posts: 2931
Location: Maryland, USA
Very elegant and totally within the intended scope of the run-time linker's intended facilities.
Once again, hat's off to you dexter1.


Top
 Profile  
 
 Post subject:
Unread postPosted: Tue Jun 08, 2004 5:22 am 
Offline
User avatar

Joined: Tue Apr 29, 2003 2:02 pm
Posts: 972
Location: Winterthur, Switzerland
dexter1 wrote:


hehehehehehhe

FANTASTISCH!!!!!!

Nostalgie, dexter1 :-)

(Ohh yeah love that Drunken Klingon fun :-) )

_________________
Shall I describe it to you? Or do you want me to get you a box?


Top
 Profile  
 
 Post subject:
Unread postPosted: Tue Jun 08, 2004 12:33 pm 
Offline
User avatar

Joined: Fri Mar 14, 2003 6:22 am
Posts: 3102
Location: living in a linux-blunderland
so, is there a problem developing with 6.5.20m and having backward and forward compatability?


Top
 Profile  
 
 Post subject:
Unread postPosted: Tue Jun 08, 2004 1:21 pm 
Offline
Moderator
Moderator
User avatar

Joined: Thu Feb 20, 2003 7:57 am
Posts: 2062
Location: Voorburg, The Netherlands
Um not particularly. It's just that i have a lot of machines who cannot run beyond 6.5.22m so might as well take that as the development platform.

But if i knew that SGI apparently has no plans yet to release the 6.5.21m update for free supportfolio subscribers, and if i knew i'd get into a lot of trouble because of these missing functions in 6.5.20m, i ofcourse wouldn't have chosen 6.5.22m or 6.5.21m as development platform.
It's just that i do have access to the latest releases and compiler updates, so that i can pamper my I2 Impact with 6.5.22m and 7.4.2m.

Plus the fact that SGI did nothing in helping people who are stuck with 6.5.20m to fix some silly extra two functions. Desperate and Disgusted i was with SGI's stance, even if they were completely correct in their answers.

But the fix is there, and we can continue with the Nekoware beta. Soon 6.5.21m will be available (i hope)

I read on the forum that 6.5.21m broke a lot with respect to blender. Quite a bummer. Care to elaborate on that? maybe we can pinpoint the culprit together...


Top
 Profile  
 
 Post subject:
Unread postPosted: Wed Jun 09, 2004 3:27 am 
Offline
Moderator
Moderator
User avatar

Joined: Fri May 09, 2003 6:10 am
Posts: 2931
Location: Maryland, USA
I just want to add again that 6.5.21 fixed alot of errors with multiprocess/multithreading that were accidentally introduced around 6.5.18. It made a huge difference in the stability of my real-time code. I could never go back pre-6.5.21 for that reason.


Top
 Profile  
 
 Post subject:
Unread postPosted: Mon Nov 15, 2004 7:36 am 
Offline
Moderator
Moderator
User avatar

Joined: Sun Mar 30, 2003 5:29 am
Posts: 2477
Location: Kabul, Afghanistan, Asia
I must have completely missed this thread. AWESOME dexter1! You are a genius par exemple!

_________________
...only chemist in .af?
Eroteme.ch - eternally unfinished and never started


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 11 posts ] 

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group