Nekochan Net

Official Chat Channel: #nekochan // irc.nekochan.net
It is currently Mon Apr 21, 2014 6:41 am

All times are UTC - 8 hours


Forum rules


Any posts concerning pirated software or offering to buy/sell/trade commercial software are subject to removal.



Post new topic Reply to topic  [ 14 posts ] 
Author Message
Unread postPosted: Wed Dec 29, 2010 1:09 pm 
Offline
User avatar

Joined: Sat May 29, 2004 1:28 pm
Posts: 28
Location: Switzerland
Hi,

I have a weird problem with one of my O300 boxes. Basic config is:

6 x O300 bricks with quad 600Mhz CPUs
1 x Router
1 x L2

When system is powered on, L2 boots and will recognize all bricks, so the 'cfg' prints the address of each brick and router.
When system is booted by 'pwr u' via L2, within 10-20 seconds one of the bricks will produce following error:

Code:
usb_control/bulk_msg: timeout
hub.c: USB device not accepting new address (error=-110)
SGIL1_RESET_DEVICE error, Connection timed out
WARNING: can't write to connection to 001c22 [0x0ffff00e]:   IRouter:write failed
L2 SMP ERROR: no response from 001?22
usb_control/bulk_msg: timeout
usb.c: USB device not accepting new address=8 (error=-110)
hub.c: USB new device connect on bus2/1/1, assigned device number 9
usb.c: USB device not accepting new address=9 (error=-110)
hub.c: USB new device connect on bus2/1/1, assigned device number 10


The last message keeps looping forever. The brick will disappear from the inventory and will not show on L2 'cfg' output anymore.

What could be the cause for this problem?

Brick itself seems to be operational, it works fine as standalone. O300

I have checked USB and Numalink cables, replaced them, inter exchanged with other bricks e.t.c., no help :(

Best Regards

Don

_________________
ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage


Last edited by Don.Key on Fri Dec 31, 2010 12:47 am, edited 1 time in total.

Top
 Profile  
 
Unread postPosted: Thu Dec 30, 2010 11:26 am 
Offline
Moderator
Moderator
User avatar

Joined: Sun Jun 06, 2004 4:55 pm
Posts: 5161
Location: NC - USA
The bottom four ports on the NUMAlink Module/Router <ports 2,3,4 and 5> route USB signals from connected compute modules to the L2 controller <from compute module to router via the NUMAlink cable, and from router to L2 via the L1 port on the router>. The top four ports on the NUMAlink Module/Router <ports 1,6,7 and 8> don't have the capability to route USB signals set between modules/bricks and the L2 controller, so those signals are routed directly from the module/brick to the L2 by USB cable <via a USB hub>. Not to belabor the obvious <or be overly simplistic>, but have you tried testing with known working USB hub in place of the one used by your O300? If you try it be sure to use a powered USB hub.


You mentioned inter exchanging with other bricks, but I wasn't sure if you were referring only to switching NUMAlink cables <so forgive me if I misunderstood>. If the inter exchanging only involved cables <and a different USB hub didn't correct the problem>, you might try switching the router port used by the problem compute module to one used by a compute module without a problem, and see if the problem follows the compute module or stays with the router port.

_________________
***********************************************************************
Welcome to ARMLand - 0/0x0d00
running...(sherwood-root 0607201829)
* InfiniteReality/Reality Software, IRIX 6.5 Release *
***********************************************************************


Top
 Profile  
 
Unread postPosted: Fri Dec 31, 2010 12:52 am 
Offline
User avatar

Joined: Sat May 29, 2004 1:28 pm
Posts: 28
Location: Switzerland
Thank you recondas,

I have tried to swap cables with another working brick, thus moving the troublesome brick to another router and usb port. Problem followed the brick. And this cannot be cables since the same cables work perfectly with another brick.

I did not, however, try to put that brick onto the router port which can route USB signals by itself, that is a very good information, thank you! I will try it today and see, perhaps the problem can be resolved that way!

Regards and happy new year!

Don

_________________
ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage


Top
 Profile  
 
Unread postPosted: Fri Dec 31, 2010 3:32 pm 
Offline
User avatar

Joined: Tue Jul 15, 2008 4:48 pm
Posts: 1887
Location: P.O. Box 121, Pymble, Sydney, NSW 2073, Australia.
Don.Key wrote:
usb.c: USB device not accepting new address=9 (error=-110)


If you type ! or shell in the L2 you get a shell in a Linux/PPC machine with busybox.
Code:
M1000707-001-L2>shell
sh-2.03# uname -a
Linux 192.168.0.179 2.4.7-sgil2 #1 Mon Feb 28 14:51:03 CST 2005 ppc unknown


I typed "uname -a" on mine and got a revision number of 2.4.7 ... So I went to kernel.org and downloaded that version.
I then poked around in ./linux/drivers/usb/usb.c and found:


Code:
/*
 * By the time we get here, the device has gotten a new device ID
 * and is in the default state. We need to identify the thing and
 * get the ball rolling..
 *
 * Returns 0 for success, != 0 for error.
 */
int usb_new_device(struct usb_device *dev)
{
        int err;

        /* USB v1.1 5.5.3 */
        /* We read the first 8 bytes from the device descriptor to get to */
        /*  the bMaxPacketSize0 field. Then we set the maximum packet size */
        /*  for the control pipe, and retrieve the rest */
        dev->epmaxpacketin [0] = 8;
        dev->epmaxpacketout[0] = 8;

        err = usb_set_address(dev);
        if (err < 0) {
                err("USB device not accepting new address=%d (error=%d)",
                        dev->devnum, err);
                clear_bit(dev->devnum, &dev->bus->devmap.devicemap);
                dev->devnum = -1;
                return 1;
        }


It is the only instance of that message in the file, so I had a look into usb_set_address( ... ) and found that it constructs a USB Request Block (URB) on the control channel.. I dug deeper into the code and found nothing especially useful or interesting...

The error code -110 comes from one of the lower routines negating the error code as it returns it.. The actual error code from errno is..

Code:
#define ETIMEDOUT       110     /* Connection timed out */


Which seems consistent with earlier messages in the code block posted in the original post above.
Make of that what you will... I have no further ideas without a USB bus analyser attached to it.

Quote:
Problem followed the brick.

If you take a magnifying glass and look at the motherboard closely starting from around the USB port on the problematic o300, do you see any lifted tracks, broken pins, worn through connectors, bulging or scorching, corrosion, dust or anything like that..?

R.

_________________
死の神はりんごだけ食べる

開いた括弧は必ず閉じる -- あるプログラマー

:Tezro: :Tezro: :Onyx2R: :Onyx2RE: :Onyx2: :O3x04R: :O3x0: :O200: :Octane: :Octane2: :O2: :O2: :Indigo2IMP: :PI: :PI: :1600SW: :1600SW: :Indy: :Indy: :Indy: :Indy: :Indy:
:hpserv: J5600, 2 x Mac, 3 x SUN, Alpha DS20E, Alpha 800 5/550, 3 x RS/6000, Amiga 4000 VideoToaster, Amiga4000 -030, 733MHz Sam440 AmigaOS 4.1 update 1.

Sold: :Indy: :Indy: :Indy: :Indigo: Tandem Himalaya S-Series Nonstop S72000 ServerNet.

@PymbleSoftware
Cortex ---> http://www.facebook.com/pages/Cortex-th ... 11?sk=info
Minnie ---> http://www.facebook.com/pages/Minnie-th ... 02?sk=info
Book ----> http://pymblesoftware.com/book/
Github ---> https://github.com/pymblesoftware
Visit http://www.pymblesoftware.com
Search for "Pymble", "InstaElf", "CryWhy" or "Cricket Score Sheet" in the iPad App store or search for "Pymble" or "CryWhy" in the iPhone App store.


Top
 Profile  
 
Unread postPosted: Sat Jan 01, 2011 5:16 am 
Offline
User avatar

Joined: Sat May 29, 2004 1:28 pm
Posts: 28
Location: Switzerland
Hi,

Okay, switching to the router port which has USB routing build in did not resolve the problem, it is still the same. PymbleSoftware: The board seems to be all ok and the test with USB over Numalink would eliminate the possibility of bad port.

Now here is another interesting fact: I have two o300 servers, on BOTH of them, bricks with same serial exhibit same problem. What is the chance of two bricks going bad at the same time? So on both machines, 001c22 will drop the connection to L2. Bricks alone themselves seem to be okay. Funky stuff, eh?

Changing Numalink, USB cables, Router ports, e.t.c. does not help.

I did some more research, it looks like the brick will drop USB connection when it is powered on and regain it when off, here is sequence of events:

oink-001-L2 Is a L2 controller
001c22-L1 is a problematic brick

I have tried to keep other bricks out of the picture and only to work with the problematic one to show the issues.

oink-001-L2
Code:
oink-001-L2>cfg
L2 192.168.1.11: - 001 (LOCAL)
L1 192.168.1.11:0:0      - 001c36
L1 192.168.1.11:2:0      - 001c24
L1 192.168.1.11:1:0      - 001c22
L1 192.168.1.11:3:0      - 001r20
L1 192.168.1.11:5:0      - 001c18
L1 192.168.1.11:4:0      - 001c16
L1 192.168.1.11:6:0      - 001c14


001c22-L1
Code:
001c22-L1>cfg
:0  - 001c22
001c22-L1>date
12/31/2010 14:07:19 CST
001c22-L1>pwr u


about 15 secs pass

oink-001-L2
Code:
usb_control/bulk_msg: timeout
hub.c: USB device not accepting new address (error=-110)
SGIL1_RESET_DEVICE error, Connection timed out
WARNING: can't write to connection to 001c22 [0x0ffff00e]:   IRouter:write failed
usb.c: USB disconnect on device 90
sgil1.c: SGI L1 disconnected, minor: 65 device: 2.90
INFO: closed connection to 001c22
hub.c: USB new device connect on bus2/1/1, assigned device number 91
usb_control/bulk_msg: timeout
usb.c: USB device not accepting new address=91 (error=-110)
hub.c: USB new device connect on bus2/1/1, assigned device number 92
usb.c: USB device not accepting new address=92 (error=-110)
hub.c: USB new device connect on bus2/1/1, assigned device number 93
usb_control/bulk_msg: timeout
usb.c: USB device not accepting new address=93 (error=-110)
hub.c: USB new device connect on bus2/1/1, assigned device number 94
usb.c: USB device not accepting new address=94 (error=-32)
oink-001-L2>cfg
L2 192.168.1.11: - 001 (LOCAL)
L1 192.168.1.11:0:0      - 001c36
L1 192.168.1.11:2:0      - 001c24
L1 192.168.1.11:3:0      - 001r20
L1 192.168.1.11:5:0      - 001c18
L1 192.168.1.11:4:0      - 001c16
L1 192.168.1.11:6:0      - 001c14


001c22-L1
Code:
001c22-L1>date
12/31/2010 14:07:54 CST
001c22-L1>pwr d


oink-001-L2
Code:
hub.c: USB new device connect on bus2/1/1, assigned device number 95
Manufacturer: Silicon Graphics, Inc.
Product: SN1 L1 System Controller
SerialNumber: 00000000
sgil1.c: SGI L1 connected, minor: 65 device: 2.95
INFO: opened USB device at b2;p1/1;d95 (/dev/sgil1_1)
oink-001-L2>cfg
L2 192.168.1.11: - 001 (LOCAL)
L1 192.168.1.11:0:0      - 001c36
L1 192.168.1.11:2:0      - 001c24
L1 192.168.1.11:1:0      - 001c22
L1 192.168.1.11:3:0      - 001r20
L1 192.168.1.11:5:0      - 001c18
L1 192.168.1.11:4:0      - 001c16
L1 192.168.1.11:6:0      - 001c14


001c22-L1
Code:
001c22-L1>log
12/31/10 14:07:01 USB0: opened
12/31/10 14:07:01 USB0: registered for events
12/31/10 14:07:20 power up (COMMAND)
12/31/10 14:07:27 IP45 ASIC clock 08
12/31/10 14:07:27 IP45 XBridge clock 04
12/31/10 14:07:28 reset again MIPS
12/31/10 14:07:33 IP45 ASIC clock 08
12/31/10 14:07:33 IP45 XBridge clock 04
12/31/10 14:07:48 USB-R: USB:connection lost
12/31/10 14:07:48 UNREG: 30004c80 0 7
12/31/10 14:07:48 USB0: unregistered
12/31/10 14:07:48 USB-W: USB:connection lost
12/31/10 14:07:48 USB-W: IRouter:write failed
12/31/10 14:07:49 USB0-R: IRouter:read failed - read error
12/31/10 14:07:49 USB0: waiting on open
12/31/10 14:07:56 power down (COMMAND)
12/31/10 14:08:00 USB0: opened
12/31/10 14:08:00 USB0: registered for events



So, all I can say here is: huh??


Top
 Profile  
 
Unread postPosted: Sat Jan 01, 2011 3:13 pm 
Offline
User avatar

Joined: Tue Jul 15, 2008 4:48 pm
Posts: 1887
Location: P.O. Box 121, Pymble, Sydney, NSW 2073, Australia.
Don.Key wrote:

oink-001-L2
Code:
usb_control/bulk_msg: timeout
hub.c: USB device not accepting new address (error=-110)
SGIL1_RESET_DEVICE error, Connection timed out
WARNING: can't write to connection to 001c22 [0x0ffff00e]:   IRouter:write failed
usb.c: USB disconnect on device 90
sgil1.c: SGI L1 disconnected, minor: 65 device: 2.90
INFO: closed connection to 001c22
hub.c: USB new device connect on bus2/1/1, assigned device number 91
usb_control/bulk_msg: timeout
usb.c: USB device not accepting new address=91 (error=-110)
hub.c: USB new device connect on bus2/1/1, assigned device number 92
usb.c: USB device not accepting new address=92 (error=-110)
hub.c: USB new device connect on bus2/1/1, assigned device number 93
usb_control/bulk_msg: timeout
usb.c: USB device not accepting new address=93 (error=-110)
hub.c: USB new device connect on bus2/1/1, assigned device number 94
usb.c: USB device not accepting new address=94 (error=-32)



New error code at the end of that is:
#define EPIPE 32 /* Broken pipe */

I don't know what is going on here... I take it you have done obvious things like the "reboot_l2" command..

You might want to talk to 001c22s L1 directly and make sure its in the same partition as the others..
Code:
001c01-L1>brick
rack: 001, slot: 01, partition: none, type: C (IP45) [2MB flash], serial:KJD687, source: EEPROM


Just for laughs, take inventory of all the o300s and check their partition..

You might want to adjust the l1dbg settings.. I run with mine off but anyway..

Code:
001c01-L1>l1dbg
L1 irouter debugging is off
L1 junkbus debugging is slab mask 0x0, mask 0x0
L1 environmental debugging is off
L1 port interrupt debugging is off
L1 i2c interrupt debugging is off
L1 voltage margin debugging is off
L1 PROM request debugging is off
L1 irouter req/rsp suspend is 300, event suspend is 50
L1 data dump on PPP error is off
L1 malloc failure PANICs are disabled
001c01-L1>


I really don't know what is going on here... Time for GoBI to weigh in with a nugget of wisdom..

R.

_________________
死の神はりんごだけ食べる

開いた括弧は必ず閉じる -- あるプログラマー

:Tezro: :Tezro: :Onyx2R: :Onyx2RE: :Onyx2: :O3x04R: :O3x0: :O200: :Octane: :Octane2: :O2: :O2: :Indigo2IMP: :PI: :PI: :1600SW: :1600SW: :Indy: :Indy: :Indy: :Indy: :Indy:
:hpserv: J5600, 2 x Mac, 3 x SUN, Alpha DS20E, Alpha 800 5/550, 3 x RS/6000, Amiga 4000 VideoToaster, Amiga4000 -030, 733MHz Sam440 AmigaOS 4.1 update 1.

Sold: :Indy: :Indy: :Indy: :Indigo: Tandem Himalaya S-Series Nonstop S72000 ServerNet.

@PymbleSoftware
Cortex ---> http://www.facebook.com/pages/Cortex-th ... 11?sk=info
Minnie ---> http://www.facebook.com/pages/Minnie-th ... 02?sk=info
Book ----> http://pymblesoftware.com/book/
Github ---> https://github.com/pymblesoftware
Visit http://www.pymblesoftware.com
Search for "Pymble", "InstaElf", "CryWhy" or "Cricket Score Sheet" in the iPad App store or search for "Pymble" or "CryWhy" in the iPhone App store.


Top
 Profile  
 
Unread postPosted: Mon Jan 03, 2011 3:03 pm 
Offline
User avatar

Joined: Sat May 29, 2004 1:28 pm
Posts: 28
Location: Switzerland
Well,

'brick' shows that the brick is in the same partition, rack e.t.c. like all other machines.
I did 'serial clear' and it was reassigned by L2 immediately.

Still, no avail, both 001c22 bricks will not work with both O300 systems, only as stand alone. Can that be some kind of licensing issue?

_________________
ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage


Top
 Profile  
 
Unread postPosted: Mon Jan 03, 2011 3:13 pm 
Offline
User avatar

Joined: Tue Jul 15, 2008 4:48 pm
Posts: 1887
Location: P.O. Box 121, Pymble, Sydney, NSW 2073, Australia.
Don.Key wrote:
Well,

'brick' shows that the brick is in the same partition, rack e.t.c. like all other machines.
I did 'serial clear' and it was reassigned by L2 immediately.

Still, no avail, both 001c22 bricks will not work with both O300 systems, only as stand alone. Can that be some kind of licensing issue?


Not that I am aware of... Licensing usually comes with trumpets and horns and the phone numbers of the sales department... MIPS Pro certainly does.

Did you turn on all the debugging and capture it..? There might be something interesting in there
....or perhaps its time for someone else to step in with more o300 experience..

R.

_________________
死の神はりんごだけ食べる

開いた括弧は必ず閉じる -- あるプログラマー

:Tezro: :Tezro: :Onyx2R: :Onyx2RE: :Onyx2: :O3x04R: :O3x0: :O200: :Octane: :Octane2: :O2: :O2: :Indigo2IMP: :PI: :PI: :1600SW: :1600SW: :Indy: :Indy: :Indy: :Indy: :Indy:
:hpserv: J5600, 2 x Mac, 3 x SUN, Alpha DS20E, Alpha 800 5/550, 3 x RS/6000, Amiga 4000 VideoToaster, Amiga4000 -030, 733MHz Sam440 AmigaOS 4.1 update 1.

Sold: :Indy: :Indy: :Indy: :Indigo: Tandem Himalaya S-Series Nonstop S72000 ServerNet.

@PymbleSoftware
Cortex ---> http://www.facebook.com/pages/Cortex-th ... 11?sk=info
Minnie ---> http://www.facebook.com/pages/Minnie-th ... 02?sk=info
Book ----> http://pymblesoftware.com/book/
Github ---> https://github.com/pymblesoftware
Visit http://www.pymblesoftware.com
Search for "Pymble", "InstaElf", "CryWhy" or "Cricket Score Sheet" in the iPad App store or search for "Pymble" or "CryWhy" in the iPhone App store.


Top
 Profile  
 
Unread postPosted: Mon Jan 03, 2011 4:19 pm 
Offline
Moderator
Moderator
User avatar

Joined: Sun Jun 06, 2004 4:55 pm
Posts: 5161
Location: NC - USA
Don.Key wrote:
'brick' shows that the brick is in the same partition, rack e.t.c. like all other machines. I did 'serial clear' and it was reassigned by L2 immediately. Still, no avail, both 001c22 bricks will not work with both O300 systems, only as stand alone. Can that be some kind of licensing issue?


--- "both 001c22 bricks" --- That seems more than just a little curious <especially since the L2 was able to assign a serial number>. Is this system newly composited from discreet parts, or if it is a pre-existing configuration, have you recently updated the L1 or L2 firmware?

While we're on the subject, what firmware versions are on the L2 and each of the bricks in that system <including an hinv -vm, L2 serial all, and L2 env probably wouldn't hurt either>. An L2 controller I picked up from eBay had firmware several revisions older than the compute modules I connected it to, and that L2 could not correctly ID those compute modules - the compute module rack.brick.slot would include spurious brick identifiers, like 001b01 <instead of 001c01>. Flashing the L2 to a newer revision corrected the problem - but if you do select your L2 firmware version carefully viewtopic.php?f=3&t=16719306&

Just for arguments sake <especially since you mention neither 001c22 will work correctly> you might try assigning a different slot ID, as in something other than "22" <if it were me I'd try a numerically smaller slot number>. I'm aware of SGI rack and slot assignment guidelines SGI provides, but this is strictly a rule-out-the-improbable exercise.

BTW - other than not being recognized by the L2, does 001c22 get linked to the rest of the system once everything is booted?

_________________
***********************************************************************
Welcome to ARMLand - 0/0x0d00
running...(sherwood-root 0607201829)
* InfiniteReality/Reality Software, IRIX 6.5 Release *
***********************************************************************


Top
 Profile  
 
Unread postPosted: Thu Jan 13, 2011 6:58 pm 
Offline
User avatar

Joined: Mon Sep 12, 2005 7:32 pm
Posts: 352
Location: New York, USA
I was seeing issues similar to that when the Dallas chip was failing in my R brick....The good news is that's a relatively easy fix if that's the case. Does the L1 on your R brick or anything else fail to come up once in awhile (throw an error rather than saying L1 Running). Are you running your O300 on 110V perchance? It may be coincidence but after that conversion is when all of this oddity started for me.

-Jesse

_________________
:Onyx2R: :O3x06R: :ChallengeL: :Onyx: :O2000: :PI: :Octane: :1600SW: :1600SW: :Indigo2IMP: :A350: :Indy: :Indy: :Indy: (4x Challenge S)


Top
 Profile  
 
Unread postPosted: Sat Jan 15, 2011 2:22 am 
Offline
User avatar

Joined: Sat May 29, 2004 1:28 pm
Posts: 28
Location: Switzerland
Hi,

This took a while since I had no time to play around with systems, but here is the report:

The bricks themselves work fine as standalone, they also work in twin config, connected directly by NUMALink Cable.
I have tried, without success following:

Updated the L1 and L2 firmware to the 1.48 code.
Changed brick slot code.

Jesse, I have never seen issue with L1 on those bricks and I use 220v (Europe).

_________________
ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage


Top
 Profile  
 
Unread postPosted: Sat Jan 15, 2011 5:52 am 
Offline
User avatar

Joined: Mon Sep 12, 2005 7:32 pm
Posts: 352
Location: New York, USA
Yeah it sounds like you could be chasing another gremlin, but it was worth asking the question.

So you were not able to upgrade the code or it didn't fix your issue?

-Jesse

_________________
:Onyx2R: :O3x06R: :ChallengeL: :Onyx: :O2000: :PI: :Octane: :1600SW: :1600SW: :Indigo2IMP: :A350: :Indy: :Indy: :Indy: (4x Challenge S)


Top
 Profile  
 
Unread postPosted: Sat Jan 15, 2011 7:26 am 
Offline
User avatar

Joined: Sat May 29, 2004 1:28 pm
Posts: 28
Location: Switzerland
Whops, not very clear message :)

I was able to both upgrade code and change brick slot but it had no effect at the initial issue :(

_________________
ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage


Top
 Profile  
 
Unread postPosted: Sat Jan 15, 2011 9:23 am 
Offline
User avatar

Joined: Tue Sep 14, 2004 2:03 am
Posts: 481
Location: Hampshire, UK
I suspect you have problem with the IO8 controller in the brick 001c22, not the router. Please swap the IO8 controller from one brick to the other and see if it moves with it.

There are USB controllers on the IO8 which do the communication of the L1 network which seem to have this problem.

If you can't replace the IO8, downgrade the L1 firmware on it and see if it go's away - I had a problem when running 1.44.0 firmware on a machine that I couldn't clear, but as a temporary workround, downgrading it to 1.32.6 I think fixed it, until I replaced the board.

Hope this helps.

tjsgifan.

_________________
In order of use at the moment..... :Fuel: :O3000:

Currently looking to buy good :Fuel: and :O2: :O2+: machine.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 14 posts ] 

All times are UTC - 8 hours


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group