Nekochan Net

Official Chat Channel: #nekochan // irc.nekochan.net
It is currently Tue Sep 07, 2010 7:46 am

All times are UTC - 8 hours [ DST ]


Forum rules


Any posts concerning pirated software or offering to buy/sell/trade commercial software are subject to removal.



Post new topic Reply to topic  [ 13 posts ] 
Author Message
 Post subject: Linux / IRIX combination makes my file system go bonk...
PostPosted: Mon Jan 25, 2010 9:46 am 
Offline

Joined: Mon Jan 04, 2010 3:58 pm
Posts: 61
Location: New York - But Not That New York
I've mentioned in earlier posts in the hardware forum work I am doing on two Altix clusters (running SLES 9). To be brief the older cluster has 48gb ram and a TP900 with 7x 146gb storage. The newer one has 64gb ram and 16x 146gb storage (in two TP900s). I started copying home directories from the old array to one of the new arrays recently after building them into RAID-5 using mdadm.

I noticed that the home directories were taking a very long time, and using a fair bit more space than they used before. At first I thought the space difference might just be a file cluster size issue, so I didn't worry about it too much. Then eventually the home directories (which did not finish copying) became enormous - some 10 times their original size.

A little investigation lead me to what was causing this.

It appears that every user on the system who has logged in to an SGI workstation in person has a directory "Desktop" in their home. In that Desktop directory is a symlink that is named the same as their user name - hence if I had a user "Bob Smith" called "smithb", his home directory would be /home/smithb. Furthermore the problem link would be /home/smithb/Desktop/smithb, pointing at /home/smithb.

I was originally using scp to copy the files from one cluster to another, and I specifically chose the "-r" option for recursive copying. Unfortunately, scp would traverse the directories to the problem link, which lead it back to the home directory. Hence pretty soon poor Mr. Smith's home directory had directories like /home/smithb/Desktop/smithb/Desktop/smithb/Desktop/smithb/Desktop ... and so on. I never bothered to figure out just how many levels down it went before I killed the scp jobs.

I have since killed off those symlinks; at least for the users whose home directories I plan to copy any time soon. It does appear that IRIX will re-create those links whenever needed so I don't think we're losing anything by unlinking them.

_________________
Current flock, after the Altix 3000 and Altix 350:
:Fuel: :Fuel: :Fuel: :Fuel: :Octane: :Octane: :O2: :Octane2:


Top
 Profile E-mail  
 
 Post subject: Re: Linux / IRIX combination makes my file system go bonk...
PostPosted: Mon Jan 25, 2010 10:10 am 
Offline
Moderator
Moderator
User avatar

Joined: Mon Jun 06, 2005 9:53 pm
Posts: 2236
Location: USA
Good reminder about scp. Scp ain't Unix. :D There's a general scp issue when using the -r option and encountering symbolic links.

Assuming you have enough disk space on the original system, I would have kicked everyone off the machine (single user mode if possible) and made a tarball of the whole /home directory and then copied that over. Copying a single large file is much more efficient with respect to network overhead than copying hundreds or thousands of files.

Alternately, assuming that everything is on a local, trusted net, you could have kicked everyone off the systems, mounted via NFS, and done a direct tar create/extract as follows (executed from the old machine, with the new machine's /home NFS mounted on the old machine as /newhome):

(cd /home ; tar cvf . - ) | (cd /newhome ; tar xvf - )

You could also do the same type of thing with xfsdump, cpio, etc., modifying the command lines accordingly.


Top
 Profile  
 
 Post subject: Re: Linux / IRIX combination makes my file system go bonk...
PostPosted: Mon Jan 25, 2010 10:12 am 
Offline
Site Admin
Site Admin
User avatar

Joined: Thu Jan 23, 2003 2:31 am
Posts: 7629
Location: Pleasanton, California
josehill wrote:
Assuming you have enough disk space on the original system, I would have kicked everyone off the machine (single user mode if possible) and made a tarball of the whole /home directory and then copied that over.


That's the same method I used when migrating Nekochan from IRIX to FreeBSD - rolled tarballs of directories I needed to move and sftp'd them over afterwards. If space is tight, compressing with gzip/bzip may help.

_________________
私のホバークラフト は鰻が一杯です。
IRIX Release 4.0.5 IP12 Version 06151813 System V
Copyright 1987-1992 Silicon Graphics, Inc.
All Rights Reserved.


Top
 Profile  
 
 Post subject: Re: Linux / IRIX combination makes my file system go bonk...
PostPosted: Mon Jan 25, 2010 5:55 pm 
Online
User avatar

Joined: Sat May 20, 2006 4:52 pm
Posts: 355
Location: Champaign, IL
I usually use rsync when copying lots of files over. If you specify the "-a" flag it not only preserves user, group, owner and time, but also copies symlinks as symlinks instead of following them. There are lots of other useful flags, which the man page of course explains :-)

_________________
:O3200: :O2000: :Octane2: :Octane: :O2: :Indigo2IMP: :PI: :1600SW: And gobs of other hardware...


Top
 Profile E-mail  
 
 Post subject: Re: Linux / IRIX combination makes my file system go bonk...
PostPosted: Mon Jan 25, 2010 6:53 pm 
Offline
User avatar

Joined: Thu Nov 08, 2007 5:35 pm
Posts: 545
Location: Lynnwood, WA
If available, I like fs dump/restore utilities such as xfsdump, ufsdump, zfs send, etc. You can send the data across a network connection or create a file and restore from the file.

_________________
FS: :O2000: :O200: :Fuel: :Indigo: :Octane2:

:O3000: :O200: :Fuel: :Fuel: :Indigo: :Indigo: :Octane: :O2: :1600SW: :Indigo2: :Indigo2: :Indigo2: :Indigo2IMP: :Indy: :Indy: <--challenge S


Top
 Profile E-mail  
 
 Post subject: Re: Linux / IRIX combination makes my file system go bonk...
PostPosted: Tue Jan 26, 2010 1:49 am 
Offline
User avatar

Joined: Fri Oct 09, 2009 2:44 am
Posts: 47
Location: Toulouse, southern France
josehill wrote:
Good reminder about scp. Scp ain't Unix. :D


Ahem. scp is the rcp protocol over an ssh connection, to the letter. So while it's a shame rcp is quite limited, I wouldn't pretend scp isn't unixy...

_________________
:Indigo: :Indigo2: :Indigo2IMP: :Indy: :Indy: :Indy: :O2: :O2: :Octane: :Octane: :O200: - :O200: :Fuel: among ~150 machines...
Looking for IP26 and IP28...


Top
 Profile E-mail  
 
 Post subject: Re: Linux / IRIX combination makes my file system go bonk...
PostPosted: Tue Jan 26, 2010 11:29 am 
Offline
Moderator
Moderator
User avatar

Joined: Mon Jun 06, 2005 9:53 pm
Posts: 2236
Location: USA
miod wrote:
josehill wrote:
Good reminder about scp. Scp ain't Unix. :D


Ahem. scp is the rcp protocol over an ssh connection, to the letter. So while it's a shame rcp is quite limited, I wouldn't pretend scp isn't unixy...

Ahem. I didn't say that it wasn't unixy, just that it wasn't Unix. ;)


Top
 Profile  
 
 Post subject: Re: Linux / IRIX combination makes my file system go bonk...
PostPosted: Tue Jan 26, 2010 1:27 pm 
Offline
User avatar

Joined: Sat Nov 10, 2007 4:31 am
Posts: 156
Location: Durham/Newcastle, UK
For system to system, rsync is the best tool (if available), as you can always Control-C it if necessary and restart from where you left off (the -a option covers most bases). Otherwise, you can use the basic ssh and tar commands to transfer data across the network, preserving permissions, groups, devices and links... something like:

tar cf - /source | ssh user@host "cd /dest ; tar xf -"

_________________
:Indigo2IMP: [ R10k-195/MaxImpact +TRAMs/896Mb/73Gb-15krpm/G160/6.5.22 ]
:Octane2: [ Dual R12k-400/V8/2Gb/73Gb-15krpm/6.5.29 ]


Top
 Profile E-mail  
 
 Post subject: Re: Linux / IRIX combination makes my file system go bonk...
PostPosted: Wed Jan 27, 2010 12:30 am 
Offline
Moderator
Moderator
User avatar

Joined: Sun Mar 30, 2003 5:29 am
Posts: 2384
Location: Kabul, Afghanistan, Asia
I found tarballing stuff, while inconvenient, is easiest. Everything else is prone to hick-ups. Even rsync (or especially?). Then again, I don't do heterogenous anymore. I mostly just use OpenSolaris and a bit of Win7, which gets stuff pushed to it via a FAT partition.

_________________
覇気元
Eroteme.org


Top
 Profile  
 
 Post subject: Re: Linux / IRIX combination makes my file system go bonk...
PostPosted: Thu Jan 28, 2010 12:09 pm 
Offline

Joined: Mon Jan 04, 2010 3:58 pm
Posts: 61
Location: New York - But Not That New York
josehill wrote:

Alternately, assuming that everything is on a local, trusted net, you could have kicked everyone off the systems, mounted via NFS, and done a direct tar create/extract as follows (executed from the old machine, with the new machine's /home NFS mounted on the old machine as /newhome):


We are on a very local, very trusted, very small, network here. While I often criticize the network admins above me for their heavy-handed firewall rules, it does keep out the riff-raff that could interfere with my work. And in this case not only are the two clusters on the same network, they are in the same room (closet, actually, but that is another story) and hooked up to the same switch.

So I decided to go ahead with the NFS option. I mounted the NFS shares from the old cluster on to the new cluster and started copying away. In this case I was only looking to copy two home directories - my own and my boss's - and we were both logged out at the time. It took a while since the directories were so large, but it seems pretty content now. I've even rebooted the new cluster a few times just to make sure it would be content and things look good.

_________________
Current flock, after the Altix 3000 and Altix 350:
:Fuel: :Fuel: :Fuel: :Fuel: :Octane: :Octane: :O2: :Octane2:


Top
 Profile E-mail  
 
 Post subject: Re: Linux / IRIX combination makes my file system go bonk...
PostPosted: Sat Jan 30, 2010 5:36 am 
Offline
User avatar

Joined: Wed Dec 07, 2005 5:46 pm
Posts: 595
Location: Overmere, Belgium
nekonoko wrote:
That's the same method I used when migrating Nekochan from IRIX to FreeBSD


Eh? Wuh? Wha? Gah? :shock:

_________________
while (!asleep()) sheep++;


Top
 Profile  
 
 Post subject: Re: Linux / IRIX combination makes my file system go bonk...
PostPosted: Sat Jan 30, 2010 8:03 am 
Offline
Moderator
Moderator
User avatar

Joined: Mon Jun 06, 2005 9:53 pm
Posts: 2236
Location: USA
Alver wrote:
Eh? Wuh? Wha? Gah? :shock:

I guess you haven't been paying attention! Check out viewtopic.php?f=2&t=16722649


Top
 Profile  
 
 Post subject: Re: Linux / IRIX combination makes my file system go bonk...
PostPosted: Mon Feb 08, 2010 4:35 am 
Offline
User avatar

Joined: Wed Dec 07, 2005 5:46 pm
Posts: 595
Location: Overmere, Belgium
josehill wrote:
Alver wrote:
Eh? Wuh? Wha? Gah? :shock:

I guess you haven't been paying attention! Check out viewtopic.php?f=2&t=16722649

Yeah, haven't been around much lately. Somehow I was convinced that nekochan itself was still running on a production IRIX system...

_________________
while (!asleep()) sheep++;


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 13 posts ] 

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group