Nekochan Net

Official Chat Channel: #nekochan // irc.nekochan.net
It is currently Tue Jul 22, 2014 11:48 pm

All times are UTC - 8 hours


Forum rules


Any posts concerning pirated software or offering to buy/sell/trade commercial software are subject to removal.



Post new topic Reply to topic  [ 59 posts ]  Go to page 1, 2, 3, 4  Next

Best tape archiver tool?
tar 56%  56%  [ 9 ]
cpio 6%  6%  [ 1 ]
xfsdump 38%  38%  [ 6 ]
Total votes : 16
Author Message
 Post subject: Tape drives
Unread postPosted: Sat Nov 03, 2012 9:31 pm 
Offline
User avatar

Joined: Wed Feb 19, 2003 1:54 pm
Posts: 975
It is said that you should spend as much in storage than in backups.

Some companies went completely out of business due to an inappropriate/in-existent backup process (notice: I said "process", not software or script). Today, I will review how I'm currently archiving "nearline" backups, but I will purposely not cover storage, software and tape inventory in this thread, simply hardware, the reason for sharing this is not simply that it could potentially benefit others, it is also that, in my own selfishness perhaps, I hope some might help me improve the current implementation.

As far as the software is concerned, it is supposed to fit one's needs; therefore the solution you adopt on any system should fit your own needs and retention policy; my needs are likely to be very different from yours; which is why I'm not going to give details on which software is being used; although I will give the following short overview:

Backups are handled in multiple ways here, nearline storage gets replicated remotely, and backups are performed on an "large SGI", we use Amanda for some servers, as it worked well for us in the past; but also some specific storage archival software (commercial) which works in adequately with our NFS and FCP servers (no, not ftp; fcp: fiber channel protocol).

The tape drive used is a HP LTO-4 (model 1760, mounted externally, scsi id #3, no terminator). For the sake of testing, I plugged this same drive on a few machines using few different cards and performed a few tests (using a LTO-4 tape), I will review the results I got, archiving a 45GB directory.

Test #1: External Tezro* #1 "IO9" Qlogic 12160, gnu tar 1.26, no compression.
Integral SCSI controller 1: Version QL12160, low voltage differential
Tape drive: unit 3 on SCSI controller 1: unknown
Time to tape: 31m14s (24MB/s).

Test #2: LSI 22320-R on Tezro* #1, gnu tar 1.26, no compression.
Integral SCSI controller 4: Version LS1030, low voltage differential
Tape drive: unit 3 on SCSI controller 4: unknown
Time to tape: 29m43s (25MB/s).

Test #3 (Irix control group): Tezro* #1, tar to local disk (on Qlogic 12160), gnu tar 1.26, no compression.
Integral SCSI controller 0: Version QL12160, low voltage differential
Disk drive: unit 1 on SCSI controller 0 (XVM Local Disk) (primary path)
Disk drive: unit 2 on SCSI controller 0 (XVM Local Disk) (primary path)
Time to tar: 33m36s (22MB/s)

Test #4: dd of the tarball created during test #3 to the tape drive, block size of 64k, tape drive on LSI 22320-R on Tezro* #1 (using Irix's dd).
Integral SCSI controller 4: Version LS1030, low voltage differential
Tape drive: unit 3 on SCSI controller 4: unknown
Time to dd: 19m07s (39MB/s)

Test #5 (for reference): Qlogic 12160 on Altix 350, gnu tar 1.23, no compression.
scsi 0:1:3:0: Sequential-Access HP Ultrium 4-SCSI W51D PQ: 0 ANSI: 5
scsi(0:1:3:0): Sync: period 9, offset 14, Wide, DT, Tagged queuing: depth 31
Time to tape: 24m43s (30MB/s)

Test #6 (Linux control group): tar to local disk, Qlogic 12160 on Altix 350, gnu tar 1.23, no compression.
ata1.00: ATA-8: WDC WD5002ABYS-02B1B0, 02.03B03, max UDMA/133
scsi 1:0:0:0: Direct-Access ATA WDC WD5002ABYS-0 02.0 PQ: 0 ANSI: 5
Time to tar: 26m59s (28MB/s)

Test #7: LSI 22320-R on Tezro* #1, xfsdump to tape.
Integral SCSI controller 4: Version LS1030, low voltage differential
Tape drive: unit 3 on SCSI controller 4: unknown
Time to tape: 20m01s (37MB/s)

Test #8: External Tezro** #2 "IO9" Qlogic 12160, gnu tar 1.26, no compression.
Integral SCSI controller 1: Version QL12160, low voltage differential
Tape drive: unit 3 on SCSI controller 1: unknown
Time to tape: 31m30s (24MB/s)

Test #9: External Tezro** #2 "IO9" Qlogic 12160, xfsdump to tape.
Integral SCSI controller 1: Version QL12160, low voltage differential
Tape drive: unit 3 on SCSI controller 1: unknown
Time to tape: 30m31s (24.5MB/s)

Test #10: External Tezro** #2 "IO9" Qlogic 12160, cpio (using Irix's cpio).
Integral SCSI controller 1: Version QL12160, low voltage differential
Tape drive: unit 3 on SCSI controller 1: unknown
Time to tape: 29m11s (25.5MB/s)

The cpu load on the tezro was 100% on only one cpu during tar (~10% on the altix, with 2% IOwait), so I think this is the current bottleneck. Tar (at least the GNU implementation) can not be threaded, however, thanks to hundreds of thousands of users using it (on different architectures and operating systems), it is reliable, a few efforts to replace GNU tar have been done already and it seems that all failed so far.

During the xfsdump test, the load on the tezro was shared amongst processors, which explains the discrepancy; nevertheless, less people run xfs (and xfsdump) than tar, so I'm not completely sure this would be more reliable than tar.

Regarding APD (asynchronous personality daemon), the tests were done on a tezro with apd installed, but disabled. For consistency, test #8 was done on a tezro where apd has never been installed.

I welcome comments, thoughts, spare tapes and concerns.

Notes:
(*) tezro #1 where apd has been installed but disabled.
(**) tezro #2 where apd has never been installed, identical hardware as tezro #1, except drives: tezro #1 has a pair of 300GB seagate and tezro #2 has a pair of 300GB HP/Compaq drives; which are a little slower than their seagate counterpart; the IOwait is noticeably higher on tezro #2 than #1 during archival.
gnu tar options: tar cvf /dev/rmt/tapedevice /path/to/directory
xfsdump options: xfsdump -o -F -l0 -J -f /dev/rmt/tapedevice -L 20121104 -s path/to/directory /
cpio options: find /path/to/directory -print | cpio -oKv > /dev/rmt/tapedevice
_______________________________________________________________________________________________________
Edit (Sat Nov 3 22:58:37 PDT 2012): added tests #6 and #7, comments on xfsdump.
Edit (Sun Nov 4 10:20:41 PST 2012): added tests #8, #9, #10, comments on apd, backup arguments used.
_______________________________________________________________________________________________________

_________________
:Onyx2:


Last edited by mia on Sun Nov 04, 2012 12:46 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Tape drives
Unread postPosted: Sun Nov 04, 2012 3:16 am 
Offline

Joined: Tue Feb 24, 2004 4:10 pm
Posts: 9444
Thank you for the valuable information (xfsdump for me) .... and the kick in the ass reminder to do backups !!!


Top
 Profile  
 
 Post subject: Re: Tape drives
Unread postPosted: Sun Nov 04, 2012 5:09 am 
Offline

Joined: Sat Nov 12, 2011 3:18 am
Posts: 330
Location: Tokyo
Is that with SGI APD or without ?

Thanks

_________________
[click for links to hinv] JP: :Fuel: |:Octane2: |:O2: | :Indy: || PL: [ :Fuel: :O2: :O2+: :Indy: ]


Top
 Profile  
 
 Post subject: Re: Tape drives
Unread postPosted: Sun Nov 04, 2012 6:08 am 
Offline
Moderator
Moderator
User avatar

Joined: Sun Jun 06, 2004 4:55 pm
Posts: 5190
Location: NC - USA
Thanks for sharing - that's very useful info.
mia wrote:
My conclusions on the results above are as follows: 37MB/s is bearable, if not satisfactory.
Be interesting to see how a fiber channel connected tape drive would fit into those results. An FC connection might also allow the use of a fiber channel switch to share a single tape drive between multiple systems.

To test both of those ideas I recently acquired (but haven't yet tested) a 1GB FC-to-SCSI bridge from the Keeper. That one is a bare PCB I plan to install in a 2U tape drive enclosure, but FC-to-SCSI bridges can be had in free-standing rackmount enclosures. I haven't done any research into compatibility or function, so solely as an illustration of the type of device I'm describing, here's an example.

kubatyszko wrote:
Is that with SGI APD or without ?
Good question. I've speculated that APD might have been at least partially a marketing tool, so it would be interesting to see the results of testing with and without APD.

_________________
***********************************************************************
Welcome to ARMLand - 0/0x0d00
running...(sherwood-root 0607201829)
* InfiniteReality/Reality Software, IRIX 6.5 Release *
***********************************************************************


Top
 Profile  
 
 Post subject: Re: Tape drives
Unread postPosted: Sun Nov 04, 2012 8:30 am 
Offline
User avatar

Joined: Wed Feb 19, 2003 1:54 pm
Posts: 975
I would like to move to LTO-5 or LTO-6, although, external or rackmount (no library) FC drives are difficult to find, even LTO-4 ones; it's all SAS now. Note that I have APD installed but disabled. In any case, I don't think APD supports LTO-4 drives.

_________________
:Onyx2:


Top
 Profile  
 
 Post subject: Re: Tape drives
Unread postPosted: Sun Nov 04, 2012 12:47 pm 
Offline
User avatar

Joined: Wed Feb 19, 2003 1:54 pm
Posts: 975
added more tests to my original post.

_________________
:Onyx2:


Top
 Profile  
 
 Post subject: Re: Tape drives
Unread postPosted: Wed Nov 07, 2012 2:06 am 
Offline
User avatar

Joined: Fri Apr 01, 2011 6:45 am
Posts: 71
I'm also using a HP LTO-4 (model 1760, mounted externally, scsi id #5, no terminator) with my tezro. The entry for it in my /var/sysgen/master.d/scsi looks like that:

Code:
 /* HP LTO4 / Ultrium 4 */
   { DATTAPE, TPDAT, 2, 9, "HP", "Ultrium 4", 0, 0, {0},
          MTCAN_BSF|MTCAN_BSR|MTCAN_APPEND|MTCAN_SETMK|MTCAN_PART|MTCAN_PREV|
          MTCAN_SYNC|MTCAN_SPEOD|MTCAN_CHKRDY|MTCAN_VAR|MTCAN_SETSZ|
          MTCAN_SILI|MTCAN_AUDIO|MTCAN_SEEK|MTCAN_CHTYPEANY|MTCAN_COMPRESS,
          40, 5*60, 20*60, 3*3600, 512, 256*512, 512*512,
          tpsc_default_dens_count, tpsc_default_hwg_dens_names,
          tpsc_default_alias_dens_names,
          {0}, 0, 0, 0,
          0, (u_char *)0 },


mt status says:

Code:
tezro 1#mt status
        Controller: SCSI
        Device: HP: Ultrium 4-SCSI  W54D
        Status: 0x20262
        Drive type: DAT
        Media : READY, writable, at BOT


Top
 Profile  
 
 Post subject: Re: Tape drives
Unread postPosted: Wed Nov 07, 2012 9:27 am 
Offline
Moderator
Moderator
User avatar

Joined: Sun Jun 06, 2004 4:55 pm
Posts: 5190
Location: NC - USA
hhoffman wrote:
I'm also using a HP LTO-4 (model 1760, mounted externally, scsi id #5, no terminator) with my tezro. The entry for it in my /var/sysgen/master.d/scsi looks like that:

Thanks! I added your SCSI definition file to those listed in the LTO Anyone thread. Can I ask if your LTO-4 definition file originated with HP?

_________________
***********************************************************************
Welcome to ARMLand - 0/0x0d00
running...(sherwood-root 0607201829)
* InfiniteReality/Reality Software, IRIX 6.5 Release *
***********************************************************************


Top
 Profile  
 
 Post subject: Re: Tape drives
Unread postPosted: Wed Nov 07, 2012 10:16 am 
Offline
User avatar

Joined: Wed Feb 19, 2003 1:54 pm
Posts: 975
Amazingly, this provided file doesn't work with my 1760:
Code:
root@plum:~# tar cvf /dev/rmt/tps4d3 /tmp/tape_benchmark
tar: /dev/rmt/tps4d3: Cannot write: Invalid argument


This doesn't seem to be a tar issue:
Code:
root@plum:~# dd if=/dev/zero of=/dev/rmt/tps4d3
Write error: Invalid argument


However, changing this:
DATTAPE, TPDAT, 2, 9, "HP", "Ultrium 4", 0, 0, {0},
[...no changes here...]
40, 5*60, 20*60, 3*3600, 512, 256*512, 512*512,

to this:

TPUNKNOWN, TPUNKNOWN, 2, 9, "HP", "Ultrium 4", 0, 0, {0},
[...no changes here...]
40, 5*60, 20*60, 20*60, 3*3600, 512, 512*512,


Makes it work, why?

This is what "mt blksize" reports with hhoffman's definition file:
Code:
 Recommended tape I/O size: 262144 bytes (512 512-byte blocks)                 
 Minimum block size: 1 byte(s)                                                 
 Maximum block size: 16777215 bytes                                           
 Current block size: Variable

This is with the one I use (and works):
Code:
 Recommended tape I/O size: 262144 bytes (512 512-byte blocks)
 Minimum block size: 1 byte(s)
 Maximum block size: 16777215 bytes
 Current block size: 512 byte(s)

My complete definition file:
Code:
         { TPUNKNOWN, TPUNKNOWN, 2, 9, "HP", "Ultrium 4", 0, 0, {0},
           MTCAN_BSF|MTCAN_BSR|MTCAN_APPEND|MTCAN_SETMK|MTCAN_PART|MTCAN_PREV|
             MTCAN_SYNC|MTCAN_SPEOD|MTCAN_CHKRDY|MTCAN_VAR|MTCAN_SETSZ|
             MTCAN_SILI|MTCAN_AUDIO|MTCAN_SEEK|MTCAN_CHTYPEANY|MTCAN_COMPRESS,
           40, 5*60, 20*60, 20*60, 3*3600, 512, 512*512,
           tpsc_default_dens_count, tpsc_default_hwg_dens_names,
             tpsc_default_alias_dens_names,
           {0}, 0, 0, 0,
           0, (u_char *)0 },

_________________
:Onyx2:


Top
 Profile  
 
 Post subject: Re: Tape drives
Unread postPosted: Wed Nov 07, 2012 11:18 am 
Offline
Moderator
Moderator
User avatar

Joined: Sun Jun 06, 2004 4:55 pm
Posts: 5190
Location: NC - USA
mia wrote:
Code:
TPUNKNOWN, TPUNKNOWN, 2, 9, "HP", "Ultrium 4", 0, 0, {0},
My guess is the edits to cause LTO to be identified as TPUNKNOWN are going to cause more than your share of headaches.

mia wrote:
Amazingly, this provided file doesn't work with my 1760

Is this with the Tezro that has APD installed, or the one you mentioned that didn't have APD?

If you haven't, I'd suggest trying it on the non-APD Tezro, and if possible, with a *copy* of the original unedited /var/sysgen/master.d/scsi.

If you have a copy of the original unedited /var/sysgen/master.d/scsi
  • restore a *copy* of the original unedited master.d/scsi;
  • as root run autoconfig;
  • reboot with the LTO-4 attached and running;
  • as root, run ./MAKEDEV tape. followed by mt status;
  • double check that the "device" line in mt status matches the one in hhoffman's mt status (the firmware revision "W54D" might be different);
  • paste a copy of hhofman's LTO-4 device file into the copy of master.d/scsi and save the change;
  • re-run autoconfig and reboot again;
  • with the LTO-4 still attached and running, re-run ./MAKEDEV tape;
  • then run mt status and post the results here.
If your mt status matches the one hhoffman posted, I'd suggest first testing the drive by running a small back up via the IRIX back up GUI. If that works then try tar.

_________________
***********************************************************************
Welcome to ARMLand - 0/0x0d00
running...(sherwood-root 0607201829)
* InfiniteReality/Reality Software, IRIX 6.5 Release *
***********************************************************************


Top
 Profile  
 
 Post subject: Re: Tape drives
Unread postPosted: Wed Nov 07, 2012 6:19 pm 
Offline
User avatar

Joined: Wed Feb 19, 2003 1:54 pm
Posts: 975
Recondas, Hhoffman,

I'm as surprised and confused by this discrepancy; I have removed APD on this machine; yet I am unsure why I do not get identical results, similarily I get identical results on another Tezro, but this could be related to the fact that both (and the others) were installed from the same Irix netinstall.

Recondas, I have followed your procedure, to the same outcome.

Hhoffman, which throughput do you get, when writing to a tape, either with tar or xfsdump (or dd, really), and which scsi card are you using?

My mt status reports this following firmware: Ultrium 4-SCSI W51D, while Hhoffman is running W54D. This is the only possible explanation at this time.

Please note that the latest firmware for this drive is: W62D.

_________________
:Onyx2:


Top
 Profile  
 
 Post subject: Re: Tape drives
Unread postPosted: Wed Nov 07, 2012 8:01 pm 
Offline
User avatar

Joined: Wed Feb 19, 2003 1:54 pm
Posts: 975
I've upgraded the drive's firmware to W62D, which is the latest firmware available for the 1760; and the repeated the test on a very fast Linux/amd64 machine; the results are astounding: 19m04s (39MB/s) for a tar archive of this same directory (which was placed on very fast storage).

I have redone the "tar" test using this new firmware, on the Tezro, and was able to do it in 24m00s (31 MB/s) which is really good, so I'm fairly certain the firmware changed a few things, so I'm going to try different scsi parameters now.

Stay tuned.

_________________
:Onyx2:


Top
 Profile  
 
 Post subject: Re: Tape drives
Unread postPosted: Wed Nov 07, 2012 8:52 pm 
Offline
User avatar

Joined: Wed Feb 19, 2003 1:54 pm
Posts: 975
I've repeated the tests, this time with a different tar blocking factor, as such:

Quote:
timex tar cvb 4096 -f /dev/rmt/tps4d3 /path/to/files


Tezro: 15m53s (47MB/s)
"Very fast" Intel*: 19m04s (39MB/s)

Now I'm happy. I'll run multiple tests overnight to collect some statistics with different blocking factors; this is really much better, now I can write a 800GB tape in less than 5 hours. The payload written to tape is hardly compressible, so I'm not going to bother trying compression on this dataset, this will be the subject of another test.

(*) 8 cpus, pci-e scsi card, tons of ram, raid-6, a disgusting marvel of technology; yet beaten by a Tezro.

_________________
:Onyx2:


Top
 Profile  
 
 Post subject: Re: Tape drives
Unread postPosted: Thu Nov 08, 2012 8:56 am 
Offline
User avatar

Joined: Fri Apr 01, 2011 6:45 am
Posts: 71
Quote:
Can I ask if your LTO-4 definition file originated with HP?

Yes of course. I tried to get support from hp for the unit with IRIX, but they where not willing to help. My version for the SCSI file simply came out of the LTO 3 version in the 'LTO Anyone thread'.
Anyway, with my scsi file entry, I noticed some issues with tar (original IRIX version) and mt. But cpio (with the IRIX backup interface) and gnu tar worked fine. With mia's version, these issues disappeared.


Top
 Profile  
 
 Post subject: Re: Tape drives
Unread postPosted: Thu Nov 08, 2012 8:58 am 
Offline
User avatar

Joined: Wed Feb 19, 2003 1:54 pm
Posts: 975
Hhoffman,

Glad it helped.

Could you tell us which throughput you currently get with tar and/or cpio/xfsdump? Finally, could you describe those tar & mt issues?

Thanks!

_________________
:Onyx2:


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 59 posts ]  Go to page 1, 2, 3, 4  Next

All times are UTC - 8 hours


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group