Octane2/Onyx350/4 - ACARD ARS-2160H/ARS-2320S with Samsung SSD

SGI hardware problems, solutions, tips, hacks, etc.
Forum rules
Any posts concerning pirated software or offering to buy/sell/trade commercial software are subject to removal.
jwhat
Posts: 317
Joined: Sat Aug 09, 2003 6:25 pm
Location: Australia

Octane2/Onyx350/4 - ACARD ARS-2160H/ARS-2320S with Samsung SSD

Unread postby jwhat » Wed Dec 06, 2017 3:02 am

Hi Performance Testers,

quite some time ago I swapped out SCSI HD from Octane2 and replaced it with ACARD ARS-2160H with SAMSUNG SSD (EVO 840 250GB).

Since I got Onyx350/4 boxes I got a couple of additional ACARD ARS-2320S adaptors and put SAMSUNG SSD EVO 850 1TB in each of these and also updated EVO 840 to same.

My motivation to move where two fold:
1. Better performance
2. Loss avoidance of my important IRIX software collection

I never really did any performance testing of ACARD/SSD combination, so thought it would be worth while to see results:

Firstly some original benchmarks:

From Dr. Dave DAE-R thread sometime ago (on accelerated 425Hz Octane): viewtopic.php?f=3&t=5820&p=57892&hilit=diskperf+dae+r#p57892

Code: Select all

DeepThought 6# diskperf -D -W -n "Root Disk" -c512m /dmconvert/test
#---------------------------------------------------------
# Disk Performance Test Results Generated By Diskperf V1.2
#
# Test name     : Root Disk
# Test date     : Sun May  1 03:05:12 2005
# Test machine  : IRIX64 DeepThought 6.5 07080050 IP30
# Test type     : XFS data subvolume
# Test path     : /dmconvert/test
# Request sizes : min=16384 max=4194304
# Parameters    : direct=1 time=10 scale=1.000 delay=0.000
# XFS file size : 536870912 bytes
#---------------------------------------------------------
# req_size  fwd_wt  fwd_rd  bwd_wt  bwd_rd  rnd_wt  rnd_rd
#  (bytes)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)
#---------------------------------------------------------
      16384   13.36   13.57    7.58    3.01    6.63    2.46
      32768   16.59   15.68   10.68    6.08   10.59    4.17
      65536   19.51   18.63   13.05    7.95   14.53    6.77
     131072   20.44   19.75   18.59   10.61   16.80   11.43
     262144   24.59   21.79   20.32   15.90   20.26   14.54
     524288   25.64   23.70   24.38   19.68   24.13   19.10
    1048576   30.17   27.81   34.91   30.44   29.78   26.41
    2097152   33.96   31.15   33.13   28.95   33.13   28.99
    4194304   36.03   32.97   35.54   31.69   35.53   31.81
DeepThought 7#


And same test on Octane2 2x600HZ with ACARD ARS-2160H+Samsung EVO 850 SATA SSD:

Code: Select all

porcipine 6# diskperf -D -W -n "Samsung SSD via ACARD ARS-2160H" -c512m test512
#---------------------------------------------------------
# Disk Performance Test Results Generated By Diskperf V1.2
#
# Test name     : Samsung SSD via ACARD ARS-2160H
# Test date     : Wed Dec  6 20:53:20 2017
# Test machine  : IRIX64 porcipine 6.5 01090133 IP30
# Test type     : XFS data subvolume
# Test path     : test512
# Request sizes : min=16384 max=4194304
# Parameters    : direct=1 time=10 scale=1.000 delay=0.000
# XFS file size : 536870912 bytes
#---------------------------------------------------------
# req_size  fwd_wt  fwd_rd  bwd_wt  bwd_rd  rnd_wt  rnd_rd
#  (bytes)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)
#---------------------------------------------------------
      16384   23.72   20.60   23.68   19.23   23.65   19.06
      32768   29.40   24.97   29.43   23.81   29.27   23.78
      65536   32.97   28.98   33.00   28.25   32.95   28.18
     131072   35.42   30.35   35.45   29.63   35.45   29.59
     262144   36.97   31.18   36.95   30.42   36.97   30.33
     524288   37.67   31.46   37.62   30.65   37.68   30.64
    1048576   38.03   31.42   38.04   30.84   38.04   30.83
    2097152   38.23   31.01   38.23   30.73   38.21   30.70
    4194304   38.31   30.06   38.32   29.93   38.30   29.92


Summary - top throughput is similar and pretty well saturates 40MB/sec SCSI bus of Octane but SSD performance is much flatter and consistent across the overall set.

And here is very slow SCSI disk result from some time back... (same thread as above)

Code: Select all

# diskperf -W -D -n "SCSI Disk" -c4m /usr/people/devel/tmp/test
#---------------------------------------------------------
# Disk Performance Test Results Generated By Diskperf V1.2
#
# Test name : SCSI Disk
# Test date : Sat Oct 1 21:54:15 2005
# Test machine : IRIX64 porcipine 6.5 01062343 IP30
# Test type : XFS data subvolume
# Test path : /usr/people/devel/tmp/test
# Request sizes : min=16384 max=4194304
# Parameters : direct=1 time=10 scale=1.000 delay=0.000
# XFS file size : 4194304 bytes
#---------------------------------------------------------
# req_size fwd_wt fwd_rd bwd_wt bwd_rd rnd_wt rnd_rd
# (bytes) (MB/s) (MB/s) (MB/s) (MB/s) (MB/s) (MB/s)
#---------------------------------------------------------
16384 2.22 18.99 3.10 3.14 2.63 3.00
32768 4.07 19.20 6.86 7.47 4.59 5.13
65536 6.70 19.11 7.43 7.57 7.27 7.02
131072 9.78 19.14 11.22 31.56 8.44 10.30
262144 9.61 18.96 7.44 15.25 5.99 13.94
524288 11.21 19.13 7.60 16.13 6.43 16.06
1048576 12.53 19.17 5.93 16.82 6.22 17.55
2097152 12.91 19.24 6.67 17.46 7.09 17.84
4194304 13.65 19.06 0.00 0.00 6.46 18.35


Same test with Octane2 2x600MHz + ACARD ARS-2160H+Samsung EVO 850 1TB

Code: Select all

porcipine 7# diskperf -W -D -n "Samsung SSD via ACARD ARS-2160H" -c4m test4m
#---------------------------------------------------------
# Disk Performance Test Results Generated By Diskperf V1.2
#
# Test name     : Samsung SSD via ACARD ARS-2160H
# Test date     : Wed Dec  6 21:36:50 2017
# Test machine  : IRIX64 porcipine 6.5 01090133 IP30
# Test type     : XFS data subvolume
# Test path     : test4m
# Request sizes : min=16384 max=4194304
# Parameters    : direct=1 time=10 scale=1.000 delay=0.000
# XFS file size : 4194304 bytes
#---------------------------------------------------------
# req_size  fwd_wt  fwd_rd  bwd_wt  bwd_rd  rnd_wt  rnd_rd
#  (bytes)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)
#---------------------------------------------------------
      16384   20.92   20.64   23.74   18.84   23.60   19.20
      32768   28.68   25.00   29.55   23.97   29.41   23.94
      65536   32.86   28.96   33.07   28.29   32.97   28.24
     131072   35.38   30.47   35.51   29.70   35.48   29.68
     262144   37.00   31.29   37.02   30.45   36.97   30.42
     524288   37.67   31.55   37.71   30.71   37.71   30.78
    1048576   38.04   31.45   38.05   30.80   38.05   30.89
    2097152   38.23   31.04   38.23   30.74   38.22   30.70
    4194304   38.32   30.11    0.00    0.00   38.33   29.93


And here are internal Onyx350/4 (4x1GHz+4x800MHz) SCSI which is Ultra 160 (I believe) with ARS-2320S ... so aiming to get something like 120MB/sec...

Code: Select all

dubious 8# diskperf -W -D -n "Samsung SSD via ACARD ARS-2320S" -c4m test4m
#---------------------------------------------------------
# Disk Performance Test Results Generated By Diskperf V1.2
#
# Test name     : Samsung SSD via ACARD ARS-2320S
# Test date     : Wed Dec  6 21:50:21 2017
# Test machine  : IRIX64 dubious 6.5 01090133 IP35
# Test type     : XFS data subvolume
# Test path     : test4m
# Request sizes : min=16384 max=4194304
# Parameters    : direct=1 time=10 scale=1.000 delay=0.000
# XFS file size : 4194304 bytes
#---------------------------------------------------------
# req_size  fwd_wt  fwd_rd  bwd_wt  bwd_rd  rnd_wt  rnd_rd
#  (bytes)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)
#---------------------------------------------------------
      16384   37.62   43.30   43.31   38.01   43.05   37.59
      32768   60.14   60.25   62.72   54.57   62.74   54.50
      65536   78.97   85.92   80.79   79.40   80.69   79.69
     131072   90.62   95.47   91.30   87.67   91.14   87.05
     262144   97.69  101.54   98.15   93.01   98.07   93.05
     524288  101.72  104.32  101.64   89.70  101.77   95.98
    1048576  102.83  104.41  102.77   97.80  102.76   98.91
    2097152  100.65  101.71  100.50   98.50  100.62   98.44
    4194304   95.25   95.97    0.00    0.00   95.22   94.50


I do not have historical Onyx350/4 HD results but for purpose of comparison, Ian Mapleson posted some Ultra 160 (on Fuel which is Chimera based) results here: viewtopic.php?f=3&t=14305

With this set of numbers...

Code: Select all

root@Mai /0 # diskperf -W -D -n ultra160 -t10 -c100m /tmp/test
#---------------------------------------------------------
# Disk Performance Test Results Generated By Diskperf V1.2
#
# Test name     : ultra160
# Test date     : Mon May 21 10:38:00 2007
# Test machine  : IRIX64 Mai 6.5 07202013 IP35
# Test type     : XFS data subvolume
# Test path     : /tmp/test
# Request sizes : min=16384 max=4194304
# Parameters    : direct=1 time=10 scale=1.000 delay=0.000
# XFS file size : 104857600 bytes
#---------------------------------------------------------
# req_size  fwd_wt  fwd_rd  bwd_wt  bwd_rd  rnd_wt  rnd_rd
#  (bytes)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)
#---------------------------------------------------------
      16384   22.42   53.30   12.93   18.94    7.21    3.18
      32768   36.62   53.49   18.95   21.03   12.87    5.98
      65536   47.01   53.17   27.14   23.50   20.39   10.57
     131072   51.77   53.29   33.18   18.23   27.10   17.45
     262144   53.04   53.55   40.19   32.78   35.33   25.89
     524288   53.52   53.40   42.26   35.07   39.34   34.75
    1048576   52.61   53.49   46.86   42.51   42.38   40.23
    2097152   53.43   52.90   47.30   45.61   47.37   45.16
    4194304   53.05   53.42   50.36   49.49   49.80   48.86


So while the Ultra 160 numbers are not as flat as Octane ones, the result with SSD are very good compared to HDD test.

And finally a goldie but oldie... again from Dr. Dave: viewtopic.php?f=14&t=10217&p=77847&hilit=paris.mpeg#p77847

Code: Select all

dubious 8# time dmconvert -f qt -p video paris.mpeg out.mov
21.259u 4.888s 0:26.25 99.5% 0+0k 4019+6io 32pf+0w


So down from 50 something sec to less than half that with faster disk and CPU (test is single threaded so expect combination of nearly double CPU speed and faster disk account for result).

Conclusion, old SGIs can benefit from new SSD technologies ;-)

Cheers,

jwhat.
jwhat - ask questions, provide answers

Jack Luminous
Donor
Donor
Posts: 200
Joined: Mon Sep 26, 2011 12:59 am
Location: PARIS

Re: Octane2/Onyx350/4 - ACARD ARS-2160H/ARS-2320S with Samsung SSD

Unread postby Jack Luminous » Thu Dec 07, 2017 5:17 am

Interesting!

For comparaison sake, here are some diskperf result I posted long ago about FC-AL enclosure plugged into FC-AL XIO card in an Octane2. Perf is for a single disk as the enclosure is not raided or stripped etc. I think it would be a cheaper way to achieve fast I/O than using SSD.


Code: Select all

Cobalt 1# diskperf -W -D -t10 -n"D9 Stone IBM HUS153030VLF400" -c500 /stor/stor4/test
---------------------------------------------------------
# Disk Performance Test Results Generated By Diskperf V1.2
#
# Test name     : D9 Stone IBM HUS153030VLF400
# Test date     : Thu Apr 17 13:56:31 2014
# Test machine  : IRIX64 Cobalt 6.5 10070055 IP30
# Test type     : XFS data subvolume
# Test path     : /stor/stor4/test
# Request sizes : min=16384 max=4194304
# Parameters    : direct=1 time=10 scale=1.000 delay=0.000
# XFS file size : 4194304 bytes
#---------------------------------------------------------
# req_size  fwd_wt  fwd_rd  bwd_wt  bwd_rd  rnd_wt  rnd_rd
#  (bytes)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)
#---------------------------------------------------------
      16384   17.94   49.04   11.17    4.16    9.67    7.39
      32768   38.08   65.29   16.03    8.64   15.06   20.34
      65536   58.71   80.96   20.11   18.32   21.76   66.04
     131072   76.33   89.51   44.85   25.04   35.60   63.83
     262144   88.06   94.74   41.77   38.87   53.89   86.92
     524288   93.78   97.12   66.22   42.22   68.88   73.76
    1048576   93.97   97.40   84.42   51.85   86.43   67.91
    2097152   96.05   99.56   56.90   61.94   85.07   74.62
    4194304   96.20   78.81    0.00    0.00   80.74   87.23
:Octane2: :Octane: :Octane: :O2:

jwhat
Posts: 317
Joined: Sat Aug 09, 2003 6:25 pm
Location: Australia

Re: Octane2/Onyx350/4 - ACARD ARS-2160H/ARS-2320S with Samsung SSD

Unread postby jwhat » Thu Dec 07, 2017 8:29 am

Hi Jack,

That is good single disk result.

Only one problem....

Need fibre channel kit.

I used to have this: viewtopic.php?f=14&t=13590&p=7311841#p7311841

But it is long gone now via eBay, though I still have controllers (HBAs for Octane2, Onyx350/4 and Mac) and brocade switch.

My last fibre channel install used Promise VTRAK E610 series with 4Gb FC connectivity and 3Gb SAS/SATA but it was connected up to Mac not SGI box, this mutated into current Areca DAS RAID also connected to Mac.

Hence my interest in NFS performance.

But agree that for SGI that path to best performance would be via FC. Would love to test against FC, but do not think it’s viable due to $$ cost and logistics nightmare of shipping FC Chassis.

This stuff is really heavy !

Out of curiosity, did you set up XVM array and what throughput are you getting ?

Cheers,

jwhat.
jwhat - ask questions, provide answers

User avatar
jan-jaap
Donor
Donor
Posts: 4939
Joined: Thu Jun 17, 2004 11:35 am
Location: Wijchen, The Netherlands
Contact:

Re: Octane2/Onyx350/4 - ACARD ARS-2160H/ARS-2320S with Samsung SSD

Unread postby jan-jaap » Thu Dec 07, 2017 8:57 am

With a stripe of two cheapo 2TB SATA disks I get ~ 200MB/s: viewtopic.php?f=14&t=16724207#p7330958

Basically, I get the same streaming speed per disk for the SATA disks using an LSI HBA as you get from the SSD in the SCSI adapter, but I have two disks, and the SSD wins for smaller block sizes (duh)
:PI: :Indigo: :Indigo: :Indy: :Indy: :Indy: :Indigo2: :Indigo2: :Indigo2IMP: :Octane: :Octane2: :O2: :O2+: Image :Fuel: :Tezro: :4D70G: :Skywriter: :PWRSeries: :Crimson: :ChallengeL: :Onyx: :O200: :Onyx2: :O3x02L:
To accentuate the special identity of the IRIS 4D/70, Silicon Graphics' designers selected a new color palette. The machine's coating blends dark grey, raspberry and beige colors into a pleasing harmony. (IRIS 4D/70 Superworkstation Technical Report)

jwhat
Posts: 317
Joined: Sat Aug 09, 2003 6:25 pm
Location: Australia

Re: Octane2/Onyx350/4 - ACARD ARS-2160H/ARS-2320S with Samsung SSD

Unread postby jwhat » Thu Dec 07, 2017 9:20 am

Hi Jan-Jaap,

Really nice job on install, presume as you have removed IO9 that chassis is Numalink’ed to main boot box.

Saw a few postings on LSI board and no guestion on it being great way to get good disk performance with low effort and cost (compared to FC).

I have some 6Gb SAS SSDs (again installed in Mac as 3 disk RAID 5), would be interesting to try these out with LSI...

Maybe off to eBay, again ;-)

Cheers,

jwhat.
jwhat - ask questions, provide answers

User avatar
jan-jaap
Donor
Donor
Posts: 4939
Joined: Thu Jun 17, 2004 11:35 am
Location: Wijchen, The Netherlands
Contact:

Re: Octane2/Onyx350/4 - ACARD ARS-2160H/ARS-2320S with Samsung SSD

Unread postby jan-jaap » Fri Dec 08, 2017 12:33 am

Yes, this is a dual module O350 and the second module never had an IO board, or SCSI backplane for that matter.

The LSI SAS3041X-R is limited to 3Gbit SATA, though I think I've heard Ian Mapleson mention it could do 6Gbit with a FW update; possibly SAS 6Gbit only. Also keep in mind IRIX is limited to 2TB max LUN size, so any disk or SAN volume > 2TB can't fx-ed. It is possible to assemble a bigger filesystem using XVM and LUNs smaller than 2TB.

When I assembled this system (it still runs 24/7 ...) SSDs of acceptable size for bulk storage were not affordable. Since you can put at most two SATA disks in the O350 slave module, each of which can be at most 2TB, I ended up with this configuration.

Other versions of the LSI SAS board exist, with up to 8 external SAS connections. I've thought about sourcing an external SAS enclosure (something like a DELL MD1000) and using that instead. That would be the way to many times 2TB in one XVM volume. But I cannot justify this at work (where the system is currently hosted) and at home I already have a large ZFS server using more modern and economic hardware.

There's one driver for the LSI 4Gbit FC board, the LSI U320 SCSI and the LSI SAS boards. The controllers implement some sort off switching fabric and are probably very similar. There's probably some limitation there; I wouldn't expect the 8-way SAS board to sustain 48Gb/s of IO bandwidth (8x 6Gbit) or even 24Gb/s when it's SCSI brother is limited to 2 buses at 320MB/s and it's FC version to 2 buses at 4Gb/s. Oh, and the U320 SCSI board performance is awful. The QL12160 is nearly as fast.
:PI: :Indigo: :Indigo: :Indy: :Indy: :Indy: :Indigo2: :Indigo2: :Indigo2IMP: :Octane: :Octane2: :O2: :O2+: Image :Fuel: :Tezro: :4D70G: :Skywriter: :PWRSeries: :Crimson: :ChallengeL: :Onyx: :O200: :Onyx2: :O3x02L:
To accentuate the special identity of the IRIS 4D/70, Silicon Graphics' designers selected a new color palette. The machine's coating blends dark grey, raspberry and beige colors into a pleasing harmony. (IRIS 4D/70 Superworkstation Technical Report)

jwhat
Posts: 317
Joined: Sat Aug 09, 2003 6:25 pm
Location: Australia

Re: Octane2/Onyx350/4 - ACARD ARS-2160H/ARS-2320S with Samsung SSD

Unread postby jwhat » Fri Dec 08, 2017 9:25 am

Hi Jan-Jaap,

Interesting about LSI all having same driver. I have LSI LSI7204XP-LC in Onyx350/4, but nothing except switch to connect it to.
I also went ebaying and ordered LSI SAS3442X-R with 4 internal / 4 external SAS/SATA lanes and as I was over excited picked up 10Gb Neterion XFrame card.

Also went looking around for 6Gb FW update, but have not found anything concrete on that, will likely ping Ian for tips on that.
No fear on breaking 2TB limit as SAS SSDs are all 800GB, I originally got them to run in Apple Xserve Pro RAID setup (based on LSI RAID controller), but RAID card refused to recognize them as disks :-( . So they are just being used as fast backup on for Mac Pro boot disk (1.6 TB SSD), which also has /Users/Shared directory on it and so is almost full.

So doing stripped test across 3 SSD on sgi would be interesting test, but bit complicated as I have not touched XVM for some time and will have to dust off manuals.

Once parts arrive will try simple SAS and 10Gbe tests to see how far you can go.

Mind you, space wise these disks are tiny as XServe has 8/10TB SAS HDDs. On my prior FC setup I had 36GB -> 72GB -> 300GB, so need for storage space kind of forces hand to move to newer tech. Cost/space of SAS vs FC HDDs drove me down Promise FC/SAS path, then likewise move to DAS/SAS, with XServe acting as AFP/NFS/SMB server in both cases. So preferred option is good NFS performance.

Cheers from Oz,

jwhat.
jwhat - ask questions, provide answers


Return to “SGI: Hardware”

Who is online

Users browsing this forum: No registered users and 2 guests