Fuel V12, R16K 900MHz (8MB L2), 4GB

[mapesdhs]

Picture links: side, front, rear, inside, PCI slots, disks, CPU

03/Feb/2015: upgraded the /ian user account SSD to a 240GB OCZ Vertex2E, ie. doubled the space as the 120GB was
almost full. Note I've tested many other models of SSD with Fuel/Tezro, including a Samsung 850 Pro, but alas for some
reason almost all SATA3 SSDs end up negotiating only a 1.5Gbit link with the SAS controller, whereas SATA2 SSDs setup
a 3Gbit link just fine. Could be that only certain firmware versions will allow for a 3Gbit link with a SATA3 device. I thought
I'd resolved this before by testing fw versions P20 vs. P21, but other tests showed this isn't the case, so it looks like I'm
going to have to at some point test the whole range of fw updates for these LSI SAS cards, see which ones work ok with
SATA3 devices and which don't. Of course though, even at 1.5Gbit, an SSD is far quicker than any rust spinner for small-
size random I/O, and beats almost all of them for sequential I/O aswell.

One other change: replaced the external SCSI backup unit with 4-bay Startech 5.25" device which holds 4x 2.5" devices.
This is filled with three SSDs and a 2.5" 750GB SATA, all used to hold clone mirror backups of the Fuel's internal drives.
Connects via a standard SAS cable to the LSI 3442's external SAS port.

08/Oct/2012 Edit: Replaced the system disk with a 60GB SSD (OCZ Vertex2E) in an ARS-2160H 68pin SCSI/SATA bridge box, but /usr and /var are
stored on a separate 120GB OCZ Vertex2E connected to the SAS/SATA card, thus allowing max speed for all accesses to those areas. This gives
the following diskperf when run within /var/tmp:

Code: Select all

# req_size  fwd_wt  fwd_rd  bwd_wt  bwd_rd  rnd_wt  rnd_rd
#  (bytes)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)
#---------------------------------------------------------
      4096   39.75   33.27   39.94   33.07   39.70   22.56
      8192   69.13   57.90   69.70   53.91   69.68   30.46
     16384  109.03   92.98  107.86   79.81  110.71   54.90
     32768  153.29  131.74  153.74  103.54  153.02   90.66
     65536  192.25  170.21  188.90  151.19  193.54  137.95
    131072  220.07  209.69  217.79  195.31  218.80  182.74
    262144  238.19  233.02  234.89  228.38  238.00  220.60
    524288  247.30  251.77  248.45  248.34  246.96  246.07
   1048576  252.74  263.24  253.12  261.81  252.85  261.06
   2097152  256.56  269.65  255.59  269.41  255.87  269.02
   4194304  257.14  274.21  262.54  273.46  257.71  273.34


I'll update the pictures later.

Meanwhile, I'm not using the DDS4 DAT or external 146GB SCSI disk anymore (got a new backup system, more on that later), and the external CDRW
isn't connected now either. And I've replaced the 3041X-R SAS card with a 3442X-R, so the system now has a 4-port external link aswell (this relates
to the new backup setup).

28/Feb/2011 Edit: Upgraded the SAS to a 600GB 15K (same model generation).

03/Jan/2011 Edit: Replaced the 300GB 10K data disk with a Seagate 450GB 15K SAS (15K.7). Moved my main /ian user account (approx. 67GB
of data atm) onto a 120GB SSD (OCZ Vertex2E 3.5").

31/Aug/2009 Edit: Replaced Maxtor system disk with 300GB 15K Fujitsu, replaced LSI 22320 SCSI card with LSI SAS3041X-R
SAS/SATA, added 1TB SATA disk.

Code: Select all

Location: /hw/module/001c01/node
            IP34 Board: barcode NED586     part 030-1707-004 rev -C
Location: /hw/module/001c01/node/cpubus/0
        IP34PIMM Board: barcode MWX852     part 030-2023-001 rev -B
Location: /hw/module/001c01/Ibrick/xtalk/13
          ASTODY Board: barcode NJY535     part 030-1726-005 rev -A
Location: /hw/module/001c01/Ibrick/xtalk/14
            IP34 Board: barcode NED586     part 030-1707-004 rev -C
Location: /hw/module/001c01/Ibrick/xtalk/15
            IP34 Board: barcode NED586     part 030-1707-004 rev -C
1 900 MHZ IP35 Processor
CPU: MIPS R16000 Processor Chip Revision: 3.0
FPU: MIPS R16010 Floating Point Chip Revision: 3.0
CPU 0 at Module 001c01/Slot 0/Slice A: 900 Mhz MIPS R16000 Processor Chip (enabled)
  Processor revision: 3.0. Scache: Size 8 MB Speed 450 Mhz  Tap 0xb
Main memory size: 4096 Mbytes
Instruction cache size: 32 Kbytes
Data cache size: 32 Kbytes
Secondary unified instruction/data cache size: 8 Mbytes
Memory at Module 001c01/Slot 0: 4096 MB (enabled)
  Bank 0 contains 1024 MB (Premium) DIMMS (enabled)
  Bank 1 contains 1024 MB (Premium) DIMMS (enabled)
  Bank 2 contains 1024 MB (Premium) DIMMS (enabled)
  Bank 3 contains 1024 MB (Premium) DIMMS (enabled)
Integral SCSI controller 2: Version QL12160, low voltage differential
Integral SCSI controller 3: Version QL12160, single ended
  Scanner: unit 7 on SCSI controller 3
Integral SCSI controller 4: Version SAS/SATA LS1068
  Disk drive: unit 0 on SCSI controller 4 (unit 0)
  Disk drive: unit 1 on SCSI controller 4 (unit 1)
  Disk drive: unit 2 on SCSI controller 4 (unit 2)
  Disk drive: unit 3 on SCSI controller 4 (unit 3)
Integral SCSI controller 0: Version QL12160, low voltage differential
  Disk drive: unit 1 on SCSI controller 0 (unit 1)
Integral SCSI controller 1: Version QL12160, single ended
  CDROM: unit 3 on SCSI controller 1
  CDROM: unit 4 on SCSI controller 1
IOC3/IOC4 serial port: tty1
IOC3/IOC4 serial port: tty2
IOC3 parallel port: plp1
Graphics board: V12
Gigabit Ethernet: eg0, module 001c01, pci_bus 1, pci_slot 3, firmware version 12.4.10
Integral Fast Ethernet: ef0, version 1, module 001c01, pci 4
Iris Audio Processor: version MAD revision 1, number 1
  PCI Adapter ID (vendor 0x1077, device 0x1216) PCI slot 1
  PCI Adapter ID (vendor 0x1000, device 0x0054) PCI slot 2
  PCI Adapter ID (vendor 0x1077, device 0x1216) PCI slot 1
  PCI Adapter ID (vendor 0x1412, device 0x1724) PCI slot 2
  PCI Adapter ID (vendor 0x10a9, device 0x0009) PCI slot 3
  PCI Adapter ID (vendor 0x10a9, device 0x0003) PCI slot 4
  PCI Adapter ID (vendor 0x1045, device 0xc861) PCI slot 5
HUB in Module 001c01/Slot 0: Revision 2 Speed 200.00 Mhz (enabled)
IP35prom in Module 001c01/Slot n0: Revision 6.210
USB controller: type OHCI


gfxinfo:

Code: Select all

Graphics board 0 is "ODYSSEY" graphics.
        Managed (":0.0") 1600x1200
        BUZZ version B.2
        PB&J version 1
        128MB memory
                Banks: 4, CAS latency: 3
         Monitor 0 type: HWP 4819
        Channel 0:
         Origin = (0,0)
         Video Output: 1600 pixels, 1200 lines, 75.00Hz (1600x1200_75)


Internal PCI cards:

Code: Select all

MAudio Revolution 7.1
SGI Copper Gbit Ethernet
QLA12160 dual-channel U160 SCSI
LSI SAS3442X-R SAS/SATA PCIX card (4 internal ports, 4 external ports via an SFF8470 socket)


SCSI/SATA Devices:

Code: Select all

  0,1: OCZ Vertex2E 60GB 2.5" SSD (fw 1.37) held in an ACARD ARS_2160 SCSI/SATA bridge box (68pin version).
  1,3: Yamaha 16/10/40 CDRW (internal).
  1,4: TOSHIBA DVD-ROM SD-M17111005 (internal).
  3,7: HP ScanJet 6300C Scanner.
  4,0: OCZ Vertex2E 120GB 2.5" SSD (dks4d0s6 90GB partition holds /var, dks4d0s7 20GB partition holds /usr).
  4,1: OCZ Vertex2E 120GB 3.5" SSD (main 'ian' user account)
  4,2: Seagate 600GB 15K SAS (ST3600057SS).
  4,4: Samsung Spinpoint F1 1TB 7200rpm SATA, Model HD103UJ.


Here are some diskperf results on the drives, done with all background processes running as normal, ie. mediad, httpd, firefox, etc.

0,1: OCZ Vertex2E 60GB 2.5" SSD. Current "system" disk (test must be executed somewhere such as /tmp, as /var and /usr are on a different device).

Code: Select all

# req_size  fwd_wt  fwd_rd  bwd_wt  bwd_rd  rnd_wt  rnd_rd
#  (bytes)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)
#---------------------------------------------------------
      4096   15.61   15.08   15.61   15.04   15.61   11.21
      8192   27.60   26.31   27.61   25.78   27.60   20.25
     16384   45.27   41.56   45.17   39.89   45.16   33.62
     32768   65.11   57.68   65.03   53.21   65.06   49.96
     65536   83.16   82.34   83.60   78.51   83.59   74.61
    131072   83.90   88.09   84.12   86.10   83.72   82.44
    262144   85.92   92.85   86.19   92.64   85.38   89.58
    524288   86.54   95.23   86.59   95.01   86.45   94.01
   1048576   85.89   95.07   86.03   95.20   85.67   94.25
   2097152   83.88   93.19   84.30   93.03   84.22   92.20
   4194304   80.13   88.45   80.15   88.15   80.08   87.66


4,0: OCZ Vertex2E 120GB 2.5" SSD, holds /usr and /var. Connected via the SAS card for max speed. Notice the much better 4K random I/O speeds.

Code: Select all

# req_size  fwd_wt  fwd_rd  bwd_wt  bwd_rd  rnd_wt  rnd_rd
#  (bytes)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)
#---------------------------------------------------------
      4096   39.75   33.27   39.94   33.07   39.70   22.56
      8192   69.13   57.90   69.70   53.91   69.68   30.46
     16384  109.03   92.98  107.86   79.81  110.71   54.90
     32768  153.29  131.74  153.74  103.54  153.02   90.66
     65536  192.25  170.21  188.90  151.19  193.54  137.95
    131072  220.07  209.69  217.79  195.31  218.80  182.74
    262144  238.19  233.02  234.89  228.38  238.00  220.60
    524288  247.30  251.77  248.45  248.34  246.96  246.07
   1048576  252.74  263.24  253.12  261.81  252.85  261.06
   2097152  256.56  269.65  255.59  269.41  255.87  269.02
   4194304  257.14  274.21  262.54  273.46  257.71  273.34


4,1: OCZ Vertex2E 120GB 3.5" SSD, holds the main /ian user account.

Code: Select all

#  req_size  fwd_wt  fwd_rd  bwd_wt  bwd_rd  rnd_wt  rnd_rd
#   (bytes)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)
#----------------------------------------------------------
       4096   41.03   32.78   40.87   32.40   40.89   18.71
       8192   71.14   57.90   71.43   55.72   71.25   35.13
      16384  112.41   92.70  112.71   84.98  112.62   62.55
      32768  155.51  134.62  155.55  114.20  155.22  101.51
      65536  195.52  178.54  194.84  165.57  194.62  148.27
     131072  222.14  214.24  222.33  207.04  221.96  192.56
     262144  239.02  239.37  239.96  236.09  238.97  227.54
     524288  249.26  255.97  250.19  254.05  249.32  250.51
    1048576  254.55  266.98  254.66  265.47  254.36  263.74
    2097152  258.39  272.93  257.60  272.10  257.53  270.96
    4194304  258.92  275.82  259.48  274.96  259.66  275.08
    8388608  259.94  277.27  261.89  276.97  260.37  276.86


4.2: Seagate 600GB 15K SAS (ST3600057SS), test not yet run.


4,3: 1TB 7200rpm SATA data disk (Samsung HD103UJ):

Code: Select all

# req_size  fwd_wt  fwd_rd  bwd_wt  bwd_rd  rnd_wt  rnd_rd
#  (bytes)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)
#---------------------------------------------------------
     16384   64.18   65.03   47.89   33.18    4.44    2.27
     32768   75.78   99.78   88.04   37.53    8.73    4.42
     65536  106.05  111.59  106.76   42.43   16.23    8.48
    131072  112.12  114.28  107.58   49.08   27.45   15.77
    262144  106.30  113.90  106.95   56.80   41.58   27.56
    524288  106.15  113.59  106.60   55.57   57.88   44.60
   1048576  112.30  113.14  107.58   79.46   74.57   62.15
   2097152  111.39  110.91  106.38   85.67   86.97   80.43
   4194304  111.30  109.20  108.82  102.21   98.86   93.09


Fujitsu 300GB 15000rpm MBA3300NC, old system disk.

Code: Select all

# req_size  fwd_wt  fwd_rd  bwd_wt  bwd_rd  rnd_wt  rnd_rd
#  (bytes)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)
#---------------------------------------------------------
     16384   59.87   68.97   30.55   27.13   10.23    5.30
     32768   80.08   93.85   34.26   33.87   17.71   10.04
     65536   94.37  114.01   56.67   38.45   29.14   18.21
    131072   92.95  117.03   43.56   43.48   32.49   31.45
    262144  102.52  124.69   44.35   44.50   51.88   48.31
    524288  107.48  124.89   66.54   67.23   72.67   67.22
   1048576  109.85  124.88   89.21   90.17   91.55   86.17
   2097152  112.99  124.93  105.37  102.76  104.32   99.83
   4194304  114.38  124.92  110.83  109.11  111.25  109.44
   8388608  114.84  119.09  114.52  115.07  114.61  114.00
  16777216  115.02  121.53  114.63  117.28  114.43  117.12


147GB 15K 'Worldview' OEM (old backup disk).

Code: Select all

# req_size  fwd_wt  fwd_rd  bwd_wt  bwd_rd  rnd_wt  rnd_rd
#  (bytes)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)
#---------------------------------------------------------
     16384   61.32   74.49   14.59    4.32    8.84    5.25
     32768   71.57   74.14   24.75    9.11   14.93    9.83
     65536   76.83   78.69   34.40   20.56   24.20   17.13
    131072   78.59   78.80   43.84   25.16   38.69   27.67
    262144   78.96   78.77   57.23   45.79   52.63   39.79
    524288   74.64   78.79   62.99   55.96   58.63   52.34
   1048576   78.83   78.73   71.28   56.21   60.90   56.55
   2097152   78.94   78.78   71.63   71.03   67.03   68.72
   4194304   78.89   78.74   76.42   71.49   74.51   71.58
   8388608   78.85   78.83   71.87   74.01   76.84   74.17
  16777216   77.99   78.83   72.37   75.36   77.69   75.38


Seagate 450GB 15K SAS (ST3450857SS), old general data disk.

Code: Select all

#  req_size  fwd_wt  fwd_rd  bwd_wt  bwd_rd  rnd_wt  rnd_rd
#   (bytes)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)
#----------------------------------------------------------
       4096   38.83   40.26   19.03   13.10    3.11    1.34
       8192   68.98   70.94   34.82   20.83    6.22    2.67
      16384  110.34  112.64   61.36   31.14   12.28    5.35
      32768  154.20  156.89   99.76   42.16   22.76   10.52
      65536  194.83  178.83  143.22   57.72   41.58   19.79
     131072  206.30  198.62  170.62   63.65   69.34   35.18
     262144  206.03  207.51  163.79   95.62   98.67   59.84
     524288  205.83  207.88   95.40   96.06   95.99   92.94
    1048576  206.58  207.95  149.61  145.27  127.19  123.02
    2097152  206.24  207.90  152.67  150.97  158.85  156.52
    4194304  206.77  207.91  184.26  171.54  180.34  176.95
    8388608  206.14  207.91  193.90  189.59  191.76  189.09


Reference info: RAID results using one dual-channel SCSI card, comparing LSI 22320 to QLA12160/66, tested
with Maxtor Atlas 15K II 146GB disks:

Code: Select all

Summary for 'n' disks on 2 channels, optimised for HD video, fwd read/write speeds only.

         | **FORWARD WRITE** | **FORWARD READ**
         |    LSI     QLA    |    LSI    QLA
-----------------------------------------------
2 disks  |  172.02   172.76  |  159.08   159.26
4 disks  |  209.30   266.74  |  186.62   237.78
6 disks  |  209.36   269.59  |  190.81   245.07
12 disks |  209.30   269.34  |  200.07   260.37


Except for using two disks, the QLA is significantly faster than the LSI.

Cheers!

Ian.

PS. For an Alias render benchmark I've done (complex scene), this system beats a dual-600 Octane2. For a simple scene though, the
Octane2 is faster. The more complex the task, the better Fuel and O3K systems run. I've accumulated data for dozens of systems, results
now available on my site!

[Richtom1]

Oh sooooooooooooo sweet Ian. Enjoy the Holidays and talk with you soon.

Sending you a PM.

Richard

[josehill]

Looks like Christmas came early to Edinburgh!

Have a great holiday season, Ian!

[R-ten-K]

Sweet system! Lucky b*stard...

The 900Mhz R16K must be rarer than hen's teeth... how did you manage the score?

[josehill]

The 900Mhz R16K must be rarer than hen's teeth... how did you manage the score?

I suspect Ian's answer might begin, "Many Bothans died..."

[mapesdhs]

(40Kbit dialup - yuck!!...)

I did a swap; my two Fuel 700/V12 systems for a 900/V10 and an Onyx2 deskside, and helping
with some other stuff.

Cheers!

Ian.

PS. Bothans?... It was a slaughter... >-)

[mapesdhs]

eMGee writes:
> That's a very nice system there Ian! Even with 8MB L2 cache memory, impressive!

Thanks!!


> So, this must be your personal system, as you mentioned. If you get the chance, and of course if you feel like it,
> could you perhaps post some pictures? All the added expansion/option cards, the whole setup, would be interesting
> to see.

Will do! Some time in January.

Ian.

[tomo]

NIce system and happy holidays!

[mapesdhs]

Hope you all had a good Christmas, etc.!!

I've finished doing the usual benchmark tests from my SGI General Performance Comparisons page (Inventor, GIMP, etc.),
and also Quake1/Quake2 tests (finally worked out how to run the tests single-buffered). I've also taken a few pics,
though the lighting/sharpness isn't all that great as the system lives under my desk, so natural light is mostly absent.

Side of system:

Front (not found a black facia for the CDRW yet):

Rear connections (just ordered a VHDCI/MicroD cable for the scanner, so the remaining SCSI socket will be used soon):

Overall internals:

PCI cards (top to bottom: Revolution 7.1, copper Gbit NIC, QLA12160/66, LSI 21320):

SCSI disks (top: Maxtor Atlas 15K II 147GB 8K147L0 68pin; bottom: Fujitsu MAT3300NP 300GB 10K 68pin. The Maxtor was a good
find as it has a valid warranty until October 2011):

The main CPU board (sorry for lack of focus, hard for my camera's flash to work well at such short range):

Happy new year everyone!!

Ian.

[guardian452]

Somebody was a very good boy this year

[directedition]

Are you open to trades? How do first born sons sound?

[mapesdhs]

sybrfreq writes:
> Somebody was a very good boy this year

Nah, somebody just worked their butt off. (ie. me)


directedition writes:
> Are you open to trades? How do first born sons sound?

Ooo, you'd have to do way better than that... would be well into the Lex Luthor scale of things. You know,
as in babes, Australia, The Universe, that sort of thing...


Meanwhile, as a final closing treat for 2008, I've just uploaded the results of of my latest benchmarking venture,
namely running Alias and Maya renders using two quite different scenes (complex scene with Alias, simple scene
with Maya; links are on the left hand side in the index frame):

http://www.sgidepot.co.uk/perfcomp.html

or if you don't want the frames layout, here are direct links to the Alias and Maya pages.

Hope you find the results interesting! Main thing to take away from the data is that, in general, rendering a complex
scene will scale much better using multiple CPUs on an O3K-class system, though the Onyx2 does rather well.

That's it for this year! Catchya next year...

Ian.

-----

Jan/09 Edit: I've moved the screenshots from here into the main announcement thread.

[JacquesT]

Very nice Ian!

If I'm ever up in Scotland (But why?! It's colder there!!! ) I'll have to pop in and see this beast!

Happy New Year!

-J

(What's next, a quad 1ghz Tezro?)

[mapesdhs]

JacquesT writes:
> Very nice Ian!

Thanks!


> If I'm ever up in Scotland (But why?! It's colder there!!! ) I'll have to pop in and see this beast!

Feel free!


> Happy New Year!

Ditto!!


> (What's next, a quad 1ghz Tezro?)

That's the plan, hehe...

Ian.

[mapesdhs]

A quick bump to help spread some useful info:-

I've replaced the LSI 22320 card with an LSI SAS3041X-R SAS/SATA card, and installed a 1TB SATA, so now my Fuel has
a 1TB SATA to act as a backup clone/copy of my movie archive disk in my PC. Here's a df (oh yeah, I also replaced the
146GB 15K Maxtor system disk with a 300GB 15K Fujitsu MBA3300NC):

Code: Select all

Filesystem             Type  kbytes      use        avail   %use  Mounted on
/dev/root               xfs 292799884  124513248  168286636  43   /
/dev/dsk/dks10d0s7      xfs 976641304   64263012  912378292   7   /video
/dev/dsk/dks0d2s7       xfs 292930940  272496640   20434300  94   /disk2


I've not yet finished copying the data over to the SATA in the Fuel from my PC, but it's pretty quick via ftp, about 75MB/sec.
An intriguing test given that the both the source and destination for this transfer is exactly the same model of drive (Samsung
Spinpoint F1 1TB 7200rpm SATA, model HD103UJ). Would be interesting to see how two of these operate as a RAID0 with
a Fuel, but that's for the future.

The SAS3041X-R just has 4 internal ports, but it's quite cheap and thus ideal for installing SATAs in a Fuel. Also note that
using the fastest possible SATA (WD VR 10K series) does work fine with Fuel/Tezro and the performance is good, beating
all SCSI 10Ks and most SCSI 15Ks (see my disks performance results page), though 2nd-hand prices are not quite as good
as 15K SCSI. OTOH, the WD VR makes less noise, uses less power and generates less heat.

I'll update the hinv, etc. in my o.p. shortly.

Ian.

PS. Am hoping to be able to test an SSD in October.