A couple of weeks back I got a new case for my PC. Previously I was using a generic mini-tower and then had an external 8-disk tower (Sans Digital TR8MB) connected via an eSATA concentrator (4 disks per channel).
It’s been working OK for years but every so often the controller would reset (especially under write loads); no data lost but annoying. Also after a power reset (eg a failure, or maintenace) then frequently one or two disks (slot 2 in both halves!!) weren’t always detected and needed reseating and re-adding to the RAID6 (yay for write-intent bitmaps, so recovery is quick!).
I spotted the Zalman MS800 tower. This has 10 bays. Not enough; I have 12x3.5 disks and 2x2.5 SSDs. But I’d previously bought some 5.25->3.5 adapters that let you put 4x3.5 disk into 3x5.25 space (1 was using one of them in the old case, and the other 2 were emergency backups incase the external tower broke).
So 9 of the slots could fit 12 disks and the 10th slot could contain both SSDs. And, hey, it was only $60 (down to $40 if I can be bothered to fill in the rebate card).
This would also mean I needed some better controllers; the old SiL eSATA card that came with the old disk tower wouldn’t work for the internal disks, so I bought 2 LSI 9211-8 cards; that’d give me 16 channels; more than enough (in theory I could have just used one card, the motherboard has 5*SATA-3Gbps ports which might have been suitable). My existing 650W PSU should be sufficient to handle all of this…
Quite amazingly it worked! I needed to fiddle with the boot stuff a fair bit but I think it’ll now boot unattended if necessary :-)
Speeds are faster than before. Now I get (using hdparm -t)
SSD: 375MB/sec (I got 226 using internal ports)
2Tbyte disks: 133MB/sec
4Tbyte disks: 145MB/sec (I was getting 125 using the old eSATA card)
The raw disk speeds don’t look that much better (in particular the 2Tbyte disks don’t show any real change). But what about RAID performance?
The two SSDs are in a RAID 1 and get 375 MB/sec
The 4*2Tbytes are in a RAID 10 and get 267 MB/sec
The 8*4Tbytes are in a RAID 6 and get 532 MB/sec
Heh, we can definitely see the benefit of load distribution for reads! This last number is much higher than the old eSATA card because there’s no contention on the port concentrator. Big win! And that’s what I’d hoped for.
Each of the disk adapters has its own fan (120mm). The case has two fans (there’s also a way of adding a fan internally if necessary, but I didn’t use that).
So how hot does this monster run? Most of the time it’s idle-ish, just running a couple of VMs plus “management” stuff (DNS, DHCP, NTP etc).
LM-sensors tells me:
Core 0: +39.0 C (high = +83.0 C, crit = +99.0 C)
Core 1: +35.0 C (high = +83.0 C, crit = +99.0 C)
Core 2: +39.0 C (high = +83.0 C, crit = +99.0 C)
Core 3: +34.0 C (high = +83.0 C, crit = +99.0 C)
Well, that’s maybe a little high for an idling machine but it’s well within tolerance. I’m happy with that.
The disks are interesting; the SMART output:
/dev/sda: 194 Temperature_Celsius ... 30
/dev/sdb: 194 Temperature_Celsius ... 30
/dev/sdc: 194 Temperature_Celsius ... 27
/dev/sdd: 194 Temperature_Celsius ... 28
/dev/sde: 194 Temperature_Celsius ... 28
/dev/sdf: 194 Temperature_Celsius ... 28
/dev/sdg: 194 Temperature_Celsius ... 27
/dev/sdh: 194 Temperature_Celsius ... 28
/dev/sdi: 194 Temperature_Celsius ... 28
/dev/sdj: 194 Temperature_Celsius ... 28
/dev/sdk: 194 Temperature_Celsius ... 28
/dev/sdl: 194 Temperature_Celsius ... 27
/dev/sdm: 194 Temperature_Celsius ... 28
/dev/sdn: 194 Temperature_Celsius ... 28
/dev/sdo: 194 Temperature_Celsius ... 40
The two SSDs run slightly hotter than the spinning rust! And the last disk (sdo) is a 1Tbyte disk sitting in an external cradle and just exposed to ambient air for cooling.
It’s quite impressive how much difference fans can make :-) The external disk is hotter than the internal disks, and even more fun the spinning rust runs cooler than the SSDs; the SSDS don’t have any forced air-flow whereas the disks have those 120mm fans drawing air over them.
(Heh, the UPS also claims power load is now down to 23%; previously it was 25% with the external drive bay).
