# Raid array verification



## Virtovo (Jan 18, 2014)

Granted, I've never run an 8 disk array before (generally four or twelve).  If anyone has an experience of an 8 disk raid 10 array, what kind of speeds would you expect to see from the misguided but widely accepted DD test?

Spec of the array:

8 x WD RE4 2TB

LSI 9621-8i

I would have expected to be able to plot an 8 disk array on a line between the four and twelve disk array.


----------



## TruvisT (Jan 18, 2014)

What kind of results are you currently getting?


----------



## Virtovo (Jan 18, 2014)

TruvisT said:


> What kind of results are you currently getting?


Right around 300 MB/s


----------



## serverian (Jan 18, 2014)

Virtovo said:


> Right around 300 MB/s


Write back?


----------



## Virtovo (Jan 18, 2014)

serverian said:


> Write back?


Yes write-back and cache enabled.


----------



## serverian (Jan 18, 2014)

Virtovo said:


> Yes write-back and cache enabled.


That card doesn't have a cache. It's a pretty low-end card so you are probably hitting its limits. For better results use a real RAID card like 9260.


----------



## Virtovo (Jan 18, 2014)

serverian said:


> That card doesn't have a cache. It's a pretty low-end card so you are probably hitting its limits. For better results use a real RAID card like 9260.


I was referring to Disk cache enabled.  Card has 512mb cache.  9261 is AFAIK pretty much the same as 9260 except a different board layout.


----------



## ndelaespada (Jan 18, 2014)

I have a server with a p400 (which is old and only has 256mb cache) with 4x1tb and I get 250mb/s.. so if you're getting only 300mb/s out of 8 disks in raid 10 you really need to consider another raid controller, better hds may also help improve i/o speed


----------



## Virtovo (Jan 18, 2014)

ndelaespada said:


> I have a server with a p400 (which is old and only has 256mb cache) with 4x1tb and I get 250mb/s.. so if you're getting only 300mb/s out of 8 disks in raid 10 you really need to consider another raid controller, better hds may also help improve i/o speed


Which is exactly why the results were confusing as I can push just under 300mb/s on a four disk H/W raid 10 array or just under 250mb/s on a four disk S/W raid 10 array.

I know the 9261 is not up there with the likes of a 9271; however I certainly wouldn't consider a 9261 a low end raid card.  Also WD RE4s aren't the latest a greatest now, but again no slouches either.


----------



## serverian (Jan 18, 2014)

Virtovo said:


> I was referring to Disk cache enabled.  Card has 512mb cache.  9261 is AFAIK pretty much the same as 9260 except a different board layout.


Oh, you have written 9621-8i instead of 9261-8i on your post. That's why I got confused. Run this command:

dd if=/dev/zero of=test bs=64k count=7k conv=fdatasync; unlink test


----------



## Virtovo (Jan 18, 2014)

serverian said:


> Oh, you have written 9621-8i instead of 9261-8i on your post. That's why I got confused. Run this command:
> 
> dd if=/dev/zero of=test bs=64k count=7k conv=fdatasync; unlink test


Ahh, total typo there, thanks for pointing it out.  It is indeed a 9261.  


```
dd if=/dev/zero of=test bs=64k count=7k conv=fdatasync; unlink test
7168+0 records in
7168+0 records out
469762048 bytes (470 MB) copied, 1.34486 s, 349 MB/s
```


----------



## serverian (Jan 18, 2014)

Virtovo said:


> Ahh, total typo there, thanks for pointing it out.  It is indeed a 9261.
> 
> 
> dd if=/dev/zero of=test bs=64k count=7k conv=fdatasync; unlink test
> ...


So, even it's hitting the 100% cache, it's still slow. I don't have experience with that specific card. I'd ask the DC is that's what they usually get with that setup.


----------



## Virtovo (Jan 18, 2014)

serverian said:


> So, even it's hitting the 100% cache, it's still slow. I don't have experience with that specific card. I'd ask the DC is that's what they usually get with that setup.


They are saying that sub 300 is normal; however based on my experience with other cards and different configs it can't be the case.  I can push around the 300 mark on a four disk array on a lesser card.  This should show a linear increase in performance (doubled), which is why I was puzzled.  I just wanted to see if anyone had any experience with an 8 disk array as I don't directly so can't quote actual numbers to the DC.


----------



## serverian (Jan 18, 2014)

Virtovo said:


> They are saying that sub 300 is normal; however based on my experience with other cards and different configs it can't be the case.  I can push around the 300 mark on a four disk array on a lesser card.  This should show a linear increase in performance (doubled), which is why I was puzzled.  I just wanted to see if anyone had any experience with an 8 disk array as I don't directly so can't quote actual numbers to the DC.


8 x Seagate ES.3 RAID10 on LSI 9271-8i:

1073741824 bytes (1.1 GB) copied, 1.52601 s, 704 MB/s


----------



## Virtovo (Jan 18, 2014)

serverian said:


> 8 x Seagate ES.3 RAID10 on LSI 9271-8i:
> 
> 1073741824 bytes (1.1 GB) copied, 1.52601 s, 704 MB/s


Well I was expecting around 600-650 from this setup.  The ES.3 are superior drives and obviously the 9271 is a superior card so that makes sense with your result.  Did you do anything special to the filesystem beyond general alignment and read-ahead?


----------



## serverian (Jan 18, 2014)

Virtovo said:


> Well I was expecting around 600-650 from this setup.  The ES.3 are superior drives and obviously the 9271 is a superior card so that makes sense with your result.  Did you do anything special to the filesystem beyond general alignment and read-ahead?


All default settings.


----------



## Virtovo (Jan 18, 2014)

serverian said:


> All default settings.


Thanks.  At least I've got something to go back to the provider with.


----------



## Virtovo (Jan 18, 2014)

Well even with trying some things I've seen in the past to cause this (such as switching the GEN3 PCIE slots to GEN2 and adjusting PCIE timings) I'm still unable to get anything better.  I've let the company know and they will be looking into it Monday.  I'll update this thread with any solution as it may server as a reference for others in the future.


----------



## Patrick (Jan 18, 2014)

You should be getting double, with 4x1TB RE4s we were getting 300 mb/s with LSI 9260-4i. 

I can take a look for you if you want, PM me over your Skype or msg me on IRC


----------



## Virtovo (Jan 18, 2014)

Patrick said:


> You should be getting double, with 4x1TB RE4s we were getting 300 mb/s with LSI 9260-4i.
> 
> I can take a look for you if you want, PM me over your Skype or msg me on IRC


It was my thinking too.  I appreciate the offer Patrick.  I've checked everything I can possibly think of and have now handed it back over to the server provider who are giving it the once over, re-configuring RAID and doing an OS install themselves.

If this heads nowhere, I'll shoot you a PM. Again, I appreciate the interest!


----------



## Virtovo (Jan 19, 2014)

Well Patrick took a look at pointed out something pretty obvious that I missed regarding the disks.  I'll ask if the drives can be swapped as the performance is not going to going any higher with the currently installed disks.


----------

