Home built NAS

[xrapidx]

I meant would be impossible if you don't want to compromise on the failure rate tollerance of RAID 5. Performance would also be less than a RAID 5 volume (theoretically and probably practically). (In my original post you'll notice I say: "Unless" )

To a home user it sounds great but from a data integrity perspective it is very bad (that is kinda the point of RAID 5 afterall). Which is probably why it isn't used anywhere else.

I can also think of sitations in which it wouldn't be possible. eg: 2x500GB drives + 2TB drive. I might be wrong but in such a setup I don't believe you would have more than 1TB of storage because mathematically you couldn't spread the parity data over enough volumes to mask a single disk failure.

My main reason for doing this - is I have quite a few 1TB discs, and they're all full - I don't want to now go and buy 5x 2TB discs just to upgrade space.... going to have to do more research on how the drobo works... I don't mind performance loss, as the space is only used for storage, but obviously data integrity is important.

 

[SilverNodashi]

Just a little slip of the tongue Gnome, build a bridge.

Anyway are these seemingly ludicrous system specs really necessary ? Its just gonna be a NAS box. I always thought one of the precious few cool things about any linux OS is that it runs well and reliably on lower specs as well.

Well, it all depends on how much the box needs todo. i.e. how many users will connect to it, how is the RAID setup, does it do encryption, what type of shares does it have (NFS, SMB, AFS, iSCSI - or all of them), does it do per-user authentication, etc. I generally don't build a NAS with less than 4GB RAM & Core2Duo / Core2Quad CPU's for better performance.

I meant would be impossible if you don't want to compromise on the failure rate tollerance of RAID 5. Performance would also be less than a RAID 5 volume (theoretically and probably practically). (In my original post you'll notice I say: "Unless" )

To a home user it sounds great but from a data integrity perspective it is very bad (that is kinda the point of RAID 5 afterall). Which is probably why it isn't used anywhere else.

I can also think of sitations in which it wouldn't be possible. eg: 2x500GB drives + 2TB drive. I might be wrong but in such a setup I don't believe you would have more than 1TB of storage because mathematically you couldn't spread the parity data over enough volumes to mask a single disk failure.

If you data is valuable, then rather use RAID6 / RAID10 (if possible) since it can withstand 2 disk failures. On a RAID5 system, with 6x 1TB / 6x 2TB HDD's, you're guaranteed to loose the whole se if one drive fails and while rebuilding another drive fails. Chances are if you brought all 6 drives at the same time, from the same supplier, they will fail shortly one after the other.

 

[CriticalConsumer]

Hmmm I am using RAID10 and got my drives from one supplier @ the same time.

so far so good - touch wood!

 

[Gnome]

On a RAID5 system, with 6x 1TB / 6x 2TB HDD's, you're guaranteed to loose the whole se if one drive fails and while rebuilding another drive fails. Chances are if you brought all 6 drives at the same time, from the same supplier, they will fail shortly one after the other.

I don't entirely agree with that statement.

The scenario you speak of is if you have 2 disks that fail entirely (eg. completely unusable). You can have 3 disks that are starting to fail, eg. CRC errors on all 3 of them (on a RAID 5 with 3 disks), so long as at least 2 of the parity blocks are recoverable over the 3 volumes that block will be completely recoverable.

Also ZFS is self healing, so if you start seeing ZFS reporting that it is encoutering errors during self healing, you know it's time to swap out the bad drive. Self healing in this case means, while the disk isn't being used a service moves randomly on the drive and scans blocks for damage. If any damage is detected it is repaired (and the ZFS statistics will show that errors have been corrected).

That allied with SMART makes me believe the chances of RAID Z1 (RAID 5) being unrecoverable becomes much lower.

Hmmm I am using RAID10 and got my drives from one supplier @ the same time.

so far so good - touch wood!

FreeNAS?

 

[CriticalConsumer]

Yip on FreeNAS.

 

[Gnome]

Yip on FreeNAS.

Ah ok, you aren't using ZFS I take it?

 

[CriticalConsumer]

Can't remember - been a year since I built this box - will check when I get home.

 

[warwickw]

What drives Make and Model are you using in your FreeNAS ?

 

[SilverNodashi]

I don't entirely agree with that statement.

The scenario you speak of is if you have 2 disks that fail entirely (eg. completely unusable). You can have 3 disks that are starting to fail, eg. CRC errors on all 3 of them (on a RAID 5 with 3 disks), so long as at least 2 of the parity blocks are recoverable over the 3 volumes that block will be completely recoverable.

You don't need to agree, don't worry I'm talking about experience and if you google around you'll see how many other companies have had this problems as well.

RAID5 can only sustain a single drive failure. Loosing a 2nd drive in the process will loose all the data. You're welcome to try this on your own NAS: Take out one drive, so the RAID array is degraded. Then, take out a 2nd drive.


ZFS, on the other hand (which is probably what you're thinking off, and using) can allow for extra parity drives, but then you're essentially using something equivalent to RAID 6 in anycase.

Also ZFS is self healing, so if you start seeing ZFS reporting that it is encoutering errors during self healing, you know it's time to swap out the bad drive. Self healing in this case means, while the disk isn't being used a service moves randomly on the drive and scans blocks for damage. If any damage is detected it is repaired (and the ZFS statistics will show that errors have been corrected).

That allied with SMART makes me believe the chances of RAID Z1 (RAID 5) being unrecoverable becomes much lower.

This is only useful if a drive gradually fails. If if dies all of the sudden, with no pre-warnings then the ZFS self-heal option won't help much. For example, when a bad batch of drive (think Seagate 7200.11) is introduced and their controller suddenly die then you'll still loose the array. Unless, you by chance have an equal amount of spare / parity drives to the amount of actual data drives. i.e. 3 parity drives & 3 data drives. Doing this on a home NAS isn't very difficult or too costly, but imagine you want to setup a 24 drive NAS (for say 22TB storage on RAID6 / ZFS2) then you'll need an additional 24 parity / spare drives to compensate for a bad drive batch. So, now you need another 4U chassis and have them linked through a SAS HBA,or use a 48bay chassis (which is large and noisy). You also need to fork out 1.5 - 2x the initial cash layout. Just cause you relied on ZFS's self heal option.

Rather spend the extra cash on a 2nd NAS, for backup / syncing the data across, and make sure the drives are a different brand / model / from different supplier / date range.


I prefer to buy a few new drives every month and then mix the drives in all the NAS's so that if a certain batch is faulty, at least only a few drives are affected, or lost, and not everything at once.



[/QUOTE]

 

[CriticalConsumer]

I am using Seagate 1,5TB 32MB cache drives.

What drives Make and Model are you using in your FreeNAS ?

 

[Gnome]

You don't need to agree, don't worry I'm talking about experience and if you google around you'll see how many other companies have had this problems as well.

RAID5 can only sustain a single drive failure. Loosing a 2nd drive in the process will loose all the data. You're welcome to try this on your own NAS: Take out one drive, so the RAID array is degraded. Then, take out a 2nd drive.

I should rephrase perhaps, I believe that a COMPLETE drive failure followed by a second COMPLETE drive failure to me seems very unlikely. In fact I think a single SUDDEN drive failure seems unlikely (I have yet to experience one on a system that runs 24/7 without moving). On disks that move around, or faulty (7200.11 series) I agree it is a risk.

I posted, in another thread, a link to hard-drive study (I got the original link from an Anandtech article). The study collected from suppliers (eg. like Esquire, Rectron, etc. not in SA however) the failure rate of hard disk drives:
- 9,71% : WD Caviar Black WD2001FASS
- 6,87% : Hitachi Deskstar 7K2000
- 4,83% : WD Caviar Green WD20EARS
- 4,35% : Seagate Barracuda LP
- 4,17% : Samsung EcoGreen F3
- 2,90% : WD Caviar Green WD20EADS

It is worth noting that Intel admitted these results are accurate (for their own SSD in the article), as well as some other manufacturers. These results are probably very close to correct.

At those failure rates you can see the low chances of single drive failure. Remember we are talking about Home use, and those stats are perfect because they are consumer drives.

Studies done by companies and google threads by corporate admins are useless to us because they don't use normal desktop drives.

ZFS, on the other hand (which is probably what you're thinking off, and using) can allow for extra parity drives, but then you're essentially using something equivalent to RAID 6 in anycase.

True,
RAID Z1 = same parity as RAID 5
RAID Z2 = same parity as RAID 6
RAID Z3 = No such RAID level but allows for 3 drive failure.

This is only useful if a drive gradually fails. If if dies all of the sudden, with no pre-warnings then the ZFS self-heal option won't help much. For example, when a bad batch of drive (think Seagate 7200.11) is introduced and their controller suddenly die then you'll still loose the array. Unless, you by chance have an equal amount of spare / parity drives to the amount of actual data drives. i.e. 3 parity drives & 3 data drives. Doing this on a home NAS isn't very difficult or too costly, but imagine you want to setup a 24 drive NAS (for say 22TB storage on RAID6 / ZFS2) then you'll need an additional 24 parity / spare drives to compensate for a bad drive batch. So, now you need another 4U chassis and have them linked through a SAS HBA,or use a 48bay chassis (which is large and noisy). You also need to fork out 1.5 - 2x the initial cash layout. Just cause you relied on ZFS's self heal option.

I agree but I find that drives usually show signs then failure. I hear you, that is corporate environment however. I simply stated for a 6 drive array, I feel RAIDz2 is overkill. Perhaps not in a corporate environment but definitely in a home environment. In a corporate environment I'd easily go for a 16 drive RAIDz3 setup.

However it is worth noting that the 7200.11 drive failures weren't synchronized based on lifetime hours. So if you had a single drive fail from that firmware problem it is unlikely it would have happened all at the same time. Not to mention that the problem only surfaced during powerup. NAS usually run 24/7 (well I know in our orginization they regularly restart the servers, so it could happen).

Rather spend the extra cash on a 2nd NAS, for backup / syncing the data across, and make sure the drives are a different brand / model / from different supplier / date range.

Remember tho, we are talking about a home setup

The problem you are faced with is this:
Not all providers use 4KB sector size drives. If you start mixing 512b sector size drives and 4kb sector size drives even in a corporate NAS you are going to have serious, very very serious performance and relaibility issues.

Western Digital CLAIMED that their 4kb sector size drives didn't experience higher return volumes but in the US a study proved that was indeed false (with almost twice the failure rate, check above the WD20EARS VS the WD20EADS, only difference, 4kb vs 512b). My opinion on that is that the drives weren't properly partitioned.

I prefer to buy a few new drives every month and then mix the drives in all the NAS's so that if a certain batch is faulty, at least only a few drives are affected, or lost, and not everything at once.

Once again, in a corporate NAS, I agree 100%, home use however I would say the financial implications are too great. Like the 7200.11 series, you just have to keep your eyes out for faulty batches.

 

[SilverNodashi]

Gnome, even if we are talking about home use, if someone values his data enough to get a NAS, then surely he should value it enough to backup that NAS as well. If he doesn't, well then that's his own fault for being hard-headed. And for listening to you

Take a look on NAS forums to see how many people loose all their data on small NAS units (2 - 5 drives) build for home usage cause they didn't back it up. And cause they used Green HDD's. Just cause my wife and I consider ourselves as semi-pro photographers and needs 30TB NAS @ home, does it make us corporate users suddenly? Most of the photos & videos on that NAS is for our own usage and never actually generated an income.

Do you consider a home-NAS-user as someone who has less than say 8TB storage?


P.S. I never actually mentioned anything about 4K drives, that's a story all on it's own.


P.S.S. To anyone else reading this: A NAS is NOT a backup device, it's purely a device which allows you store a crap load of data on one single machine and give you the ability to change failed HDD's with easy and no downtime. If you don't back up the data on the NAS, and you loose it (data loss is a reality, accept it) then you're the only one to blame. Not the sales rep at the company who sold you the NAS (and probably tried to tell you to have a backup device as well) or someone on a forum like this who suggested that only one device is sufficient cause having 2 of them (or even just any other backup medium) is too costly for home-use.

 

[ponder]

Take a look on NAS forums to see how many people loose all their data on small NAS units (2 - 5 drives) build for home usage cause they didn't back it up.

P.S.S. To anyone else reading this: A NAS is NOT a backup device...

Many people out there are under the misconception that it is, to their detriment of course.

 

[Gnome]

If he doesn't, well then that's his own fault for being hard-headed. And for listening to you

I think your problem is you live in the ideal world where everyone can afford to build 30TB NAS devices. I can hardly afford a 6 drive NAS, forget RAIDz2. That is out of my reach. I need X amount of storage and I need it as cheap as possible. If I want to build a 9 drive NAS with RAIDz2 it would cost at least R4K more. To you that must be nothing but to others that isn't worth the data.

Yes, yes NAS isn't a backup device bla bla bla. That at least is in the reach of the average user, cost wise. You seem to want to insinuate that storing data on a NAS is less safe than just a bunch of hard-drives?

I'm providing the statistics of home drive failures. They say the failure rate is X% now it is up to the user to determine if they are willing to "hard headed and listen to me" (I think of it more in terms of thinking for themselves and making an informed decision but whatever). Your argument is that by the time a single drive has failed, another will fail, etc. but rather than post proof of that you go around taking about theoretical situations that *could* occur less than 0.1% of the time for all we know (but we don't). I don't like creating these theoretical situations because to me they are useless. You make the claim that RAID 5 is pointless because the drives will all fail simultaneously one after the other, what do you base that on?

I don't know what the situation is for the people on those "NAS forums". As far as I'm concerned they may be using a flawed implementation. Don't even get me started on the possible problems (drive dropped because TLER isn't supported by drive, array dropped because a single unrecoverable block was encountered, during one of the write operations the power failed and caused incorrect parity data to be written, etc.). That is why I don't care for those forums. I only compare my NAS which uses ZFS to other NAS which also use ZFS (the same version and OS). And even then it is posted on a public forum and almost all the "I've lost all my data" threads, involving RAIDz1 with supplied logs come down to some cock up on the users end.

NAS Forums aren't a good study and it reminds me of this.

As my research Professor said, anyone whom uses techniques like that to draw a conclusion is an idiot [sic].