While it is undeniable that prudent organisations should take every possible step to protect their data, it’s worth questioning at what level and at what associated overhead should that protection take?
Do the traditional ways of thinking about resilience and availability still apply with current and emerging technologies?
Commodity HDD:
Traditionally, data protection has required a multi- tiered approach, and to a degree this remains true, but technological advances have rendered some of the traditional methods increasingly obsolete. Take RAID as an example, which, it has been argued, is becoming increasingly outdated. When dealing with Hard Disk Drives (HDDs) it absolutely makes sense to use RAID to protect the data that is held on them. If we were to lose a large unprotected HDD, for example a 10TB drive that are commonplace and readily affordable nowadays, then you also need to factor in the recovery of the data from that drive. Recovery itself would take a considerable amount of time, and associated staff costs while recovering the data from backup. Add to this lost goodwill, loss of revenue from the downtime, and other intangibles, and the choice of not having RAID on the HDD, could be hugely detrimental to the organisation.
It’s all about the Flash: But Consider, Not all Flash is Created Equal.
Okay, but what about Flash? Should we use the same RAID methods to protect our flash resident data? That depends. If you are relying on consumer grade Solid State Drives (SSDs) to give your performance levels lift, then considering RAID hardware level protection would be wise. The quality of Flash and the amount of data that can be written to consumer flash remains considerably less than Enterprise grade flash devices, with values such as PetaBytes Written (PBW) and Data Writes Per Day (DWPD) usually significantly lower in their Enterprise grade cousins. Technologies such as ‘wear levelling’ which ensure that the flash drives are written to in an even fashion are at best poor, at worst, non-existent in consumer options.
Enterprise Grade Flash – Monthly ‘Wear’ Monitoring is Essential
When using Enterprise Flash in a software defined storage environment that uses mirrored vdisks then there is little to be gained from adding the complications and latency of RAID into the mix. Device failures in Flash are several orders of magnitude less frequent than in HDD, although you should always check the wear levels on your flash devices and do this at least monthly – again most consumer grade flash lacks this ability. For Enterprise Flash, if the vdisk is mirrored in DataCore or other software defined platform, resilience looks like having another flash device available and, in the event of failure, you can add the device to the pool and purge the failed device. This can be done pre-emptively according to the results of the monthly flash monitoring.
Rethinking RAID in Enterprise Grade Flash
It is worth noting that Flash devices can typically drive IO faster than a RAID controller. For this reason alone, there should be a rethink on using RAID with Flash devices, and some available RAID controllers even have a pass-through mode for Flash devices. If you really require physically separate Flash devices with redundant copies of the same data, use the “Pool Mirror” functionality within software defined platform to add a secondary copy of the data within the storage pool itself.
Finally, never forget the value of Read Cache. If you can get the installed memory in the software defined storage nodes to approximately 1% of the capacity of the storage under management, you could see as little as 10% of the Reads reach the backend, improving read latency by up to 3 orders of magnitude. (Noting that the number of vdisks under management and features used against the vdisks (snapshots, CDP etc.) could affect the amount of memory required).
If you are already using DataCore as your software defined storage platform, then please view the DataCore Server System Memory Considerations document or contact your DataCore partner or Solutions Architect for more information before changing memory settings.
Further reading:
