Over the last year, there has been a lot of commentary about how disk (aka HDDs or magnetic media) is dead. Colorful metaphors such as “spinning rust” are
used to describe the apparent death of the HDD market, but is this really the case?
According to a report from
TrendFocus, the number of drives that shipped in 2015 declined by 16.9% (to 469 million units), however the amount of capacity that shipped increased
by more than 30% (to 538 exabytes, or 538,000 petabytes, or 538,000,000 terabytes). In other words, a lot of HDD capacity.
This is NEW capacity added to the industry on top of the already mind-blowing amount of existing capacity in the field today (estimated at over 10
zettabytes, or 10,000 exabytes, or, well, you get the idea). Eric Brewer, VP of Infrastructure at Google recently said,
“YouTube users are uploading one petabyte every day and at current growth rates they should be uploading 10 petabytes per day by 2021.”
The capacity trend certainly doesn’t show signs of slowing which is why new and improved ways of increasing HDD density are emerging (such as Helium-filled
drives, HAMR, and SMR). With these new manufacturing techniques, HDD capacities are expected to reach 20TB+ by 2020.
So, disk (HDD) isn’t exactly dead, at least from a capacity demand perspective, but it does raise some interesting questions about the ecosystem of drive
technology. Perhaps the conclusion that disk is dead is based on drive performance. There is no doubt a battle is waging in the industry. On one side we
have HDD, on the other SSD (or flash). Both have advantages and disadvantages, but must we choose between one or the other? Is it all or nothing?
MOVE TO FLASH NOW OR THE SKY WILL FALL
In addition to the commentary about disk being dead, there has been an equal amount of commentary about how the industry needs to adopt all-flash
tomorrow or the world will come to an end (slight exaggeration perhaps). This is simply an impossible proposition. According to a past Gartner report,
“It will be physically impossible to manufacture a sufficient number of SSDs to replace the existing HDD install base and produce enough to cater for
the extra storage growth.”
Even displacing 20% of the forecasted growth is a near impossibility. And to take this one step further, not only is it near impossible, it is
completely unnecessary. However, none of this implies HDD and SSD cannot coexist together in peace, they certainly can. What is needed is exactly what
Gartner said in the same report,
“Ensure that your choice of system and management software will allow for seamless integration and intelligent tiering of data among disparate
devices.”
The reason Gartner made this statement is because they know only a small percentage of an organization’s data footprint benefits from residing on
high-performance media.
THE SOLUTION TO THE PROBLEM IS SOFTWARE
One of the many things DataCore accomplishes with the hardware it manages is optimizing the placement of data across storage devices with varying
performance characteristics. This feature is known as auto-tiering and DataCore does this automatically across any storage vendor or device type whether
flash or disk based.
Over the last six years, DataCore has proven with its auto-tiering capability that only 3-5% of the data within most organizations benefit from
high-performance disk (the percentage is even less when you understand how DataCore’s Parallel I/O and cache works, but we will touch on this later). Put
another way, 95% of an organization’s I/O demand occurs within 3-5% of the data footprint.
While the 3-5% data range doesn’t radically change from day to day, the data contained within that range does. The job of DataCore’s auto-tiering engine is
to ensure the right data is on the right disk at the right time in order to deliver the right performance level at the lowest cost. No need to wait,
schedule, or perform any manual steps. By the way, the full name of DataCore’s auto-tiering feature is: fully automated, sub-LUN, real-time, read and
write-aware, heterogeneous auto-tiering. Not exactly a marketing-friendly name, but there it is.
WAIT, I THOUGHT THIS WAS ABOUT DISK, NOT FLASH
While DataCore can use flash technologies like any other disk, it doesn’t require them. To prove the point, our Director, Systems Engineering and Solution
Architecture, Jeff Slapp, performed a very simple test to demonstrate the impact that just a little bit of software can have on the overall performance of
a system. To see the outcome of his test, read his complete blog post here. A more comprehensive analysis of DataCore’s performance
can be seen on the Storage Performance Council’s website.
CONCLUSION
There are two things happening in the storage industry which has caused a lot of confusion. The first is an unawareness of the distinction between I/O
parallelization and device parallelization. DataCore has definitively proven its I/O parallelization technique is superior in performance, cost, and
efficiency. Flash is a form of device parallelization and can only improve system performance to a point. Device parallelization without I/O
parallelization will not take us where the industry is demanding we go (see Jeff Slapp’s article on Parallel Storage).
The second is a narrative being pushed on the industry which says “disk is dead.” The demonstration above proves “spinning disk is very much alive.”
Someone may argue I’m using a flash-type device in the form of RAM to serve as cache. Yes, RAM is a solid state device (a device electronic in nature), but
it is not exotic, has superior performance characteristics, and organizations already have tons of it sitting in very powerful multiprocessor servers
within their infrastructures right now. They simply need the right software to unlock its power.
Insert DataCore’s software layer between the disk and the application and immediately unbind the application from traditional storage hardware limitations.
Jeff Slapp’s complete blog post can be read
here.
