Storage Virtualization

Storage virtualization (also sometimes called software-defined storage or a virtual SAN) is the pooling of multiple physical storage arrays from SANs and making them appear as a single virtual storage device. The pool can integrate unlike storage hardware from different networks, vendors, or data centers into one logical view and manage them from a single pane of glass.

Virtualizing storage separates the storage management software from the underlying hardware infrastructure in order to provide more flexibility and scalable pools of storage resources. And not only that, but it can abstract storage hardware (arrays and disks) into virtual storage pools in the same way compute virtualization (VMWare ESX or Hyper-V) abstract compute hardware (servers) into virtual machine instances (VMs).

The Storage Network Industry Association (SNIA) even has a unique storage virtualization definition:

The application of virtualization to storage services or devices for the purpose of aggregating functions or devices, hiding complexity, or adding new capabilities to lower level storage resources.

This technology has been used over the years to solve many of the challenges with growth, managing large amounts of storage. Virtualized storage has become even more important today as the amount of data stored continues to grow exponentially every year.

Storage virtualization can easily be used to address the following top 7 challenges:

1. Vendor lock-in
2. Data migration across arrays
3. Scalability
4. Redundancy
5. Performance
6. High costs
7. Management

While benefits of storage virtualization are substantial, there are many technical hurdles to overcome. One of the top hurdles is with storage environments containing heterogeneous hardware and software components, often supplied by many different vendors.

By virtualizing storage, a system can be made more flexible because commodity hardware can be quickly added on and discovered by the virtualization software platform. These hardware resources can then be joined to a virtual pool of storage that is available to the virtualization platform.

This has driven one of the largest trends in the storage industry, which is a “scale out” approach in which hardware capacity can be added quickly in a modular fashion. Virtualization can aggregate and manage data across a wide range of physical assets in large networks or data centers. This data can be consolidated to isolate performance issues, predict and troubleshoot problems, and plan for future capacity needs.

Regardless of the storage hardware or vendor you own today, the software is the brain and layer of intelligence that determines the functions, features, services, and benefits you will be able to offer to your hosts, applications, and end users.

If you find yourself losing sleep at night due to existing SAN struggles and challenges, then this is the best time for you to evaluate storage virtualization software. The number of benefits are too many to ignore.

Your organization cannot afford to continue depending solely on traditional storage. It is time to make modernize your storage architecture.

One major reason why companies are switching to a storage virtualization model is the need to consolidate and manage all existing storage under a single console, while also leveraging a set of diverse features and functionalities.

A storage virtualization node is essentially a virtual controller that virtualizes and manages your physical storage. All your disk arrays are placed inside a “virtual pool” and thin-provisioned for maximum capacity. Once the virtual pool is created, “virtual disks” are also created in the pool and then presented to your host servers as raw LUNs to store data.

This means you can start mixing arrays from your HPE 3Par, Dell Compellent, Pure Storage, or any SAN inside your data center and consolidate them into a virtual pool. This in addition to striping your application data across all the arrays regardless of how fast or slow the disks may be.

This virtualization storage solution will automatically move cold and hot data on the fly across all arrays by simply assigning a tier number to each of array. Isn’t that amazing?

Most popular traditional SANs are not capable of auto-tiering data across competitor’s arrays. In other words, a Compellent array cannot auto-tier with a Pure or 3PAR array because it is not supported by those vendors.

The quick fix to this roadblock is solved with a storage virtualization node that can operate every possible SAN in the world. Storage virtualization software can help you accomplish what traditional SAN’s cannot do natively. In case you ever need to replace any of the expensive traditional SANs, you have the freedom to find more cost-effective alternatives and still stay mix and match your physical storage.

Don’t worry about your data migration from the old SAN to the new less costly storage. Your virtualization software platform makes this process easy and painless. With just 3 clicks, you can start migrating your data from the old clunkers to the new arrays, with little to no downtime to the applications and host operating system.

The idea of fixing storage challenges by simply adding a piece of software may sound too hard to believe. But it is true: it is that easy. Let us show you the 2 most common types of storage virtualization available today.

Keep in mind that your existing SAN is also made up of software + hardware. With storage virtualization, you are merely upgrading the software piece and keeping the same hardware. That makes this architecture easy to deploy and migrate to.

Therefore, you need to determine if you only want to virtualize existing storage or if you want to add new storage and get rid of your old legacy arrays.

Option 1: Storage Virtualization Node

The first option consists of a storage virtualization controller node. This is typically a 1u x86 server with CPU, memory, network ports, and a raid-1 for the operating system to host the virtualization software.

The next step is to interconnect your existing SAN arrays via iSCSI or FC connections to the new storage virtualization controller. Then you can connect your hosts next and begin to present virtual disks to each of the hosts.

At this point, you only need to open a single management console for your daily storage admin tasks. You still get all the enterprise-level storage services your virtualization software offers, even if the underlying software is not licensed for those features and functions.

If you ever need to bring a newer array and remove an old array, you simply click on “remove” to get rid of the old one and click on “add to pool” to start using the new storage. You can add a second node with dedicated storage to build a highly available storage group with 2 redundant active/active copies of your data.

Option 2: Converged Server SAN

This second option will need a bigger 2u x86 server with higher hardware specs to produce more punching power. You really want to max your ROI by supersizing your disks, memory and networking ports.

If you get the backplane with 24x 2.5” drives, you can pack a lot of capacity using a mixture of 4TB NVMe drives and/or SAS 10k drives. In some cases, we have seen solutions hosting up to 60TB or even 80TB of usable capacity per 2u server.

You still get the same benefits mentioned with the first option, but now you are adding a new twist by mixing new internal disks and existing external SAN arrays under the same virtual pool. You are getting the best of both worlds and still pushing 100k plus IOPS with sub-millisecond response times.

Most organizations start out with the storage virtualization node and eventually convert it to a converged server SAN without incurring a huge CAPEX when adding the internal drives for capacity.

With this setup, all of your performance issues will be a thing of the past, and you can continue to scale by adding more arrays or nodes.

• More redundancy, check.
• Lower costs, check.
• Managed storage from a single pane of glass, check.
• Performance and scalability, check.

The pioneer in storage virtualization was DataCore Software, a company founded in Fort Lauderdale in February 1998 by George Teixeira and Ziya Aral. The company was created to build software that would allow IT departments to buy commodity storage across different vendors and access them as virtual disks through block storage.

Over time, other companies have created similar storage virtualization products. However, DataCore continues to innovate, developing new technologies such as auto-tiering and synchronous mirroring, resulting in over a dozen patents for the core technology.

As of October 2018, storage virtualization has been broadly adopted, but it is not pervasive across IT departments. Market research conducted by DataCore indicates adoption of storage virtualization is about double that of hyperconverged (37% vs. 21%), with 56% of respondents also strongly considering or planning to consider software-defined storage in the next 12 months.