Automated Storage Tiering and Data Placement

In the world of data storage, performance and cost are two critical factors to optimize. The performance of your storage hardware has a direct impact on application responsiveness and, in turn, the user experience.

When you can control what data gets placed on which storage device, you can positively impact performance, and at the same time, reduce storage costs that are increasing with growing data volumes.

Every chunk of data stored on your storage infrastructure occupies space and soon fills up available capacity. But not all of the data that gets written to storage is equally important. While there is mission-critical application data that is written to and accessed by databases, there is also a bulk of data generated that is not very frequently accessed, such as snapshots, log files, analytics data, etc.

As an IT/storage administrator, to be able to manually determine what data to place on which storage and to do it in real-time with changing patterns of user access and business conditions is close to impossible. This is when IT teams rely on storage tiering and data placement automation.

DataCore provides enterprise-class software-defined storage (SDS) solutions that deliver the automation you need to dynamically tier data across different storage classes. The software leverages AI/ML-assisted automation to govern data placement based on business requirements.

DataCore offers two different solutions to choose from based on your specific needs. Each of these uses a unique approach to data tiering and placement helping you eliminate futile manual efforts and accelerate time-to-value.

In both choices, data is dynamically migrated between storage tiers based on usage patterns and organizational policies.

They ensure your fastest storage devices store only your most important data, and the less relevant data is automatically moved over to secondary storage and other archival tiers.

This allows you to optimize for cost and performance while maximizing the returns from your existing storage investments.

DataCore SANsymphony is a software-defined storage solution that aggregates resources from disparate storage devices into a virtual storage pool. Within the storage pool, SANsymphony tiers data to the appropriate storage equipment based on how frequently (hot) or infrequently (warm/cold) it is accessed.

Hot data automatically gets placed on the fastest and most performant storage, whereas warm and cold data gets tiered to commodity hardware. SANsymphony uses built-in machine learning algorithms to continuously monitor data access frequencies and dynamically migrate data between tiers.

Typically, hot data are written to the faster flash/SSDs, SAS disks store warm data, and low-cost SATA disks house colder data. If you have faster storage, such as Intel Optane-powered disks in the mix, that will assume the spot of the highest tier, followed by the rest in order of IOPS performance. You simply designate which devices belong in which tier, then sit back, relax and let SANsymphony take over.

Without any manual intervention, SANsymphony will tier data in real-time. The tiering process is automated, fully transparent to the applications, and is continuously happening.

Not just when data gets written to storage initially, but even as data access frequencies change, SANsymphony tracks these pattern changes and dynamically keeps relocating data to the appropriate storage tier as its value changes over time.

Unlike competing alternatives that force an entire volume to be pinned to a tier, SANsymphony distinguishes between blocks within a volume, so only the heavily accessed chunks migrate to faster tiers.

Automated data tiering of hot/warm/cold data

Studies show that typically more than 80% of storage capacity is occupied by dormant data and less than 20% is actively used. You can move all of these dormant blocks to cheaper storage tiers and save the costs of unnecessarily expanding your primary storage.

Data tiering, in addition to other performance-enhancing capabilities in SANsymphony such as high-speed caching, parallel I/O operations, etc., helps to augment storage responsiveness even further

DataCore FileFly software provides a quick and easy means to offload less important file contents from Windows NTFS servers and SMB shares on NetApp and Dell EMC Isilon NAS to lower-cost on-premises object storage.

The destinations include DataCore Swarm and popular S3-compatible cloud storage from AWS, Azure, GCP, Wasabi, and others. Migrated files remain accessible from their original folders and their contents recalled to primary storage when needed.

Using flexible rules and policies, you set which files are migrated and when. You can also schedule migrations outside of peak business hours to reduce I/O contention with production workloads.

For example, you may choose to migrate all files with names ending in *.mp4 (typically large video clips) that have not been accessed in the past 12 months from the Marketing file servers to DataCore Swarm object storage. And schedule the task for Friday nights at 11:30 p.m. when there is very little activity on the servers.

Once FileFly tiers a file’s contents to object storage, it leaves a stub in the on-premises NAS/filer, which serves as a pointer to the new location.

The stub achieves two purposes: 1) it allows users/applications to retain the same file path to access the file when needed 2) it signals backup tools to skip the file for backup, thereby reducing backup time and cost.

File tiering from on-premises NAS/Windows filers to object storage

FileFly includes a data assessment tool and savings calculator that helps you pre-determine how much money you can save by offloading older files to more economical storage.