In today’s data-driven business world, organizations are facing increasing pressure to find smart and affordable solutions to store the vast amounts of data being created. On one hand, there is data generated by tier 1 applications and databases that need high-performance block storage solutions. Then, there are greater volumes of unstructured data comprising user-generated documents, images, audio and video files, etc. which IT departments are looking to store on economical and user-friendly file and object storage targets. Redundant data, snapshots, and backups are also stored on inexpensive secondary storage (which is mostly object storage either on-prem or in the cloud) for long-term preservation.
One typically thinks of object storage as “cheap and deep” storage for cold data. While this might be true, there is much more to object storage than using it as a backend storage for cold data. More than just the economics of storage media and affordability, there is greater management simplicity and end-user convenience in storing unstructured data on object storage. Let’s take a closer look at how organizations can benefit from a massively scalable S3-enabled on-prem object storage platform like DataCore Swarm. The following use cases apply broadly to any industry/vertical based on IT and end-user requirements. The industries highlighted in this blog show the diversity of Swarm’s capabilities and extent of adoption within our customer base.
Simplified Video Storage, Delivery and Streaming
One of the more popular uses cases for Swarm is in the media and entertainment industry where there are surging requirements for storing video files. We are not talking about just archival and long-term preservation here. The continuous generation of video content (for example, from sports and events coverage, broadcast news reporting, video surveillance feeds, etc.) need to be stored securely and then accessed, edited, streamed, and shared for re-use and continued collaboration.
DataCore Swarm provides a robust platform for video storage and on-demand delivery to support digital video workflows. It allows users to:
- Store content securely in your data center or private cloud with built-in data resilience to handle hardware failures
- Find assets quickly with integrated search, custom metadata, and playback accessible via a web-based portal or API
- Ingest, view, and restore files efficiently through parallel uploads, web-playback range reads and partial file restore, allowing you create a smaller video clips from a larger file
- Keep content instantly accessible and available with minimal manual administration (Digital Asset Managers can directly handle video management using Swarm’s web-based content management interface)
Swarm is ideal for content delivery origin storage and long-tail delivery, enabling over-the-top (OTT) services, and can be used as the storage target for Digital Asset Management and Media Asset Management solutions.
Multi-Tenant S3 Storage for High-Performance Computing Environments
High-performance computing (HPC) environments, such as in research and analytics, industrial applications, life sciences, etc. have very particular requirements for content access and collaboration. Files require higher throughput for read/write access, simplified management across sites, tenants, and users with access controls, while seamlessly scaling to meet growing capacity demands.
Swarm provides a high-performance production infrastructure at scale simplifying content management and data access on highly resilient and cost-effective object storage. Using S3 and HTTP(S) protocols, Swarm can enable content access to internal employees and external users through an intuitive web-based content management portal.
Without having to use any separate caching layer, Swarm uses a parallel architecture of self-balancing and seal-healing nodes that distribute load and balance capacity automatically. This enables Swarm to handle many concurrent requests in parallel yielding the full throughput potential of all the drives in the system.
Swarm’s multi-tenant architecture allows administrators to create buckets with capacity and bandwidth quotas and set unique data protection policies and access controls. In addition, Swarm provides metered storage for chargebacks creating detailed billing and accounting reports for bandwidth usage, access audits and API request summaries.
Scalable Medical Image Archive and Patient Record Management for Healthcare
Healthcare organizations are facing exponentially growing digital data sets (in the form of medical images, X-rays, scans, videos, electronic health records) and stringent retention and protection requirements.
Swarm provides many data services and infrastructure management capabilities to ensure secure object and high durability to help comply with industry regulations such as HIPAA. Storage administrators can easily migrate medical images that reside on the existing primary storage to Swarm where they are consolidated in a searchable, secure (AES 256-bit encryption/SSL), and expandable pool of storage. Users are able to:
- Maintain backup of medical images and patient records on affordable object storage
- Leverage customizable metadata to make stored objects more easily accessible for search and big data analytics
- Take advantage of RESTful APIs to directly write content as object stores to Swarm and use it as a medical image archive
- Ensure business continuity 24×7 with continuous integrity checks and fast volume recovery
- Use any hardware platform and scale to hundreds of PBs easily, leveraging interoperability with leading Vendor Neutral Archives (VNA) and medical imaging applications
Eliminating Storage Silos for Enterprise IT and Managing Unstructured Data at Scale
With tighter budgets and limited resources, IT teams are always strained to get the most out of their investment in storage and other technologies. Especially in the storage context, having data on disparate systems with isolated capacity management creates hardware silos and capacity wastage. Also, there is a chance of primary storage being filled up with inactive data leading to faster capacity exhaustion and performance bottlenecks.
Swarm forms a highly resilient cluster of server nodes, from standard x86 server hardware, that have the intelligence and automation to load-balance and self-heal allowing for efficient scaling and faster recovery. Swarm runs from RAM leaving over 95% of the disk space for actual data storage needs. New server nodes can be added easily; and they start collaborating with the existing nodes in the cluster instantaneously thus enabling seamless operations.
By offloading warm and cold data to affordable object storage, resources on NAS devices can be freed up and NAS expansion costs minimized. DataCore FileFly is an optional add-on solution available with Swarm to move data from NetApp and EMC Isilon NAS devices and Windows file servers to Swarm.
Securely Accessible Storage with Built-in Data Integrity and Compliance Enablement for Government
Swarm is trusted and used by local and federal governments in many countries for its robust encryption, access controls and integrity verification functionalities. Regulatory compliance is key in such environments and Swarm ships with various security capabilities and integrations to meet compliance requirements.
- Supports data encryption in transit and at rest, SEC 17a-4(f) certification, and is FIPS-140 compliant
- Assures data immutability with Legal Hold and WORM integration (this also helps files from being accidentally deleted)
- Snaps into existing identity management systems such as LDAP, AD, ApacheDS, and Linux PAM and additionally supports token-based authentication through Amazon S3 API
- Establishes content integrity using Content-MD5, Integrity Seals, and Lifepoint Metadata Headers
- Tracks storage access and activity with comprehensive audit trails
Enabling Storage-as-a-Service for Private Cloud and Public Cloud Services
While the cloud is touted as an inexpensive object storage location for cold data, the mechanics of storing and accessing data on-demand comes with many hidden costs. For example, there are ingress and egress fees being charged by many leading public cloud providers inflating the total cost of ownership (TCO) of cloud projects. In addition to that security is a chief concern for many companies looking to preserve their data in the cloud.
Swarm overcomes all these shortcomings by providing an affordable and secure private cloud platform that you can run within your data center. If you are a managed service provider, you can use this private cloud object storage and provide storage-as-a-service to your end-customers. As we saw earlier, the multi-tenant architecture of Swarm allows service providers to easily create tenants and manage capacity quotas, licenses, and access controls to address different customer needs.
There are multiple ways you can leverage Swarm in your cloud strategy:
- As a scalable private cloud platform to host unstructured data on-premises
- Offer storage as-a-service services to your partners and clients
- Part of a hybrid cloud strategy where you use Swarm as an S3-compatible active archive and leverage other any other public/private cloud as in active archive or for DR
Swarm delivers a competitive advantage that will increase subscribers, reduce churn and optimize your OPEX and CAPEX spend.
As we saw, there are so many use cases for which Swarm can be used and so many ways it can help: simplify data management, reduce cost, automate operations, ensure data availability, and so on. Broadly these use cases can apply to any industry and any IT team looking for a cost-efficient object storage solution for data storage, delivery, collaboration, and management. Contact us to learn more about DataCore Swarm.
