Due to the astronomical growth of data over the past decade or so, enterprises are struggling to find ways to balance the needs between business growth and its ease of operation. One of the many challenges for modern IT is to archive cold data (i.e. less frequently accessed data, to relatively cheap storage such as tape, optical storage media like bluray/DVD) in order to free up space from the more expensive storage tiers, such as primary storage and all-flash arrays. In addition to freeing up space, verticals such as Government, HealthCare, Legal, Financial and Telecom have a specific need to retain their cold data for a long time for many reasons.
Chief among these:
Offsite archival of cold data offers a very cost-efficient way to fulfill this objective. Let’s talk about how IT is adopting the cloud for their data archival needs.
Today’s “state-of-the-art” archiving solutions are standalone solutions that require customers to manage multiple licenses from different vendors for data protection as well as for archival. They require multiple UIs, gateways, servers, and even writing custom scripts to accomplish tasks not native to the solution. Don’t even get me started on the complexities of managing tape libraries for archival and recovery. In most organizations, the standard practice for archival is to archive data to the tapes and ship them off-site (e.g. Iron Mountain).
In my view, cloud offers a cheaper and more agile way to address an enterprise’s data retention goals. Recovery from these archives may be done in minutes to a few hours as compared to days/weeks in a traditional tape-based archival solution. For those enterprises whose compliance goals allow them to get away from tape, our cloud archival is an order of magnitude simpler. Now, let’s discuss our storage platform integration with both Public and Private Clouds for archival and why it is way better than the current Cloud Gateway solutions for data archival.
At Cohesity, we take great pride in delivering the most comprehensive, flexible and enterprise-grade solution to our customers. Cohesity Data Platform has integration with major Public Cloud service providers, and also integrates with any Private Cloud Object Store that is AWS S3 (Simple Storage Service) compliant.
There are two main use-cases for which Cohesity leverages the power of Cloud storage:
In this post, we’re going to focus on Cloud Archival. Our Cloud Tiering solution is worthy of its own post in the future.
Here are some additional highlights of our Cloud Archival solution:
Now let’s dive deeper into our solution. In the spirit of flexibility and options for the customers, our platform supports all major public cloud services, including Amazon (AWS), Google Cloud Platform, and Microsoft Azure. You can pick the cloud provider of your choice and archive your data to the storage tier of your choice based on your needs. For example, if you are already invested in AWS then you can select either S3, S3-SIA (if you care about retrieval time and are fine with higher cost) or Glacier (if you care about cost and are patient enough to wait for a few hours before you can retrieve your data). If your performance, cost and compliance objectives are met by Google Nearline or Azure Standard Storage, then choose them instead. We don’t have a dog in this fight and strive hard to play fair with everyone. All of these are still provided by our single pane of glass!
But, what if security (and yes cost) objectives don’t allow you to push your data out of your data center but rather archive it to the private cloud that your IT staff has diligently setup? No worries, we have that covered! Cohesity can also archive to any S3-compatible object storage!
We have painstakingly tried to simplify setting up the archival workflow and managing the archival data sets when you need them. Here are the simple steps to setup the data archival:
That’s it! You are all set to start archiving your older snapshots from Cohesity to the cloud storage service of your choice! As per your requirements, you can easily use multiple cloud storage providers with different RPO and retention objectives.
By default, data-in-motion is transferred to the cloud over secure channels (HTTPS). However, for data-at-rest, we offer additional security with different keys across different cloud vaults. These keys are managed by our internal key management system that allows you to rotate your encryption keys at any point without having to re-encrypt the entire datasets with new keys. That’s how we provide industry-compliant security for your archived data.
When we archive your data to the cloud, index metadata is also stored along with the archived data set. This enables you to execute a full recovery to a totally new Cohesity cluster, in the event your primary cluster had some unrecoverable failures. In addition, this also empowers you to create another copy of your archived data from the cloud to a new Cohesity cluster. As the primary Cohesity cluster has metadata index stored for all the archived local snapshots, you will be able to search for any specific file in a VM from your on-premises Cohesity cluster and restore it from a specific cloud-archived snapshot.
There you go! A scalable, easy-to-manage, and flexible solution for long-term retention and performance that other cloud gateway solutions cannot provide!
When talking about long-term archival, the question of support for tape libraries always comes up. If you find yourself in a position where you still have to deal with tape, have no fear! We will cover how we address even that seamlessly, in a future blog post. Stay tuned with our upcoming releases packed with unprecedented innovation!
What challenges do you face when it comes to archival? Are you currently considering leveraging the cloud as an alternate solution? Let us know in the comments below how we can help you to make the transition!