Redundant Array of Inexpensive Clouds - Pt III

In my two previous articles I discussed Cloud Storage and the concept of using middleware to store multiple copies of data across different service providers. In this final part, I'd like to discuss the whole issue of security.

Using "the cloud" to store data requires a major shift in thinking; traditionally all your information would be stored locally and therefore benefit from the advantage of physical security. Not only would someone need to hack your firewall to get network access, they would then have to obtain system access too, and likely as not would be spotted (hopefully) quite quickly. So, retaining physical access to data has been a significant benefit.

Now we've obviously been trusting a form of cloud storage for some time. Email systems like Gmail, Hotmail and Yahoo have always had access to our email data and have provided limited storage capabilities but they haven't really been the foundation for running a business (although I'm sure there are organisations that have done it). Putting data into the cloud means there's always a risk of someone else getting to your data. You make someone else the guardian or gatekeeper of that data access and rely on the quality of their encryption and access controls. So, it is important to understand what facilities each infrastructure provider offers.

Amazon Web Services

Amazon have a great whitepaper on security, which can be found here. It highlights the level of physical security offered (which is high) plus details of the logical security of data. It may seem surprising that Amazon don't routinely back up data on AWS but rely instead on multiple copies in remote locations, however backup and archive should be thought of as distinct requirements. In addition, data at rest in AWS is not encrypted; users of AWS should therefore ensure their service provider offers this capability at source.

Nirvanix

Nirvanix have two white papers which discuss data security. They can be found here (registration required). As with Amazon, Nirvanix are keen to highlight the security of their facilities and adherence to Statement on Auditing Standard (SAS 70) certification. They also go further in indicating that data is stored using RAID-6 and RAID-10 protection, with backups in place too.

Summary

Both AWS and Nirvanix offer good physical security and SSL encryption for data in flight. Encryption at rest and backups are not routinely offered and therefore a cloud user should weigh up how these features are to be implemented. This takes us back to the original premise of these postings, the idea of using multiple cloud providers to add resilience and availability to cloud stored data. It also demands a set of standards for cloud storage use, which I am working on even as I write this post. Watch this space.

Redundant Array of Inexpensive Clouds - Pt II

In my previous post I started the discussion on how cloud storage could actually be useful to organisations and not be simply for consumer use.

Standards

One of the big issues that will arise is the subject of standards. To my knowledge, there is no standard so far which determines how cloud storage should be accessed and how objects should be stored. Looking at the two main infrastructure providers, Amazon and Nirvanix, the following services are offered:

Amazon

S3 (Simple Storage Service) - storage of data objects up to 5GB in size. These objects are basically files with metadata and can be accessed via HTTP or BitTorrent protocols. The application programming interface (API) uses REST/SOAP (which is standard) but follows Amazon's own standards in terms of functions to store and retrieve data.

Elastic Block Store (EBS) - this feature offers block-level storage to Amazon EC2 instances (elastic compute cloud) to store persistent data outside of the compute instance itself. Data is accessed at the block level, however it is still stored in S3.

Nirvanix

Storage Delivery Network (SDN) - provides file-based access to store and retrieve data on Nirvanix's Internet Media File System. Access is via HTTP(S) using standard REST/SOAP protocols but follow Nirvanix's proprietary API. Nirvanix also offer access to files with their CloudNAS and FTP Proxy services.

The protocols from both Amazon and Nirvanix follow standard access methods (i.e. REST/SOAP) but the format of the APIs are proprietary in nature. This means the terminology is different, command structures are different, the method of storing and retrieving objects is different and the metadata format for referencing those objects is different.

Lack of standards is a problem. Without a consistent method for storing and retrieving data, it will become necessary to program to each service provider implementation, effectively causing lock-in to that solution or creating significant overhead for development.

What about availability? Some customers may choose not to use one service provider in isolation, in order to improve the availability of data. Unfortunately this means programming to two (or potentially more) interfaces and investing time to standardise data access to those features available in both products.

What's required is middleware to sit between the service providers and the customer. The middleware would provide a set of standardized services, which would allow data to be stored in either cloud, or both depending on the requirement. This is where RAIC comes in:

RAIC-0 - data is striped across multiple Cloud Storage infrastructure providers. No redundancy is provided, however data can be stored selectively based on cost or performance.

RAIC-1 - data is replicated across multiple Cloud Storage infrastructure providers. Redundancy is provided by multiple copies (as many as required by the customer) and data can be retrieved using the cheapest or fastest service provider.

Now there are already service providers out there offering services that store data on Amazon S3 and Nirvanix SDN; companies like FreeDrive and JungleDisk, however these companies are providing cloud storage as a service rather than offering a tool which integrates the datacentre directly with S3 and SDN.

I'm proposing middleware which sits on the customer's infrastructure and provides the bridge between the internal systems and the infrastructure providers. How this middleware should work, I haven't formulated yet. Perhaps it sits on a server, perhaps it is integrated into a NAS application, or a fabric device. I guess it depends on the data itself.

At this stage there are only two cloud storage infrastructure providers (CSIPs), however barriers to entry in the market are low; just get yourself some kit and an API and off you go. I envisage that we'll see lots of companies entering the CSIP space (EMC have already set their stall out by offering Atmos as a product, they just need to now offer it as a service via Decho) and if that's the case, then competition will be fierce. As the offering count grows, then the ability to differentiate and access multiple suppliers becomes critical. When costs are forced down and access becomes transparent, then we'll truly have usable cloud storage.