Wednesday, 5 July 2006

Virtual tape libraries

I previously mentioned virtual tape libraries. Two examples of products I've been looking at are Netapp's Nearstore Virtual Tape Library and ADIC's Pathlight products. Both effectively simulate tape drives and allow a virtual tape to be exported to a physical tape. Here are some of the issues as I see it:

1. How many virtual devices can I write to? Products such as Netbackup do well having lots of drives to write to; a separate drive is needed (for instance) for each retention period (monthly/weekly/daily) and for different storage pools. This can cause issues with the ability to make most effective use of drives, especially when multiplexing. So, the more drives, the better.

2. How much data can I stream? OK, it's great having lots of virtual drives, but how much data can I actually write? The Netapp product for instance can have up to 3000 virtual drives but can only sustain 1000MB/s throughput (equivalent to about 30 LTO2 drives).

3. How is compression handled? Data written to tape will usually be compressed by the drive, resulting in variable capacity on each tape, some data will compress well, some won't. A virtual tape system that writes to physical tape must ensure that the level of compression doesn't prevent a virtual tape being written to the physical tape (imagine getting such poor compression that only 80% of data could be written to the tape, what use is that). However the flip side to this is ensuring that all the tape capacity can be utilised - it's easy to simply write 1/2 full tapes to get around the compression issue.

4. How secure is my data? So you now have multiple terabytes of data on a single virtual tape unit. How is that protected? What RAID protection is there? How is the index of data on the VTL protected? Can the index be backed up externally? One of the great benefits of tape is the portability. Can I replicate my VTL to another VTL? If so, how is this managed?

5. What is the TCO? Probably one of the most important questions. Why should I buy a VTL when I could simply buy more tape drives and create an effective media ejection policy? The VTL must be cost effective. I'll touch on TCO another time.

6. The Freedom Factor. How tied will I be to this technology? The solution may not be appropriate, the vendor may go out of business. How quickly can I extricate myself from the product.


Victor, HDS PMM said...

Hi Chris, you nailed most of the issues, but there is one that you missed that is critical to my customers and that is scalability. Most of the solutions on the market do well with small amounts of storage (under 10TB) but fall apart with lots of data. Especially those with data de-duplication features. We recently launch a VTL solution (a partnership of HDS and Diligent Technologies) that adresses all of those issues. Our data de-duplication VTL solution called ProtecTIER provides enterprise class performance with a PetaByte of data and guarantees 100% data integrity.

Chris M Evans said...

Thanks Victor, good comment, in fact, I posted on the product release just as you'd made the comment. Scalability is an issue; especially if developing a D2D2T solution. I will be looking further into the Diligent solution; I'd like to get answers to the couple of points I raised - perhaps you can answer?