Storage Predictions for 2009

It's the end of another year and of course time to the obligatory posts on predictions in the industry for the next 12 months. True to form, I've spent some time thinking and here are my top 5 ruminations for the coming year.

• EMC join SPC. EMC will finally have the epiphany we've all been dreaming of and embrace the world of generalised benchmarking. DMX will prove to be underpowered (due to the lack of SATA drives and the discovery that SSD is in fact the slowest disk technology) and be outperformed by Tony Asaro and his new Drobo appliance.
• HDS Win At Web Awards. HDS embrace new media in a game changing way and Hu wins first prize at the Weblog Awards. Barry Burke is so impressed, he defects to HDS from EMC, becoming www.thestoragedefeatist.com.
• XIV Conquers the World. IBM releases XIV-1000, by sending Moshe Yanai back in time from 2032, the time when Atmos became self-aware and took over storage for the entire world. EMC counter, sending StorageZilla (now aged 32) back to defeat Moshe in a monumental battle pitting SATA against SSD.
• JWT joins Star Trek. Jon Toigo bows out of storage to follow a career in acting. His first role as the father of Willam Riker in Star Trek XI is critically acclaimed as a work of genius.
• Convoy II: SWCSA is Released. The life of Marc Farley is brought to the big screen as he attempts to outwit the authorities and drive from the west to east coast using nothing but his blogging skills. Farley is portrayed on screen by Kris Kristofferson, reprising his role from the original cult movie.

Check back next year to see how many of these predictions did in fact come true.

Thin Provisioning or Good Practice, which is best? There's only one way to find out - Fight!

Marc Farley makes some interesting comparisons to storage purchasing decisions in a recent post. For the sake of disclosure, I do go to Costco and buy in bulk - no not 200lbs of chicken wings, but those things that can be divided and/or frozen (like salmon and coffee) - and more crucially things that don't become cheaper in price over time.

That is effectively Marc's argument; don't buy stuff you don't need yet because it will be cheaper in the future (not so with my salmon and coffee, I suggest). That's certainly true as we see a year on year reduction in storage per GB cost.

There are a number of reasons why people buy more than they need;

1. New hardware deployment time is excessive due to datacentre restrictions and change control. In some sites this delay could be 3-6 months, so people work on the assumption that it's better to have more on the floor than be in a panic to deploy at the last minute.
2. Business customers can't plan. It's a truism that everyone knows. Add on top the fact that chinese whispers inflate the original business requirement to two, three or four times more storage than actually needed.
3. Vendors give discounts. Yes, shock! Vendors will sell you storage cheaper if you buy more. I know many places that buy complete arrays up front (even DMX-4 with 1920 drives!) to avoid the deploy time and get a better price.

There are many more reasons than this but you get the idea.

I've deliberately left off one issue - the inflexibility of some storage systems in their deployment method. Although this isn't directly a reason to buy more storage, it is certainly a reason why users hoard more storage on their servers. Monolithic arrays are way too slow at executing on configuration tasks and on dynamic rebalancing, requiring too much planning and thinking time to avoid bad layout and configuration issues.

So Marc, you should have stated that thin provisioning is only one aspect of reducing storage hoarding. Good practice is another. Flexible technology is an undoubted third.

Oh and 10 house points to the first non-UK person who can explain my post title!

You Sunk My Battleship!

I spent some time today with the good folks at 3Par, as offered by Marc Farley a few posts ago. It was good to get more of a background on the product and also see what's happening in the future (although I can't talk about that!).

I think most of their technology is fairly well known (thin provisioning, wide striping etc), but two features stood out for me.

Dynamic Optimisation

Dynamic Optimisation allows a LUN to be moved around the array based on a number of parameters. One of the most interesting is the ability to change the RAID type without any outage or downtime. Think about it; you create LUNs as RAID-10 devices then realise you don't need to have that level of performance. With a couple of clicks, your LUN is changed and re-laid out as anything from RAID-5 2+1 to 8+1. The key factor here is that this is seamless, needs no outage or no host-based migration.

Compare and contrast this to a traditional "monolithic" array like a DMX-4 or USP. Well, the DMX-4 uses hypers (slices of a physical disk) which are combined up to create RAID devices. Naturally, the hypers used to create a RAID-5 device are a different size to those used on a RAID-1 device when creating a LUN of the same usable size. Re-purposing a LUN simply isn't possible; a RAID-1 LUN is a RAID-1 LUN for it's lifetime. In fact, releasing storage and recreating different LUNs although technically possible, is very rarely done as it's a complete nightmare to accomplish. If you like analogies (and I do), re-structuring an existing DMX array is a little like going back in time to a WWII RAF or Navy Operations room with lots of little models of ships being pushed around by wooden paddles, compared to today's modern ATC systems.

The way forward for monolithic or enterprise arrays has to be flexibility, especially with the need to provide storage for virtualised environments.

Thin Built In

So thin provisioning is on everyone's lips as a must have feature and 3Par claim to be the grandfather of the technology. I say claim, as TP has been done before, but that's not what I'm concerned about. What interests me more is the issue of Fat->Thin LUN migration. That is, copying "full size" LUNs onto a thin provisioned array. TP is great for new data. Create the LUNs, provision out the storage and voila!, more than 100% allocation on your array! TP relies on the fact that writing data to a LUN is a block-level process, and blocks are not necessarily written to sequentially, so there can be plenty of unwritten space on the LUN. However, copying a LUN from a standard array to a TP array will write all of the data blocks, negating the TP benefit.

3Par's arrays now have "Thin Build In", a custom ASIC which can detect unused space as data is written. This means fat->thin can be achieved as data is moved onto the array without any other intervention. It's worth thinking this one through; perhaps someone can answer whether EMC's TP implementation in 73 code and HDS's TP implementation on USP-V can do that and if not, how they expect to migrate existing data onto their arrays and still see the TP benefit.

While I'm on the subject, what happens if you defrag a TP volume? Well, data will be consolidated onto contiguous blocks of space and the location where data is moved from will get logically freed, but I don't believe products such as Diskeeper will write anything on the old data. What if it did? Well if that space was cleared, a 3Par array could reclaim this storage. So defragmenters out there - do you do this?

Anyway, thanks to the 3Par guys for the head's up on their technology; the next few years certainly are going to be interesting in this space!

Fast Food Storage Provisioning

Yesterday I discussed beating the credit crunch by getting your house in order. This was picked up my Marc Farley over at StorageRap and he posted accordingly. Marc, thanks for the additional comments, I will be reviewing the diagram accordingly based on your thoughts.

Moving on, think to yourself does this sound familiar?

Storage requests come in over time in a constant but unpredictable rate. When they arrive, you just provision them. Perhaps you check the requestor can "pay" for their storage (i.e. is authorised to request) but generally, storage is provisioned pretty much on demand. When you run out of storage, there's a minor panic and rush to place new hardware orders and then in a few weeks you're back in the game and provisioning again.

Welcome to Fast Food Storage Provisioning! I was going to use a brand name in this post, but then decided against it. After all, as these guys know, you're just asking for trouble.

How does this compare to storage? Easy. You walk into a fast food place and they're just there waiting to serve you, no questions asked, as long as you pay. They may have what you require ready, but if not, there's a panic in the kitchen area to cook what you want and so a delay in the delivery of your request. Those customers who eat food/storage every day become "overprovisioned" in both senses of the word.

Clearly Fast Food establishments have a vested interest in acquiring more customers as it builds their profits, however unless you are selling a service, storage growth is bad for the bottom line.

So, how about taking a few steps to make sure that storage is really needed?

• When do you need the storage by? Poor project planning means storage requests can be placed long before servers and HBAs have even been delivered, never mind racked and configured.
• Can the storage be delivered to you in increments? Most users who request 20TB immediately will never actually use it for days or weeks (in extreme cases may never use it all).
• Have you checked your existing server to see if you have free storage? You would be amazed how many users have free LUNs on their servers they didn't know were there.
• What exactly is your requirement in detail? How will you use what we give you? By questioning the request, you can find out if users have simply doubled the estimate of required storage given to them by the DBA. Get to the real deal on what growth is anticipated.

I'm not advocating saying no to customers, just to be confident that what you're deploying is what you need. Then you won't have that guilty feeling ordering another burger - I mean array...

Off The Grid

I've been on holiday for the last week (sunning myself and the family in Cyprus). I had no Internet access - not even TV! Although I had no laptop (or Blackberry this time) I did take my iPod Touch, now configured with the mobile version of NewsGator. As I've mentioned previously, I have a 100+ RSS feeds (which I'll publish once I get around to it) on storage and others. My backlog was about 2500 entries, so I decided to challenge myself to get up to date and read as much as possible. Clearly I didn't read them all (there were plenty that could be skipped) but I read most and it provides for an interesting cross section...

EMC - blogs are run like a military machine; co-ordinating the news relating to new product releases and mercilessly hammering the competition. EMC have more storage bloggers than any other storage company and there are some good ones out there - one of my favourites is Information Playground by Steve Todd, where he discusses the design of Clariion.

Netapp - follows a close second to EMC with lots of bloggers and lots of competitor bashing. I particularly like Alex McDonald's postings.

IBM - doing a great job running the "resistance", fighting back against the continual onslaught of Barry (A Burke). Check out Barry (Whyte) and Tony Pearson. I'd like to see more from IBM though, especially their product developers working on DS arrays and XIV.

HDS - A jolly good bloke, but not really a player in the blogosphere. Only Hu contributes regularly, but doesn't engage in any serious debate.

Sun - quite literally on another planet with their storage strategy!

Dell - bought some toys, but doesn't know how to play with them. Unfortunately the older boy who could help them play with them has left...

Now there are more companies out there and I don't think I have any blog links from Brocade, 3Par, Compellent, Emulex, Qlogic, Pillar and others although I may be wrong (it is getting late). Any RSS link offerings gladly welcome - although I might not get around to reading them before my next holiday!