Eating My Own Dog Food

After my last post relating to personal data storage, I thought I'd spend some time and check out what I was currently using.

On my main server, I store my data on a Drobo unit. The Drobo's visualisation of data is to present a 2GB LUN, regardless of the installed drives. This is slightly misleading when analysing storage utilisation as I only have 2x 1TB drives actually installed, which is around 1TB of usable space.

However, that problem aside, on analysis I see I have about 794GB of storage in use - around 79% of my capacity, which in a business environment I would consider to be close to the margin where I'd purchase more storage (depending on growth rate and deployment lead time).

Using Treesize Professional I did some initial analysis. Treesize is really quick and provides data in lots of different formats, including a bizarre format called Tree Map which uses cascading squares to indicate data types and capacities.

Immediately I realised that my Exchange backups have been writing to a single file as appended backups and that the file has grown to 321GB! I only have a round 2GB of actual email data, so I've never bothered to archive and all backups are full. Archiving won't save me that much at this point (although I could archive and reduce the daily backup size), however I will now start a new backup file and delete the old one in a few days. That gives me an instant 300GB back.

Digging further, my next biggest usage is media files. Many are home video which need processing, many are films or digitised music. I know these files need more work to get organised and there are some files which can be deleted, so I just need to put the effort in to sort them.

After that the remainder of files are ISO and EXE installation files (like various copies of Solaris and Linux distros) and could be archived to DVD. The rest are Excel, Word, Powerpoint and other miscellaneous office files which comprise only a few GB.

So in reality, my core data is probably less than 10GB. I could even consider putting this into an online service. Unfortunately the process by which I could easily isolate and selectively backup those files isn't easy - which is why a lot of the time we resort to just backing up everything.

What has all this taught me? Well, I've saved 300GB of storage at a stroke, I know how to prioritise my next group of storage saves and I know I need to selectively move out data I'd like on an online backup service.

All good. I reckon simple organisation should only take me an hour a week - the problem is finding that hour!!