Dealing With The Consequences

In a remarkable piece of coincidence, two WWII unexploded bombs were found today and caused air traffic delays. The first, at Stratford in east London, temporarily closed London City Airport. The second closed a runway at Amsterdam’s Schiphol airport. It’s amazing that over 60 years after these bombs were dropped, they are still being found; fortunately this time with no injury to anyone. I found out about both incidents because colleagues of mine flying from both City Airport and into Schiphol were delayed or had to alter their plans.

On a less serious level, but important nonetheless, we are having to live with the consequences of our data storage policies. We will have data being stored now which must be accessed in 60 years time. This will include medical and financial information, directly affecting individuals if the information cannot be retrieved. Obviously good data management practices are essential, but they will go beyond the normal storage of data we’ve been used to up until now.

Take the use of medical imagery; x-rays, CAT and PET scans, MRI scans. These all now produce complex digital images. If we store them using today’s format, how will the format and presentation of the images change in the future as we move to more complex display technology, higher resolutions and possibly 3D television screens?

If you want examples of what I mean, think of your old word processing documents. They could have been stored in an early version of Microsoft Word, WordPerfect, WordStar or other similar now defunct technology. You may be lucky and still be able to read them; you may not. Fortunately, apart from some formatting codes, most word processing documents can be opened in Notepad, WordPad or some raw file viewer which will at least allow you to recover the content. Things won’t be so easy with imaging files as their binary nature will render them useless if the software isn’t retained to open them.

I can see one of our future storage challenges will be to ensure all of our data is retained in a readable format for the future. XML addresses some of these problems and the adoption of file format standards will help. However, data will need to be refreshed as it ages. More metadata referring to the content format of files will have to be produced and software written to detect and convert unstructured files as file formats become defunct.

For many of us, we can afford to lose a few files here and there or perhaps print out the most important of our documents. For large organisations, the data management lifecycle has only just begun.