The Defrag Debate

I was asked again this week whether defragging of hard drives on Windows servers is really necessary. This is quite pertinent as the cost of deploying an enterprise-wide defrag tool can be significant and any opportunity to save money has to be a good one.

I discussed fragmentation last year (here) when looking into a problem which turned out to be the lack of free space consolidation. however I didn't discuss the potential performance impact.

So, reflecting on whether defragmentation is required or not, I'd say that in most cases the benefits are minimal. Here's why...

Firstly, hard drives have on-board cache; the larger and more "enterprise" the drive, then the larger the cache. We are also likely to see more and more hybrid drives on the market, which will have large amounts of fast memory fronting access to the old moveable components. Cache will mask the impact of fragmented files during writes as data will be written to cache and confirmed as written by the drive. The data can then be written to disk asynchronously afterwards. Obviously if cache becomes overrun, then the benefit will be negated.

Second, operating systems have built in file system performance techniques, including lazy writing and file prefetch. These features will try and minimise the latency issues of reading and writing to disk.

Third, if the data being accessed is a database, the database itself will also have lazy writer processes to asynchronously write data to disk.

Now all of the above applies directly to "traditional hard drives". In systems which have RAID controllers with onboard cache, then the issues will be less. Where storage is taken from a SAN with an enterprise or modular array, all reads and writes will occur through cache, giving the best options for performance and masking the underlying hard drive mechanics.

So, can fragmentation actually help? In fact, I have seen one instance when this occurred and it required slightly special circumstances. The file system was made up from a number of concatenated LUNs using a volume manager. As the server had multiple CPUs and the storage was SAN connected, then multiple I/Os could be issued to the volume. With more fragmentation, then the I/Os are spread across all of the LUNs and performance was increased.