Non-Persistent Disks – How to Lose Your Data in One Reboot
Though I’m familiar with the concept, I’ve never used a non-persistent disk in a production server environment. I don’t even know of any good reason to do so. I have used non-persistent disks for Linked Clones where the data is blown away on a reboot (for temp files) along with a “delta disk” for Linked Clones, but still never on a server. But here I am, wondering how the hell a SQL server, that I deployed, was configured with a independent, non-persistent disk for its data volume.
If you’re not familiar with non-persistent disks, it’s exactly what it sounds like in relation to its data…once the VM is powered down the data doesn’t persist and it looks like a brand new volume to the server when it’s powered back on. Sounds bad right? Even worse is that you must, in the web client, expand the drop-down list when adding a new disk to even see the option to change.
It all started with a typical request from the application owner…”I need more space than I originally requested.” Sure, easy fix right? Expand the drive, rescan within Windows Disk Manager, and do an Extend. But what happens when the drive is deployed as non-persistent and you try to extend? You get this message:
The disk extend operation failed: The virtual disk requires a feature not supported by this program: Hot-extend is currently supported for VMFS flat virtual disks without snapshots opened in persistent mode.
First thing I did was to ensure there were no snapshots and verified in both Snapshot Manager and checking the file name that the disk was pointing to (ensuring there weren’t any vm-000001.vmdk files).
Second step was to Google the error for any hints. Unfortunately, I didn’t find much on this particular error that helped me out (hence the reason I’m writing this post). I did learn, however, that expanding a ThickEagerZeroed disk using the Web Client or C# client would expand it by tacking on a ThickLazyZeroed disk, which I witnessed happen in my environment. Check Cormac Hogan’s post on how to expand an ThickEagerZeroed disk using the vmkfstools command.
Last but not least, I (cringe) shut down the VM to see if I could expand the drive – unaware of the presence of the non-persistent disk setting. Goodbye data – I hardly knew you.
The result of this configuration was a complete wipe of the drive, resulting in the requirement to restore the data from backup. Looking at the error now makes total sense now but a non-persistent disk never dawned on me at the time as I’ve never used one before (knowing that I deployed the server). Luckily this environment was still in a testing phase but it still has me left wondering…..how in the world did the disk mode get changed to non-persistent?