How Does Lost RAID Controller Write-Back Cache Corrupt a VHDX That Still Mounts?
Lost write-back cache corrupts a mountable VHDX when acknowledged writes never reach the member drives. A controller acknowledges a host write once it lands in DRAM cache; if that dirty cache is discarded before destaging, the VHDX still parses while specific LBA ranges read back stale.
A hardware RAID controller acknowledges a host write the moment it lands in its write-back DRAM cache, before the data is destaged to the member drives. Enterprise cache protection prevents loss from that window by design.
HP Smart Array Flash-Backed Write Cache (FBWC) uses a supercapacitor on Gen8, and a Smart Storage Battery on Gen9 and later, to flush the volatile DRAM cache into an onboard non-volatile NAND flash module on power loss, then replay it on restore. Dell PERC Non-Volatile Cache performs the same flush powered by a lithium-ion battery.
These are backup-power and NAND modules, not NVMe devices. A healthy module loses nothing on a routine power event.
Loss occurs only when that protection is degraded, bypassed, or thrown away. Three vectors account for the acknowledged-but-lost writes that sit beneath a Hyper-V volume.
- Degraded or failed cache backup unit: when the supercapacitor or battery degrades and fails, the controller disables write caching and keeps read acceleration enabled, which is the safe default rather than a loss event on its own.
- Forced write-back without protection: if an operator forces write-back caching with no functional capacitor, or enables physical-drive write cache, acknowledged writes are exposed to permanent loss on a power failure.
- Discarded preserved cache: when a Dell PERC or Broadcom MegaRAID virtual disk goes offline or member disks go missing, the controller retains the dirty acknowledged-but-not-destaged cache as preserved or pinned cache. On reboot and import it normally auto-flushes to the members, but if an operator manually discards the preserved cache through the controller interface, those writes are permanently lost and never reach the disks.
Because the guest OS already received write acknowledgment, lost or discarded dirty cache leaves stale or inconsistent on-disk LBA ranges beneath the VHDX. The VHDX header, region table, and BAT can all parse cleanly while the payload at specific offsets reads back a previous generation, so the file appears mountable while individual blocks are stale.
We image the on-disk RAID members directly through a SAS HBA on the PC-3000 Express before any controller cache re-initialization, preserving the exact on-disk state for analysis instead of letting the controller silently flush or discard pinned cache.
Why Does a Hyper-V Checkpoint Show Read Errors for Blocks It Never Wrote?
A checkpoint returns read errors for blocks it never wrote because an AVHDX differencing disk inherits unmodified blocks from its parent VHDX. As described in the VHDX on-disk structure section above, the AVHDX BAT marks every block it did not change PAYLOAD_BLOCK_NOT_PRESENT and reads it through to the parent. If an underlying RAID partial-sector failure corrupts specific parent VHDX payload block offsets, the checkpoint cannot return data it never stored, and the read fails at an address the differencing disk never wrote itself.
The corrupted offsets map back to physical stripes. HP Smart Array arrays default to a 256KB stripe, and Dell PERC defaults range from 64KB on PERC 9 and earlier to 256KB on PERC 10 and later, so a single damaged stripe can span a contiguous run of inherited payload blocks across the parent. Recovery traces each corrupted LBA range through every AVHDX BAT in the checkpoint chain and classifies each block as inherited from the parent, which requires parent VHDX repair, or overridden in the differencing disk's own BAT, which is recoverable directly from the AVHDX.
The PC-3000 Express and DeepSpar Disk Imager read the member drives; Data Extractor Express RAID Edition destripes those images and resolves the checkpoint chain in software, never against the live members. This keeps the corrupted parent stripes isolated to the blocks that actually depend on them rather than failing the entire chain.
What Does a VSS Hardware Provider Timeout Tell You About the RAID Array Underneath?
A VSS hardware provider timeout tells you the storage path under the Hyper-V host is unstable, not that a file is corrupt. The Volume Shadow Copy Service coordinates three roles for every snapshot: the requestor is the backup application that asks for the shadow copy, the writer is the application component such as the Hyper-V VSS writer that quiesces its data so the copy is consistent, and the provider is what actually creates and holds the shadow copy.
The default Windows system software provider does volume-level copy-on-write in the host. A hardware provider is different: it is a storage-array or controller-integrated plug-in that offloads snapshot creation down to the storage layer itself.
When a hardware provider times out or crashes partway through creating a snapshot, the guest VHDX or AVHDX can be left in an uncommitted, inconsistent state. That inconsistency lives beneath the file layer, in the same acknowledged-but-not-reconciled storage state as the discarded write-back cache described above: the members hold bytes that do not match what the host was told is committed.
In-place tools run at the wrong layer for this. A guest-side chkdsk and a host-side DISM operate on the filesystem and the VHDX as a file, and neither can reconcile a stripe that the array never destaged.
The provider that failed tells you where to look. Read the VSS-source log first: open Windows Event Viewer, then Windows Logs, then Application, and filter for the Volume Shadow Copy Service source. VSS-source errors such as Event ID 8193 and Event ID 12289 record failures in the snapshot path and identify which provider was in play.
- Hardware provider failure: a timeout or fault reported from the storage layer. It implicates the RAID controller and the I/O path to the members, because the snapshot work was pushed down to that storage layer and did not return in time.
- Software provider or writer failure: a host-side crash in the system software provider or an application VSS writer. It runs in the host, is independent of the storage path, and does not implicate the array on its own.
A hardware-provider error during a routine backup job, on a host whose VMs live on a RAID array, is an early warning that a member has path instability. It is a reason to image the array offline, member by member, rather than trust an in-place controller rebuild. A rebuild started against a member that is dropping off the I/O path can read a stale stripe as good and propagate it into parity, which converts a recoverable inconsistency into a spread one.
We image each member with the PC-3000 Express and DeepSpar Disk Imager, which hold slow or unstable drives on the imaging path instead of ejecting them, then reassemble the array from those images with Data Extractor Express RAID Edition. The geometry is rebuilt in software against the images, so a member with path instability is never asked to serve a live rebuild.
