What to Do When a RAID 5 Rebuild Fails

RAID 5 distributes parity across all drives to survive one drive failure. A rebuild failure occurs when the array loses one drive, begins regenerating data onto a replacement, and encounters a second error before regeneration completes. The array is now worse off than after the original failure.

• 1.One drive in the array fails or goes offline. The array enters degraded mode.
• 2.The controller serves data by computing the missing drive's contribution from parity on each read request.
• 3.An administrator inserts a replacement drive (or a hot spare activates). The controller begins rebuild: reading every sector of every surviving drive and XORing the results onto the replacement.
• 4.A second drive reports an Unrecoverable Read Error or fails outright. The controller cannot XOR the stripe where the error occurred.
• 5.The rebuild aborts. The array transitions from degraded to failed.

A stale drive contains data from before it was removed from the array. Forcing it back online causes the controller to recalculate parity using outdated blocks, silently corrupting every stripe that received writes while the drive was absent.

• 1.RAID 5 parity for each stripe is the XOR of all data blocks in that stripe.
• 2.When a drive goes offline, the controller stops including it in parity calculations and continues serving I/O using parity reconstruction.
• 3.Writes that occur while the drive is offline update the remaining drives but leave the offline drive unchanged.
• 4.If the stale drive is forced back in, the controller XORs its outdated blocks with current blocks. The resulting parity is wrong for every modified stripe.
• 5.Reads from affected stripes return silently corrupted data. The corruption is invisible until a parity scrub or until an application encounters garbage output.

The probability of hitting an Unrecoverable Read Error (URE) during a RAID 5 rebuild is a function of drive capacity, member count, and the manufacturer's published URE rate. For modern high-capacity arrays, this probability is not negligible.

• 1.Consumer drives (WD Red, Seagate IronWolf non-Pro) specify a URE rate of 1 in 10¹⁴ bits read. That equals roughly 1 unrecoverable error per 12.5 TB of sequential reads.
• 2.Enterprise drives (Seagate Exos, WD Ultrastar) specify 1 in 10¹⁵ bits, or roughly 1 error per 125 TB.
• 3.A 4-drive RAID 5 with 14 TB consumer drives rebuilds by reading 3 surviving members sequentially: 3 x 14 TB = 42 TB total reads.
• 4.At a 1-in-10¹⁴ URE rate, reading 42 TB means reading 3.36 × 10¹⁴ bits. The expected number of UREs is 3.36 (42 TB / 12.5 TB per expected error). The probability of completing that read with zero UREs is under 5%. In practice, a 4-drive RAID 5 rebuild with 14 TB consumer drives is more likely to hit a URE than not.

These numbers explain why RAID 5 is no longer recommended for drives above 2 TB in production environments. RAID 6 (dual parity) or RAID 10 (mirrored stripes) tolerate a single URE during rebuild without losing the array. For existing RAID 5 deployments, regular mdadm --action=check or controller-level patrol reads surface latent errors before they are discovered during the high-stakes rebuild window.

A degraded array has lost one drive but continues operating with parity intact; all data remains computable. A failed rebuild is a different state: partial parity has been written to the replacement drive, and the pre-rebuild data on surviving drives may be partially overwritten.