NAS and Server Recovery
TrueNAS / FreeNAS Data Recovery
We recover ZFS pools from degraded and faulted TrueNAS and FreeNAS systems. Vdev reconstruction, dataset extraction, GELI decryption (with your key), and iXsystems hardware failures. Free evaluation. No data = no charge.
How TrueNAS ZFS Pools Fail and How We Recover Them
TrueNAS (and its predecessor FreeNAS) uses ZFS as its sole filesystem. A ZFS pool is built from one or more vdevs.
When enough drives in a vdev fail that ZFS can no longer reconstruct missing data from parity, the pool status changes to FAULTED and ZFS refuses to import it. Recovery requires imaging every drive in the pool, reconstructing the vdev geometry from ZFS label data, and force-importing the pool from images to extract datasets and zvols.
ZFS checksums all data and metadata. User data gets fletcher4 by default, and deduplicated datasets default to SHA-256. This means ZFS can detect silent corruption that traditional RAID controllers miss.
The downside: when ZFS detects a checksum mismatch and cannot correct it from parity, it returns an I/O error rather than silently serving corrupt data. This is the correct behavior, but it means scrub errors on a degraded pool are a warning sign that more data may become inaccessible if another drive fails.
Common TrueNAS Failure Scenarios
zpool import. We image every member first, then reconstruct the pool from the drive images.RAIDZ1 Double Drive Failure
RAIDZ1 tolerates one drive failure. A second failure in the same vdev causes the pool to fault. If the second drive developed bad sectors gradually, the pool may have been serving data with correctable errors before the complete failure.
Failed Resilver
A resilver is ZFS's rebuild. It reads every surviving drive in the vdev and writes the missing data onto the replacement. If a surviving drive has an unreadable sector, you lose the file stored there. ZFS names that file and finishes the resilver. If a surviving drive dies outright before the resilver finishes, the pool faults.
Controller or Backplane Failure
TrueNAS systems attach drives through SAS/SATA HBAs, or through RAID cards set to HBA (passthrough) mode. If that controller or the backplane fails, the drives behind it drop off together, even though the drives themselves are still healthy. We image them through a known-good HBA.
Accidental Pool Destruction
Running zpool destroy marks the pool as destroyed, and zpool import -D can still list and import it. zpool labelclear is a different story. It overwrites each label's configuration and its uberblocks with zeros.
Scrub Errors Accumulating
ZFS scrubs verify every block checksum on every drive. If scrub reports uncorrectable errors, data blocks are failing and ZFS cannot reconstruct them from parity. This is a precursor to pool failure if another drive drops.
Boot Drive Failure
TrueNAS CORE and SCALE install the OS on a separate boot pool. If only the boot drive fails, the data pool is unaffected. Reinstall TrueNAS on a new boot drive, reimport the pool, and you're back in. The exception is a legacy GELI pool. It also needs its encryption key, and FreeNAS kept that key in the system dataset.
Deduplication Table Load After Pool Import
Enabling ZFS deduplication creates a Deduplication Table (DDT) that maps every unique data block to its physical location. Holding it in ARC takes roughly 1 to 5 GB of RAM per 1 TB of deduplicated data. The DDT is stored on disk and read on demand, so a pool whose DDT does not fit in RAM still imports, but iXsystems documents that loading the table on demand can take days after an import or reboot, and every deduplicated write or deletion becomes a random disk read in the meantime. Because the DDT is not needed to read existing data, a strictly read-only import with `zpool import -o readonly=on` is the recovery path on undersized hardware.
ZFS Pool Import I/O Errors
`zpool import` I/O errors indicate ZFS detected checksum mismatches or unreadable sectors it cannot reconstruct from parity. Each vdev label carries a 128 KiB uberblock ring. Each slot is the pool's sector size, clamped to no less than 1 KiB and no more than 8 KiB. So an ashift=9 pool holds 128 uberblocks, and a common ashift=12 pool holds 32. We go looking for the older uberblocks and try the import from an earlier Transaction Group with `zpool import -T <txg_id>`, which rolls the pool back to that TXG. The zpool-import man page warns that a pool imported at an inconsistent TXG may contain uncorrectable checksum errors. Anything written after that TXG is gone. We image all drives before attempting any import command so the originals are never modified.
When SLOG or L2ARC Cache Drives Fail
A failed L2ARC (read cache) SSD doesn't cause data loss. ZFS treats L2ARC as a disposable performance layer; removing a dead cache drive & reimporting the pool restores full access with only a temporary read performance drop.
SLOG (ZFS Intent Log) failures are different. The SLOG is never read from during normal operations; it acts purely as a persistent backup for in-memory synchronous transactions. If the SLOG device dies with uncommitted synchronous writes, ZFS refuses to import the pool because the intent log is incomplete.
We bypass the missing log device with zpool import -m -f, which discards the log device so the pool mounts from the healthy data drives. Any recent transactions that lived only in that log are gone. The lost writes are limited to whatever synchronous I/O was in flight at the moment of failure.
If your pool won't import after a cache or log drive failure, NAS recovery follows the same imaging-first approach we use across all ZFS systems. We image every pool member with write-blocking before running any import command, so the originals are never modified. SATA members go through PC-3000. No data recovered means no charge.
OS-Layer Failure vs Data-Pool Failure Triage
A TrueNAS appliance that won't boot looks like a lost array. The WebUI is your window into the system, so when the middleware fails to start and the shares drop offline, the box looks dead. In these three scenarios the ZFS data pool is untouched.
Before running any pool command, image every member so the originals are never modified, then confirm which layer actually failed. See our ZFS pool recovery guide for the data-layer path.
Bad Update: Boot Environment Rollback
TrueNAS stores ZFS boot environments (BEs) on the boot-pool, separate from the data pool. An update creates a NEW boot environment and leaves the previous one intact, so a failed upgrade is reversible without touching data. On TrueNAS CORE (FreeBSD), you manage BEs from the System > Boot web UI. On TrueNAS SCALE (Debian Linux), current documentation puts them under System > Boot in the web UI as well. Activate the prior BE, reboot, and the working OS returns with the data pool never modified.
config.db Corruption
TrueNAS stores system configuration (users, shares, network, services) in a SQLite database at /data/freenas-v1.db. That file holds settings, not file data, so a corrupt config.db leaves the ZFS data pool untouched. Restore a saved config backup or reset the config, then re-import the existing pool through the TrueNAS WebUI or API.
Feature-Flag Mismatch After Upgrade
Upgrading TrueNAS doesn't turn on new pool features by itself. Running zpool upgrade afterward enables newer OpenZFS pool feature flags, for example block_cloning, introduced in OpenZFS 2.2.0. Once a feature that an older OpenZFS release doesn't support is active, that release can't import the pool read-write, and that includes the release inside a previous boot environment. This presents as "pool won't import," but the data is intact. The fix is a version match: import under an OpenZFS at least as new as the pool's flags. This is not pool reconstruction.
Feature-Flag Rollback After zpool upgrade
Boot environment rollback doesn't undo zpool upgrade. OpenZFS feature flags can't be disabled once they're enabled. An enabled feature doesn't stop older software from importing the pool. Once it becomes active, though, the older OpenZFS release in a previous boot environment can't import the pool read-write. If the feature isn't read-only compatible, that release can't import it at all.
For a pool that's already been upgraded, the fix is an OpenZFS version at least as new as the active flags, not a BE rollback. The data sits on disk the whole time. The only thing blocking it is the OpenZFS version at the import layer.
When the pool won't mount and the version history is unclear, we image every member and reconstruct the import path on a workstation with a current OpenZFS build. That's server recovery work. We image every member with write-blocking, so the originals are never modified. SATA members go through PC-3000. No data recovered means no charge.
Which Apps Datasets Does TrueNAS SCALE Put on Your Pool?
TrueNAS SCALE embeds a container-apps framework and creates a dedicated system dataset on the data pool you select for Apps. Apps can also be assigned to a separate dedicated pool.
This is a SCALE-only concern. TrueNAS CORE runs on FreeBSD and has no Apps engine at all; it uses iocage jails and plugins, and TrueNAS MINI units running CORE have no Apps layer either. Get the version boundary right before you touch the pool.
K3s Era: SCALE 24.04 and Earlier
Through TrueNAS SCALE 24.04 (Dragonfish) and earlier, the Apps engine was K3s, an embedded Kubernetes distribution. It created the ix-applications dataset on the chosen data pool. K3s also provisions child ZFS datasets for persistent volumes through the OpenEBS ZFS LocalPV provisioner (zfs-localpv), so each PVC becomes another dataset under that tree.
Docker Era: SCALE 24.10 and Later
TrueNAS SCALE 24.10 (Electric Eel) replaced K3s with Docker and migrated apps to a new ix-apps dataset. TrueNAS mounts that dataset at /mnt/.ix-apps. A box on 24.10 or later runs Docker with ix-apps, not K3s with ix-applications. Before reconstruction, we check which engine and dataset name that box used, so we match the app datasets to the right layout.
TrueNAS CORE: No Apps Layer
TrueNAS CORE (FreeBSD) has no K3s, no Docker, and no ix-applications or ix-apps dataset. Containerized workloads on CORE run in iocage jails and plugins. If the box in front of you is CORE, none of this applies and the pool has no Apps system dataset to audit.
Re-Importing a Pool That Holds Apps Datasets
When Apps are assigned to the pool that holds your files, the apps dataset and its container and PVC child datasets share that pool, so container lifecycle activity runs alongside your files.
Your user datasets are separate from the ix-applications or ix-apps hierarchy.
On a live TrueNAS appliance, re-import the pool through the TrueNAS WebUI or middleware API, not a bare CLI zpool import. A bare CLI import on the running appliance bypasses the middleware and can leave the appliance's view of the pool inconsistent with what the WebUI expects. See our ZFS pool recovery guide for the data-layer path, and our Proxmox VE recovery page for another host that layers container storage on ZFS.
In the lab we image every member first with write-blocking, running SATA members through PC-3000, then reconstruct the pool from the images. We skip the ix-applications or ix-apps child datasets and snapshots, or export them separately. All of it happens in-house at our Austin, TX lab, and the originals are never modified. No data recovered means no charge.
ZFS On-Disk Structures Relevant to Recovery
zpool import -T <txg_id>.Understanding ZFS internals is required for targeted recovery. For a deeper technical treatment, see our ZFS pool recovery guide.
With the storage controller in HBA pass-through mode, OpenZFS on TrueNAS manages the member block devices directly instead of consuming a hardware controller's virtual-disk abstraction, so pool geometry is read back out of the ZFS labels on each imaged member. A controller-managed parity array keeps the equivalent geometry in a vendor metadata region on the same drives: Dell PERC and LSI/Broadcom MegaRAID write a SNIA Disk Data Format variant to the trailing sectors, and traditional HP Smart Array P-series and E-series controllers write a RAID Information Sector at the start of each member. Either layout travels with the drives, so the structural differences between ZFS and hardware RAID decide which structures get parsed from the images, not whether the original card is still alive.
ZFS Labels and Uberblocks
- Each drive has four ZFS label copies: two at the beginning (L0, L1) and two at the end (L2, L3) of the device
- Labels contain the pool name, GUID, vdev tree, and a 128 KiB uberblock ring, which holds the root pointers into the ZFS object tree. Each slot in that ring is the pool's sector size, clamped between 1 KiB and 8 KiB. That gives you 128 uberblocks at ashift=9 and 32 at the common ashift=12
- OpenZFS ranks uberblocks by transaction group (txg) number. When two carry the same txg, it breaks the tie on the uberblock timestamp. The winner is the most recent consistent state of the pool
- If labels are corrupted, we scan for uberblocks at known offsets across the drive to find a valid root pointer
MOS, DSL, and Dataset Objects
- The Meta Object Set (MOS) is the root of the ZFS object tree; the uberblock points to it
- The Dataset and Snapshot Layer (DSL) tracks all datasets, zvols, and snapshots in the pool
- Each dataset has its own object set containing dnodes (ZFS inodes) that map files to their on-disk block pointers
- If the MOS is damaged, we traverse the block pointer tree manually using known dnode sizes and indirect block structures
Deduplication Tables, TXG Rollback, and Import Mechanics
When ZFS deduplication is enabled, every unique data block gets an entry in the Deduplication Table (DDT). The DDT is stored on disk as a ZFS internal object and read on demand; holding it in ARC takes roughly 1 to 5 GB of RAM per 1 TB of deduplicated data.
A pool whose DDT does not fit in RAM still imports, but iXsystems documents that loading the table on demand can take days after an import or reboot, and every deduplicated write or deletion becomes a random disk read. Because the DDT is not needed to read existing data, a strictly read-only import with `zpool import -o readonly=on` is the recovery path on undersized hardware.
ZFS pools are transactional; every change is batched into a Transaction Group (TXG). The uberblock points to the latest valid TXG. Each vdev label carries a 128 KiB uberblock ring, and each slot in it is the pool's sector size, clamped between 1 KiB and 8 KiB. That works out to 128 uberblocks on an ashift=9 pool and 32 on a common ashift=12 pool. ZFS skips any uberblock that fails verification and uses the newest valid one. If the metadata that uberblock leads to is damaged, the pool won't import without recovery mode (zpool import -F), and recovery mode discards the last few transactions.
In recovery, we scan the uberblock ring for the most recent valid root pointer and force import from an earlier TXG using zpool import -T <txg_id>. That rolls the pool back to that TXG, and you lose any writes committed after it. The zpool-import man page warns that a pool imported at an inconsistent TXG may contain uncorrectable checksum errors.
dRAID Vdevs: Distributed Parity, Fixed Stripe Width, and Sequential Resilver
dRAID is an OpenZFS vdev topology that distributes parity and spare capacity across every child drive instead of reserving one idle physical hot spare. It shipped in OpenZFS 2.1.0, and TrueNAS first supported it in 23.10 (Cobia). Recovery starts the same way every other pool does, with sector-by-sector imaging of every member.
The geometry is written with suffix tags, not bare positional numbers: draid[parity]:[data]d:[children]c:[spares]s. A vdev created as draid2:4d:11c:1s carries parity level 2, four data devices per redundancy group, eleven total children, and one distributed spare. ZFS parses that configuration string at pool-create time and writes the resulting geometry into each member's vdev label as discrete nvlist fields (nparity, draid_ndata, draid_nchildren, draid_nspares). You read them back with zdb -l.
dRAID uses a fixed stripe width and pads with zeros as necessary, so the minimum allocation size rises with the data-device count. With eight data devices and 4 KiB sectors, the minimum allocation is 32 KiB. That padding is what makes a fully sequential resilver possible: the rebuild reads across the children and completes faster than a RAIDZ healing resilver, at the cost of higher I/O load on the entire array. OpenZFS exposes zfs_rebuild_vdev_limit for the maximum bytes in flight per leaf vdev during a sequential resilver, and zfs_rebuild_scrub_enabled to scrub automatically once the sequential resilver finishes. The scrub is the pass that verifies the blocks after the rebuild.
Fault tolerance equals the parity level P. Losing more than P children before the rebuild finishes exceeds the vdev's fault tolerance and the data in that vdev is gone. Distributed spares shorten the rebuild window; they do not make the pool immune to simultaneous hardware failure, and they do not change the rule that imaging with write-blocking comes before any zpool import or forced resilver on a pool you cannot afford to lose.
A declustered layout is more work to reconstruct offline, not less. The fixed stripe width removes the variable-width problem that makes RAIDZ block mapping awkward, but the parity groups and the spare capacity are spread across all of the children, so knowing the member order is not enough to locate a block. The geometry has to be recovered from the labels before anything can be mapped. dRAID is a vdev-level topology, so a dRAID pool carries the same deduplication table RAM overhead and follows the same zpool import -T <txg_id> rollback path as a RAIDZ or mirror pool. Related failure detail lives on our ZFS pool import I/O error page.
- Image every member of the dRAID vdev sector-by-sector with write-blocking before any import or resilver attempt, including members that have already faulted. SATA members go through PC-3000 hardware.
- Read the geometry fields back from each member's vdev label with
zdb -lagainst the image files: parity level, data devices per redundancy group, child count, and distributed spare count. - Reconstruct the vdev in software against the images, never against the live members, so the original drives are read-only for the entire job.
- Locate a valid uberblock and import read-only from the images. If the pool won't import from its newest valid uberblock, roll back with
zpool import -T <txg_id>. - Extract datasets and zvols. All of this runs in-house at the Austin, TX lab, and no data recovered means no charge.
ZFS Native Encryption and Legacy GELI Pools
Current TrueNAS releases encrypt inside ZFS, at the dataset and zvol level; iXsystems documents dataset encryption as AES-256-GCM from TrueNAS 12.0 and states that TrueNAS no longer supports GELI encryption. Older pools can still carry GELI, which encrypts whole disk partitions below ZFS. Without the key material, the raw disk data is indistinguishable from random bytes on either layer.
- Legacy GELI key material: the recovery key was an optional keyfile that FreeNAS generated, provided for download, and then wiped from the system. If nobody downloaded it, it is not sitting on the appliance waiting to be found, so whatever key material you did export is what the recovery depends on.
- ZFS native encryption: encryption happens at the dataset or zvol level, with the master key wrapped by a user passphrase or keyfile. We do not need FreeBSD GELI tools for these pools, but we do need the passphrase, the keyfile, or the encryption key JSON exported from the UI.
- Recovery with key material: If you provide the correct key (GELI encryption key, GELI recovery key, or ZFS encryption passphrase), we attach the encryption layer to the drive images and import the pool normally. The decryption happens on our workstation.
Recovery Methodology for IT Administrators
1. Drive Imaging
We image every drive in the TrueNAS system with write-blocking. SATA drives go through PC-3000, and SAS drives go through SAS HBAs.
For drives with bad sectors, we capture healthy sectors first using head maps, then retry damaged areas. We also image the boot drive or drives to pull the TrueNAS configuration. On a legacy GELI system whose system dataset sat on the boot pool, the boot drive also holds the GELI encryption key, and we pull that too.
2. ZFS Pool Reconstruction
We read ZFS labels from each drive image to determine pool geometry: vdev type (mirror, RAIDZ1/2/3), member ordering, and ashift (sector size alignment). The pool is imported read-only from the images.
If the pool refuses to import, we locate historical uberblocks and attempt import from an earlier transaction group. For pools with corrupted spacemaps, we traverse the block pointer tree manually to locate dataset objects.
3. Dataset and Zvol Extraction
Individual datasets (file-level shares), zvols (block-level iSCSI targets or VM storage), and snapshots are extracted from the reconstructed pool. Each dataset is verified by checking file counts and sizes against what ZFS metadata reports. For zvols used as VM storage, we also verify the guest filesystem integrity (NTFS, ext4, XFS).
How Much Does TrueNAS Recovery Cost?
Same transparent model: per-drive imaging based on each drive's condition, plus a $400-$800 ZFS pool reconstruction fee that includes dataset extraction. No data recovered means no charge.
| Service Tier | Price Range (Per Drive) | Description |
|---|---|---|
| Logical / Firmware Imaging | $250-$900 | Firmware module damage, SMART threshold failures, or filesystem corruption on individual pool members. |
| Mechanical (Head Swap / Motor) | $1,200-$1,50050% deposit | Donor parts consumed during transplant. SAS drives require SAS-specific donors. |
| ZFS Pool Reconstruction | $400-$800per pool | Vdev reconstruction, pool import, dataset/zvol extraction. |
No Data = No Charge: If we recover nothing from your TrueNAS system, you owe $0. Free evaluation, no obligation.
Before sending drives: export your ZFS encryption key or passphrase, or the GELI key material on an older pool, if your pool is encrypted. Without the key, encrypted data is unrecoverable.
TrueNAS Recovery; Common Questions
My TrueNAS pool shows FAULTED and will not import. Can you recover the data?
Should I try to replace a failed drive and resilver before contacting you?
My TrueNAS pool uses GELI encryption. Can you still recover the data?
Does TrueNAS CORE vs SCALE matter for recovery?
What happens if a drive fails during a TrueNAS RAIDZ expansion?
Why won't my TrueNAS SCALE pool import after using Apps (K3s/Docker)?
Can you recover VMware or Proxmox virtual machines from a failed TrueNAS iSCSI Zvol?
Why does my deduplicated TrueNAS pool take so long to settle after import?
Can a failed SLOG SSD destroy data that was already written to the pool?
What do ZFS pool import I/O errors mean, and can they be fixed?
My TrueNAS SCALE dataset is locked with ZFS native encryption. Can you unlock it?
My zfs send | zfs receive replication died mid-stream. Did I lose data?
My TrueNAS pool faulted when the SAS HBA failed, but the drives are fine. Can you recover it?
My TrueNAS won't boot or the WebUI is down after an update. Is my data gone?
Why won't my TrueNAS pool import after I upgraded TrueNAS?
Can you recover a TrueNAS pool built on dRAID vdevs?
Since 2008
Established
As Featured In
Related services
Need Recovery for Other Devices?
Data Recovery Standards & Verification
Our Austin lab operates on a transparency-first model. We use industry-standard recovery tools, including PC-3000 and DeepSpar, combined with strict environmental controls to maintain drive integrity. This approach allows us to serve clients nationwide with consistent technical standards.
Localized Clean Zone
Open-drive work is performed in a 0.02 micron ULPA-filtered laminar clean bench.
Transparent History
Serving clients nationwide via mail-in service since 2008. Our lead engineer holds PC-3000 and HEX Akademia certifications for hard drive firmware repair and mechanical recovery.
Media Coverage
Our repair work has been covered by The Wall Street Journal and Business Insider, with CBC News reporting on our pricing transparency. Louis Rossmann has testified in Right to Repair hearings in multiple states and founded the Repair Preservation Group.
Aligned Incentives
Our "No Data, No Charge" policy means we assume the risk of the recovery attempt, not the client.
Technical Oversight
Louis Rossmann
Our engineers review all lab protocols to maintain technical accuracy and honest service. Since 2008, his focus has been on clear technical communication and accurate diagnostics rather than sales-driven explanations.
We believe in showing the bench rather than just describing it. Open-drive work runs on a 0.02 micron ULPA-filtered laminar clean bench, and we filmed it.
See the particle counter test at the benchReady to recover your TrueNAS system?
Free evaluation. No data = no charge. Mail-in from anywhere in the U.S.