hashicorp/nomad · error

failed to read snapshot hashes: %v

Error message

failed to read snapshot hashes: %v

What it means

During snapshot archive parsing, the SHA256SUMS entry inside the tar could not be read from the underlying stream. The library throws this when io.Copy of the checksum file into an in-memory buffer fails, which almost always means the gzip/tar stream is corrupt or truncated, or the destination state (temp files, disk) is failing. It is wrapped by CopySnapshot/Restore as "failed to read snapshot file".

Source

Thrown at helper/snapshot/archive.go:222

			// turn made the snapshot verification fail. By explicitly reading the
			// whole thing first we ensure that we calculate the correct hash
			// independent of how json.Decode works internally.
			buf, err := io.ReadAll(io.TeeReader(archive, metaHash))
			if err != nil {
				return fmt.Errorf("failed to read snapshot metadata: %v", err)
			}
			if err := json.Unmarshal(buf, &metadata); err != nil {
				return fmt.Errorf("failed to decode snapshot metadata: %v", err)
			}

		case "state.bin":
			if _, err := io.Copy(io.MultiWriter(snap, snapHash), archive); err != nil {
				return fmt.Errorf("failed to read or write snapshot data: %v", err)
			}

		case "SHA256SUMS":
			if _, err := io.Copy(&shaBuffer, archive); err != nil {
				return fmt.Errorf("failed to read snapshot hashes: %v", err)
			}

		default:
			return fmt.Errorf("unexpected file %q in snapshot", hdr.Name)
		}
	}

	// Verify all the hashes.
	if err := hl.DecodeAndVerify(&shaBuffer); err != nil {
		return fmt.Errorf("failed checking integrity of snapshot: %v", err)
	}

	return nil
}

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Re-obtain the snapshot file from a healthy source (leader API /v1/agent/snapshot or a verified backup) since the current one is likely truncated or corrupt.
2. Verify the file's integrity before restoring: gunzip -t snapshot.gz (or snapshot.Verify in Go) and compare its sha-256 checksum header against the transfer.
3. Check disk space and I/O health on the node performing the restore; a failing temp disk can abort the read pipeline.
4. If transferring manually, re-download in binary mode over a resumable protocol (HTTPS/S3) and compare sizes/checksums with the leader's copy.

Example fix

// before: restoring a partially downloaded snapshot directly
f, _ := os.Open("snapshot.bin")
snapshot.Restore(logger, f, r)

// after: validate first, fall back to re-fetch if corrupt
f, _ := os.Open("snapshot.bin")
if _, err := snapshot.Verify(f); err != nil {
    f.Close()
    return fmt.Errorf("snapshot corrupt, re-download required: %w", err)
}
f.Seek(0, 0)
snapshot.Restore(logger, f, r)

// Validate the snapshot stream before restore/copy
func validateSnapshot(in io.Reader) error {
    f, ok := in.(*os.File)
    if !ok || f == nil {
        return fmt.Errorf("need a seekable snapshot file")
    }
    if st, err := f.Stat(); err != nil || st.Size() == 0 {
        return fmt.Errorf("snapshot file empty or unreadable")
    }
    if _, err := snapshot.Verify(f); err != nil {
        return fmt.Errorf("snapshot stream corrupt: %w", err)
    }
    f.Seek(0, 0)
    return nil
}

When it happens

Trigger: Calling snapshot.Verify, snapshot.CopySnapshot, or snapshot.Restore on a snapshot whose tar archive's SHA256SUMS member cannot be read: truncated download, corrupt gzip stream mid-checksums-file, or an I/O error on the source reader (e.g. failed HTTP body read).

Common situations: Downloaded snapshot files cut off by network interruptions; snapshots transferred via FTP in ASCII mode; storage backend (S3/NFS/disk) returning short reads; restoring an old or hand-edited snapshot archive.

Related errors

• failed to read snapshot: %w
• Failed to copy snapshot to temporary file: %v
• failed to finalize snapshot: %v
• failed reading snapshot: %v
• failed to read snapshot metadata: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/c08ca20caa6cc276. Report an issue: GitHub.