hashicorp/nomad · error

failed to decode snapshot metadata: %v

Error message

failed to decode snapshot metadata: %v

What it means

This error is returned when the meta.json bytes read from the snapshot tar archive fail json.Unmarshal into *raft.SnapshotMeta. Unlike [1942], the stream read succeeded but the bytes are not valid JSON matching the expected raft.SnapshotMeta schema. This means the snapshot's metadata section is corrupt or was produced by an incompatible format.

Source

Thrown at helper/snapshot/archive.go:212

			return fmt.Errorf("failed reading snapshot: %v", err)
		}

		switch hdr.Name {
		case "meta.json":
			// Previously we used json.Decode to decode the archive stream. There are
			// edgecases in which it doesn't read all the bytes from the stream, even
			// though the json object is still being parsed properly. Since we
			// simultaneously feeded everything to metaHash, our hash ended up being
			// different than what we calculated when creating the snapshot. Which in
			// turn made the snapshot verification fail. By explicitly reading the
			// whole thing first we ensure that we calculate the correct hash
			// independent of how json.Decode works internally.
			buf, err := io.ReadAll(io.TeeReader(archive, metaHash))
			if err != nil {
				return fmt.Errorf("failed to read snapshot metadata: %v", err)
			}
			if err := json.Unmarshal(buf, &metadata); err != nil {
				return fmt.Errorf("failed to decode snapshot metadata: %v", err)
			}

		case "state.bin":
			if _, err := io.Copy(io.MultiWriter(snap, snapHash), archive); err != nil {
				return fmt.Errorf("failed to read or write snapshot data: %v", err)
			}

		case "SHA256SUMS":
			if _, err := io.Copy(&shaBuffer, archive); err != nil {
				return fmt.Errorf("failed to read snapshot hashes: %v", err)
			}

		default:
			return fmt.Errorf("unexpected file %q in snapshot", hdr.Name)
		}
	}

	// Verify all the hashes.

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Validate the snapshot with 'consul snapshot inspect <file>'; if it fails, the snapshot is corrupt
2. Re-take the snapshot from a healthy raft peer / re-download from the source
3. Do not hand-edit snapshot archives; meta.json must be a valid raft.SnapshotMeta JSON document
4. Verify checksums of the backup file against known-good values to rule out transfer corruption

Example fix

// before
f, _ := os.Open(backupPath)
err := snapshot.Restore(logOut, f)
// after
f, _ := os.Open(backupPath)
if err := snapshot.Verify(f); err != nil {
    return fmt.Errorf("backup at %s is corrupt; use a known-good snapshot: %w", backupPath, err)
}
_, _ = f.Seek(0, 0)
return snapshot.Restore(logOut, f)

// validate before restoring: 'consul snapshot inspect' equivalent check
func validateSnapshotFile(path string) error {
    f, err := os.Open(path)
    if err != nil { return err }
    defer f.Close()
    _, err = snapshot.Verify(f) // fails fast on corrupt meta.json
    return err
}

func isMetaDecodeErr(err error) bool {
    return err != nil && strings.Contains(err.Error(), "failed to decode snapshot metadata")
}

if err := snapshot.Restore(logOut, snap); err != nil {
    if strings.Contains(err.Error(), "failed to decode snapshot metadata") {
        return fmt.Errorf("snapshot meta.json is corrupt or from an incompatible format; restore from a known-good snapshot: %w", err)
    }
    return err
}

When it happens

Trigger: read() (via Restore/CopySnapshot/snapshot Open) on a snapshot whose meta.json entry contains malformed, truncated, or non-JSON bytes — corruption of the archive contents, a mismatched/custom archive, or manual tampering with the tar entries.

Common situations: Snapshot file partially corrupted after a crash during copy; hand-assembled or edited snapshot archives; restoring an artifact that is not a Consul snapshot (e.g. a generic Raft snapshot of different vintage or another tool's tar) into Consul; bit-rot on backup media.

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• failed reading snapshot: %v
• failed to read snapshot hashes: %v
• failed checking integrity of snapshot: %v
• failed to encode bootstrap environment: %w
• error parsing fingerprint output as json: %w

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/c195b227f0e9ffe8. Report an issue: GitHub.