What causes "failed to create FSM: %w" in hashicorp/nomad?

Inspect the wrapped cause (%w) in the message — fix the underlying dummyFSM/state-store initialization error it reports.

hashicorp/nomad · error

failed to create FSM: %w

Error message

failed to create FSM: %w

What it means

RestoreFromArchive first constructs a dummy Nomad FSM that will ingest the snapshot archive. If dummyFSM fails to create that FSM, the function aborts with "failed to create FSM: %w". This happens before any snapshot bytes are read, so the archive itself is not the cause — FSM construction (internal state store setup) failed.

Source

Thrown at helper/raftutil/snapshot.go:24

import (
	"fmt"
	"io"
	"os"

	"github.com/hashicorp/go-hclog"
	"github.com/hashicorp/nomad/helper/snapshot"
	"github.com/hashicorp/nomad/nomad"
	"github.com/hashicorp/nomad/nomad/state"
	"github.com/hashicorp/nomad/nomad/structs"
	"github.com/hashicorp/raft"
)

func RestoreFromArchive(archive io.Reader, filter *nomad.FSMFilter) (raft.FSM, *state.StateStore, *raft.SnapshotMeta, error) {
	logger := hclog.L()

	fsm, err := dummyFSM(logger)
	if err != nil {
		return nil, nil, nil, fmt.Errorf("failed to create FSM: %w", err)
	}

	// r is closed by RestoreFiltered, w is closed by CopySnapshot
	r, w := io.Pipe()

	errCh := make(chan error)
	metaCh := make(chan *raft.SnapshotMeta)

	go func() {
		meta, err := snapshot.CopySnapshot(archive, w)
		if err != nil {
			errCh <- fmt.Errorf("failed to read snapshot: %w", err)
		} else {
			metaCh <- meta
		}
	}()

	err = fsm.RestoreWithFilter(r, filter)

View on GitHub (pinned to 482b49bf1a)

Solutions

Inspect the wrapped cause (%w) in the message — fix the underlying dummyFSM/state-store initialization error it reports.
Rebuild the tool from the same Nomad version as the snapshot to avoid schema-init incompatibilities.
Retry in a normal writable environment (ensure TMPDIR/working dir exists and is writable) if the store needs temp files.
If transient (resource exhaustion), retry with adequate memory.

Example fix

// before: state store init fails because working dir is missing
snap, err := os.Open("snap.tar") // then RestoreFromArchive -> "failed to create FSM: ..."
// after: ensure a valid writable working environment first
if err := os.MkdirAll(workDir, 0o755); err != nil { log.Fatal(err) }
fsm, store, meta, err := raftutil.RestoreFromArchive(snap, nil)

Defensive patterns

Strategy: try-catch

Validate before calling

// Sanity-check the environment before restoring
if fi, err := os.Stat("."); err != nil || !fi.IsDir() {
    return fmt.Errorf("working directory unavailable")
}

Try / catch

fsm, store, meta, err := raftutil.RestoreFromArchive(archive, filter)
if err != nil {
    if strings.Contains(err.Error(), "failed to create FSM") {
        return fmt.Errorf("environment/FSM init failed (not a snapshot problem): %w", err)
    }
    return err
}

Prevention

Run the snapshot tool in an environment with a writable working directory and adequate memory.
Match the tool's Nomad version to the cluster that produced the snapshot.
Read the wrapped cause first — this error always indicates FSM construction, never archive contents.
Pin the tool binary version in automation to avoid schema-init drift.

When it happens

Trigger: Calling RestoreFromArchive (directly or via RedactSnapshot / NewHarnessFromSnapshot / the nomad-snapshot Run command) when dummyFSM returns an error from its internal state.NewStateStore or similar setup failure.

Common situations: Running the tool in an environment where the state store cannot initialize (e.g. missing working directory, memory/resources constraints); a build/version mismatch where the FSM constructor depends on misconfigured logger or schema initialization.

Related errors

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/9a7a9311a2e792d5. Report an issue: GitHub.