What causes "error making snapshot: %v" in hashicorp/nomad?

Retry the snapshot request after confirming the allocation is still running (nomad alloc status ); transient races with restarts often resolve on a fresh attempt.

hashicorp/nomad · error

error making snapshot: %v

Error message

error making snapshot: %v

What it means

This error is returned by the Nomad agent's allocSnapshot HTTP endpoint (command/agent/alloc_endpoint.go:516) when the filesystem snapshot of an allocation fails. After resolving the allocation's filesystem via AllocFS, the endpoint calls allocFS.Snapshot(resp) to stream a tar-style snapshot back to the client; any error from that snapshot operation is wrapped with this message. It means the alloc was found and reachable, but the underlying archive/stream operation against the client driver failed.

Source

Thrown at command/agent/alloc_endpoint.go:516

		}
	}

	return reply, rpcErr
}

func (s *HTTPServer) allocSnapshot(allocID string, resp http.ResponseWriter, req *http.Request) (any, error) {
	var secret string
	s.parseToken(req, &secret)
	if !s.agent.Client().ValidateMigrateToken(allocID, secret) {
		return nil, structs.ErrPermissionDenied
	}

	allocFS, err := s.agent.Client().GetAllocFS(allocID)
	if err != nil {
		return nil, fmt.Errorf(allocNotFoundErr)
	}
	if err := allocFS.Snapshot(resp); err != nil {
		return nil, fmt.Errorf("error making snapshot: %v", err)
	}
	return nil, nil
}

func (s *HTTPServer) allocStats(allocID string, resp http.ResponseWriter, req *http.Request) (any, error) {

	// Build the request and parse the ACL token
	task := req.URL.Query().Get("task")
	args := cstructs.AllocStatsRequest{
		AllocID: allocID,
		Task:    task,
	}
	s.parse(resp, req, &args.QueryOptions.Region, &args.QueryOptions)

	// Determine the handler to use
	useLocalClient, useClientRPC, useServerRPC := s.rpcHandlerForAlloc(allocID)

	// Make the RPC

View on GitHub (pinned to 482b49bf1a)

Solutions

Retry the snapshot request after confirming the allocation is still running (nomad alloc status <allocID>); transient races with restarts often resolve on a fresh attempt.
Check the Nomad client logs on the node running the alloc for the underlying driver/filesystem error reported at snapshot time.
Verify the task directory and any host volumes still exist on the client node and are readable by the Nomad agent.
If the alloc is terminal or was rescheduled, target the new allocation ID instead.

Example fix

// before: snapshotting a possibly-stale alloc
snap, err := client.AllocFS().Snapshot(allocID)
// after: verify alloc state first
alloc, _, err := client.Allocations().Info(allocID, nil)
if err != nil { return err }
if alloc.ClientStatus != "running" { return fmt.Errorf("alloc %s is %s; cannot snapshot", allocID, alloc.ClientStatus) }
snap, err := client.AllocFS().Snapshot(allocID)

Defensive patterns

Strategy: retry

Validate before calling

alloc, _, err := client.Allocations().Info(allocID, nil)
if err != nil { return err }
if alloc.ClientStatus != "running" {
    return fmt.Errorf("alloc %s not snapshot-able (status=%s)", allocID, alloc.ClientStatus)
}

Try / catch

var snap io.ReadCloser
for attempt := 0; attempt < 3; attempt++ {
    snap, err = client.AllocFS().Snapshot(allocID, nil)
    if err == nil { break }
    if strings.Contains(err.Error(), "error making snapshot") {
        time.Sleep(time.Duration(attempt+1) * time.Second)
        continue
    }
    return err
}

Prevention

Check the allocation's ClientStatus is running before requesting a snapshot.
Avoid snapshotting allocs during known restarts/reschedules; prefer stable, long-running allocations.
Monitor client-node disk and permissions on task directories.
Log and inspect the wrapped inner error from the HTTP response to distinguish transient vs permanent failures.

When it happens

Trigger: Calling the client-facing HTTP API GET /v1/client/allocation/<allocID>/snapshot (via ClientAllocRequest -> allocSnapshot) when allocFS.Snapshot(resp) returns an error — e.g. the allocation's task driver cannot produce a filesystem snapshot, the alloc's files changed or vanished mid-archive, or the streaming response failed mid-write.

Common situations: Operator runs `nomad alloc fs <alloc> ...` snapshot-style access or tools that download an alloc's filesystem while the task is being restarted/rescheduled; the alloc's host volume or task directory was cleaned up concurrently; driver-level filesystem errors (permissions, disk full) during snapshot creation.

Related errors

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/668de0465cad4274. Report an issue: GitHub.