hashicorp/nomad · error

failed to read stream

Error message

failed to read stream: %v

What it means

Indicates the error channel from decodeStreamOutput yielded an error after snapshot.Restore was initiated — i.e. reading the client's streamed snapshot input failed partway through (decode or read error on the pipe). Returns 400 to the client.

Solutions

1. Verify the snapshot file is complete and valid (nomad operator snapshot inspect)
2. Re-run the restore over a reliable connection
3. Check client logs for the streaming/upload error to find the root cause
4. Confirm no proxy is buffering or truncating large request bodies

err := client.Operator().SnapshotRestore(r, nil, nil)
if err != nil && strings.Contains(err.Error(), "failed to read stream") {
    // input stream was cut: re-open reader and retry
}

When it happens

Trigger: The client-side stream (from 'nomad operator snapshot restore' or decodeStreamOutput's decoder goroutine) hits a decode error, EOF mid-stream, or connection drop; the goroutine forwards it via errCh.

Common situations: Network drop between client and leader during restore; the pushed file was truncated or is not a valid Nomad snapshot archive.

Related errors

• failed to decode input
• could not create snapshot
• could not delete
• failed to open a streaming connection
• failed to stream snapshot

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/6767774fdda27280. Report an issue: GitHub.

Appendix: source

Thrown at nomad/operator_endpoint.go:704

		return
	} else if !aclObj.IsManagement() {
		handleFailure(403, structs.ErrPermissionDenied)
		return
	}

	op.srv.setQueryMeta(&reply.QueryMeta)

	reader, errCh := decodeStreamOutput(decoder)

	err = snapshot.Restore(op.logger.Named("snapshot"), reader, op.srv.raft)
	if err != nil {
		handleFailure(500, fmt.Errorf("failed to restore from snapshot: %v", err))
		return
	}

	err = <-errCh
	if err != nil {
		handleFailure(400, fmt.Errorf("failed to read stream: %v", err))
		return
	}

	// This'll be used for feedback from the leader loop.
	timeoutCh := time.After(time.Minute)

	lerrCh := make(chan error, 1)

	select {
	// Reassert leader actions and update all leader related state
	// with new state store content.
	case op.srv.reassertLeaderCh <- lerrCh:

	// We might have lost leadership while waiting to kick the loop.
	case <-timeoutCh:
		handleFailure(500, fmt.Errorf("timed out waiting to re-run leader actions"))

	// Make sure we don't get stuck during shutdown

View on GitHub (pinned to 482b49bf1a)