hashicorp/nomad · error

failed to read stream: %v

Error message

failed to read stream: %v

What it means

Indicates the error channel from decodeStreamOutput yielded an error after snapshot.Restore was initiated — i.e. reading the client's streamed snapshot input failed partway through (decode or read error on the pipe). Returns 400 to the client.

Source

Thrown at nomad/operator_endpoint.go:704

		return
	} else if !aclObj.IsManagement() {
		handleFailure(403, structs.ErrPermissionDenied)
		return
	}

	op.srv.setQueryMeta(&reply.QueryMeta)

	reader, errCh := decodeStreamOutput(decoder)

	err = snapshot.Restore(op.logger.Named("snapshot"), reader, op.srv.raft)
	if err != nil {
		handleFailure(500, fmt.Errorf("failed to restore from snapshot: %v", err))
		return
	}

	err = <-errCh
	if err != nil {
		handleFailure(400, fmt.Errorf("failed to read stream: %v", err))
		return
	}

	// This'll be used for feedback from the leader loop.
	timeoutCh := time.After(time.Minute)

	lerrCh := make(chan error, 1)

	select {
	// Reassert leader actions and update all leader related state
	// with new state store content.
	case op.srv.reassertLeaderCh <- lerrCh:

	// We might have lost leadership while waiting to kick the loop.
	case <-timeoutCh:
		handleFailure(500, fmt.Errorf("timed out waiting to re-run leader actions"))

	// Make sure we don't get stuck during shutdown

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Verify the snapshot file is complete and valid (nomad operator snapshot inspect)
2. Re-run the restore over a reliable connection
3. Check client logs for the streaming/upload error to find the root cause
4. Confirm no proxy is buffering or truncating large request bodies

err := client.Operator().SnapshotRestore(r, nil, nil)
if err != nil && strings.Contains(err.Error(), "failed to read stream") {
    // input stream was cut: re-open reader and retry
}

When it happens

Trigger: The client-side stream (from 'nomad operator snapshot restore' or decodeStreamOutput's decoder goroutine) hits a decode error, EOF mid-stream, or connection drop; the goroutine forwards it via errCh.

Common situations: Network drop between client and leader during restore; the pushed file was truncated or is not a valid Nomad snapshot archive.

Related errors

• failed to decode input: %v
• missing AllocID
• <remote node streaming RPC error relayed from ack.Error>
• failed to open a streaming connection: %v
• missing target region

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/6767774fdda27280. Report an issue: GitHub.