hashicorp/nomad · error

failed to decode input: %v

Error message

failed to decode input: %v

What it means

Inside decodeStreamOutput's goroutine, when decoder.Decode(&wrapper) fails while reading a streamed snapshot from the RPC connection, the pipe writer is closed with this error and the error is forwarded on errCh. It surfaces as a 400 in snapshotRestore's 'failed to read stream' path.

Source

Thrown at nomad/operator_endpoint.go:840

	reply.QueryMeta.Index, _ = op.srv.State().LatestIndex()
	op.srv.setQueryMeta(&reply.QueryMeta)

	return nil
}

func decodeStreamOutput(decoder *codec.Decoder) (io.Reader, <-chan error) {
	pr, pw := io.Pipe()
	errCh := make(chan error, 1)

	go func() {
		defer close(errCh)

		for {
			var wrapper cstructs.StreamErrWrapper

			err := decoder.Decode(&wrapper)
			if err != nil {
				pw.CloseWithError(fmt.Errorf("failed to decode input: %v", err))
				errCh <- err
				return
			}

			if len(wrapper.Payload) != 0 {
				_, err = pw.Write(wrapper.Payload)
				if err != nil {
					pw.CloseWithError(err)
					errCh <- err
					return
				}
			}

			if errW := wrapper.Error; errW != nil {
				if errW.Message == io.EOF.Error() {
					pw.CloseWithError(io.EOF)
				} else {
					pw.CloseWithError(errors.New(errW.Message))

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Verify the snapshot file integrity before pushing (checksum / nomad operator snapshot inspect)
2. Re-run the restore over a stable connection
3. Check for proxies that cut long uploads; increase request-body timeouts
4. Look at errCh logs to distinguish EOF (connection drop) vs decode (corrupt data) errors

fi, err := os.Stat(snapPath)
if err != nil || fi.Size() == 0 { return errors.New("snapshot file missing or empty") }
sum, _ := snapChecksum(snapPath)
if sum != expectedSum { return errors.New("snapshot checksum mismatch") }

if err != nil && strings.Contains(err.Error(), "failed to read stream") {
    // decode/read of pushed stream failed: revalidate file and retry
}

When it happens

Trigger: The msgpack decoder hits EOF (connection closed early), malformed bytes, or a wrapped StreamErrWrapper carrying a server-side error payload.

Common situations: Client upload interrupted mid-restore; the pushed snapshot bytes are not valid msgpack stream framing; network proxy truncating the request body.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• failed to read stream: %v
• missing AllocID
• <remote node streaming RPC error relayed from ack.Error>
• failed to open a streaming connection: %v
• missing target region

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/919054e88280cbb5. Report an issue: GitHub.