What causes "unexpected token after valid JSON: %v" in hashicorp/terraform?

Ensure the reader contains exactly one JSON document with no trailing tokens.

unexpected token after valid JSON

Error message

unexpected token after valid JSON: %v

What it means

Returned by ParseJson when the JSON decoder reads one valid JSON value from the io.Reader but then finds a non-EOF token immediately after. ParseJson expects the entire stream to be exactly one JSON document, so any trailing content is rejected.

Solutions

Ensure the reader contains exactly one JSON document with no trailing tokens.
If the input is NDJSON or a stream, decode in a loop with decoder.Decode and handle each value separately instead of using ParseJson.
Trim or re-buffer the input to a single document before calling ParseJson.
Log the offending token (already included via %v) to identify which caller is producing extra content.

Example fix

// before
val, err := ParseJson(bytes.NewReader(raw)) // raw = `{...}{...}`
// after
for dec := json.NewDecoder(bytes.NewReader(raw)); ; {
    var v interface{}
    if err := dec.Decode(&v); err == io.EOF { break } else if err != nil { return err }
    _ = v
}

Defensive patterns

Strategy: validation

Validate before calling

// Ensure the reader contains exactly one JSON document before calling ParseJson.
func singleDocument(raw []byte) error {
    dec := json.NewDecoder(bytes.NewReader(raw))
    var v interface{}
    if err := dec.Decode(&v); err != nil {
        return err
    }
    if _, err := dec.Token(); err != io.EOF {
        return fmt.Errorf("expected EOF after one JSON document")
    }
    return nil
}

Try / catch

jv, err := ParseJson(r)
if err != nil {
    if strings.Contains(err.Error(), "unexpected token after valid JSON") {
        // switch to streaming decode for NDJSON inputs
    }
    return err
}

Prevention

Do not feed NDJSON or concatenated documents into ParseJson; use a decode loop instead.
Reset/recreate buffers between uses to avoid stale trailing bytes.
Validate upstream that the source emits exactly one framed JSON document.

When it happens

Trigger: Passing a reader that contains concatenated JSON documents, a JSON value followed by trailing whitespace-garbage, or a stream that was not fully consumed because of a partial write. Also triggered by feeding a JSON array/object stream when a single scalar value was expected and vice versa in some callers.

Common situations: Piping output from a tool that emits newline-delimited JSON (NDJSON) into a caller expecting one document; buffer reuse leaving stale bytes; reading from a connection that delivered more than one framed message.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of hashicorp/terraform@d32a084675 (2026-08-11). Data as JSON: /api/errors/14d4a608b456ea52. Report an issue: GitHub.

Appendix: source

Thrown at internal/command/jsonformat/structured/change.go:327

	for key, value := range values {
		out[key] = unmarshalGeneric(value)
	}
	return out
}

func ParseJson(reader io.Reader) (interface{}, error) {
	decoder := json.NewDecoder(reader)
	decoder.UseNumber()

	var jv interface{}
	if err := decoder.Decode(&jv); err != nil {
		return nil, err
	}

	// The JSON decoder should have consumed the entire input stream, so
	// we should be at EOF now.
	if token, err := decoder.Token(); err != io.EOF {
		return nil, fmt.Errorf("unexpected token after valid JSON: %v", token)
	}

	return jv, nil
}

View on GitHub (pinned to d32a084675)