ErrLookup › Background articles › "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it

"failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it

Errors like "failed to unmarshal bgp state", "unmarshalling response object", or "failed to parse ... args: unexpected end of JSON input" all come from Go's json.Unmarshal failing to decode bytes into an expected struct. Developers hit them when an HTTP API returns HTML instead of JSON, a response is truncated or empty, a struct doesn't match the server's current schema, or a config/state file was hand-edited or corrupted. This article explains the mechanics behind the json-unmarshal-failed error family and the fixes that apply across libraries.

Distilled from 119 documented records across 23 repositories.

Background

This family sits at the boundary where raw bytes become typed data. A Go program has already obtained a payload — an HTTP response body, stdout captured from a command exec'd in a pod, a line from a log file, a value read from a bolt database or embedded JSON asset — and hands it to encoding/json's Unmarshal with a pointer to a target struct. When the bytes are not valid JSON, or are valid JSON whose shape and field types don't fit the target struct, Unmarshal returns a *json.SyntaxError or *json.UnmarshalTypeError, and the calling library wraps it with a family-style message: cilium's "failed to unmarshal bgp state from %s: %w", pulumi's "unmarshalling response object: %w", sops's "Could not unmarshal input data", containerd's "failed to unmarshal object with key %q: %w". The %w wrapping is deliberate: errors.Is and errors.As still reach the original json error, which names the offending byte offset or field type.

Crucially, an unmarshal error is almost never a JSON-parser bug — it is a symptom of something upstream. The records cluster into a few upstream shapes. Network-path interception is the biggest: proxies, captive portals, SSO gateways, and load balancers substitute an HTML error or login page for the JSON the client expected, and charmbracelet/crush's "unmarshal response: %w: %s" even appends the raw body to the message for exactly this diagnosis. Truncation is second: an interrupted download, a killed CLI mid-write, a short read deadline, or ENOSPC yields an empty or cut-off body ("unexpected end of JSON input"). Schema drift is third: the client was compiled against one version of an API or peer process — cilium-cli vs. the Cilium agent, the tailscale Go client vs. a differently-aged tailscaled, Jaeger query vs. documents written by an older collector — and fields were added, renamed, or retyped. Finally, corrupted or hand-edited persisted state (pulumi stack tags and cloud state, lima's event log, containerd's bolt metadata) fails the same way.

From the caller's side the failure looks uniform — a wrapped json error — but the right fix depends on where the bytes came from, and the records show libraries handling this differently. Some treat it as fatal with no recovery; others build in fallbacks, like cilium's metrics loader that retries a failed perDeployNodeMetrics decode as a flat metric list but gives up with "error unmarshalling file" on structural (syntax) errors; Tencent/WeKnora's Notion client treats a missing or empty data_sources key as a legitimate empty result, erroring only on a present-but-mismatched shape. Some surfaces are inherently version-skew detectors: tailscale documents decode failures on localapi endpoints as signals that client and daemon builds diverge, and cilium's in-pod exec helpers fail when the agent inside the pod is older than the CLI invoking it. Others are compile-time defects rather than runtime ones — alibaba/open-code-review unmarshals go:embed'd template JSON, so a parse failure means the source tree shipped a malformed manifest and the fix ends with a rebuild.

Two practical consequences follow for any library in this family. First, the wrapped json error's own text is the most diagnostic string available: "invalid character '<'" means HTML, "unexpected end of JSON input" means empty or truncated, and an UnmarshalTypeError naming a field means a shape or version mismatch. Second, because the check usually happens after a successful read (sometimes even after a 200 status, as in several web-search providers and the GraphQL client in beads), the error often appears where a developer expects HTTP errors — so checking status and Content-Type before decoding, or capturing the raw body on failure, is the recurring prevention advice across these records.

Common causes

What usually fixes it

Go deeper

Documented occurrences

…and 99 more across the corpus — use search.

Honest provenance: generated on 2026-09-02 from AI-assisted analysis of the linked records. See how records are made.