grpc-ecosystem/grpc-gateway · error
%s: unexpected token %v
Error message
%s: unexpected token %v
What it means
This error is returned when a token read as a top-level object key is not a JSON string (merge.go:195-197). In a well-formed JSON object every key is a string, so in practice this means the decoder's token stream got out of sync with expectations — the document is structurally invalid even if individual tokens parsed. The unexpected token's value is included in the message to aid debugging.
Source
Thrown at openapiv3-merge/internal/merge/merge.go:197
extras: newOrderedObject(),
}
dec := json.NewDecoder(bytes.NewReader(in.Data))
dec.UseNumber()
tok, err := dec.Token()
if err != nil {
return nil, fmt.Errorf("%s: %w", in.Name, err)
}
if delim, ok := tok.(json.Delim); !ok || delim != '{' {
return nil, fmt.Errorf("%s: expected JSON object at top level", in.Name)
}
for dec.More() {
tok, err := dec.Token()
if err != nil {
return nil, fmt.Errorf("%s: %w", in.Name, err)
}
key, ok := tok.(string)
if !ok {
return nil, fmt.Errorf("%s: unexpected token %v", in.Name, tok)
}
var raw json.RawMessage
if err := dec.Decode(&raw); err != nil {
return nil, fmt.Errorf("%s: %q: %w", in.Name, key, err)
}
switch key {
case "openapi":
if err := json.Unmarshal(raw, &d.OpenAPI); err != nil {
return nil, fmt.Errorf("%s: openapi: %w", in.Name, err)
}
case "info":
d.Info = raw
case "servers":
d.Servers = raw
case "paths":
obj, err := decodeOrderedObject(raw)
if err != nil {
return nil, fmt.Errorf("%s: paths: %w", in.Name, err)View on GitHub (pinned to a58a4436a3)
Solutions
- Validate the file with a strict JSON parser (json.Valid, or jsonlint) — a non-string key token indicates structural corruption that validation will catch.
- Remove JSONC/JSON5 comments and trailing commas before passing the data (convert to standard JSON first).
- Do not concatenate spec files as text; merge them via Merge with separate Inputs instead.
- If keys are generated programmatically, ensure they are always emitted as quoted JSON strings.
Example fix
// before
combined := append(bytes.TrimSuffix(specA, []byte("}")), '{', ...) // string-concat merge
// after
merged, err := merge.Merge([]merge.Input{
{Name: "a", Data: specA},
{Name: "b", Data: specB},
}) Defensive patterns
Strategy: validation
Validate before calling
func strictJSONCheck(data []byte) error {
if !json.Valid(data) {
return errors.New("structurally invalid JSON: check braces, commas, and that all keys are quoted strings")
}
return nil
} Try / catch
if err != nil && strings.Contains(err.Error(), "unexpected token") {
return fmt.Errorf("spec is not a well-formed JSON object; validate with jsonlint: %w", err)
} Prevention
- Strip JSONC/JSON5 comments and trailing commas before merging.
- Merge documents through Merge Inputs, never by concatenating file text.
- Always emit generated keys as quoted JSON strings.
- Add a lint step (jsonlint / json.Valid) to CI before running merge tooling.
When it happens
Trigger: Calling Merge with Input data whose object structure is malformed such that dec.Token() inside the key-reading loop yields a non-string token (e.g. a Delim like '}' or ':' appearing where a key is expected, due to mismatched braces, duplicate/stray delimiters, or hand-edited JSON).
Common situations: Hand-edited or machine-concatenated JSON with mismatched braces; a naive string-merge of two spec files; a converter that emitted invalid object structure; JSON with comments (JSONC/JSON5) confusing a prior preprocessor that only partially stripped them.
Understand the failure class
- Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.
Related errors
AI-assisted analysis of grpc-ecosystem/grpc-gateway@a58a4436a3 (2026-09-02).
Data as JSON: /api/errors/0e9eb11bd0231b83.
Report an issue: GitHub.