What causes "unexpected delimiter %q" in JuliusBrussee/caveman?

Run json.Valid on the input before parseJSONTOON to reject structurally broken JSON early.

unexpected delimiter

Error message

unexpected delimiter %q

What it means

readJSONTOONValue hits its default case when the json.Decoder yields a delimiter token that the current parse context does not allow — for example a '{' or '}' appearing where a value or a specific closing bracket is expected. The library cannot map that delimiter to any toonValue production, so it reports the offending delimiter with %q.

Solutions

Run json.Valid on the input before parseJSONTOON to reject structurally broken JSON early.
Print the offending input around the failure position (use json.Decoder.InputOffset) to locate the misplaced delimiter.
Fix the JSON producer so delimiters are balanced and correctly nested.

Defensive patterns

Strategy: validation

Validate before calling

dec := json.NewDecoder(bytes.NewReader(input))
if err := dec.Decode(&json.RawMessage{}); err != nil {
    return fmt.Errorf("invalid JSON at offset %d: %w", dec.InputOffset(), err)
}

Try / catch

v, err := parseJSONTOON(input)
if err != nil {
    var off int64
    // use a pre-pass decoder's InputOffset to locate the misplaced delimiter
    _ = off
    return fmt.Errorf("toon encode failed: %w", err)
}

Prevention

Validate JSON structure before encoding; reject documents with mismatched delimiters early.
Avoid concatenating JSON fragments manually — build payloads with encoding/json marshaling.
Log dec.InputOffset() on failure to pinpoint the bad delimiter.

When it happens

Trigger: Input JSON with structurally misplaced delimiters, e.g. "[{}]" handled by a scalar branch, "{" closed by ']'{, or a bare '}' at the top level of a value being parsed by parseJSONTOON.

Common situations: Concatenating JSON fragments incorrectly; string templates that inject a closing brace too early; encoding a value that is a bare delimiter-bearing JSON document instead of a value.

Understand the failure class

Background: JSON parse error: "Unexpected token" / "not valid JSON" / "failed to parse" — what JSON parsers are really complaining about — this error's family across 45 libraries.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/78a5435f1e6532c3. Report an issue: GitHub.

Appendix: source

Thrown at engine/compressors/toon_encode.go:179

		case '[':
			var arr []toonValue
			for dec.More() {
				val, err := readJSONTOONValue(dec)
				if err != nil {
					return toonValue{}, err
				}
				arr = append(arr, val)
			}
			end, err := dec.Token()
			if err != nil {
				return toonValue{}, err
			}
			if d, ok := end.(json.Delim); !ok || d != ']' {
				return toonValue{}, fmt.Errorf("array not closed")
			}
			return toonValue{kind: toonArray, arr: arr}, nil
		default:
			return toonValue{}, fmt.Errorf("unexpected delimiter %q", t)
		}
	case nil:
		return toonValue{kind: toonNull}, nil
	case bool:
		return toonValue{kind: toonBool, b: t}, nil
	case json.Number:
		return toonValue{kind: toonNumber, s: t.String()}, nil
	case string:
		return toonValue{kind: toonString, s: t}, nil
	default:
		return toonValue{}, fmt.Errorf("unsupported token %T", tok)
	}
}

func writeTOONValue(b *strings.Builder, v toonValue, name string, indent int, opt EncodeOptions) bool {
	switch v.kind {
	case toonObject:
		if name == "" {

View on GitHub (pinned to 3ee70a1026)