JuliusBrussee/caveman · error

unsupported token %T

Error message

unsupported token %T

What it means

readJSONTOONValue switches on the token type from json.Decoder.Token() and handles json.Delim, nil, bool, json.Number, and string. Any other token type falls into the default branch and is rejected with this error, since the TOON value model has no representation for it.

Solutions

1. Confirm the input is plain JSON parsed with encoding/json's json.Decoder.
2. Check your Go toolchain version and pin to a known-good version if a stdlib upgrade introduced this.
3. If you extended the token handling, add a case for the new token type in the switch.

v, err := parseJSONTOON(input)
if err != nil {
    if strings.Contains(err.Error(), "unsupported token") {
        return fmt.Errorf("toon encoder cannot handle this token type; report with Go version %s", runtime.Version())
    }
    return err
}

When it happens

Trigger: Only reachable if the decoder returns a token type outside the handled set — in practice this is a defensive guard; it can surface if the input feeds something the standard decoder maps to an unhandled token type in a future Go version or via a custom decoder.

Common situations: Rare; typically seen only after Go stdlib changes or when the parse function is refactored to feed tokens from a non-standard source.

Understand the failure class

Background: JSON parse error: "Unexpected token" / "not valid JSON" / "failed to parse" — what JSON parsers are really complaining about — this error's family across 45 libraries.

Related errors

• array not closed
• unexpected delimiter
• not losslessly TOON-encodable (invalid JSON, or…
• object key is %T
• duplicate object key

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/1d93dfc06c68a6c0. Report an issue: GitHub.

Appendix: source

Thrown at engine/compressors/toon_encode.go:190

				return toonValue{}, err
			}
			if d, ok := end.(json.Delim); !ok || d != ']' {
				return toonValue{}, fmt.Errorf("array not closed")
			}
			return toonValue{kind: toonArray, arr: arr}, nil
		default:
			return toonValue{}, fmt.Errorf("unexpected delimiter %q", t)
		}
	case nil:
		return toonValue{kind: toonNull}, nil
	case bool:
		return toonValue{kind: toonBool, b: t}, nil
	case json.Number:
		return toonValue{kind: toonNumber, s: t.String()}, nil
	case string:
		return toonValue{kind: toonString, s: t}, nil
	default:
		return toonValue{}, fmt.Errorf("unsupported token %T", tok)
	}
}

func writeTOONValue(b *strings.Builder, v toonValue, name string, indent int, opt EncodeOptions) bool {
	switch v.kind {
	case toonObject:
		if name == "" {
			if len(v.obj) == 0 {
				b.WriteString("{}\n")
				return true
			}
			return writeTOONObject(b, v.obj, indent, opt)
		}
		if !safeTOONKey(name) {
			return false
		}
		if len(v.obj) == 0 {
			writeIndent(b, indent)

View on GitHub (pinned to 3ee70a1026)