not losslessly TOON-encodable (invalid JSON, or…

Error message

not losslessly TOON-encodable (invalid JSON, or nested/non-uniform shape); keep JSON

What it means

EncodeTOON only accepts input that the TOON compressor can encode losslessly: valid JSON with a flat/uniform structure. If the compressor reports failure (invalid JSON or nested/non-uniform shapes that TOON cannot represent losslessly), the engine refuses to guess and tells you to keep JSON. This is a deliberate fail-closed guard, not a corrupt-output condition.

Solutions

Validate the input parses as JSON before calling EncodeTOON
Check the shape: TOON needs uniform arrays/flat structures; if nested, keep JSON as the message says
Catch the error and fall back to sending the original JSON untouched
Pre-flatten or reshape the payload if TOON encoding is required

Example fix

out, err := engine.EncodeTOON(input)
if err != nil {
    out = input // keep JSON
}

Defensive patterns

Strategy: fallback

Validate before calling

var probe any
if err := json.Unmarshal(input, &probe); err != nil { /* not JSON; skip TOON */ }

Try / catch

out, err := engine.EncodeTOON(input)
if err != nil {
  out = input // fall back to JSON
}

Prevention

Only request TOON for flat/uniform JSON payloads
Treat EncodeTOON failure as a signal to keep JSON, per the error message
Pre-validate JSON parses before attempting TOON encoding

When it happens

Trigger: Calling EncodeTOON with non-JSON bytes, JSON with nested objects/arrays inside uniform arrays, or otherwise irregular shapes that compressors.NewTOON().Compress rejects.

Common situations: Passing arbitrary user JSON to a TOON tool expecting it always works; trying to TOON-encode deeply nested API responses; piping truncated/invalid JSON.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/710a89dca91c5e33. Report an issue: GitHub.

Appendix: source

Thrown at engine/toon.go:15

package engine

import (
	"encoding/json"
	"fmt"

	"github.com/JuliusBrussee/caveman/engine/compressors"
)

// EncodeTOON converts JSON to the lossless TOON subset without CCR or size
// gating. It is for explicit agent/CLI requests, not automatic proxy routing.
func (e *Engine) EncodeTOON(input []byte) ([]byte, error) {
	out, ok := compressors.NewTOON().Compress(input)
	if !ok {
		return nil, fmt.Errorf("not losslessly TOON-encodable (invalid JSON, or nested/non-uniform shape); keep JSON")
	}
	return out, nil
}

// DecodeTOON converts TOON back to compact JSON. Malformed TOON fails closed.
func (e *Engine) DecodeTOON(input []byte) ([]byte, error) {
	v, ok := compressors.DecodeTOON(input)
	if !ok {
		return nil, fmt.Errorf("not valid TOON")
	}
	out, err := json.Marshal(v)
	if err != nil {
		return nil, err
	}
	return out, nil
}

View on GitHub (pinned to 3ee70a1026)