not losslessly TOON-encodable (invalid JSON, or nested/non-u

Error message

not losslessly TOON-encodable (invalid JSON, or nested/non-uniform shape); keep JSON

What it means

toonEncodeBytes converts JSON to the lossless TOON subset and fails closed: the input is either not valid JSON or its shape (deep nesting, non-uniform arrays) falls outside the proven round-trip subset. Instead of emitting a lossy approximation it returns an error so the caller keeps the original JSON.

Source

Thrown at engine/cmd/caveman-engine/main.go:613

	return model
}

func parsePositiveInt(name, raw string) int {
	n, err := strconv.Atoi(raw)
	if err != nil || n < 1 {
		fatal("%s must be a positive integer", name)
	}
	return n
}

// toonEncodeBytes converts JSON to the lossless TOON subset. It fails closed:
// any input that is not valid JSON, or whose shape is outside the proven
// round-trip subset (deeply nested / non-uniform), returns an error rather than a
// lossy approximation — so the caller keeps JSON instead of trusting bad TOON.
func toonEncodeBytes(input []byte) ([]byte, error) {
	out, ok := compressors.NewTOON().Compress(input)
	if !ok {
		return nil, fmt.Errorf("not losslessly TOON-encodable (invalid JSON, or nested/non-uniform shape); keep JSON")
	}
	return out, nil
}

// toonDecodeBytes converts TOON back to compact JSON. Malformed TOON fails closed
// with an error; it never emits partial or guessed JSON.
func toonDecodeBytes(input []byte) ([]byte, error) {
	v, ok := compressors.DecodeTOON(input)
	if !ok {
		return nil, fmt.Errorf("not valid TOON")
	}
	out, err := json.Marshal(v)
	if err != nil {
		return nil, err
	}
	return out, nil
}

View on GitHub (pinned to 27d5a3981a)

Solutions

Validate/parse the input as JSON before attempting TOON encoding.
On failure, fall back to storing/sending the original JSON — that is the designed behavior, not a bug.
Normalize the JSON upstream (uniform array shapes, flattened structures) if TOON gains matter.
Check the TOON encoder's documented subset to see which shapes qualify.

Example fix

// before
enc, err := toonEncodeBytes(data)

// after: treat failure as keep-JSON
enc, err := toonEncodeBytes(data)
if err != nil { enc = data } // keep original JSON

Defensive patterns

Strategy: fallback

Validate before calling

func isJSON(b []byte) bool {
    return json.Valid(b)
}

Try / catch

enc, err := toonEncodeBytes(input)
if err != nil {
    enc = input // keep original JSON — designed fallback
}

Prevention

Validate json.Valid(input) before attempting TOON encoding.
Keep JSON shapes uniform (consistent array element types) if TOON gains matter.
Treat encode failure as a normal branch, never as a hard error.

When it happens

Trigger: Calling toonEncodeBytes on non-JSON bytes, or on JSON with nested objects / mixed-type arrays that the TOON encoder's Compress rejects (returns ok=false).

Common situations: Piping arbitrary text or log lines (not JSON) into a TOON-encoded path; JSON documents with heterogeneous array elements or deep structures after an upstream format change.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/56bbc6a6c61718c4. Report an issue: GitHub.