{"record":{"id":"56bbc6a6c61718c4","repo":"JuliusBrussee/caveman","slug":"not-losslessly-toon-encodable-invalid-json-or-ne","errorCode":null,"errorMessage":"not losslessly TOON-encodable (invalid JSON, or nested/non-uniform shape); keep JSON","messagePattern":"not losslessly TOON-encodable \\(invalid JSON, or nested/non-uniform shape\\); keep JSON","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"info","filePath":"engine/cmd/caveman-engine/main.go","lineNumber":613,"sourceCode":"\treturn model\n}\n\nfunc parsePositiveInt(name, raw string) int {\n\tn, err := strconv.Atoi(raw)\n\tif err != nil || n < 1 {\n\t\tfatal(\"%s must be a positive integer\", name)\n\t}\n\treturn n\n}\n\n// toonEncodeBytes converts JSON to the lossless TOON subset. It fails closed:\n// any input that is not valid JSON, or whose shape is outside the proven\n// round-trip subset (deeply nested / non-uniform), returns an error rather than a\n// lossy approximation — so the caller keeps JSON instead of trusting bad TOON.\nfunc toonEncodeBytes(input []byte) ([]byte, error) {\n\tout, ok := compressors.NewTOON().Compress(input)\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"not losslessly TOON-encodable (invalid JSON, or nested/non-uniform shape); keep JSON\")\n\t}\n\treturn out, nil\n}\n\n// toonDecodeBytes converts TOON back to compact JSON. Malformed TOON fails closed\n// with an error; it never emits partial or guessed JSON.\nfunc toonDecodeBytes(input []byte) ([]byte, error) {\n\tv, ok := compressors.DecodeTOON(input)\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"not valid TOON\")\n\t}\n\tout, err := json.Marshal(v)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n","sourceCodeStart":595,"sourceCodeEnd":631,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/27d5a3981a347890211bb1bf2439e5c821a63bc9/engine/cmd/caveman-engine/main.go#L595-L631","documentation":"toonEncodeBytes converts JSON to the lossless TOON subset and fails closed: the input is either not valid JSON or its shape (deep nesting, non-uniform arrays) falls outside the proven round-trip subset. Instead of emitting a lossy approximation it returns an error so the caller keeps the original JSON.","triggerScenarios":"Calling toonEncodeBytes on non-JSON bytes, or on JSON with nested objects / mixed-type arrays that the TOON encoder's Compress rejects (returns ok=false).","commonSituations":"Piping arbitrary text or log lines (not JSON) into a TOON-encoded path; JSON documents with heterogeneous array elements or deep structures after an upstream format change.","solutions":["Validate/parse the input as JSON before attempting TOON encoding.","On failure, fall back to storing/sending the original JSON — that is the designed behavior, not a bug.","Normalize the JSON upstream (uniform array shapes, flattened structures) if TOON gains matter.","Check the TOON encoder's documented subset to see which shapes qualify."],"exampleFix":"// before\nenc, err := toonEncodeBytes(data)\n\n// after: treat failure as keep-JSON\nenc, err := toonEncodeBytes(data)\nif err != nil { enc = data } // keep original JSON","handlingStrategy":"fallback","validationCode":"func isJSON(b []byte) bool {\n    return json.Valid(b)\n}","typeGuard":null,"tryCatchPattern":"enc, err := toonEncodeBytes(input)\nif err != nil {\n    enc = input // keep original JSON — designed fallback\n}","preventionTips":["Validate json.Valid(input) before attempting TOON encoding.","Keep JSON shapes uniform (consistent array element types) if TOON gains matter.","Treat encode failure as a normal branch, never as a hard error."],"tags":["toon","json","encode","fail-closed"],"backgroundTag":null,"analyzedSha":"27d5a3981a347890211bb1bf2439e5c821a63bc9","analyzedAt":"2026-08-15T09:26:11.751Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}