{"record":{"id":"c56f305985a42268","repo":"antonmedv/fx","slug":"invalid-character-code-point-d-in-string","errorCode":null,"errorMessage":"Invalid character code point %d in string","messagePattern":"Invalid character code point (.+?) in string","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/jsonx/json.go","lineNumber":241,"sourceCode":"\t\t\t\t\t\ts += string(p.char)\n\t\t\t\t\t}\n\t\t\t\t\t_, err := strconv.ParseInt(s, 16, 32)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\tpanic(fmt.Sprintf(\"Invalid Unicode escape sequence '\\\\u%s'\", s))\n\t\t\t\t\t}\n\t\t\t\tcase '\"', '\\\\', '/', 'b', 'f', 'n', 'r', 't':\n\t\t\t\tdefault:\n\t\t\t\t\tpanic(fmt.Sprintf(\"Invalid escape sequence '\\\\%c'\", p.char))\n\t\t\t\t}\n\t\t\t}\n\t\t} else if p.char == '\\\\' {\n\t\t\tescaped = true\n\t\t} else if p.char == '\"' {\n\t\t\tbreak\n\t\t} else if p.char == 0 {\n\t\t\tpanic(\"Unexpected end of input in string\")\n\t\t} else if rune(p.char) > unicode.MaxRune {\n\t\t\tpanic(fmt.Sprintf(\"Invalid character code point %d in string\", p.char))\n\t\t}\n\t\tp.next()\n\t}\n\n\tstr := string(p.data[start:p.end])\n\tp.next()\n\n\treturn str\n}\n\nfunc (p *JsonParser) parseMinus() *Node {\n\tstart := p.end - 1\n\tp.next()\n\tswitch p.char {\n\tcase '0', '1', '2', '3', '4', '5', '6', '7', '8', '9':\n\t\treturn p.parseNumber(start)\n\t}\n\tif !p.strict {","sourceCodeStart":223,"sourceCodeEnd":259,"githubUrl":"https://github.com/antonmedv/fx/blob/4f31cd3a0c5d66f1b4290a2719bab14a5cee8ebe/internal/jsonx/json.go#L223-L259","documentation":"scanString read a byte whose value, interpreted as a rune, exceeds unicode.MaxRune (0x10FFFF) — i.e. invalid UTF-8 / an out-of-range code point inside a string literal. The library rejects it because valid JSON strings must contain valid Unicode.","triggerScenarios":"Parse on input containing corrupted multi-byte UTF-8 sequences inside a string (e.g. a byte >= 0xF5 or broken continuation) such as `\"\\xFF\\xFF\"`-style binary data embedded in the JSON.","commonSituations":"Binary files accidentally parsed as JSON, latin-1/GBK encoded files read as UTF-8, database blobs pasted into payloads, charset mismatch between producer (ISO-8859-1) and consumer expecting UTF-8.","solutions":["Transcode the input to valid UTF-8 before parsing (e.g. golang.org/x/text/encoding transform from the actual source encoding)","Sanitize with strings.ToValidUTF8(s, \"\\uFFFD\") prior to Parse","Fix the producer to emit UTF-8 (set charset headers / file encodings correctly)","Remove embedded binary data from JSON; use base64 encoding for binary payloads"],"exampleFix":"// before\nnode := jsonx.Parse(rawLatin1Bytes, true)\n// after\nutf8Bytes, _ := charmap.ISO8859_1.NewDecoder().Bytes(rawLatin1Bytes)\nnode := jsonx.Parse(utf8Bytes, true)","handlingStrategy":"validation","validationCode":"import \"unicode/utf8\"\nif !utf8.Valid(b) {\n\treturn errors.New(\"input is not valid UTF-8; transcode before parsing\")\n}","typeGuard":null,"tryCatchPattern":"defer func() {\n\tif r := recover(); r != nil {\n\t\tif msg, ok := r.(string); ok && strings.Contains(msg, \"Invalid character code point\") {\n\t\t\terr = errors.New(\"invalid UTF-8 in input; sanitize with strings.ToValidUTF8\")\n\t\t}\n\t}\n}()","preventionTips":["Run utf8.Valid on all external input before parsing","Declare and honor charset headers; transcode latin-1/GBK sources","Base64-encode binary data instead of embedding raw bytes in JSON","Standardize producer output on UTF-8"],"tags":["json","unicode","utf-8","encoding"],"backgroundTag":"invalid-utf8-input","analyzedSha":"4f31cd3a0c5d66f1b4290a2719bab14a5cee8ebe","analyzedAt":"2026-09-02T02:17:47.344Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}