{"record":{"id":"2a01b88efdae1eff","repo":"vitessio/vitess","slug":"invalid-hex-digit-in-u-escape-q","errorCode":null,"errorMessage":"invalid hex digit in \\u escape: %q","messagePattern":"invalid hex digit in \\\\u escape: %q","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"go/mysql/json/marshal.go","lineNumber":441,"sourceCode":"\t\t\tdst = append(dst, '\\f')\n\t\t\ti++\n\t\tcase 'n':\n\t\t\tdst = append(dst, '\\n')\n\t\t\ti++\n\t\tcase 'r':\n\t\t\tdst = append(dst, '\\r')\n\t\t\ti++\n\t\tcase 't':\n\t\t\tdst = append(dst, '\\t')\n\t\t\ti++\n\t\tcase 'u':\n\t\t\ti++ // skip 'u'\n\t\t\tif i+4 > len(src) {\n\t\t\t\treturn dst, errors.New(\"truncated \\\\u escape in JSON string\")\n\t\t\t}\n\t\t\tr := parseHex4(src[i : i+4])\n\t\t\tif r < 0 {\n\t\t\t\treturn dst, fmt.Errorf(\"invalid hex digit in \\\\u escape: %q\", src[i:i+4])\n\t\t\t}\n\t\t\ti += 4\n\n\t\t\t// Handle UTF-16 surrogate pairs.\n\t\t\tif utf16.IsSurrogate(r) {\n\t\t\t\tif i+6 <= len(src) && src[i] == '\\\\' && src[i+1] == 'u' {\n\t\t\t\t\tr2 := parseHex4(src[i+2 : i+6])\n\t\t\t\t\tif r2 >= 0 {\n\t\t\t\t\t\tcombined := utf16.DecodeRune(r, r2)\n\t\t\t\t\t\tif combined != utf8.RuneError {\n\t\t\t\t\t\t\tdst = utf8.AppendRune(dst, combined)\n\t\t\t\t\t\t\ti += 6\n\t\t\t\t\t\t\tcontinue\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\t// Lone surrogate: encode as replacement character.\n\t\t\t\tdst = utf8.AppendRune(dst, utf8.RuneError)","sourceCodeStart":423,"sourceCodeEnd":459,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/mysql/json/marshal.go#L423-L459","documentation":"When unescaping a JSON string body, a \\u escape must be followed by exactly four hexadecimal digits. parseHex4 returned -1 because one of the four bytes after \\u is not a valid hex character, so unescapeJSON rejects the string with this error. This keeps malformed or corrupted JSON strings out of the generated SQL instead of silently encoding garbage.","triggerScenarios":"Writing a JSON string that contains an escape sequence where the four characters after \\\\u are not all hex digits, e.g. \\\\u12z4, \\\\u+123, \\\\u 1F6, or a shortened escape like \\\\u12 that was padded with other text. Reached via writeStringContent -> unescapeJSON when serializing a string value or an object key through the JSON-to-SQL writer.","commonSituations":"JSON produced by buggy custom escape routines that emit \\\\u followed by decimal code points instead of hex; double-encoding layers (an escaped \\\\u0041 turned into \\\\\\\\u0041 then corrupted); manually edited JSON fixtures with typos; corrupted binary payloads where bytes after \\\\u were replaced.","solutions":["Validate the document with encoding/json (json.Valid/Unmarshal) before passing it to the writer; the standard parser rejects malformed \\\\u escapes earlier with clearer context.","Fix the escape to use exactly four hex digits (e.g. \\\\u0041) or, better, re-encode the source data with json.Marshal so escapes are generated correctly.","Track down the producer emitting non-hex \\\\u sequences (custom escaper, template engine, log munging) and correct it.","If the data was corrupted in transit, checksum or length-validate payloads before parsing."],"exampleFix":"// before: decimal code point in escape\ns := \"{\\\\\"k\\\\\": \\\\\\\\u12345}\" // invalid\n// after: proper 4-hex-digit escape\ns := \"{\\\\\"k\\\\\": \\\\\\\\u1234}\"","handlingStrategy":"validation","validationCode":"// reject malformed \\\\u escapes before writing\nfor i := 0; i+1 < len(s); i++ {\n\tif s[i] == '\\\\' && s[i+1] == 'u' {\n\t\tif i+6 > len(s) || !isHex4(s[i+2:i+6]) {\n\t\t\treturn errors.New(\"invalid \\\\u escape\")\n\t\t}\n\t}\n}","typeGuard":"func hasValidUnicodeEscapes(s []byte) bool {\n\tfor i := 0; i+1 < len(s); i++ {\n\t\tif s[i] == '\\\\' && s[i+1] == 'u' {\n\t\t\tif i+6 > len(s) {\n\t\t\t\treturn false\n\t\t\t}\n\t\t\tfor _, c := range s[i+2 : i+6] {\n\t\t\t\tif !((c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F')) {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\t\t\t}\n\t\t\ti += 5\n\t\t}\n\t}\n\treturn true\n}","tryCatchPattern":"if err := writeJSONAsSQL(input); err != nil {\n\tif strings.Contains(err.Error(), \"invalid hex digit in \\\\u escape\") {\n\t\treturn fmt.Errorf(\"rejected payload: malformed unicode escape: %w\", err)\n\t}\n\treturn err\n}","preventionTips":["Generate escapes with json.Marshal only","Never hand-roll \\\\u escaping in templates or log mungers","Decode-then-reencode suspicious payloads before passing them through","Test with emoji/CJK data to exercise unicode escapes"],"tags":["json","unicode","escaping","parsing"],"backgroundTag":"invalid-json-escape-sequence","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}