{"record":{"id":"e91d3b1763efe5b6","repo":"gastownhall/beads","slug":"invalid-json-unicode-escape","errorCode":null,"errorMessage":"invalid JSON Unicode escape","messagePattern":"invalid JSON Unicode escape","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/migration/legacysqlite/reader.go","lineNumber":655,"sourceCode":"\nfunc checkJSONSurrogates(raw string) error {\n\tfor i := 0; i < len(raw); i++ {\n\t\tif raw[i] != '\\\\' {\n\t\t\tcontinue\n\t\t}\n\t\tif i+1 >= len(raw) {\n\t\t\treturn fmt.Errorf(\"truncated JSON escape\")\n\t\t}\n\t\tif raw[i+1] != 'u' {\n\t\t\ti++\n\t\t\tcontinue\n\t\t}\n\t\tif i+6 > len(raw) {\n\t\t\treturn fmt.Errorf(\"truncated JSON Unicode escape\")\n\t\t}\n\t\tcode, err := strconv.ParseUint(raw[i+2:i+6], 16, 16)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"invalid JSON Unicode escape\")\n\t\t}\n\t\tswitch {\n\t\tcase code >= 0xd800 && code <= 0xdbff:\n\t\t\tnext := i + 6\n\t\t\tif next+6 > len(raw) || raw[next] != '\\\\' || raw[next+1] != 'u' {\n\t\t\t\treturn fmt.Errorf(\"lone high UTF-16 surrogate escape\")\n\t\t\t}\n\t\t\tlow, err := strconv.ParseUint(raw[next+2:next+6], 16, 16)\n\t\t\tif err != nil || low < 0xdc00 || low > 0xdfff {\n\t\t\t\treturn fmt.Errorf(\"lone high UTF-16 surrogate escape\")\n\t\t\t}\n\t\t\ti = next + 5\n\t\tcase code >= 0xdc00 && code <= 0xdfff:\n\t\t\treturn fmt.Errorf(\"lone low UTF-16 surrogate escape\")\n\t\tdefault:\n\t\t\ti += 5\n\t\t}\n\t}","sourceCodeStart":637,"sourceCodeEnd":673,"githubUrl":"https://github.com/gastownhall/beads/blob/71377f276968b452ee607177637970a4ff888584/internal/migration/legacysqlite/reader.go#L637-L673","documentation":"When a \\uXXXX escape parses but its four hex characters are not valid hexadecimal (e.g. '\\uZZZZ'), strconv.ParseUint fails and the migration rejects the value with 'invalid JSON Unicode escape'. JSON requires exactly four hex digits.","triggerScenarios":"raw[i+2:i+6] contains characters outside 0-9a-fA-F for a \\u escape.","commonSituations":"Data written by non-conformant serializers, manual string mangling, or corruption in legacy TEXT columns.","solutions":["Fix the malformed \\uXXXX sequence in the legacy row to use valid hex digits.","Re-encode the text as plain UTF-8 instead of escape sequences.","Re-export from the original source with a standards-compliant JSON encoder."],"exampleFix":"// before\n\"key\": \"\\u00zz\"\n// after\n\"key\": \"\\u00ff\"","handlingStrategy":"validation","validationCode":"m := regexp.MustCompile(`\\\\u[0-9a-fA-F]{4}`); cleaned := m.ReplaceAllStringFunc(raw, func(e string) string { return string(rune(mustHex(e))) })","typeGuard":"func validUnicodeEscapes(s string) bool { for _, m := range regexp.MustCompile(`\\\\u....`).FindAllString(s, -1) { if _, err := strconv.ParseUint(m[2:], 16, 16); err != nil { return false } }; return true }","tryCatchPattern":null,"preventionTips":["Emit JSON only via encoding/json or equivalent","Normalize legacy text to UTF-8 before storing","Lint hex escapes during data export"],"tags":["json","unicode","escaping","migration"],"backgroundTag":"invalid-json-escape","analyzedSha":"71377f276968b452ee607177637970a4ff888584","analyzedAt":"2026-08-30T18:55:39.744Z","schemaVersion":2},"datasetVersion":"2026-08-30T23:17:21.991Z"}