{"record":{"id":"4a61ee3bcdb8e114","repo":"gastownhall/beads","slug":"lone-low-utf-16-surrogate-escape","errorCode":null,"errorMessage":"lone low UTF-16 surrogate escape","messagePattern":"lone low UTF-16 surrogate escape","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/migration/legacysqlite/reader.go","lineNumber":669,"sourceCode":"\t\t\treturn fmt.Errorf(\"truncated JSON Unicode escape\")\n\t\t}\n\t\tcode, err := strconv.ParseUint(raw[i+2:i+6], 16, 16)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"invalid JSON Unicode escape\")\n\t\t}\n\t\tswitch {\n\t\tcase code >= 0xd800 && code <= 0xdbff:\n\t\t\tnext := i + 6\n\t\t\tif next+6 > len(raw) || raw[next] != '\\\\' || raw[next+1] != 'u' {\n\t\t\t\treturn fmt.Errorf(\"lone high UTF-16 surrogate escape\")\n\t\t\t}\n\t\t\tlow, err := strconv.ParseUint(raw[next+2:next+6], 16, 16)\n\t\t\tif err != nil || low < 0xdc00 || low > 0xdfff {\n\t\t\t\treturn fmt.Errorf(\"lone high UTF-16 surrogate escape\")\n\t\t\t}\n\t\t\ti = next + 5\n\t\tcase code >= 0xdc00 && code <= 0xdfff:\n\t\t\treturn fmt.Errorf(\"lone low UTF-16 surrogate escape\")\n\t\tdefault:\n\t\t\ti += 5\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc validateCurrentTextBytes(issue *types.Issue) error {\n\tif len(issue.Payload) > currentTextBytes {\n\t\treturn fmt.Errorf(\"legacy SQLite issue payload is %d bytes (current TEXT maximum %d)\", len(issue.Payload), currentTextBytes)\n\t}\n\twaiters := issueops.FormatJSONStringArray(issue.Waiters)\n\tif len(waiters) > currentTextBytes {\n\t\treturn fmt.Errorf(\"legacy SQLite issue waiters serialize to %d bytes (current TEXT maximum %d)\", len(waiters), currentTextBytes)\n\t}\n\treturn nil\n}\n","sourceCodeStart":651,"sourceCodeEnd":687,"githubUrl":"https://github.com/gastownhall/beads/blob/71377f276968b452ee607177637970a4ff888584/internal/migration/legacysqlite/reader.go#L651-L687","documentation":"A low surrogate (U+DC00–U+DFFF) must never appear without a preceding high surrogate. If checkJSONSurrogates encounters a \\uDC00–\\uDFFF escape on its own, it rejects the string with 'lone low UTF-16 surrogate escape' since it cannot be decoded to valid Unicode.","triggerScenarios":"A raw metadata string contains a \\uDC00–\\uDFFF escape that was not preceded by a paired high surrogate escape.","commonSituations":"Substrings cut from the middle of surrogate pairs, broken UTF-16 to JSON converters, or manually edited legacy rows.","solutions":["Precede the low surrogate with its matching high surrogate to form a valid pair.","Replace the lone low surrogate with the intended character in UTF-8.","Strip the invalid escape or substitute U+FFFD before migrating."],"exampleFix":"// before\n\"text\": \"\\ude00\"\n// after\n\"text\": \"\\ud83d\\ude00\"","handlingStrategy":"validation","validationCode":"if regexp.MustCompile(`(^|[^\\\\]|\\\\u[0-9a-fA-F]{4}(?<!\\\\ud[89ab]..))\\\\ud[c-f][0-9a-fA-F]{2}`).MatchString(raw) { return errors.New(\"lone low surrogate\") }","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Avoid slicing strings mid-surrogate-pair","Encode astral characters as UTF-8 literals","Pre-scan legacy data for DC00-DFFF escapes"],"tags":["json","utf-16","surrogate","migration"],"backgroundTag":"lone-surrogate-escape","analyzedSha":"71377f276968b452ee607177637970a4ff888584","analyzedAt":"2026-08-30T18:55:39.744Z","schemaVersion":2},"datasetVersion":"2026-08-30T23:17:21.991Z"}