{"record":{"id":"6386bdccc2798d20","repo":"gastownhall/beads","slug":"legacy-sqlite-issue-s-metadata-w","errorCode":null,"errorMessage":"legacy SQLite issue %s metadata: %w","messagePattern":"legacy SQLite issue (.+?) metadata: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/migration/legacysqlite/reader.go","lineNumber":514,"sourceCode":"\tif issue.DueAt, err = parseOptionalTime(\"due_at\", x.dueAt); err != nil {\n\t\treturn fmt.Errorf(\"legacy SQLite issue %s: %w\", issue.ID, err)\n\t}\n\tif issue.DeferUntil, err = parseOptionalTime(\"defer_until\", x.deferUntil); err != nil {\n\t\treturn fmt.Errorf(\"legacy SQLite issue %s: %w\", issue.ID, err)\n\t}\n\treturn nil\n}\n\n// applyMetadataAndWaiters validates the legacy metadata and waiters JSON blobs\n// (well-formed and free of unpaired surrogates) and assigns them to issue.\n// metadata is stored verbatim unless it is the empty object; waiters is decoded.\nfunc (x legacyExtras) applyMetadataAndWaiters(issue *types.Issue) error {\n\tif x.metadata.Valid && x.metadata.String != \"\" && !json.Valid([]byte(x.metadata.String)) {\n\t\treturn fmt.Errorf(\"legacy SQLite issue %s has invalid metadata JSON\", issue.ID)\n\t}\n\tif x.metadata.Valid && x.metadata.String != \"\" {\n\t\tif err := checkJSONSurrogates(x.metadata.String); err != nil {\n\t\t\treturn fmt.Errorf(\"legacy SQLite issue %s metadata: %w\", issue.ID, err)\n\t\t}\n\t}\n\tif x.metadata.Valid && x.metadata.String != \"\" && x.metadata.String != \"{}\" {\n\t\tissue.Metadata = []byte(x.metadata.String)\n\t}\n\tif x.waiters.Valid && x.waiters.String != \"\" {\n\t\tif !json.Valid([]byte(x.waiters.String)) {\n\t\t\treturn fmt.Errorf(\"issue %s waiters: invalid JSON\", issue.ID)\n\t\t}\n\t\tif err := checkJSONSurrogates(x.waiters.String); err != nil {\n\t\t\treturn fmt.Errorf(\"issue %s waiters: %w\", issue.ID, err)\n\t\t}\n\t\twaiters, err := decodeWaiters(x.waiters.String)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"issue %s waiters: %w\", issue.ID, err)\n\t\t}\n\t\tissue.Waiters = waiters\n\t}","sourceCodeStart":496,"sourceCodeEnd":532,"githubUrl":"https://github.com/gastownhall/beads/blob/71377f276968b452ee607177637970a4ff888584/internal/migration/legacysqlite/reader.go#L496-L532","documentation":"After confirming the metadata blob is valid JSON, applyMetadataAndWaiters runs checkJSONSurrogates and wraps any failure as 'legacy SQLite issue <id> metadata: <reason>'. The metadata contains unpaired Unicode surrogate code points, which are invalid in well-formed JSON text and would corrupt downstream storage/encoding.","triggerScenarios":"Migrating a legacy DB whose issues.metadata JSON is parseable but contains lone \\uD800-\\uDFFF surrogate escapes or raw surrogate bytes from broken UTF-16 to UTF-8 conversion.","commonSituations":"Data imported from Windows/UTF-16 tooling with broken encoding conversion; old exporters that split surrogate pairs; string truncation cutting a surrogate pair in half.","solutions":["Locate the offending surrogate escape in the metadata for the reported issue ID and repair the text (pair the surrogate or replace with the correct character / U+FFFD), then re-run migration.","Re-encode the metadata column from its original source with a correct UTF-16 to UTF-8 conversion.","Strip or replace unpaired surrogates with SQL/SQLite string functions or a small repair script before migrating."],"exampleFix":"-- before (lone surrogate)\nUPDATE issues SET metadata='{\"title\": \"\\uD83D\"}' WHERE id='bd-5';\n-- after (complete surrogate pair)\nUPDATE issues SET metadata='{\"title\": \"\\uD83D\\uDE00\"}' WHERE id='bd-5';","handlingStrategy":"validation","validationCode":"if strings.ContainsFunc(metadata, func(r rune) bool { return r >= 0xD800 && r <= 0xDFFF }) || hasLoneSurrogateEscapes(metadata) { /* re-encode before migrate */ }","typeGuard":null,"tryCatchPattern":"if err := checkJSONSurrogates(metadata); err != nil {\n  // replace unpaired surrogates (e.g. with utf8.RuneError) before migrating\n}","preventionTips":["Ensure any external import into the legacy DB performs correct UTF-16 to UTF-8 conversion.","Avoid truncating text columns, which can split surrogate pairs.","Audit for \\uD800-\\uDFFF escape sequences before migrating.","Keep data flows UTF-8 end to end."],"tags":["sqlite","migration","json","unicode","encoding"],"backgroundTag":"unpaired-surrogate-json","analyzedSha":"71377f276968b452ee607177637970a4ff888584","analyzedAt":"2026-08-30T18:55:39.744Z","schemaVersion":2},"datasetVersion":"2026-08-30T23:17:21.991Z"}