What causes "lone low UTF-16 surrogate escape" in gastownhall/beads?

Precede the low surrogate with its matching high surrogate to form a valid pair.

lone low UTF-16 surrogate escape

Error message

lone low UTF-16 surrogate escape

What it means

A low surrogate (U+DC00–U+DFFF) must never appear without a preceding high surrogate. If checkJSONSurrogates encounters a \uDC00–\uDFFF escape on its own, it rejects the string with 'lone low UTF-16 surrogate escape' since it cannot be decoded to valid Unicode.

Source

Thrown at internal/migration/legacysqlite/reader.go:669

			return fmt.Errorf("truncated JSON Unicode escape")
		}
		code, err := strconv.ParseUint(raw[i+2:i+6], 16, 16)
		if err != nil {
			return fmt.Errorf("invalid JSON Unicode escape")
		}
		switch {
		case code >= 0xd800 && code <= 0xdbff:
			next := i + 6
			if next+6 > len(raw) || raw[next] != '\\' || raw[next+1] != 'u' {
				return fmt.Errorf("lone high UTF-16 surrogate escape")
			}
			low, err := strconv.ParseUint(raw[next+2:next+6], 16, 16)
			if err != nil || low < 0xdc00 || low > 0xdfff {
				return fmt.Errorf("lone high UTF-16 surrogate escape")
			}
			i = next + 5
		case code >= 0xdc00 && code <= 0xdfff:
			return fmt.Errorf("lone low UTF-16 surrogate escape")
		default:
			i += 5
		}
	}
	return nil
}

func validateCurrentTextBytes(issue *types.Issue) error {
	if len(issue.Payload) > currentTextBytes {
		return fmt.Errorf("legacy SQLite issue payload is %d bytes (current TEXT maximum %d)", len(issue.Payload), currentTextBytes)
	}
	waiters := issueops.FormatJSONStringArray(issue.Waiters)
	if len(waiters) > currentTextBytes {
		return fmt.Errorf("legacy SQLite issue waiters serialize to %d bytes (current TEXT maximum %d)", len(waiters), currentTextBytes)
	}
	return nil
}

View on GitHub (pinned to 71377f2769)

Solutions

Precede the low surrogate with its matching high surrogate to form a valid pair.
Replace the lone low surrogate with the intended character in UTF-8.
Strip the invalid escape or substitute U+FFFD before migrating.

Example fix

// before
"text": "\ude00"
// after
"text": "\ud83d\ude00"

Defensive patterns

Strategy: validation

Validate before calling

if regexp.MustCompile(`(^|[^\\]|\\u[0-9a-fA-F]{4}(?<!\\ud[89ab]..))\\ud[c-f][0-9a-fA-F]{2}`).MatchString(raw) { return errors.New("lone low surrogate") }

Prevention

Avoid slicing strings mid-surrogate-pair
Encode astral characters as UTF-8 literals
Pre-scan legacy data for DC00-DFFF escapes

When it happens

Trigger: A raw metadata string contains a \uDC00–\uDFFF escape that was not preceded by a paired high surrogate escape.

Common situations: Substrings cut from the middle of surrogate pairs, broken UTF-16 to JSON converters, or manually edited legacy rows.

Related errors

AI-assisted analysis of gastownhall/beads@71377f2769 (2026-08-30). Data as JSON: /api/errors/4a61ee3bcdb8e114. Report an issue: GitHub.