gastownhall/beads · error

invalid JSON Unicode escape

Error message

invalid JSON Unicode escape

What it means

When a \uXXXX escape parses but its four hex characters are not valid hexadecimal (e.g. '\uZZZZ'), strconv.ParseUint fails and the migration rejects the value with 'invalid JSON Unicode escape'. JSON requires exactly four hex digits.

Source

Thrown at internal/migration/legacysqlite/reader.go:655

func checkJSONSurrogates(raw string) error {
	for i := 0; i < len(raw); i++ {
		if raw[i] != '\\' {
			continue
		}
		if i+1 >= len(raw) {
			return fmt.Errorf("truncated JSON escape")
		}
		if raw[i+1] != 'u' {
			i++
			continue
		}
		if i+6 > len(raw) {
			return fmt.Errorf("truncated JSON Unicode escape")
		}
		code, err := strconv.ParseUint(raw[i+2:i+6], 16, 16)
		if err != nil {
			return fmt.Errorf("invalid JSON Unicode escape")
		}
		switch {
		case code >= 0xd800 && code <= 0xdbff:
			next := i + 6
			if next+6 > len(raw) || raw[next] != '\\' || raw[next+1] != 'u' {
				return fmt.Errorf("lone high UTF-16 surrogate escape")
			}
			low, err := strconv.ParseUint(raw[next+2:next+6], 16, 16)
			if err != nil || low < 0xdc00 || low > 0xdfff {
				return fmt.Errorf("lone high UTF-16 surrogate escape")
			}
			i = next + 5
		case code >= 0xdc00 && code <= 0xdfff:
			return fmt.Errorf("lone low UTF-16 surrogate escape")
		default:
			i += 5
		}
	}

View on GitHub (pinned to 71377f2769)

Solutions

1. Fix the malformed \uXXXX sequence in the legacy row to use valid hex digits.
2. Re-encode the text as plain UTF-8 instead of escape sequences.
3. Re-export from the original source with a standards-compliant JSON encoder.

Example fix

// before
"key": "\u00zz"
// after
"key": "\u00ff"

m := regexp.MustCompile(`\\u[0-9a-fA-F]{4}`); cleaned := m.ReplaceAllStringFunc(raw, func(e string) string { return string(rune(mustHex(e))) })

func validUnicodeEscapes(s string) bool { for _, m := range regexp.MustCompile(`\\u....`).FindAllString(s, -1) { if _, err := strconv.ParseUint(m[2:], 16, 16); err != nil { return false } }; return true }

When it happens

Trigger: raw[i+2:i+6] contains characters outside 0-9a-fA-F for a \u escape.

Common situations: Data written by non-conformant serializers, manual string mangling, or corruption in legacy TEXT columns.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• truncated JSON Unicode escape
• legacy SQLite issue %s metadata: %w
• truncated JSON escape
• parsing legacy config: %w
• legacy SQLite issue %s has invalid metadata JSON

AI-assisted analysis of gastownhall/beads@71377f2769 (2026-08-30). Data as JSON: /api/errors/e91d3b1763efe5b6. Report an issue: GitHub.