gastownhall/beads · error

legacy SQLite issue %s metadata: %w

Error message

legacy SQLite issue %s metadata: %w

What it means

After confirming the metadata blob is valid JSON, applyMetadataAndWaiters runs checkJSONSurrogates and wraps any failure as 'legacy SQLite issue <id> metadata: <reason>'. The metadata contains unpaired Unicode surrogate code points, which are invalid in well-formed JSON text and would corrupt downstream storage/encoding.

Source

Thrown at internal/migration/legacysqlite/reader.go:514

	if issue.DueAt, err = parseOptionalTime("due_at", x.dueAt); err != nil {
		return fmt.Errorf("legacy SQLite issue %s: %w", issue.ID, err)
	}
	if issue.DeferUntil, err = parseOptionalTime("defer_until", x.deferUntil); err != nil {
		return fmt.Errorf("legacy SQLite issue %s: %w", issue.ID, err)
	}
	return nil
}

// applyMetadataAndWaiters validates the legacy metadata and waiters JSON blobs
// (well-formed and free of unpaired surrogates) and assigns them to issue.
// metadata is stored verbatim unless it is the empty object; waiters is decoded.
func (x legacyExtras) applyMetadataAndWaiters(issue *types.Issue) error {
	if x.metadata.Valid && x.metadata.String != "" && !json.Valid([]byte(x.metadata.String)) {
		return fmt.Errorf("legacy SQLite issue %s has invalid metadata JSON", issue.ID)
	}
	if x.metadata.Valid && x.metadata.String != "" {
		if err := checkJSONSurrogates(x.metadata.String); err != nil {
			return fmt.Errorf("legacy SQLite issue %s metadata: %w", issue.ID, err)
		}
	}
	if x.metadata.Valid && x.metadata.String != "" && x.metadata.String != "{}" {
		issue.Metadata = []byte(x.metadata.String)
	}
	if x.waiters.Valid && x.waiters.String != "" {
		if !json.Valid([]byte(x.waiters.String)) {
			return fmt.Errorf("issue %s waiters: invalid JSON", issue.ID)
		}
		if err := checkJSONSurrogates(x.waiters.String); err != nil {
			return fmt.Errorf("issue %s waiters: %w", issue.ID, err)
		}
		waiters, err := decodeWaiters(x.waiters.String)
		if err != nil {
			return fmt.Errorf("issue %s waiters: %w", issue.ID, err)
		}
		issue.Waiters = waiters
	}

View on GitHub (pinned to 71377f2769)

Solutions

  1. Locate the offending surrogate escape in the metadata for the reported issue ID and repair the text (pair the surrogate or replace with the correct character / U+FFFD), then re-run migration.
  2. Re-encode the metadata column from its original source with a correct UTF-16 to UTF-8 conversion.
  3. Strip or replace unpaired surrogates with SQL/SQLite string functions or a small repair script before migrating.

Example fix

-- before (lone surrogate)
UPDATE issues SET metadata='{"title": "\uD83D"}' WHERE id='bd-5';
-- after (complete surrogate pair)
UPDATE issues SET metadata='{"title": "\uD83D\uDE00"}' WHERE id='bd-5';
Defensive patterns

Strategy: validation

Validate before calling

if strings.ContainsFunc(metadata, func(r rune) bool { return r >= 0xD800 && r <= 0xDFFF }) || hasLoneSurrogateEscapes(metadata) { /* re-encode before migrate */ }

Try / catch

if err := checkJSONSurrogates(metadata); err != nil {
  // replace unpaired surrogates (e.g. with utf8.RuneError) before migrating
}

Prevention

When it happens

Trigger: Migrating a legacy DB whose issues.metadata JSON is parseable but contains lone \uD800-\uDFFF surrogate escapes or raw surrogate bytes from broken UTF-16 to UTF-8 conversion.

Common situations: Data imported from Windows/UTF-16 tooling with broken encoding conversion; old exporters that split surrogate pairs; string truncation cutting a surrogate pair in half.

Related errors

AI-assisted analysis of gastownhall/beads@71377f2769 (2026-08-30). Data as JSON: /api/errors/6386bdccc2798d20. Report an issue: GitHub.