gastownhall/beads · critical

journal: record %s for %s after seq counter heal: %w

Error message

journal: record %s for %s after seq counter heal: %w

What it means

After a duplicate-key failure, insertEventRow heals the event seq counter and retries the INSERT exactly once; if the retry also fails the error is wrapped as 'journal: record <op> for <id> after seq counter heal'. By design this means the duplicate persisted after the counter was raised past the high-water mark — a real bug (two writers minting the same seq), not a transient condition.

Source

Thrown at internal/storage/issueops/journal.go:476

		// A duplicate seq means the counter is BEHIND the journal — it was
		// restored, hand-edited, or copied from another workspace. Left alone
		// that wedges the instance permanently: every later mutation re-mints
		// the same taken seq and fails, and because the journal row shares the
		// mutation's transaction, the user's write fails with it. Raise the
		// counter past the high-water mark and retry exactly once; a second
		// duplicate is a real bug and must surface, not spin.
		if !dberrors.IsDuplicateKey(err) {
			return fmt.Errorf("journal: record %s for %s: %w", op, issueID, err)
		}
		if healErr := healEventSeqCounter(ctx, tx); healErr != nil {
			return healErr
		}
		seq, err = nextEventSeq(ctx, tx)
		if err != nil {
			return err
		}
		if err := insert(seq); err != nil {
			return fmt.Errorf("journal: record %s for %s after seq counter heal: %w", op, issueID, err)
		}
	}
	return nil
}

// healEventSeqCounter seeds the counter row if it is missing and raises it to
// the journal's high-water mark, so the next allocation cannot collide. VALUES +
// GREATEST, not INSERT ... SELECT MAX(): in Dolt a literal+aggregate SELECT over
// an empty table yields zero rows, so an INSERT ... SELECT would seed nothing on
// a fresh journal. GREATEST also makes this safe to call on a counter that is
// already ahead — it never moves the counter backwards.
func healEventSeqCounter(ctx context.Context, tx DBTX) error {
	if _, err := tx.ExecContext(ctx, "INSERT IGNORE INTO bd_events_seq (id, next_seq) VALUES (0, 0)"); err != nil {
		return fmt.Errorf("journal: seed seq counter: %w", err)
	}
	if _, err := tx.ExecContext(ctx, `
		UPDATE bd_events_seq
		SET next_seq = GREATEST(next_seq, COALESCE((SELECT MAX(seq) FROM bd_events_journal), 0))

View on GitHub (pinned to 71377f2769)

Solutions

1. Ensure only one writer (or proper locking) targets the database — the journal shares the caller's transaction
2. Verify bd_events_seq was not manually modified; let healEventSeqCounter manage it
3. Compare the retried seq against MAX(seq) in bd_events_journal to confirm the counter state
4. Report as a bug with logs — a second duplicate is explicitly meant to surface, not spin

Example fix

// before (operator manual fix, causes repeats)
UPDATE bd_events_seq SET next_seq = 1 WHERE id = 0;
// after (let the library heal; never hand-edit)
-- no manual SQL; retry the failing operation and report if it recurs

var next, maxSeq int64
_ = tx.QueryRowContext(ctx, "SELECT next_seq FROM bd_events_seq WHERE id = 0").Scan(&next)
_ = tx.QueryRowContext(ctx, "SELECT COALESCE(MAX(seq),0) FROM bd_events_journal").Scan(&maxSeq)
if next <= maxSeq { return errors.New("seq counter behind journal; do not hand-edit, retry write to trigger heal") }

if err := doWrite(ctx, tx); err != nil {
    if strings.Contains(err.Error(), "after seq counter heal") {
        // genuine bug: log full state for report, do NOT retry blindly
        log.Printf("journal seq conflict after heal: %v", err)
        return err
    }
    return err
}

When it happens

Trigger: Two transactions concurrently insert journal rows with the same seq even after healing; a stale/incorrect bd_events_seq row that GREATEST-based healing doesn't fix (e.g. journal MAX(seq) read lags an in-flight uncommitted duplicate); manual tampering with bd_events_seq.

Common situations: Multiple app instances writing to one shared Dolt database with serialization gaps; operator manually editing bd_events_seq to a low value; driver-level snapshot isolation causing MAX(seq) to miss committed rows.

Related errors

• journal: seed seq counter: %w
• journal: raise seq counter to high-water mark: %w
• lock busy: held by another process
• lock already held by another process
• lock already held by another process

AI-assisted analysis of gastownhall/beads@71377f2769 (2026-08-30). Data as JSON: /api/errors/8c258a45415d2a60. Report an issue: GitHub.