What causes "proposalrun: seq %d prev_hash does not link to the prior row" in JuliusBrussee/caveman?

Serialize appends: allocate Seq and read the chain tail inside one transaction (SELECT ... FOR UPDATE on the tail row or an advisory lock), so PrevHash is always the committed predecessor.

JuliusBrussee/caveman · critical

proposalrun: seq %d prev_hash does not link to the prior row

Error message

proposalrun: seq %d prev_hash does not link to the prior row

What it means

Each Run row stores PrevHash = the previous row's RowHash (empty string for seq 1). After sorting and confirming contiguity, VerifyChain compares each row's stored PrevHash against the RowHash of the row before it. A mismatch means the linkage field itself is broken: the row was inserted with the wrong predecessor hash (e.g. read a stale chain head under a race), was edited in place, or was spliced in from another chain.

Source

Thrown at shared/platform/proposalrun/proposalrun.go:357

//   - each row's row_hash recomputes from its own stored fields (no tampering).
//
// It returns nil for an empty slice (a proposal with no runs yet is consistent).
func VerifyChain(runs []Run) error {
	if len(runs) == 0 {
		return nil
	}
	sorted := make([]Run, len(runs))
	copy(sorted, runs)
	sort.Slice(sorted, func(i, j int) bool { return sorted[i].Seq < sorted[j].Seq })

	prevHash := ""
	for i, r := range sorted {
		wantSeq := int64(i + 1)
		if r.Seq != wantSeq {
			return fmt.Errorf("proposalrun: seq %d out of order (expected %d)", r.Seq, wantSeq)
		}
		if r.PrevHash != prevHash {
			return fmt.Errorf("proposalrun: seq %d prev_hash does not link to the prior row", r.Seq)
		}
		got, err := RowHash(r.PrevHash, r.Seq, r.Action, r.Detail, r.CostUSD, r.CreatedAt)
		if err != nil {
			return fmt.Errorf("proposalrun: seq %d recompute: %w", r.Seq, err)
		}
		if got != r.RowHash {
			return fmt.Errorf("proposalrun: seq %d row_hash mismatch (tampered)", r.Seq)
		}
		prevHash = r.RowHash
	}
	return nil
}

View on GitHub (pinned to 27d5a3981a)

Solutions

Serialize appends: allocate Seq and read the chain tail inside one transaction (SELECT ... FOR UPDATE on the tail row or an advisory lock), so PrevHash is always the committed predecessor.
Repair by recomputing PrevHash and RowHash forward from the first broken row (re-chain) — but treat the gap as a tamper event and investigate first.
Add a UNIQUE(org_id, seq) constraint and a trigger rejecting UPDATEs to prev_hash/row_hash so drift cannot recur.

Example fix

// before
head := store.GetTailHash(orgID)            // outside the write transaction
row := buildRun(action, detail, head)       // racy: another writer may append in between
store.Insert(row)

// after
tx := db.Begin()
tx.Exec(`SELECT row_hash FROM proposal_runs WHERE org_id=$1 ORDER BY seq DESC LIMIT 1 FOR UPDATE`, orgID)
head := /* that row_hash */
row := buildRun(action, detail, head)
tx.Insert(row)
tx.Commit()

Defensive patterns

Strategy: validation

Validate before calling

// Detect a broken link before the verifier: compare stored PrevHash to prior RowHash.
func linksConsistent(sorted []proposalrun.Run) bool {
    prev := ""
    for _, r := range sorted {
        if r.PrevHash != prev { return false }
        prev = r.RowHash
    }
    return true
}

Try / catch

if err := proposalrun.VerifyChain(runs); err != nil {
    if strings.Contains(err.Error(), "prev_hash does not link") {
        // stop writes, diff against a replica/backup, then re-chain from the broken row
    }
}

Prevention

Allocate Seq and read the tail RowHash inside one transaction (FOR UPDATE on the tail or an advisory lock).
Make the table append-only for the app role; forbid UPDATE on prev_hash/row_hash.
Alert on this error: it indicates either a writer race or tampering, both needing investigation.

When it happens

Trigger: Two writers append concurrently: both read the same tail RowHash and one overwrites/interleaves, so a row links to a hash that is not its true predecessor. Or an UPDATE statement touches prev_hash. Or rows from two organizations/chains were merged into one ListRuns result.

Common situations: Missing unique constraint on (org, seq) letting concurrent appends race; an operator 'fixing' a row with SQL UPDATE; copying rows between environments; test fixtures hand-built with fabricated hashes.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/4e1df9686206b521. Report an issue: GitHub.