gastownhall/beads · error

legacy SQLite schema drift in %s hidden column

Error message

legacy SQLite schema drift in %s hidden column

What it means

verifyTable reads PRAGMA table_xinfo for each table and rejects any column whose 'hidden' flag is non-zero. Non-zero hidden values indicate generated, virtual, or internal (e.g. STRICT/virtual-table shadow) columns that are not part of the audited legacy layout — evidence the table shape deviates from the exact contract this package migrates. The error names the offending table.

Source

Thrown at internal/migration/legacysqlite/reader.go:343

}

func verifyTable(ctx context.Context, db *sql.Tx, table, want string) error {
	rows, err := db.QueryContext(ctx, "PRAGMA table_xinfo("+table+")")
	if err != nil {
		return err
	}
	defer rows.Close()
	var got []string
	for rows.Next() {
		var cid, hidden int
		var name, typ string
		var notNull, pk int
		var defaultValue sql.NullString
		if err := rows.Scan(&cid, &name, &typ, &notNull, &defaultValue, &pk, &hidden); err != nil {
			return err
		}
		if hidden != 0 {
			return fmt.Errorf("legacy SQLite schema drift in %s hidden column", table)
		}
		defaultText := "-"
		if defaultValue.Valid {
			defaultText = defaultValue.String
		}
		got = append(got, fmt.Sprintf("%s|%s|%d|%s|%d", name, typ, notNull, defaultText, pk))
	}
	if err := rows.Err(); err != nil {
		return err
	}
	if strings.Join(got, " ") != want {
		return fmt.Errorf("legacy SQLite schema drift in %s", table)
	}
	return nil
}

func verifyFKs(ctx context.Context, db *sql.Tx, table string) error {
	rows, err := db.QueryContext(ctx, "PRAGMA foreign_key_list("+table+")")

View on GitHub (pinned to 71377f2769)

Solutions

  1. Compare the table definition: sqlite3 beads.db ".schema issues" and remove/revert generated or virtual columns so the layout matches the documented legacy schema
  2. Restore the original database from a backup taken before the schema was modified
  3. Recreate the table with the canonical legacy schema and copy rows into it, then retry the export
  4. Check with the bd project which release produced the file; an unexpected producer may have introduced drift

Example fix

-- inspect for generated columns
sqlite3 beads.db "PRAGMA table_xinfo(issues);"  -- last column (hidden) must be 0 for every row
-- fix: rebuild the table without the generated column
sqlite3 beads.db "CREATE TABLE issues_new (...same legacy columns...); INSERT INTO issues_new SELECT <cols> FROM issues; DROP TABLE issues; ALTER TABLE issues_new RENAME TO issues;"
Defensive patterns

Strategy: validation

Validate before calling

// pre-flight: ensure no hidden/generated columns in any legacy table
func checkNoHiddenColumns(dbPath string, tables []string) error {
	db, err := sql.Open("sqlite3", dbPath+"?mode=ro"); if err != nil { return err }
	defer db.Close()
	for _, t := range tables {
		rows, err := db.Query("PRAGMA table_xinfo(" + t + ")"); if err != nil { return err }
		for rows.Next() {
			var cid, hidden int; var name, typ string; var nn, pk int; var d sql.NullString
			if err := rows.Scan(&cid, &name, &typ, &nn, &d, &pk, &hidden); err != nil { rows.Close(); return err }
			if hidden != 0 { rows.Close(); return fmt.Errorf("%s: hidden column %s", t, name) }
		}
		rows.Close()
	}
	return nil
}

Try / catch

if err := legacysqlite.Export(ctx, src, out, os.Stdout); err != nil {
	if strings.Contains(err.Error(), "hidden column") {
		return fmt.Errorf("database has generated/virtual columns; restore original schema or rebuild the table before export")
	}
	return err
}

Prevention

When it happens

Trigger: Export -> read -> verify -> verifyTable: a row from PRAGMA table_xinfo(<table>) has hidden != 0 for any of metadata, issues, dependencies, labels, comments. Occurs when the table was created as a virtual table, has generated columns, or was modified by tooling.

Common situations: The database was produced or altered by a different tool or schema generator that added generated columns; a virtual-table-based shim replaced a real table; custom patches to the legacy schema.

Related errors

AI-assisted analysis of gastownhall/beads@71377f2769 (2026-08-30). Data as JSON: /api/errors/dbf572dc9ad54716. Report an issue: GitHub.