benbjohnson/litestream · error

begin transaction: %w

Error message

begin transaction: %w

What it means

populateTable wraps a failed db.Begin() in this error. Rows are inserted in batches of BatchSize, each batch wrapped in a transaction for throughput. If a transaction cannot be started, no batch is inserted and population aborts.

Source

Thrown at cmd/litestream-test/populate.go:177

	duration := time.Since(startTime)
	finalSize, _ := getDatabaseSize(c.DB)

	slog.Info("Population complete",
		"duration", duration,
		"final_size_mb", finalSize/1024/1024,
		"throughput_mb_per_sec", fmt.Sprintf("%.2f", float64(finalSize)/1024/1024/duration.Seconds()),
	)

	return nil
}

func (c *PopulateCommand) populateTable(ctx context.Context, db *sql.DB, tableName string, rowCount int) error {
	data := make([]byte, c.RowSize)

	for i := 0; i < rowCount; i += c.BatchSize {
		tx, err := db.Begin()
		if err != nil {
			return fmt.Errorf("begin transaction: %w", err)
		}

		stmt, err := tx.Prepare(fmt.Sprintf(`
			INSERT INTO %s (data, text_field, int_field, float_field, timestamp)
			VALUES (?, ?, ?, ?, ?)
		`, tableName))
		if err != nil {
			tx.Rollback()
			return fmt.Errorf("prepare statement: %w", err)
		}

		batchEnd := i + c.BatchSize
		if batchEnd > rowCount {
			batchEnd = rowCount
		}

		for j := i; j < batchEnd; j++ {
			cryptorand.Read(data)

View on GitHub (pinned to 4ed7a308f6)

Solutions

1. Check the wrapped SQLite error for SQLITE_BUSY and increase the busy timeout in the DSN (e.g. ?_pragma=busy_timeout(5000)).
2. Stop competing writers (other litestream-test runs, monitoring connections) during population.
3. Verify the database file and hot journal are writable and not corrupted.
4. Retry the run; SQLITE_BUSY is often transient under contention.

Example fix

// before
db, _ := sql.Open("sqlite", c.DB)
// after
db, _ := sql.Open("sqlite", c.DB+"?_pragma=busy_timeout(5000)")

// increase busy timeout so concurrent locks don't fail Begin
db, err := sql.Open("sqlite", c.DB+"?_pragma=busy_timeout(10000)")
if err != nil {
    return err
}

tx, err := db.Begin()
if err != nil {
    if strings.Contains(err.Error(), "SQLITE_BUSY") {
        time.Sleep(250 * time.Millisecond)
        tx, err = db.Begin() // bounded retry
    }
    if err != nil {
        return fmt.Errorf("begin transaction: %w", err)
    }
}

When it happens

Trigger: db.Begin() returns an error: the connection pool has no usable connection (driver issue), the database is locked by another writer exceeding the busy timeout, or the database is in a broken state after an I/O error.

Common situations: A concurrent litestream or backup process holding the write lock; running with database_sqlite driver connection limits exhausted; a prior crashed run leaving a hot journal that cannot be recovered.

Related errors

• open database: %w
• open database: %w
• set journal mode: %w
• create table %s: %w
• populate table %s: %w

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/82c2e1b11b60141d. Report an issue: GitHub.