hashicorp/nomad · error

failed to write data at key %s: %v

Error message

failed to write data at key %s: %v

What it means

boltdd.Bucket.Put wraps a failure from the underlying bolt bucket's Put when writing the serialized bytes at the given key. The value encoded fine, but the boltdb transaction rejected the write. Since bolt only allows one write transaction, this usually means the write is happening outside a writable transaction or the DB is in a read-only/invalid state.

Source

Thrown at helper/boltdd/boltdd.go:332

	if err := codec.NewEncoder(&buf, structs.MsgpackHandle).Encode(val); err != nil {
		return fmt.Errorf("failed to encode passed object: %v", err)
	}

	// Hash for skipping unnecessary writes
	hashKey := string(key)
	hashVal := blake2b.Sum256(buf.Bytes())

	// lastHash value or nil if it hasn't been hashed yet
	lastHash := b.bm.getHash(hashKey)

	// If the hashes are equal, skip the write
	if bytes.Equal(hashVal[:], lastHash) {
		return nil
	}

	// New value: write it to the underlying boltdb
	if err := b.boltBucket.Put(key, buf.Bytes()); err != nil {
		return fmt.Errorf("failed to write data at key %s: %v", key, err)
	}

	// New value written, store hash (bucket path map was created above)
	b.bm.setHash(hashKey, hashVal[:])

	return nil

}

// Get value by key from boltdb or return an ErrNotFound error if key not
// found.
func (b *Bucket) Get(key []byte, obj interface{}) error {
	// Get the raw data from the underlying boltdb
	data := b.boltBucket.Get(key)
	if data == nil {
		return NotFound(string(key))
	}

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Inspect the wrapped %v cause (e.g. ErrTxNotWritable, ErrDatabaseNotOpen) to identify the transaction/state problem.
2. Ensure all Put calls occur inside the single active write transaction managed by boltdd and before it is committed.
3. Verify the DB file is writable, the disk is not full, and the DB is not opened with ReadOnly:true.
4. Check for code paths calling Put after Close or after restore/rollback, and serialize writers behind one goroutine/lock.

Example fix

// before
go func() { bucket.Put(key, val) }() // outside write tx / concurrent
// after
err := db.Update(func(tx interface{}) error {
    return bucket.Put(key, val) // inside managed write transaction
})

// before writing: ensure DB is open and writable
db.View() // will fail fast if DB is not open; instead track state:
if dbClosed || !inWriteTx {
    return errors.New("cannot Put: no active write transaction / db closed")
}

if err := b.Put(key, val); err != nil {
    var pe *boltdbWriteErr // or inspect wrapped cause
    if errors.Is(err, bolt.ErrTxNotWritable) || errors.Is(err, bolt.ErrDatabaseNotOpen) {
        // route write through the single writer goroutine / reopen db
    }
    return fmt.Errorf("put %q: %w", key, err)
}

When it happens

Trigger: Calling Put after the DB was opened read-only (bolt.ErrDatabaseNotOpen / ErrDatabaseReadOnly); calling Put outside the single allowed write transaction (bolt.ErrTxNotWritable) or after that transaction committed/rolled back; key or value exceeding bolt limits; disk full / I/O error while writing the page.

Common situations: Concurrent writers racing for bolt's single write lock; calling Put after Close or from a goroutine outside the managed transaction; read-only file system or full disk on the node; snapshot restore running concurrently with writes.

Related errors

• failed to write task_runner state: %v
• failed to encode passed object: %v
• failed to open raft store %v: %v
• node update failed: %v
• CSI Plugin loaded incorrectly

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/6faf339d9a6a6b6e. Report an issue: GitHub.