weaviate/weaviate · critical

create commit logger

Error message

create commit logger

What it means

init() wraps MakeCommitLoggerThunk failures as "create commit logger". After restoring state, the commit logger must create a new raw commit file (it never appends to an existing one). Failure here means the filesystem would not allow a new commit log to be created/opened.

Source

Thrown at adapters/repos/db/vector/hnsw/startup.go:47

	"github.com/weaviate/weaviate/entities/storobj"
	ent "github.com/weaviate/weaviate/entities/vectorindex/hnsw"
)

func (h *hnsw) init(cfg Config) error {
	h.pools = newPools(h.maximumConnectionsLayerZero, h.visitedListPoolMaxSize)

	// Restore from disk. This loads existing snapshot/sorted/raw files into
	// in-memory state.
	if err := h.restoreFromDisk(); err != nil {
		return errors.Wrapf(err, "restore hnsw index %q", cfg.ID)
	}

	// Create commit logger for future writes. The logger unconditionally
	// creates a new raw file and never appends to an existing one — see
	// createNewCommitFile's comment for why.
	cl, err := cfg.MakeCommitLoggerThunk()
	if err != nil {
		return errors.Wrap(err, "create commit logger")
	}
	h.commitLog = cl

	// report the vector_index_size at server startup.
	// otherwise on server restart, prometheus reports
	// a vector_index_size of 0 until more vectors are
	// added.
	h.metrics.SetSize(len(h.nodes))

	return nil
}

// restoreFromDisk loads the HNSW state from commit log files using compact.Loader.
// A truncated/corrupt WAL file is logged for diagnostic purposes but does not
// change the commit logger's behavior: the commit logger always starts a new
// raw file anyway, so there is no "append to the corrupted file" path to avoid.
func (h *hnsw) restoreFromDisk() error {
	beforeAll := time.Now()

View on GitHub (pinned to 75aa4b6d11)

Solutions

1. Make the data directory writable by the weaviate process user (chown/chmod or fix the volume mount)
2. Free disk space if ENOSPC
3. Raise the open-file limit (ulimit -n / container limits) when many shards open files at once
4. Ensure the configured PERSISTENCE_DATA_PATH points to a writable persistent volume, not a read-only overlay

Example fix

// before: read-only mount in docker-compose
volumes: []
// after
volumes:
  - weaviate_data:/var/lib/weaviate
PERSISTENCE_DATA_PATH=/var/lib/weaviate

// pre-start: verify the data path is writable
path := os.Getenv("PERSISTENCE_DATA_PATH")
if err := os.MkdirAll(path, 0o755); err != nil { log.Fatalf("data path not writable: %v", err) }
if f, err := os.CreateTemp(path, "wtest"); err != nil { log.Fatalf("no write access: %v", err) } else { f.Close(); os.Remove(f.Name()) }

cl, err := cfg.MakeCommitLoggerThunk()
if err != nil {
  h.logger.Errorf("commit logger init failed: %v", err)
  return errors.Wrap(err, "create commit logger")
}

When it happens

Trigger: MakeCommitLoggerThunk fails creating/opening the new raw commit file — read-only filesystem, no write permission on the shard directory, disk full, or too many open files (ulimit).

Common situations: Container running with read-only rootfs and no writable volume mount; wrong user/permissions after restoring backups as root; ENOSPC on the data disk; fd exhaustion at startup with many shards.

Related errors

• create commit logger directory
• create commit log file
• browse commit logger directory
• restore hnsw index %q
• stat file: %w

AI-assisted analysis of weaviate/weaviate@75aa4b6d11 (2026-09-04). Data as JSON: /api/errors/3485b3d489c99ca8. Report an issue: GitHub.