weaviate/weaviate · critical

create commit logger directory

Error message

create commit logger directory

What it means

NewCommitLogger creates the per-index HNSW commit log directory (<rootPath>/<name>.hnsw.commitlog.d) via fs.MkdirAll before opening a fresh WAL file. This error wraps the OS error when that directory cannot be created, so the commit logger (and thus the shard's HNSW index) cannot be initialized.

Source

Thrown at adapters/repos/db/vector/hnsw/commit_logger.go:86

		id:                   name,
		logger:               logger,
		fs:                   common.NewOSFS(),
		maintenanceCallbacks: maintenanceCallbacks,

		// can be overwritten using functional options
		maxSizeIndividual: defaultCommitLogSize / 5,
	}

	for _, o := range opts {
		if err := o(l); err != nil {
			return nil, err
		}
	}

	// Ensure the commit log directory exists
	dir := commitLogDirectory(rootPath, name)
	if err := l.fs.MkdirAll(dir, os.ModePerm); err != nil {
		return nil, errors.Wrap(err, "create commit logger directory")
	}

	// Always start a fresh raw commit log file. Reusing an existing file is a
	// footgun: getCurrentCommitLogFileName used to select the append target
	// by highest parsed timestamp, which let it hand back a .snapshot /
	// .sorted / .condensed file and the next AddNode would corrupt it (block
	// CRCs become invalid on next SnapshotReader load). Creating a new file
	// every startup eliminates the class of bug — the append path can never
	// land on anything but a freshly created raw file that we own.
	//
	// Any existing zero-byte raw files from previous startups that the
	// compactor hasn't yet absorbed are pruned first so the directory never
	// accumulates more than one empty raw file at a time.
	fd, fileName, err := createNewCommitFile(rootPath, name, l.fs, l.logger)
	if err != nil {
		return nil, err
	}

View on GitHub (pinned to 75aa4b6d11)

Solutions

1. Check and fix permissions/ownership of PERSISTENCE_DATA_PATH so the Weaviate process user can write to it.
2. Verify the volume is actually mounted (Kubernetes PVC, docker -v) and not read-only; free disk space if full.
3. Ensure no regular file exists at <rootPath>/<name>.hnsw.commitlog.d; remove or rename it if so.
4. Correct the rootPath configuration and restart Weaviate.

Example fix

// before: rootPath points to a read-only or nonexistent location
NewCommitLogger("/var/lib/weaviate", "Collection__shard0", logger, cbs)
// after: ensure path exists and is writable before init
if err := os.MkdirAll(persistencePath, 0o755); err != nil {
    log.Fatalf("persistence path not writable: %v", err)
}
NewCommitLogger(persistencePath, "Collection__shard0", logger, cbs)

// before starting Weaviate, verify the data path is writable
import "os"
func assertWritableDataPath(root string) error {
    if err := os.MkdirAll(root, 0o755); err != nil {
        return fmt.Errorf("data path %q not creatable: %w", root, err)
    }
    probe := filepath.Join(root, ".write-probe")
    if err := os.WriteFile(probe, []byte("ok"), 0o644); err != nil {
        return fmt.Errorf("data path %q not writable: %w", root, err)
    }
    return os.Remove(probe)
}

logger, err := hnsw.NewCommitLogger(rootPath, name, log, cbs)
if err != nil {
    if strings.Contains(err.Error(), "create commit logger directory") {
        // surface as misconfiguration: check PERSISTENCE_DATA_PATH perms/mount
    }
    return fmt.Errorf("shard init failed: %w", err)
}

When it happens

Trigger: Calling NewCommitLogger(rootPath, name, ...) when MkdirAll on the commit-log directory fails: parent path does not exist and cannot be created, permission denied on rootPath, disk full, or rootPath is a file instead of a directory.

Common situations: PERSISTENCE_DATA_PATH misconfigured to a non-writable location; running the container as a non-root user against a host-mounted volume owned by another user; read-only filesystem (disk full -> remounted read-only); Kubernetes PVC not mounted; corrupted path where a file occupies the directory name.

Understand the failure class

Background: mkdir permission denied (EACCES): failed to create directory errors explained — this error's family across 32 libraries.

Related errors

• create commit logger
• create commit log file
• browse commit logger directory
• restore hnsw index %q
• stat file: %w

AI-assisted analysis of weaviate/weaviate@75aa4b6d11 (2026-09-04). Data as JSON: /api/errors/a02a7b0e2298ec17. Report an issue: GitHub.