{"record":{"id":"a02a7b0e2298ec17","repo":"weaviate/weaviate","slug":"create-commit-logger-directory","errorCode":null,"errorMessage":"create commit logger directory","messagePattern":"create commit logger directory","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"adapters/repos/db/vector/hnsw/commit_logger.go","lineNumber":86,"sourceCode":"\t\tid:                   name,\n\t\tlogger:               logger,\n\t\tfs:                   common.NewOSFS(),\n\t\tmaintenanceCallbacks: maintenanceCallbacks,\n\n\t\t// can be overwritten using functional options\n\t\tmaxSizeIndividual: defaultCommitLogSize / 5,\n\t}\n\n\tfor _, o := range opts {\n\t\tif err := o(l); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\t// Ensure the commit log directory exists\n\tdir := commitLogDirectory(rootPath, name)\n\tif err := l.fs.MkdirAll(dir, os.ModePerm); err != nil {\n\t\treturn nil, errors.Wrap(err, \"create commit logger directory\")\n\t}\n\n\t// Always start a fresh raw commit log file. Reusing an existing file is a\n\t// footgun: getCurrentCommitLogFileName used to select the append target\n\t// by highest parsed timestamp, which let it hand back a .snapshot /\n\t// .sorted / .condensed file and the next AddNode would corrupt it (block\n\t// CRCs become invalid on next SnapshotReader load). Creating a new file\n\t// every startup eliminates the class of bug — the append path can never\n\t// land on anything but a freshly created raw file that we own.\n\t//\n\t// Any existing zero-byte raw files from previous startups that the\n\t// compactor hasn't yet absorbed are pruned first so the directory never\n\t// accumulates more than one empty raw file at a time.\n\tfd, fileName, err := createNewCommitFile(rootPath, name, l.fs, l.logger)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n","sourceCodeStart":68,"sourceCodeEnd":104,"githubUrl":"https://github.com/weaviate/weaviate/blob/75aa4b6d11f8818305aafd4440b4e32794f7ca04/adapters/repos/db/vector/hnsw/commit_logger.go#L68-L104","documentation":"NewCommitLogger creates the per-index HNSW commit log directory (<rootPath>/<name>.hnsw.commitlog.d) via fs.MkdirAll before opening a fresh WAL file. This error wraps the OS error when that directory cannot be created, so the commit logger (and thus the shard's HNSW index) cannot be initialized.","triggerScenarios":"Calling NewCommitLogger(rootPath, name, ...) when MkdirAll on the commit-log directory fails: parent path does not exist and cannot be created, permission denied on rootPath, disk full, or rootPath is a file instead of a directory.","commonSituations":"PERSISTENCE_DATA_PATH misconfigured to a non-writable location; running the container as a non-root user against a host-mounted volume owned by another user; read-only filesystem (disk full -> remounted read-only); Kubernetes PVC not mounted; corrupted path where a file occupies the directory name.","solutions":["Check and fix permissions/ownership of PERSISTENCE_DATA_PATH so the Weaviate process user can write to it.","Verify the volume is actually mounted (Kubernetes PVC, docker -v) and not read-only; free disk space if full.","Ensure no regular file exists at <rootPath>/<name>.hnsw.commitlog.d; remove or rename it if so.","Correct the rootPath configuration and restart Weaviate."],"exampleFix":"// before: rootPath points to a read-only or nonexistent location\nNewCommitLogger(\"/var/lib/weaviate\", \"Collection__shard0\", logger, cbs)\n// after: ensure path exists and is writable before init\nif err := os.MkdirAll(persistencePath, 0o755); err != nil {\n    log.Fatalf(\"persistence path not writable: %v\", err)\n}\nNewCommitLogger(persistencePath, \"Collection__shard0\", logger, cbs)","handlingStrategy":"validation","validationCode":"// before starting Weaviate, verify the data path is writable\nimport \"os\"\nfunc assertWritableDataPath(root string) error {\n    if err := os.MkdirAll(root, 0o755); err != nil {\n        return fmt.Errorf(\"data path %q not creatable: %w\", root, err)\n    }\n    probe := filepath.Join(root, \".write-probe\")\n    if err := os.WriteFile(probe, []byte(\"ok\"), 0o644); err != nil {\n        return fmt.Errorf(\"data path %q not writable: %w\", root, err)\n    }\n    return os.Remove(probe)\n}","typeGuard":null,"tryCatchPattern":"logger, err := hnsw.NewCommitLogger(rootPath, name, log, cbs)\nif err != nil {\n    if strings.Contains(err.Error(), \"create commit logger directory\") {\n        // surface as misconfiguration: check PERSISTENCE_DATA_PATH perms/mount\n    }\n    return fmt.Errorf(\"shard init failed: %w\", err)\n}","preventionTips":["Validate PERSISTENCE_DATA_PATH is writable at container startup before opening shards.","Run the container with a UID matching the volume's owner.","Ensure Kubernetes PVCs are mounted read-write and sized with headroom.","Never place a file where a shard directory is expected."],"tags":["go","filesystem","hnsw","commit-log","startup"],"backgroundTag":"mkdir-permission-denied","analyzedSha":"75aa4b6d11f8818305aafd4440b4e32794f7ca04","analyzedAt":"2026-09-04T14:58:20.392Z","contentChangedAt":"2026-09-04T14:58:20.392Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}