{"record":{"id":"3485b3d489c99ca8","repo":"weaviate/weaviate","slug":"create-commit-logger","errorCode":null,"errorMessage":"create commit logger","messagePattern":"create commit logger","errorType":"console","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"adapters/repos/db/vector/hnsw/startup.go","lineNumber":47,"sourceCode":"\t\"github.com/weaviate/weaviate/entities/storobj\"\n\tent \"github.com/weaviate/weaviate/entities/vectorindex/hnsw\"\n)\n\nfunc (h *hnsw) init(cfg Config) error {\n\th.pools = newPools(h.maximumConnectionsLayerZero, h.visitedListPoolMaxSize)\n\n\t// Restore from disk. This loads existing snapshot/sorted/raw files into\n\t// in-memory state.\n\tif err := h.restoreFromDisk(); err != nil {\n\t\treturn errors.Wrapf(err, \"restore hnsw index %q\", cfg.ID)\n\t}\n\n\t// Create commit logger for future writes. The logger unconditionally\n\t// creates a new raw file and never appends to an existing one — see\n\t// createNewCommitFile's comment for why.\n\tcl, err := cfg.MakeCommitLoggerThunk()\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"create commit logger\")\n\t}\n\th.commitLog = cl\n\n\t// report the vector_index_size at server startup.\n\t// otherwise on server restart, prometheus reports\n\t// a vector_index_size of 0 until more vectors are\n\t// added.\n\th.metrics.SetSize(len(h.nodes))\n\n\treturn nil\n}\n\n// restoreFromDisk loads the HNSW state from commit log files using compact.Loader.\n// A truncated/corrupt WAL file is logged for diagnostic purposes but does not\n// change the commit logger's behavior: the commit logger always starts a new\n// raw file anyway, so there is no \"append to the corrupted file\" path to avoid.\nfunc (h *hnsw) restoreFromDisk() error {\n\tbeforeAll := time.Now()","sourceCodeStart":29,"sourceCodeEnd":65,"githubUrl":"https://github.com/weaviate/weaviate/blob/75aa4b6d11f8818305aafd4440b4e32794f7ca04/adapters/repos/db/vector/hnsw/startup.go#L29-L65","documentation":"init() wraps MakeCommitLoggerThunk failures as \"create commit logger\". After restoring state, the commit logger must create a new raw commit file (it never appends to an existing one). Failure here means the filesystem would not allow a new commit log to be created/opened.","triggerScenarios":"MakeCommitLoggerThunk fails creating/opening the new raw commit file — read-only filesystem, no write permission on the shard directory, disk full, or too many open files (ulimit).","commonSituations":"Container running with read-only rootfs and no writable volume mount; wrong user/permissions after restoring backups as root; ENOSPC on the data disk; fd exhaustion at startup with many shards.","solutions":["Make the data directory writable by the weaviate process user (chown/chmod or fix the volume mount)","Free disk space if ENOSPC","Raise the open-file limit (ulimit -n / container limits) when many shards open files at once","Ensure the configured PERSISTENCE_DATA_PATH points to a writable persistent volume, not a read-only overlay"],"exampleFix":"// before: read-only mount in docker-compose\nvolumes: []\n// after\nvolumes:\n  - weaviate_data:/var/lib/weaviate\nPERSISTENCE_DATA_PATH=/var/lib/weaviate","handlingStrategy":"validation","validationCode":"// pre-start: verify the data path is writable\npath := os.Getenv(\"PERSISTENCE_DATA_PATH\")\nif err := os.MkdirAll(path, 0o755); err != nil { log.Fatalf(\"data path not writable: %v\", err) }\nif f, err := os.CreateTemp(path, \"wtest\"); err != nil { log.Fatalf(\"no write access: %v\", err) } else { f.Close(); os.Remove(f.Name()) }","typeGuard":null,"tryCatchPattern":"cl, err := cfg.MakeCommitLoggerThunk()\nif err != nil {\n  h.logger.Errorf(\"commit logger init failed: %v\", err)\n  return errors.Wrap(err, \"create commit logger\")\n}","preventionTips":["Mount a writable persistent volume at PERSISTENCE_DATA_PATH","Raise nofile ulimit for many-shard deployments","Monitor disk usage; alert before ENOSPC","Run the container as the same user that owns the data dir","Smoke-test writability in the container entrypoint before starting the server"],"tags":["go","hnsw","startup","commit-log","filesystem"],"backgroundTag":"commit-log-create-failed","analyzedSha":"75aa4b6d11f8818305aafd4440b4e32794f7ca04","analyzedAt":"2026-09-04T14:58:20.392Z","contentChangedAt":"2026-09-04T14:58:20.392Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}