{"record":{"id":"9fec425625c0aebd","repo":"weaviate/weaviate","slug":"parse-header-w","errorCode":null,"errorMessage":"parse header: %w","messagePattern":"parse header: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"adapters/repos/db/lsmkv/segment.go","lineNumber":316,"sourceCode":"\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"mmap file: %w\", err)\n\t\t}\n\t\tcontents = contents2\n\t\tunMapContents = true\n\t} else { // read the file into memory if it's small enough and we have enough memory\n\t\tmeteredF := diskio.NewMeteredReader(file, diskio.MeteredReaderCallback(metrics.ReadObserver(\"readSegmentFile\")))\n\t\tbufio.NewReader(meteredF)\n\t\tcontents, err = io.ReadAll(meteredF)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"read file: %w\", err)\n\t\t}\n\t\tunMapContents = false\n\t\treadFromMemory = true\n\t\tuseBloomFilter = false\n\t}\n\theader, err := segmentindex.ParseHeader(contents[:segmentindex.HeaderSize])\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"parse header: %w\", err)\n\t}\n\n\tif err := segmentindex.CheckExpectedStrategy(header.Strategy); err != nil {\n\t\treturn nil, fmt.Errorf(\"unsupported strategy in segment: %w\", err)\n\t}\n\n\tif header.Version >= segmentindex.SegmentV1 && cfg.enableChecksumValidation {\n\t\tfile.Seek(0, io.SeekStart)\n\t\theaderSize := int64(segmentindex.HeaderSize)\n\t\tif header.Strategy == segmentindex.StrategyInverted {\n\t\t\theaderSize += int64(segmentindex.HeaderInvertedSize)\n\t\t}\n\t\tsegmentFile := segmentindex.NewSegmentFile(segmentindex.WithReader(file))\n\t\tif err := segmentFile.ValidateChecksum(size, headerSize); err != nil {\n\t\t\treturn nil, fmt.Errorf(\"validate segment %q: %w\", path, err)\n\t\t}\n\t}\n","sourceCodeStart":298,"sourceCodeEnd":334,"githubUrl":"https://github.com/weaviate/weaviate/blob/75aa4b6d11f8818305aafd4440b4e32794f7ca04/adapters/repos/db/lsmkv/segment.go#L298-L334","documentation":"Thrown in newSegment (segment.go:316) when segmentindex.ParseHeader fails on the first HeaderSize bytes of the segment contents. ParseHeader validates the magic/version framing, so failure means the bytes do not form a valid LSM segment header.","triggerScenarios":"Opening a segment whose first bytes are not a valid header: truncated/empty file (contents shorter than segmentindex.HeaderSize), file written by an incompatible or corrupted write path, bitrot flipping magic/version bytes, or pointing Weaviate at a file that isn't a segment.","commonSituations":"Disk corruption or partial write after a crash leaving a zero-length or garbage segment; manually copying segment files between nodes/versions with mismatched segment formats; restoring a backup into the wrong location; storage layer silently returning zeros (misconfigured volume).","solutions":["Check the file size and first bytes (hexdump) — if empty or garbage, the segment is corrupt; remove it and restore the shard from backup.","Verify the segment was produced by a compatible Weaviate version; don't hand-copy segment files across versions or clusters.","Check disk health (SMART, filesystem fsck) — bitrot is the usual root cause of magic-byte mismatch.","If only one shard/segment is affected, drop the shard and rebuild from replicas or re-ingest the data.","Enable checksum validation (enableChecksumValidation) so corrupt segments are caught with clearer errors at load time."],"exampleFix":"// before: hand-copied segment from an old version into the shard dir\ncp ~/old-node/segment-7 /var/lib/weaviate/.../shard/segment-7\n// after: restore the shard properly from a backup\nweaviate backup restore --backend filesystem --id backup-1 && restart weaviate","handlingStrategy":"validation","validationCode":"func looksLikeSegmentFile(path string) error {\n    f, err := os.Open(path)\n    if err != nil {\n        return err\n    }\n    defer f.Close()\n    buf := make([]byte, segmentindex.HeaderSize)\n    n, err := io.ReadFull(f, buf)\n    if err != nil {\n        return fmt.Errorf(\"segment %q too small for header (%d bytes): %w\", path, n, err)\n    }\n    return segmentindex.ParseHeader(buf)\n}","typeGuard":"func isValidSegmentHeader(b []byte) bool {\n    if len(b) < segmentindex.HeaderSize {\n        return false\n    }\n    return segmentindex.ParseHeader(b[:segmentindex.HeaderSize]) == nil\n}","tryCatchPattern":"header, err := segmentindex.ParseHeader(contents[:segmentindex.HeaderSize])\nif err != nil {\n    return nil, fmt.Errorf(\"segment %q has corrupt/foreign header — restore from backup: %w\", path, err)\n}","preventionTips":["Only use Weaviate's own backup/restore tooling to move segment files; never hand-copy between versions or nodes.","Enable filesystem checksumming (ZFS/btrfs) or periodic fsck to catch bitrot.","Enable enableChecksumValidation so segment corruption is detected with checksum detail.","Verify restored backup directories contain complete, unmixed bucket files before startup."],"tags":["go","lsmkv","corruption","parse","segment"],"backgroundTag":"corrupt-file-header","analyzedSha":"75aa4b6d11f8818305aafd4440b4e32794f7ca04","analyzedAt":"2026-09-04T14:58:20.392Z","contentChangedAt":"2026-09-04T14:58:20.392Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}