{"record":{"id":"2cb7d3f7a0d44914","repo":"thanos-io/thanos","slug":"sync","errorCode":null,"errorMessage":"sync","messagePattern":"sync","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/block/indexheader/binary_reader.go","lineNumber":162,"sourceCode":"\n\tif err := ir.CopyPostingsOffsets(bw.PostingOffsetsWriter(), buf); err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := bw.writer.Flush(); err != nil {\n\t\treturn nil, errors.Wrap(err, \"flush\")\n\t}\n\n\tif err := bw.WriteTOC(); err != nil {\n\t\treturn nil, errors.Wrap(err, \"write index header TOC\")\n\t}\n\n\tif err := bw.writer.Flush(); err != nil {\n\t\treturn nil, errors.Wrap(err, \"flush\")\n\t}\n\n\tif err := bw.writer.Sync(); err != nil {\n\t\treturn nil, errors.Wrap(err, \"sync\")\n\t}\n\n\tif tmpFilename != \"\" {\n\t\t// Create index-header in atomic way, to avoid partial writes (e.g during restart or crash of store GW).\n\t\treturn nil, os.Rename(tmpFilename, filename)\n\t}\n\n\treturn bw.Buffer(), nil\n}\n\ntype chunkedIndexReader struct {\n\tctx  context.Context\n\tpath string\n\tsize uint64\n\tbkt  objstore.BucketReader\n\ttoc  *index.TOC\n}\n","sourceCodeStart":144,"sourceCodeEnd":180,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/block/indexheader/binary_reader.go#L144-L180","documentation":"After flushing, WriteBinary calls bw.writer.Sync() (fsync) to durably persist the index-header bytes before the atomic rename into place. This error wraps an fsync failure: the OS could not flush dirty pages of the file to stable storage.","triggerScenarios":"WriteBinary -> bw.writer.Sync() returning an error such as EIO, ENOSPC, or EROFS from fsync on the temp index-header file, right before os.Rename(tmpFilename, filename).","commonSituations":"Underlying disk failure or full device surfaced only at fsync; Kubernetes nodes with failing disks; virtualized/cloud storage with intermittent EIO; read-only remount after filesystem error.","solutions":["Check dmesg for fsync/EIO errors on the device and run a filesystem check.","Verify free disk space (fsync can fail with ENOSPC when metadata needs flushing).","Confirm the volume is still mounted read-write.","Retry after storage recovers; the temp-file + rename design means no partial index-header was published, so retrying is safe."],"exampleFix":"// before\nif err := bw.writer.Sync(); err != nil {\n    return nil, errors.Wrap(err, \"sync\")\n}\n// after\nif err := bw.writer.Sync(); err != nil {\n    os.Remove(tmpFilename)\n    return nil, errors.Wrapf(err, \"fsync index-header %s (check disk health/space)\", tmpFilename)\n}","handlingStrategy":"try-catch","validationCode":"// Go: check device is writable and healthy before fsync-heavy batch work\nfunc deviceHealthy(dir string) error {\n    var st syscall.Statfs_t\n    if err := syscall.Statfs(dir, &st); err != nil { return err }\n    if st.Flags&syscall.ST_RDONLY != 0 { return errors.New(\"filesystem mounted read-only\") }\n    return nil\n}","typeGuard":null,"tryCatchPattern":"// Go\nif err := buildIndexHeader(); err != nil {\n    var se syscall.Errno\n    if errors.As(err, &se) && (se == syscall.EIO || se == syscall.ENOSPC || se == syscall.EROFS) {\n        log.Error(\"fsync failed; storage problem\", \"errno\", se)\n        // page node / alert, do not hot-retry\n        return err\n    }\n    return retryWriteBinary()\n}","preventionTips":["Run disk health monitoring (SMART, node_exporter) on nodes running Thanos components.","Alert on read-only filesystem remounts — fsync fails immediately on EROFS.","Treat fsync errors as node-level incidents, not transient retries.","Keep the atomic .tmp+rename flow intact so failed syncs never publish partial headers."],"tags":["io","fsync","durability","thanos"],"backgroundTag":"file-write-failed","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}