{"record":{"id":"b9a748d834db94b6","repo":"thanos-io/thanos","slug":"read-chunk-dir","errorCode":null,"errorMessage":"read chunk dir","messagePattern":"read chunk dir","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/shipper/shipper.go","lineNumber":588,"sourceCode":"\t\treturn metas[i].MinTime < metas[j].MinTime\n\t})\n\n\tif len(failedBlocks) > 0 {\n\t\terr = ErrorSyncBlockCorrupted\n\t}\n\treturn metas, failedBlocks, err\n}\n\nfunc hardlinkBlock(src, dst string) error {\n\tchunkDir := filepath.Join(dst, block.ChunksDirname)\n\n\tif err := os.MkdirAll(chunkDir, 0750); err != nil {\n\t\treturn errors.Wrap(err, \"create chunks dir\")\n\t}\n\n\tfis, err := os.ReadDir(filepath.Join(src, block.ChunksDirname))\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"read chunk dir\")\n\t}\n\tfiles := make([]string, 0, len(fis))\n\tfor _, fi := range fis {\n\t\tfiles = append(files, fi.Name())\n\t}\n\tfor i, fn := range files {\n\t\tfiles[i] = filepath.Join(block.ChunksDirname, fn)\n\t}\n\tfiles = append(files, block.MetaFilename, block.IndexFilename)\n\n\tfor _, fn := range files {\n\t\tif err := os.Link(filepath.Join(src, fn), filepath.Join(dst, fn)); err != nil {\n\t\t\treturn errors.Wrapf(err, \"hard link file %s\", fn)\n\t\t}\n\t}\n\treturn nil\n}\n","sourceCodeStart":570,"sourceCodeEnd":606,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/shipper/shipper.go#L570-L606","documentation":"hardlinkBlock reads the source block's chunks directory (os.ReadDir on <src>/chunks) to enumerate the chunk files to hardlink into the staging area. Failure here means the source block directory is missing its chunks subdirectory or is unreadable, so the block cannot be staged for upload and the error is wrapped as \"read chunk dir\".","triggerScenarios":"Sync()/upload() staging a block whose <block-dir>/chunks directory does not exist, was deleted, or cannot be read due to permissions or I/O error.","commonSituations":"A partially written or manually truncated block (missing chunks dir); external tooling (cleanup scripts, compaction) removing files while the shipper runs; wrong permissions after copying blocks between hosts; NFS stale handles.","solutions":["Verify the source block directory contains a readable chunks/ subdirectory (ls <block>/chunks).","Remove the corrupted/incomplete block from the data dir so the source re-materializes it, or restore it from a healthy replica.","Check for cleanup jobs or concurrent compaction deleting chunk files while the shipper uploads; pause them or exclude the shipper's dir.","Check mount health and permissions on the data directory if ReadDir fails with EACCES/ESTALE."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"chunksDir := filepath.Join(blockDir, block.ChunksDirname)\nif fi, err := os.Stat(chunksDir); err != nil || !fi.IsDir() {\n\treturn fmt.Errorf(\"block %s missing chunks dir\", blockDir)\n}","typeGuard":"func hasChunksDir(blockDir string) bool {\n\tfi, err := os.Stat(filepath.Join(blockDir, \"chunks\"))\n\treturn err == nil && fi.IsDir()\n}","tryCatchPattern":"if err := sh.Sync(ctx); err != nil {\n\tif strings.Contains(err.Error(), \"read chunk dir\") {\n\t\tlevel.Warn(logger).Log(\"msg\", \"block missing chunks, removing\", \"err\", err)\n\t}\n\treturn err\n}","preventionTips":["Never run external cleanup scripts against the live TSDB block directory.","Verify block integrity (chunks dir present) before pointing a shipper at copied blocks.","Avoid NFS for TSDB data; stale handles cause ReadDir failures.","Alert on blocks with missing or empty chunks/ directories."],"tags":["filesystem","corrupt-data","chunks","upload"],"backgroundTag":"directory-not-found","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}