{"record":{"id":"9a0ddea357b93d4a","repo":"thanos-io/thanos","slug":"create-chunks-dir","errorCode":null,"errorMessage":"create chunks dir","messagePattern":"create chunks dir","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/shipper/shipper.go","lineNumber":583,"sourceCode":"\t\t\treturn nil, nil, errors.Wrapf(err, \"read metadata for block %v\", dir)\n\t\t}\n\t\tmetas = append(metas, m)\n\t}\n\tsort.Slice(metas, func(i, j int) bool {\n\t\treturn metas[i].MinTime < metas[j].MinTime\n\t})\n\n\tif len(failedBlocks) > 0 {\n\t\terr = ErrorSyncBlockCorrupted\n\t}\n\treturn metas, failedBlocks, err\n}\n\nfunc hardlinkBlock(src, dst string) error {\n\tchunkDir := filepath.Join(dst, block.ChunksDirname)\n\n\tif err := os.MkdirAll(chunkDir, 0750); err != nil {\n\t\treturn errors.Wrap(err, \"create chunks dir\")\n\t}\n\n\tfis, err := os.ReadDir(filepath.Join(src, block.ChunksDirname))\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"read chunk dir\")\n\t}\n\tfiles := make([]string, 0, len(fis))\n\tfor _, fi := range fis {\n\t\tfiles = append(files, fi.Name())\n\t}\n\tfor i, fn := range files {\n\t\tfiles[i] = filepath.Join(block.ChunksDirname, fn)\n\t}\n\tfiles = append(files, block.MetaFilename, block.IndexFilename)\n\n\tfor _, fn := range files {\n\t\tif err := os.Link(filepath.Join(src, fn), filepath.Join(dst, fn)); err != nil {\n\t\t\treturn errors.Wrapf(err, \"hard link file %s\", fn)","sourceCodeStart":565,"sourceCodeEnd":601,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/shipper/shipper.go#L565-L601","documentation":"During upload, the Shipper stages each block via hardlinkBlock, which first creates the chunks directory (<staging dst>/chunks) with os.MkdirAll. If that mkdir fails, the upload of the block cannot proceed and the error is wrapped with \"create chunks dir\". This is almost always an OS-level problem: permissions, a full disk, or the destination path not being writable.","triggerScenarios":"Calling upload() (via Sync) when os.MkdirAll(<dst>/chunks, 0750) fails because the staging/destination parent dir is unwritable, the path is invalid, or the filesystem is full/read-only.","commonSituations":"Data directory volume mounted read-only (e.g. k8s readOnlyRootFilesystem); disk full during large compaction uploads; wrong ownership/permissions on the local data dir after running the container as a different UID; SELinux/AppArmor blocking writes.","solutions":["Check permissions and ownership of the shipper's local data directory and ensure the process UID can create directories there (0750 needs write on parent).","Verify the volume backing the data dir is writable and not full (df -h; mount | grep <path>).","In Kubernetes, add an emptyDir/PVC for the data dir and drop readOnlyRootFilesystem or add writableVolumeMounts.","Retry after fixing; if mkdir fails transiently due to NFS, remount the share."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"func canCreateDir(parent string) error {\n\tfi, err := os.Stat(parent)\n\tif err != nil {\n\t\treturn err\n\t}\n\tif !fi.IsDir() {\n\t\treturn fmt.Errorf(\"%s is not a directory\", parent)\n\t}\n\treturn os.Access(parent, os.O_RDWR)\n}","typeGuard":null,"tryCatchPattern":"if err := sh.Sync(ctx); err != nil {\n\tif strings.Contains(err.Error(), \"create chunks dir\") {\n\t\tlevel.Error(logger).Log(\"msg\", \"staging dir unwritable\", \"err\", err)\n\t}\n\treturn err\n}","preventionTips":["Pre-create the data dir with correct ownership before starting the process.","Set disk-full alerts on the volume backing the shipper data dir.","Avoid read-only root filesystems without a writable mount for the data dir.","Pin a dedicated UID/GID for the shipper across deployments."],"tags":["filesystem","permissions","disk","upload"],"backgroundTag":"mkdir-permission-denied","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}