{"record":{"id":"112d9401ea319b43","repo":"juicedata/juicefs","slug":"list-all-blocks-s","errorCode":null,"errorMessage":"list all blocks: %s","messagePattern":"list all blocks: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cmd/gc.go","lineNumber":357,"sourceCode":"\t\tcsize, _ := strconv.Atoi(parts[2])\n\t\tif cobj {\n\t\t\tstats.addObject(gcStateTrash, obj.Size())\n\t\t} else if pobj {\n\t\t\tstats.addObject(gcStatePending, obj.Size())\n\t\t} else if isLeakedBlock(indx, csize, int(size), chunkConf.BlockSize) {\n\t\t\tif csize == chunkConf.BlockSize {\n\t\t\t\tlogger.Warnf(\"size of slice %d is larger than expected: %d > %d\", cid, indx*chunkConf.BlockSize+csize, size)\n\t\t\t} else {\n\t\t\t\tlogger.Warnf(\"size of slice %d is %d, but expect %d\", cid, indx*chunkConf.BlockSize+csize, size)\n\t\t\t}\n\t\t\tfoundLeaked(obj)\n\t\t} else {\n\t\t\tstats.addObject(gcStateUsed, obj.Size())\n\t\t}\n\t}\n\twaitLeakedObj()\n\tif err := <-listErr; err != nil {\n\t\treturn errors.Errorf(\"list all blocks: %s\", err)\n\t}\n\tstats.slices.Done()\n\treturn nil\n}\n\nconst (\n\tgcStateUsed    uint8 = 0\n\tgcStatePending uint8 = 1\n\tgcStateTrash   uint8 = 2\n)\n\ntype gcStats struct {\n\tprogress      *utils.Progress\n\tdeletedSlices *utils.Bar\n\tcleanedFiles  *utils.DoubleSpinner\n\tslices        *utils.Bar\n\tdelayedSlices *utils.DoubleSpinner\n\tcleanedSlices *utils.DoubleSpinner","sourceCodeStart":339,"sourceCodeEnd":375,"githubUrl":"https://github.com/juicedata/juicefs/blob/c9a67b23e8e08ec23ec331aa6f1675e2319e921c/cmd/gc.go#L339-L375","documentation":"Wrapped error in the in-memory GC path (`gcInMemory`) when the goroutine that lists all chunk objects (`scanGcChunkObjects` over the object store) reports an error on its `listErr` channel. It means the full enumeration of `chunks/` objects did not complete, so leaked-object detection is unreliable and GC aborts rather than deleting anything based on partial data.","triggerScenarios":"`juicefs gc` (without --ext-sort) listing objects from S3/minio when object.ListAll fails: expired/incorrect credentials, bucket removed, network timeout, listing pagination errors, or a wrapped 'list chunk prefix ...' error from scanGcChunkObjectsPrefix.","commonSituations":"AWS S3 credentials expired mid-run; MinIO test env container stopped; bucket policy denies ListObjects; large buckets causing listing timeouts; rate limiting (SlowDown) from the object store on very large buckets.","solutions":["Read the wrapped cause after 'list all blocks:' — it names the object store error (auth, 404, timeout).","Validate object storage credentials and bucket existence: `juicefs status META-URL` or a direct s3 listing.","Re-run with fewer threads (`--threads`) if the store is throttling or timing out.","Check bucket IAM/policy allows ListObjects/HeadObject.","For very large buckets, use the external-sort mode (`--ext-sort`) which streams prefixes instead of holding everything in memory."],"exampleFix":"// before\njuicefs gc s3://bucket\n// after: verify access then retry with backoff\naws s3 ls s3://bucket/chunks/ && juicefs gc s3://bucket","handlingStrategy":"validation","validationCode":"aws s3 ls s3://$BUCKET/chunks/ >/dev/null 2>&1 || { echo 'object store listing not permitted'; exit 1; }","typeGuard":null,"tryCatchPattern":"out, err := exec.Command(\"juicefs\", \"gc\", metaURL).CombinedOutput()\nif err != nil && strings.Contains(string(out), \"list all blocks:\") {\n    // surface embedded object-store error; fix credentials/bucket then retry\n}","preventionTips":["Rotate credentials before expiry; use instance profiles","Pre-check bucket list permission with a probe listing","Throttle --threads on large buckets","Use --ext-sort for very large volumes"],"tags":["go","object-storage","s3","gc","listing"],"backgroundTag":"http-error-response","analyzedSha":"c9a67b23e8e08ec23ec331aa6f1675e2319e921c","analyzedAt":"2026-09-06T17:55:48.476Z","contentChangedAt":"2026-09-06T17:55:48.476Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}