What causes "list all blocks: %s" in juicedata/juicefs?

Read the wrapped cause after 'list all blocks:' — it names the object store error (auth, 404, timeout).

list all blocks: %s

Error message

list all blocks: %s

What it means

Wrapped error in the in-memory GC path (`gcInMemory`) when the goroutine that lists all chunk objects (`scanGcChunkObjects` over the object store) reports an error on its `listErr` channel. It means the full enumeration of `chunks/` objects did not complete, so leaked-object detection is unreliable and GC aborts rather than deleting anything based on partial data.

Source

Thrown at cmd/gc.go:357

		csize, _ := strconv.Atoi(parts[2])
		if cobj {
			stats.addObject(gcStateTrash, obj.Size())
		} else if pobj {
			stats.addObject(gcStatePending, obj.Size())
		} else if isLeakedBlock(indx, csize, int(size), chunkConf.BlockSize) {
			if csize == chunkConf.BlockSize {
				logger.Warnf("size of slice %d is larger than expected: %d > %d", cid, indx*chunkConf.BlockSize+csize, size)
			} else {
				logger.Warnf("size of slice %d is %d, but expect %d", cid, indx*chunkConf.BlockSize+csize, size)
			}
			foundLeaked(obj)
		} else {
			stats.addObject(gcStateUsed, obj.Size())
		}
	}
	waitLeakedObj()
	if err := <-listErr; err != nil {
		return errors.Errorf("list all blocks: %s", err)
	}
	stats.slices.Done()
	return nil
}

const (
	gcStateUsed    uint8 = 0
	gcStatePending uint8 = 1
	gcStateTrash   uint8 = 2
)

type gcStats struct {
	progress      *utils.Progress
	deletedSlices *utils.Bar
	cleanedFiles  *utils.DoubleSpinner
	slices        *utils.Bar
	delayedSlices *utils.DoubleSpinner
	cleanedSlices *utils.DoubleSpinner

View on GitHub (pinned to c9a67b23e8)

Solutions

Read the wrapped cause after 'list all blocks:' — it names the object store error (auth, 404, timeout).
Validate object storage credentials and bucket existence: `juicefs status META-URL` or a direct s3 listing.
Re-run with fewer threads (`--threads`) if the store is throttling or timing out.
Check bucket IAM/policy allows ListObjects/HeadObject.
For very large buckets, use the external-sort mode (`--ext-sort`) which streams prefixes instead of holding everything in memory.

Example fix

// before
juicefs gc s3://bucket
// after: verify access then retry with backoff
aws s3 ls s3://bucket/chunks/ && juicefs gc s3://bucket

Defensive patterns

Strategy: validation

Validate before calling

aws s3 ls s3://$BUCKET/chunks/ >/dev/null 2>&1 || { echo 'object store listing not permitted'; exit 1; }

Try / catch

out, err := exec.Command("juicefs", "gc", metaURL).CombinedOutput()
if err != nil && strings.Contains(string(out), "list all blocks:") {
    // surface embedded object-store error; fix credentials/bucket then retry
}

Prevention

Rotate credentials before expiry; use instance profiles
Pre-check bucket list permission with a probe listing
Throttle --threads on large buckets
Use --ext-sort for very large volumes

When it happens

Trigger: `juicefs gc` (without --ext-sort) listing objects from S3/minio when object.ListAll fails: expired/incorrect credentials, bucket removed, network timeout, listing pagination errors, or a wrapped 'list chunk prefix ...' error from scanGcChunkObjectsPrefix.

Common situations: AWS S3 credentials expired mid-run; MinIO test env container stopped; bucket policy denies ListObjects; large buckets causing listing timeouts; rate limiting (SlowDown) from the object store on very large buckets.

Understand the failure class

Background: "API error: {status}" and "HTTP 401/403/404/429/5xx" errors: non-2xx HTTP responses explained — this error's family across 27 libraries.

Related errors

AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06). Data as JSON: /api/errors/112d9401ea319b43. Report an issue: GitHub.