juicedata/juicefs · error

list chunk prefix %s: %s

Error message

list chunk prefix %s: %s

What it means

Wrapped error from scanGcChunkObjectsPrefix when object.ListAll (a full paginated listing of one chunk prefix) fails. The prefix and original store error are included. This makes the GC scan for that prefix incomplete, so the error is propagated to abort GC rather than risk misclassifying leaked objects.

Source

Thrown at cmd/gc.go:585

					}
				}
			} else if strings.Contains(strings.TrimPrefix(key, prefix), "/") {
				return errors.Errorf("delimiter list returned nested object %s", key)
			}
			marker = key
		}
		if !hasMore {
			break
		}
		token = nextToken
	}
	return nil
}

func scanGcChunkObjectsPrefix(ctx context.Context, blob object.ObjectStorage, prefix string, handle func(object.Object) error) error {
	objs, err := object.ListAll(ctx, blob, prefix, "", true, false)
	if err != nil {
		return errors.Errorf("list chunk prefix %s: %s", prefix, err)
	}
	for obj := range objs {
		if obj == nil {
			return errors.Errorf("list chunk prefix %s failed", prefix)
		}
		if err := handle(obj); err != nil {
			return err
		}
	}
	return nil
}

func filterGcObject(ctx context.Context, blob object.ObjectStorage, obj object.Object, maxMtime time.Time, scanned *utils.Bar, skipped *utils.DoubleSpinner) (object.Object, bool) {
	if obj.IsDir() {
		return nil, false
	}
	if obj.Size() == 0 || obj.Mtime().Unix() == 0 {
		headObj, err := blob.Head(ctx, obj.Key())

View on GitHub (pinned to c9a67b23e8)

Solutions

  1. Check the underlying error after 'list chunk prefix:' — fix auth/endpoint/bucket accordingly.
  2. Verify credentials with a plain list: `aws s3 ls s3://bucket/chunks/` or `mc ls`.
  3. Retry with fewer parallel listings (`--threads`) to avoid throttling.
  4. Fix network path (proxy/firewall/VPN) if timeouts recur.
  5. Confirm the volume's storage config with `juicefs status` if the endpoint changed.

Example fix

// before: failing due to stale keys
juicefs gc $META
// after
export AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=...
juicefs gc $META
Defensive patterns

Strategy: retry

Validate before calling

aws s3 ls s3://$BUCKET/chunks/ --max-items 1 >/dev/null 2>&1 || { echo 'listing failed: check creds/endpoint'; exit 1; }

Try / catch

if m := regexp.MustCompile(`list chunk prefix ([0-9a-f]{2}): (.*)`).FindStringSubmatch(out); m != nil {
    // m[1] = failing prefix, m[2] = store error; handle by cause
}

Prevention

When it happens

Trigger: `juicefs gc` enumerating chunks/<xx>/ when ListObjects against the bucket fails: auth failure, bucket missing, throttling, network timeout, or unsupported listing options by the backend (ListAll wraps backend errors).

Common situations: Expired cloud credentials; minio container down; S3 request-rate throttling on huge prefixes; proxy/firewall dropping long listings; misconfigured storage endpoint in the volume format.

Understand the failure class

Background: "API request failed": what wrapped HTTP errors from external APIs mean and how to find the real cause — this error's family across 29 libraries.

Related errors

AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06). Data as JSON: /api/errors/d9bcaebb0a389bd9. Report an issue: GitHub.