juicedata/juicefs · error

list chunk prefix %s failed

Error message

list chunk prefix %s failed

What it means

Defensive error from scanGcChunkObjectsPrefix: after a successful ListAll, the results channel yielded a nil object. Since nil entries would crash or corrupt GC accounting, the function treats it as a listing failure and returns this error naming the prefix. Indicates a bug or race in the object-store list implementation rather than a user mistake.

Source

Thrown at cmd/gc.go:589

			}
			marker = key
		}
		if !hasMore {
			break
		}
		token = nextToken
	}
	return nil
}

func scanGcChunkObjectsPrefix(ctx context.Context, blob object.ObjectStorage, prefix string, handle func(object.Object) error) error {
	objs, err := object.ListAll(ctx, blob, prefix, "", true, false)
	if err != nil {
		return errors.Errorf("list chunk prefix %s: %s", prefix, err)
	}
	for obj := range objs {
		if obj == nil {
			return errors.Errorf("list chunk prefix %s failed", prefix)
		}
		if err := handle(obj); err != nil {
			return err
		}
	}
	return nil
}

func filterGcObject(ctx context.Context, blob object.ObjectStorage, obj object.Object, maxMtime time.Time, scanned *utils.Bar, skipped *utils.DoubleSpinner) (object.Object, bool) {
	if obj.IsDir() {
		return nil, false
	}
	if obj.Size() == 0 || obj.Mtime().Unix() == 0 {
		headObj, err := blob.Head(ctx, obj.Key())
		if err != nil {
			logger.Warnf("head %s: %s", obj.Key(), err)
			scanned.Increment()
			skipped.IncrInt64(obj.Size())

View on GitHub (pinned to c9a67b23e8)

Solutions

1. Note the backend type in the volume format and check for known driver bugs; upgrade JuiceFS to the latest release.
2. Retry the GC to rule out a transient shutdown race.
3. If reproducible with a specific backend, run with `--debug` and file an issue with the backend name.
4. Test the backend with a direct listing tool to confirm data integrity under chunks/.

Example fix

// upgrade client and retry
juicefs version
juicefs gc $META   # after upgrading to latest juicefs

// caller-side nil guard mirroring the library check
if obj == nil {
    return fmt.Errorf("nil object from listing")
}

if strings.Contains(out, "list chunk prefix") && strings.Contains(out, "failed") {
    // backend pushed nil object; upgrade juicefs / report backend bug
}

When it happens

Trigger: A storage backend's List implementation pushing a nil `object.Object` into the results channel (backend bug or shutdown race), seen while GC scans chunks/<xx>/.

Common situations: Third-party S3-compatible backends with buggy list code; object store connection torn down mid-iteration; JuiceFS version with a known object-storage driver bug.

Understand the failure class

Background: "invalid response format", "malformed payload", "missing data field": when an API returns 200 but the response shape is wrong — this error's family across 23 libraries.

Related errors

• list all blocks: %s
• list chunks from %s: %s
• delimiter list returned nested object %s
• list chunk prefix %s: %s
• produce object records: %s

AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06). Data as JSON: /api/errors/ddb1b60bae93e7f4. Report an issue: GitHub.