What causes "delimiter list returned nested object %s" in juicedata/juicefs?

List the offending key shown in the message and inspect its path under chunks/.

delimiter list returned nested object %s

Error message

delimiter list returned nested object %s

What it means

Invariant-violation error from walkGcChunkObjectPrefixes: when listing with a delimiter, any key that still contains '/' after stripping the prefix should be a common-prefix (directory) handled by the recursive branch. Hitting an object key that is 'nested' but wasn't returned as a prefix means the object-storage backend violated the delimiter listing contract or the key layout is unexpected.

Source

Thrown at cmd/gc.go:570

			if key == prefix {
				marker = key
				continue
			}
			if obj.IsDir() {
				if _, ok := seenPrefixes[key]; !ok {
					seenPrefixes[key] = struct{}{}
					if depth == 1 {
						if err := fn(key); err != nil {
							return err
						}
					} else {
						if err := walkGcChunkObjectPrefixes(ctx, blob, key, depth-1, false, fn); err != nil {
							return err
						}
					}
				}
			} else if strings.Contains(strings.TrimPrefix(key, prefix), "/") {
				return errors.Errorf("delimiter list returned nested object %s", key)
			}
			marker = key
		}
		if !hasMore {
			break
		}
		token = nextToken
	}
	return nil
}

func scanGcChunkObjectsPrefix(ctx context.Context, blob object.ObjectStorage, prefix string, handle func(object.Object) error) error {
	objs, err := object.ListAll(ctx, blob, prefix, "", true, false)
	if err != nil {
		return errors.Errorf("list chunk prefix %s: %s", prefix, err)
	}
	for obj := range objs {
		if obj == nil {

View on GitHub (pinned to c9a67b23e8)

Solutions

List the offending key shown in the message and inspect its path under chunks/.
Remove or rename manually placed/misnamed objects that contain unexpected slashes.
If using a non-AWS S3-compatible store, test its delimiter listing; use a spec-conformant backend or update it.
Upgrade JuiceFS client so chunk key layout matches what gc expects.
Report persistent occurrences with the backend name — it indicates a delimiter-conformance bug.

Example fix

// manual cleanup of misplaced object
s3cmd ls s3://bucket/chunks/ab/   # find unexpected nested key
s3cmd rm s3://bucket/chunks/ab/unexpected/nested/key

Defensive patterns

Strategy: validation

Validate before calling

# ensure chunks/ has only two-level layout: chunks/<hex>/<name>
aws s3 ls s3://$BUCKET/chunks/ --recursive | awk -F'chunks/' '{n=split($2,a,"/"); if(n!=2) print $2}'

Try / catch

if strings.Contains(out, "delimiter list returned nested object") {
    // inspect the reported key; clean misplaced objects; verify backend delimiter conformance
}

Prevention

Never manually add/rename objects under chunks/
Use spec-conformant S3 backends for delimiter listing
Keep JuiceFS clients up to date so key layout matches

When it happens

Trigger: An object storage backend that does not implement delimiter listing correctly (returns nested keys as objects instead of common prefixes), or objects under chunks/ whose names don't match the expected flat hashed-prefix layout (e.g. manually copied objects with extra slashes in the key).

Common situations: Using a custom or third-party S3-compatible store with non-conformant ListObjectsV2 delimiter behavior; operator copied/renamed objects inside chunks/ creating unexpected nesting; version mismatch where an old JuiceFS wrote a different key layout.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06). Data as JSON: /api/errors/dc071d998f83ffc5. Report an issue: GitHub.