{"record":{"id":"dc071d998f83ffc5","repo":"juicedata/juicefs","slug":"delimiter-list-returned-nested-object-s","errorCode":null,"errorMessage":"delimiter list returned nested object %s","messagePattern":"delimiter list returned nested object (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cmd/gc.go","lineNumber":570,"sourceCode":"\t\t\tif key == prefix {\n\t\t\t\tmarker = key\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif obj.IsDir() {\n\t\t\t\tif _, ok := seenPrefixes[key]; !ok {\n\t\t\t\t\tseenPrefixes[key] = struct{}{}\n\t\t\t\t\tif depth == 1 {\n\t\t\t\t\t\tif err := fn(key); err != nil {\n\t\t\t\t\t\t\treturn err\n\t\t\t\t\t\t}\n\t\t\t\t\t} else {\n\t\t\t\t\t\tif err := walkGcChunkObjectPrefixes(ctx, blob, key, depth-1, false, fn); err != nil {\n\t\t\t\t\t\t\treturn err\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t} else if strings.Contains(strings.TrimPrefix(key, prefix), \"/\") {\n\t\t\t\treturn errors.Errorf(\"delimiter list returned nested object %s\", key)\n\t\t\t}\n\t\t\tmarker = key\n\t\t}\n\t\tif !hasMore {\n\t\t\tbreak\n\t\t}\n\t\ttoken = nextToken\n\t}\n\treturn nil\n}\n\nfunc scanGcChunkObjectsPrefix(ctx context.Context, blob object.ObjectStorage, prefix string, handle func(object.Object) error) error {\n\tobjs, err := object.ListAll(ctx, blob, prefix, \"\", true, false)\n\tif err != nil {\n\t\treturn errors.Errorf(\"list chunk prefix %s: %s\", prefix, err)\n\t}\n\tfor obj := range objs {\n\t\tif obj == nil {","sourceCodeStart":552,"sourceCodeEnd":588,"githubUrl":"https://github.com/juicedata/juicefs/blob/c9a67b23e8e08ec23ec331aa6f1675e2319e921c/cmd/gc.go#L552-L588","documentation":"Invariant-violation error from walkGcChunkObjectPrefixes: when listing with a delimiter, any key that still contains '/' after stripping the prefix should be a common-prefix (directory) handled by the recursive branch. Hitting an object key that is 'nested' but wasn't returned as a prefix means the object-storage backend violated the delimiter listing contract or the key layout is unexpected.","triggerScenarios":"An object storage backend that does not implement delimiter listing correctly (returns nested keys as objects instead of common prefixes), or objects under chunks/ whose names don't match the expected flat hashed-prefix layout (e.g. manually copied objects with extra slashes in the key).","commonSituations":"Using a custom or third-party S3-compatible store with non-conformant ListObjectsV2 delimiter behavior; operator copied/renamed objects inside chunks/ creating unexpected nesting; version mismatch where an old JuiceFS wrote a different key layout.","solutions":["List the offending key shown in the message and inspect its path under chunks/.","Remove or rename manually placed/misnamed objects that contain unexpected slashes.","If using a non-AWS S3-compatible store, test its delimiter listing; use a spec-conformant backend or update it.","Upgrade JuiceFS client so chunk key layout matches what gc expects.","Report persistent occurrences with the backend name — it indicates a delimiter-conformance bug."],"exampleFix":"// manual cleanup of misplaced object\ns3cmd ls s3://bucket/chunks/ab/   # find unexpected nested key\ns3cmd rm s3://bucket/chunks/ab/unexpected/nested/key","handlingStrategy":"validation","validationCode":"# ensure chunks/ has only two-level layout: chunks/<hex>/<name>\naws s3 ls s3://$BUCKET/chunks/ --recursive | awk -F'chunks/' '{n=split($2,a,\"/\"); if(n!=2) print $2}'","typeGuard":null,"tryCatchPattern":"if strings.Contains(out, \"delimiter list returned nested object\") {\n    // inspect the reported key; clean misplaced objects; verify backend delimiter conformance\n}","preventionTips":["Never manually add/rename objects under chunks/","Use spec-conformant S3 backends for delimiter listing","Keep JuiceFS clients up to date so key layout matches"],"tags":["go","object-storage","invariant-violation","delimiter-listing","gc"],"backgroundTag":"internal-invariant-violation","analyzedSha":"c9a67b23e8e08ec23ec331aa6f1675e2319e921c","analyzedAt":"2026-09-06T17:55:48.476Z","contentChangedAt":"2026-09-06T17:55:48.476Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}