{"record":{"id":"c3a639447a9b56c7","repo":"quickwit-oss/quickwit","slug":"listed-object-is-not-under-storage-prefix","errorCode":null,"errorMessage":"listed object `{}` is not under storage prefix `{}`: {error}","messagePattern":"listed object `(.+?)` is not under storage prefix `(.+?)`: (.+?)","errorType":"exception","errorClass":"StorageError","httpStatus":null,"severity":"error","filePath":"quickwit/quickwit-storage/src/prefix_storage.rs","lineNumber":135,"sourceCode":"            if prefix == Path::new(\"\") {\n                return Ok(objects);\n            }\n            let prefix_bytes = prefix.as_os_str().as_encoded_bytes();\n            let mut relative_objects = Vec::with_capacity(objects.len());\n            for mut object in objects {\n                match object.path.strip_prefix(prefix) {\n                    Ok(relative_path) => {\n                        object.path = relative_path.to_path_buf();\n                        relative_objects.push(object);\n                    }\n                    Err(error) => {\n                        let is_under_prefix = object\n                            .path\n                            .as_os_str()\n                            .as_encoded_bytes()\n                            .starts_with(prefix_bytes);\n                        if !is_under_prefix {\n                            return Err(StorageErrorKind::Internal.with_error(anyhow::anyhow!(\n                                \"listed object `{}` is not under storage prefix `{}`: {error}\",\n                                object.path.display(),\n                                prefix.display()\n                            )));\n                        }\n                    }\n                }\n            }\n            Ok(relative_objects)\n        }\n\n        let storage_prefix = self.prefix.clone();\n        self.storage\n            .list(&self.prefix.join(prefix))\n            .map(move |objects_res| {\n                let objects = objects_res?;\n                strip_prefix_from_objects(objects, &storage_prefix)\n            })","sourceCodeStart":117,"sourceCodeEnd":153,"githubUrl":"https://github.com/quickwit-oss/quickwit/blob/a39730c5cdcd1a4fe798403737ae293999ea21f8/quickwit/quickwit-storage/src/prefix_storage.rs#L117-L153","documentation":"`PrefixStorage` wraps another storage and guarantees every listed object lies under its configured prefix. During `strip_prefix_from_objects`, if an object's path returned by the inner storage does not start with the expected prefix bytes, this Internal invariant-violation error is raised (the {error} carries the underlying strip failure).","triggerScenarios":"Calling `list` on a PrefixStorage whose inner storage returns objects outside the prefix — e.g. after the prefix/root URI changed while the underlying bucket still holds old keys, or the inner storage ignores the prefix filter.","commonSituations":"Changing `storageUri`/root or per-index prefix config so stale objects no longer match; a buggy S3-compatible backend that does not honor the `prefix` list parameter; mixing prefixes that overlap (e.g. `idx` vs `index`).","solutions":["Verify the configured prefix matches the actual key layout in the bucket and that no config change broke it","List the bucket at the prefix directly and find the offending object(s) outside it; move or delete them","Update quickwit config so the index root URI matches where the data actually lives","If a backend ignores the prefix parameter, fix/replace the S3-compatible implementation"],"exampleFix":"// before: root changed from /indexes to /data but old keys remain at /indexes/...\nroot: s3://bucket/data\n// after: migrate keys to match config, or point root back\nroot: s3://bucket/indexes","handlingStrategy":"validation","validationCode":"let root_prefix = index_root_uri_path();\nfor key in bucket_keys_under(root_prefix) {\n    assert!(key.starts_with(root_prefix), \"key {key} escapes prefix {root_prefix}\");\n}","typeGuard":null,"tryCatchPattern":"match storage.list(&prefix).try_collect::<Vec<_>>().await {\n    Ok(objs) => use(objs),\n    Err(e) if e.kind() == StorageErrorKind::Internal => check_prefix_config_and_bucket_layout(),\n    Err(e) => return Err(e.into()),\n}","preventionTips":["Keep index root URIs and physical bucket key layout in sync when migrating data","Avoid overlapping/ambiguous prefixes like `idx` and `index` in the same bucket","Copy keys to the new prefix before changing config during migrations","Verify any S3-compatible backend honors the `prefix` list parameter"],"tags":["storage","prefix","listing","invariant"],"backgroundTag":"internal-invariant-violation","analyzedSha":"a39730c5cdcd1a4fe798403737ae293999ea21f8","analyzedAt":"2026-09-08T13:19:37.784Z","contentChangedAt":"2026-09-08T13:19:37.784Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}