quickwit-oss/quickwit · error · StorageError

listed object ` ` is not under storage prefix

Error message

listed object `{}` is not under storage prefix `{}`: {error}

What it means

`PrefixStorage` wraps another storage and guarantees every listed object lies under its configured prefix. During `strip_prefix_from_objects`, if an object's path returned by the inner storage does not start with the expected prefix bytes, this Internal invariant-violation error is raised (the {error} carries the underlying strip failure).

Solutions

1. Verify the configured prefix matches the actual key layout in the bucket and that no config change broke it
2. List the bucket at the prefix directly and find the offending object(s) outside it; move or delete them
3. Update quickwit config so the index root URI matches where the data actually lives
4. If a backend ignores the prefix parameter, fix/replace the S3-compatible implementation

Example fix

// before: root changed from /indexes to /data but old keys remain at /indexes/...
root: s3://bucket/data
// after: migrate keys to match config, or point root back
root: s3://bucket/indexes

let root_prefix = index_root_uri_path();
for key in bucket_keys_under(root_prefix) {
    assert!(key.starts_with(root_prefix), "key {key} escapes prefix {root_prefix}");
}

match storage.list(&prefix).try_collect::<Vec<_>>().await {
    Ok(objs) => use(objs),
    Err(e) if e.kind() == StorageErrorKind::Internal => check_prefix_config_and_bucket_layout(),
    Err(e) => return Err(e.into()),
}

When it happens

Trigger: Calling `list` on a PrefixStorage whose inner storage returns objects outside the prefix — e.g. after the prefix/root URI changed while the underlying bucket still holds old keys, or the inner storage ignores the prefix filter.

Common situations: Changing `storageUri`/root or per-index prefix config so stale objects no longer match; a buggy S3-compatible backend that does not honor the `prefix` list parameter; mixing prefixes that overlap (e.g. `idx` vs `index`).

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• failed to URL-decode listed object key
• listing objects is not supported for storage
• storage factory and config backends should match
• `append_records` should be called with `position_opt: None`
• attempted to merge splits with different doc mapping uid

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/c3a639447a9b56c7. Report an issue: GitHub.

Appendix: source

Thrown at quickwit/quickwit-storage/src/prefix_storage.rs:135

            if prefix == Path::new("") {
                return Ok(objects);
            }
            let prefix_bytes = prefix.as_os_str().as_encoded_bytes();
            let mut relative_objects = Vec::with_capacity(objects.len());
            for mut object in objects {
                match object.path.strip_prefix(prefix) {
                    Ok(relative_path) => {
                        object.path = relative_path.to_path_buf();
                        relative_objects.push(object);
                    }
                    Err(error) => {
                        let is_under_prefix = object
                            .path
                            .as_os_str()
                            .as_encoded_bytes()
                            .starts_with(prefix_bytes);
                        if !is_under_prefix {
                            return Err(StorageErrorKind::Internal.with_error(anyhow::anyhow!(
                                "listed object `{}` is not under storage prefix `{}`: {error}",
                                object.path.display(),
                                prefix.display()
                            )));
                        }
                    }
                }
            }
            Ok(relative_objects)
        }

        let storage_prefix = self.prefix.clone();
        self.storage
            .list(&self.prefix.join(prefix))
            .map(move |objects_res| {
                let objects = objects_res?;
                strip_prefix_from_objects(objects, &storage_prefix)
            })

View on GitHub (pinned to a39730c5cd)