{"record":{"id":"5d7f453d49e256a4","repo":"quickwit-oss/quickwit","slug":"the-prefix-should-have-been-prepended-to-the-key-b","errorCode":null,"errorMessage":"The prefix should have been prepended to the key before this method call.","messagePattern":"The prefix should have been prepended to the key before this method call\\.","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"quickwit/quickwit-storage/src/object_storage/s3_compatible_storage.rs","lineNumber":391,"sourceCode":"    fn key(&self, relative_path: &Path) -> String {\n        // FIXME: This may not work on Windows.\n        let prefix = self.prefix.to_string_lossy();\n        let relative_path = relative_path.to_string_lossy();\n        if prefix.is_empty() {\n            relative_path.to_string()\n        } else if relative_path.is_empty() {\n            prefix.to_string()\n        } else if prefix.ends_with('/') {\n            format!(\"{prefix}{relative_path}\")\n        } else {\n            format!(\"{prefix}/{relative_path}\")\n        }\n    }\n\n    fn relative_path(&self, key: &str) -> PathBuf {\n        // FIXME: This may not work on Windows.\n        let relative_key = strip_storage_prefix(key, &self.prefix)\n            .expect(\"The prefix should have been prepended to the key before this method call.\");\n        PathBuf::from(relative_key)\n    }\n\n    async fn put_single_part_single_try<'a>(\n        &'a self,\n        bucket: &'a str,\n        key: &'a str,\n        payload: Box<dyn crate::PutPayload>,\n        len: u64,\n    ) -> Result<(), Retry<StorageError>> {\n        // For MD5 uploads, compute Content-MD5 before streaming the body.\n        // The AWS SDK no-ops ChecksumAlgorithm::Md5, so MD5 must be sent via\n        // the legacy Content-MD5 header (same as the multipart path does per part).\n        let content_md5: Option<String> = self\n            .maybe_compute_part_md5(payload.as_ref(), 0..len)\n            .await\n            .map_err(|err| Retry::Permanent(StorageError::from(err)))?\n            .map(|digest| BASE64_STANDARD.encode(digest.0));","sourceCodeStart":373,"sourceCodeEnd":409,"githubUrl":"https://github.com/quickwit-oss/quickwit/blob/a39730c5cdcd1a4fe798403737ae293999ea21f8/quickwit/quickwit-storage/src/object_storage/s3_compatible_storage.rs#L373-L409","documentation":"`relative_path` strips the configured storage prefix from an object key via `strip_storage_prefix` and panics with this message if the key does not start with that prefix. The function's contract is that callers (e.g. `bulk_delete_multi`) have already applied the prefix when building keys, so a missing prefix is an internal invariant violation rather than a user-facing error. It converts S3 full keys back into storage-relative paths.","triggerScenarios":"A key passed to `relative_path` that does not begin with `self.prefix` — e.g. an object listed in the bucket that was created outside quickwit (manually uploaded or by an older deployment with a different prefix) and then targeted by a bulk delete, or a code path that composes keys without applying the prefix.","commonSituations":"Mixed-prefix buckets: the container/bucket holds objects under several prefixes (multiple quickwit indexes or manual uploads) and an operation iterates raw keys assuming the instance's prefix; also occurs after prefix configuration changes on an existing bucket, or on Windows paths due to the noted FIXME.","solutions":["Ensure all keys passed to delete/list operations are built by prepending the storage prefix (use the same key-building helper used by put operations).","If objects may live under other prefixes, filter keys by `key.starts_with(&self.prefix)` before calling `relative_path` instead of letting it panic.","Verify the storage `prefix` config matches the one used when the objects were written; a mismatched prefix causes both misses and this panic.","If you control the code, change `strip_storage_prefix` handling to return an error (e.g. skip or log) rather than expect, for externally influenced inputs."],"exampleFix":"// before\nlet relative_key = strip_storage_prefix(key, &self.prefix)\n    .expect(\"The prefix should have been prepended to the key before this method call.\");\n// after (caller-side guard)\nif !key.starts_with(self.prefix.as_str()) { return PathBuf::from(key); }\nlet relative_key = strip_storage_prefix(key, &self.prefix)\n    .expect(\"The prefix should have been prepended to the key before this method call.\");","handlingStrategy":"validation","validationCode":"// Guard every key before any operation that funnels into relative_path\nfn is_prefixed(key: &str, prefix: &str) -> bool { key.starts_with(prefix) }\n// skip keys that do not belong to this storage instance\nkeys.retain(|k| is_prefixed(k, &storage_prefix));","typeGuard":"fn strip_prefix_safe<'a>(key: &'a str, prefix: &'a str) -> Option<&'a str> {\n    key.strip_prefix(prefix)\n}","tryCatchPattern":"// This panics rather than returning an error; pre-filter instead of catching\nlet relative = match strip_storage_prefix(key, &prefix) {\n    Some(k) => PathBuf::from(k),\n    None => { tracing::warn!(\"skipping key with foreign prefix: {key}\"); continue; }\n};","preventionTips":["Always build object keys by prepending the storage prefix through a single shared helper.","Do not assume every key returned by a bucket listing belongs to the current quickwit instance; filter by prefix.","Keep the configured prefix stable across deployments that share a bucket.","Run delete/list operations only over keys produced by this instance's own key-building code."],"tags":["rust","s3","prefix-mismatch","invariant-violation"],"backgroundTag":"internal-invariant-violation","analyzedSha":"a39730c5cdcd1a4fe798403737ae293999ea21f8","analyzedAt":"2026-09-08T13:19:37.784Z","contentChangedAt":"2026-09-08T13:19:37.784Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}