{"record":{"id":"3a71c1f207ca2997","repo":"quickwit-oss/quickwit","slug":"failed-to-url-decode-listed-object-key-encoded-k","errorCode":null,"errorMessage":"failed to URL-decode listed object key `{encoded_key}`: {error}","messagePattern":"failed to URL-decode listed object key `(.+?)`: (.+?)","errorType":"exception","errorClass":"StorageError","httpStatus":null,"severity":"error","filePath":"quickwit/quickwit-storage/src/object_storage/s3_compatible_storage.rs","lineNumber":1224,"sourceCode":"            None => {\n                warn!(\"listed object has no last modified time, skipping\");\n                continue;\n            }\n        };\n        object_metadata.push(ObjectMetadata {\n            path: relative_path,\n            size: ByteSize(size_bytes),\n            last_modified,\n        });\n    }\n    Ok(object_metadata)\n}\n\nfn decode_list_object_key(encoded_key: &str) -> StorageResult<Cow<'_, str>> {\n    percent_decode_str(encoded_key)\n        .decode_utf8()\n        .map_err(|error| {\n            StorageErrorKind::Internal.with_error(anyhow::anyhow!(\n                \"failed to URL-decode listed object key `{encoded_key}`: {error}\"\n            ))\n        })\n}\n\n/// Strips a storage prefix from an S3 key without interpreting it as a filesystem path.\nfn strip_storage_prefix<'a>(key: &'a str, storage_prefix: &Path) -> Option<&'a str> {\n    let prefix = storage_prefix.to_string_lossy();\n    if prefix.is_empty() {\n        return Some(key);\n    }\n    if key == prefix {\n        return Some(\"\");\n    }\n    if prefix.ends_with('/') {\n        key.strip_prefix(prefix.as_ref())\n    } else {\n        let prefix_with_separator = format!(\"{prefix}/\");","sourceCodeStart":1206,"sourceCodeEnd":1242,"githubUrl":"https://github.com/quickwit-oss/quickwit/blob/a39730c5cdcd1a4fe798403737ae293999ea21f8/quickwit/quickwit-storage/src/object_storage/s3_compatible_storage.rs#L1206-L1242","documentation":"When listing objects, S3 keys are URL-encoded by the service; Quickwit percent-decodes each key back into a path. If the encoded key is not valid percent-encoding or is not valid UTF-8 after decoding, this Internal error is raised. It means a listed object key cannot be interpreted as a storage path.","triggerScenarios":"Calling `list` on an S3-compatible storage when a listed key contains invalid percent-escapes (e.g. a lone `%` not followed by two hex digits) or bytes that are not valid UTF-8 after decoding.","commonSituations":"Objects uploaded directly to the bucket by other tools with raw/non-UTF8 key names; a non-AWS S3 implementation that double-encodes or leaves keys unencoded inconsistently; manually renamed objects containing stray `%` characters.","solutions":["Find and rename/re-upload the offending object in the bucket so its key is valid UTF-8 percent-encodable","List the bucket directly (aws s3 ls) around the reported prefix to identify the bad key","Avoid uploading objects with raw percent signs or non-UTF8 names into the Quickwit index bucket","If a backend mis-encodes keys, fix or upgrade that S3-compatible implementation"],"exampleFix":"// before (raw key with stray % uploaded out-of-band)\nmy-index/hotcache-100%.bin\n// after\naws s3 mv s3://bucket/my-index/hotcache-100%.bin s3://bucket/my-index/hotcache-100pct.bin","handlingStrategy":"try-catch","validationCode":"// client-side pre-check before uploading out-of-band objects\nif !key.is_ascii() || key.contains('%') { reject_or_rename(key); }","typeGuard":null,"tryCatchPattern":"match storage.list(&prefix).try_collect::<Vec<_>>().await {\n    Ok(objs) => use(objs),\n    Err(e) if e.kind() == StorageErrorKind::Internal => inspect_bucket_keys_for_bad_encoding(),\n    Err(e) => return Err(e.into()),\n}","preventionTips":["Never upload objects with raw percent signs or non-UTF8 keys into a Quickwit bucket","Restrict direct bucket writes to tools that URL-encode keys like S3 does","Periodically audit bucket keys for stray '%'-sequences","If using a non-AWS S3 implementation, verify its key-encoding behavior matches AWS"],"tags":["s3","url-decode","listing","storage"],"backgroundTag":"invalid-url-format","analyzedSha":"a39730c5cdcd1a4fe798403737ae293999ea21f8","analyzedAt":"2026-09-08T13:19:37.784Z","contentChangedAt":"2026-09-08T13:19:37.784Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}