What causes "failed to find dest_path {:?}" in quickwit-oss/quickwit?

Ensure the path exists in the RamStorage before copying — add it via RamStorageBuilder/`put` in test setup

quickwit-oss/quickwit · error · StorageError

failed to find dest_path {:?}

Error message

failed to find dest_path {:?}

What it means

`RamStorage::copy_to` fetches the whole in-memory payload via `get_data`; if the path is absent from the in-memory map, it raises a NotFound StorageError with the (misleading) message "failed to find dest_path". Despite the wording, `path` is the source path whose contents are copied to `output`.

Source

Thrown at quickwit/quickwit-storage/src/ram_storage.rs:101

impl Storage for RamStorage {
    async fn check_connectivity(&self) -> anyhow::Result<()> {
        Ok(())
    }

    async fn put(
        &self,
        path: &Path,
        payload: Box<dyn crate::PutPayload>,
    ) -> crate::StorageResult<()> {
        let payload_bytes = payload.read_all().await?;
        self.put_data(path, payload_bytes).await;
        Ok(())
    }

    async fn copy_to(&self, path: &Path, output: &mut dyn SendableAsync) -> StorageResult<()> {
        let payload_bytes = self.get_data(path).await.ok_or_else(|| {
            StorageErrorKind::NotFound
                .with_error(anyhow::anyhow!("failed to find dest_path {:?}", path))
        })?;
        output.write_all(&payload_bytes).await?;
        output.flush().await?;
        Ok(())
    }

    async fn get_slice(&self, path: &Path, range: Range<usize>) -> StorageResult<OwnedBytes> {
        let payload_bytes = self.get_data(path).await.ok_or_else(|| {
            StorageErrorKind::NotFound
                .with_error(anyhow::anyhow!("failed to find dest_path {:?}", path))
        })?;
        Ok(payload_bytes.slice(range.start..range.end))
    }

    async fn get_slice_stream(
        &self,
        path: &Path,
        range: Range<usize>,

View on GitHub (pinned to a39730c5cd)

Solutions

Ensure the path exists in the RamStorage before copying — add it via RamStorageBuilder/`put` in test setup
Check for typos or a path/prefix mismatch between the writer and the reader of the RAM storage
If simulating deletion, expect NotFound and assert on StorageErrorKind::NotFound rather than treating it as a bug
Verify you are reading from the same RamStorage instance that was written to

Example fix

// before
let ram = RamStorage::default();
storage.copy_to(&Path::from("meta.json"), &mut out).await?;
// after
let ram = RamStorage::builder().put("meta.json", payload).build();
ram.copy_to(&Path::from("meta.json"), &mut out).await?;

Defensive patterns

Strategy: try-catch

Validate before calling

let exists = ram_storage.exists(&path).await.unwrap_or(false);
if !exists { seed_or_fail(path); }

Try / catch

match storage.copy_to(&path, &mut out).await {
    Ok(()) => {},
    Err(e) if e.kind() == StorageErrorKind::NotFound => seed_ram_storage_or_skip(path),
    Err(e) => return Err(e.into()),
}

Prevention

Seed RamStorage via RamStorageBuilder in tests before any read
Reuse the same RamStorage instance for writes and reads in a test
Remember the message says dest_path but the missing path is the source
Account for delete/bulk_delete side effects before later reads

When it happens

Trigger: Calling `Storage::copy_to` on a RamStorage for a key that was never inserted or was removed (e.g. after `bulk_delete` or `delete`).

Common situations: Unit/integration tests where the RamStorage was not prepopulated with the expected file (RamStorageBuilder missing the entry); metastore tests simulating storage failure; reading a file from the wrong storage instance.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/f9cfffaa270062f5. Report an issue: GitHub.