nautechsystems/nautilus_trader · error · anyhow::Error

replace_existing for remote streaming paths requires a non-e

Error message

replace_existing for remote streaming paths requires a non-empty prefix

What it means

Thrown by FeatherQueryResult::from_uri when replace_existing is requested for a remote object store with an empty base_path. Deleting existing data on a remote store requires an explicit prefix to scope the deletion; deleting with no prefix on a remote store would be dangerously broad, so it is only allowed for local paths (where the catalog directory itself scopes it).

Source

Thrown at crates/persistence/src/backend/feather.rs:302

        rotation_config: RotationConfig,
        included_types: Option<HashSet<String>>,
        flush_interval_ms: Option<u64>,
        replace_existing: bool,
    ) -> anyhow::Result<Self> {
        let normalized_uri = crate::parquet::normalize_path_to_uri(uri);
        if normalized_uri.starts_with("file://") {
            let path = crate::parquet::file_uri_to_native_path(&normalized_uri);
            std::fs::create_dir_all(&path)
                .with_context(|| format!("Failed to create streaming directory '{path}'"))?;
        }
        let location = create_object_store_location_from_path(&normalized_uri, storage_options)?;
        let is_local = matches!(location.kind, ObjectStoreLocationKind::Local);
        let store = location.object_store;
        let base_path = location.base_path;

        if replace_existing {
            let prefix = if base_path.is_empty() {
                anyhow::ensure!(
                    is_local,
                    "replace_existing for remote streaming paths requires a non-empty prefix",
                );
                None
            } else {
                Some(Path::from(base_path.clone()))
            };
            let runtime = nautilus_common::live::get_runtime();
            runtime.block_on(async {
                let mut objects = store.list(prefix.as_ref());
                let mut paths = Vec::new();
                while let Some(result) = objects.next().await {
                    paths.push(result?.location);
                }

                for path in paths {
                    store.delete(&path).await?;
                }

View on GitHub (pinned to 18893faf8b)

Solutions

Provide a non-empty base_path/prefix in the object store location when using replace_existing with a remote store.
Set replace_existing=false if you don't need to delete prior data on the remote store.
If targeting local paths only, keep the empty base_path but confirm location.kind is Local.
Scope deletions by writing into a dedicated prefix directory and replacing only that.

Example fix

// before
let uri = "s3://bucket"; // empty prefix + replace_existing
// after
let uri = "s3://bucket/data/strategy_a"; // non-empty prefix for replace_existing

Defensive patterns

Strategy: validation

Validate before calling

if replace_existing && !is_local && base_path.is_empty() {
    panic!("remote replace_existing requires a non-empty base_path prefix");
}

Type guard

fn can_replace_existing(kind: &ObjectStoreLocationKind, base_path: &str) -> bool {
    matches!(kind, ObjectStoreLocationKind::Local) || !base_path.is_empty()
}

Try / catch

match FeatherQueryResult::from_uri(uri, replace_existing) {
    Ok(r) => use(r),
    Err(e) if e.to_string().contains("non-empty prefix") => eprintln!("add a base path to the remote URI"),
    Err(e) => return Err(e.into()),
}

Prevention

Always include a key/prefix in remote object store URIs
Only use replace_existing=true with a scoped prefix
Avoid copying local-path configs directly to remote stores
Prefer writing into dedicated per-strategy prefixes for safe replacement

When it happens

Trigger: Constructing a streaming query/write URI against a remote object store (S3, GCS, etc.) with replace_existing=true and an empty base_path prefix.

Common situations: Config mistakes where the remote URI omitted the bucket key/prefix portion; copying local-path configuration to a cloud store without adding the base path; tests that always used local paths now pointed at remote storage.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/6f46909a3cd64e14. Report an issue: GitHub.