quickwit-oss/quickwit · error · io::Error (InvalidInput)

split folder name should match the format `<split_id>.split`

Error message

split folder name should match the format `<split_id>.split`: got `{}`

What it means

While restoring the split cache from disk, open() iterates subdirectories of the cache and derives the split id from the folder name via split_id_from_split_folder, which expects the `<split_id>.split` format. Any directory whose name does not match this convention triggers an InvalidInput I/O error.

Source

Thrown at quickwit/quickwit-indexing/src/split_store/indexing_split_cache.rs:467

        let mut read_dir = tokio::fs::read_dir(&split_store_folder).await?;
        while let Some(dir_entry) = read_dir.next_entry().await? {
            let metadata = dir_entry.metadata().await?;
            let dir_path: PathBuf = dir_entry.path();

            if metadata.is_file() {
                warn!(
                    "unexpected file found in split cache directory: `{}`",
                    dir_path.display()
                );
                continue;
            }

            let split_id = split_id_from_split_folder(&dir_path).ok_or_else(|| {
                let error_msg = format!(
                    "split folder name should match the format `<split_id>.split`: got `{}`",
                    dir_path.display()
                );
                io::Error::new(io::ErrorKind::InvalidInput, error_msg)
            })?;

            let split_folder = SplitFolder::create(split_id, &dir_entry.path()).await?;
            split_folders.push(split_folder);
        }

        let mut inner_local_split_store = InnerSplitCache {
            split_store_folder: split_store_folder.clone(),
            split_registry: SplitFolderRegistry::with_quota(space_quota),
        };

        split_folders.sort_by_key(|split_folder| split_folder.created_at);

        // We record all `split_folder`, sorted by `creation_time`.
        for split_folder in split_folders {
            let split_id = split_folder.split_id.clone();
            if !inner_local_split_store
                .make_room_and_record_split(split_folder)

View on GitHub (pinned to a39730c5cd)

Solutions

1. Inspect the cache directory and rename the offending folder to `<split_id>.split`, or remove it if it is stale.
2. Only place split folders produced by Quickwit in the cache directory; move extracted/migrated folders elsewhere.
3. If the data is stale/recoverable from storage, wipe the cache directory and let Quickwit re-download splits.

Example fix

// cache dir contains: 03F8QW2X/ (invalid)
mv data/indexing/03F8QW2X data/indexing/03F8QW2X.split
// or delete the stale folder
rm -r data/indexing/03F8QW2X

fn is_valid_split_folder(name: &str) -> bool {
    let Some(stem) = name.strip_suffix(".split") else { return false };
    !stem.is_empty()
}
// before booting the indexer, scan the cache dir:
// for entry in fs::read_dir(cache_dir)? { assert!(is_valid_split_folder(entry.file_name())) }

fn split_id_from_name(name: &str) -> Option<&str> {
    name.strip_suffix(".split").filter(|s| !s.is_empty())
}

match cache.open().await {
    Ok(cache) => cache,
    Err(e) if format!("{}").contains("split folder name should match") => {
        // clean invalid folders, then retry
        clean_invalid_folders(cache_dir)?;
        cache.open().await?
    }
    Err(e) => return Err(e.into()),
}

When it happens

Trigger: Starting an indexer whose cache directory (default per indexing directory setting, e.g. ./data/indexing) contains a subdirectory not named `<split_id>.split`, such as a leftover temp folder, a manually created directory, or an old-format split folder.

Common situations: Manual cleanup or copying of the indexing cache directory left stray folders; upgrades changing folder naming; operators unpacking splits by hand into the cache directory.

Understand the failure class

Background: "invalid id" errors: invalid identifier format — why libraries reject IDs before lookup, and how to fix them — this error's family across 37 libraries.

Related errors

• failed to render config file template: environment variable
• failed to create {index_name} index: {error}
• `metastore_read_replica_uri` must be set when the `metastore
• the filename limit is too small
• couldn't find parent for {}

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/77182d01c3ce68b6. Report an issue: GitHub.