quickwit-oss/quickwit · error · IndexServiceError

path based file sources are limited to a local usage, please

Error message

path based file sources are limited to a local usage, please use the CLI command `quickwit tool local-ingest` to ingest data from a specific file or setup a notification based file source

What it means

The Quickwit server rejects source configs that use the file source with a filepath parameter. Path-based file sources only work for local, single-node usage via the CLI (quickwit tool local-ingest); through the server API they are disallowed to avoid nondeterministic reads on distributed deployments. The check is applied on create and update source endpoints.

Source

Thrown at quickwit/quickwit-serve/src/index_api/source_resource.rs:60

        .and(warp::post())
        .and(extract_config_format())
        .and(warp::body::content_length_limit(1024 * 1024))
        .and(warp::filters::body::bytes())
        .and(with_arg(index_service))
        .then(create_source)
        .map(log_failure("failed to create source"))
        .and(extract_format_from_qs())
        .map(into_rest_api_response)
        .boxed()
}

#[allow(clippy::result_large_err)]
fn check_source_type(source_params: &SourceParams) -> Result<(), IndexServiceError> {
    // Note: This check is performed here instead of the source config serde
    // because many tests use the file source, and can't store that config in
    // the metastore without going through the validation.
    if let SourceParams::File(FileSourceParams::Filepath(_)) = source_params {
        return Err(IndexServiceError::InvalidConfig(anyhow::anyhow!(
            "path based file sources are limited to a local usage, please use the CLI command \
             `quickwit tool local-ingest` to ingest data from a specific file or setup a \
             notification based file source"
        )));
    }
    Ok(())
}

#[utoipa::path(
    post,
    tag = "Sources",
    path = "/indexes/{index_id}/sources",
    request_body = VersionedSourceConfig,
    responses(
        // We return `VersionedSourceConfig` as it's the serialized model view.
        (status = 200, description = "Successfully created source.", body = VersionedSourceConfig)
    ),
    params(

View on GitHub (pinned to a39730c5cd)

Solutions

1. Use `quickwit tool local-ingest` to ingest directly from the local file instead of creating a server source.
2. Switch to a notification-based source (e.g. Kafka, Kinesis, SNS/SQS, GCP Pub/Sub) that pushes data to Quickwit.
3. If files land in object storage, use an appropriate supported source or ingestion via the ingest API.

Example fix

// before
source_params:
  file:
    filepath: /var/log/data.json
// after (e.g. SQS notification source)
source_params:
  sqs:
    queue_url: https://sqs.us-east-1.amazonaws.com/123/my-queue

fn uses_path_file_source(source_config: &serde_yaml::Value) -> bool {
    source_config["source_params"]["file"]["filepath"].as_str().is_some()
}

fn is_api_rejected_source(params: &SourceParams) -> bool {
    matches!(params, SourceParams::File(FileSourceParams::Filepath(_)))
}

match client.create_source(index_id, source_config).await {
    Err(e) if e.to_string().contains("path based file sources") => {
        Err(anyhow!("use `quickwit tool local-ingest` or a notification-based source"))
    }
    r => r,
}

When it happens

Trigger: POST /indexes/{index_id}/sources or PUT .../sources with a source config whose source_params is {file: {filepath: ...}}.

Common situations: Migrating a locally tested file source config to a server deployment; following older tutorials/examples that used file sources over the API; copy-pasting a CLI-only source config into an API call.

Related errors

• `source_id` in config file does not match source_id from que
• Encountered unknown command: code {other}
• the `compactor` service can only be enabled when `enable_sta
• failed to parse service `{service_str}`. supported services
• failed to compile VRL script: {formatter}

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/f110d13a148098de. Report an issue: GitHub.