quickwit-oss/quickwit · error · anyhow::Error

stdin can only be used as source through the CLI command `qu

Error message

stdin can only be used as source through the CLI command `quickwit tool local-ingest`

What it means

The stdin source is only meaningful when documents are piped into the `quickwit tool local-ingest` CLI command. When a source config is loaded from user config or a source update (`validate_and_build`), a `SourceParams::Stdin` config is rejected because stdin has no meaning for a running cluster-managed source. The loader bails to prevent nonsensical persistent source definitions.

Source

Thrown at quickwit/quickwit-config/src/source_config/serialize.rs:108

impl SourceConfigForSerialization {
    /// Checks the validity of the `SourceConfig` as a "deserializable source".
    ///
    /// Two remarks:
    /// - This does not check connectivity, it just validate configuration, without performing any
    ///   IO. See `check_connectivity(..)`.
    /// - This is used each time the `SourceConfig` is deserialized (at creation but also during
    ///   communications with the metastore). When ingesting from stdin, we programmatically create
    ///   an invalid `SourceConfig` and only use it locally.
    fn validate_and_build(self) -> anyhow::Result<SourceConfig> {
        if !RESERVED_SOURCE_IDS.contains(&self.source_id.as_str()) {
            validate_identifier("source", &self.source_id)?;
        }
        let num_pipelines = NonZeroUsize::new(self.num_pipelines)
            .ok_or_else(|| anyhow::anyhow!("`desired_num_pipelines` must be strictly positive"))?;
        match &self.source_params {
            SourceParams::Stdin => {
                bail!(
                    "stdin can only be used as source through the CLI command `quickwit tool \
                     local-ingest`"
                );
            }
            SourceParams::File(_)
            | SourceParams::Kafka(_)
            | SourceParams::Kinesis(_)
            | SourceParams::Pulsar(_) => {
                // TODO consider any validation opportunity
            }
            SourceParams::PubSub(_)
            | SourceParams::Ingest
            | SourceParams::IngestApi
            | SourceParams::IngestCli
            | SourceParams::Vec(_)
            | SourceParams::Void(_) => {}
        }
        match &self.source_params {

View on GitHub (pinned to a39730c5cd)

Solutions

Remove the stdin source config from the index; use it only as a transient source for `quickwit tool local-ingest --stdin`.
Replace it with a real source type (file, kafka, kinesis, pulsar, gcp-pubsub) appropriate for your ingestion.
For one-off local ingestion, pipe documents: `cat docs.json | quickwit tool local-ingest --index ...` with a stdin source config passed only to the CLI.

Example fix

# before
source_configs:
  - source_id: stdin-src
    source_type: stdin

# after
source_configs:
  - source_id: my-file-source
    source_type: file
    params:
      path: /data/docs.json

Defensive patterns

Strategy: validation

Validate before calling

if source_config.source_params == SourceParams::Stdin && !running_local_ingest_cli {
    return Err(anyhow!(
        "stdin source is only valid for `quickwit tool local-ingest`"
    ));
}

Type guard

fn is_persistable(params: &SourceParams) -> bool {
    !matches!(params, SourceParams::Stdin)
}

Try / catch

match load_source_config_from_user_config(&user_config) {
    Ok(sc) => persist(sc),
    Err(e) if e.to_string().contains("local-ingest") => {
        eprintln!("use `quickwit tool local-ingest` for stdin sources");
    }
    Err(e) => return Err(e),
}

Prevention

Never place a stdin source in index source_configs intended for a cluster.
Reserve stdin configs for ad-hoc `quickwit tool local-ingest` invocations only.
Lint index configs to reject `source_type: stdin` outside CLI examples.

When it happens

Trigger: Defining a source in an index config (or via the source config update API/CLI) with `source_type: stdin` and loading it through `load_source_config_from_user_config` or `load_source_config_update`.

Common situations: Copy-pasting an example ingest config that used stdin for `quickwit tool local-ingest` into an index's source_configs; trying to run a server-managed stdin source in docker.

Understand the failure class

Background: "is not a compatible type" / "cannot merge" errors: when a value's type doesn't match what the library requires — this error's family across 65 libraries.

Related errors

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/a03fe71fb5baed66. Report an issue: GitHub.