quickwit-oss/quickwit · error

`searcher.use_metastore_read_replica` cannot be enabled on a

Error message

`searcher.use_metastore_read_replica` cannot be enabled on a node running the `metastore` service

What it means

If a node runs both the primary `metastore` service and a searcher configured with `use_metastore_read_replica: true`, a deadlock occurs: the searcher waits for a READY read replica while the read replica waits for the co-hosted primary metastore to become READY. `validate_metastore_read_replica` detects this combination and rejects the config before startup.

Source

Thrown at quickwit/quickwit-config/src/node_config/serialize.rs:447

fn validate_metastore_read_replica(node_config: &NodeConfig) -> anyhow::Result<()> {
    let read_replica_enabled =
        node_config.is_service_enabled(QuickwitService::MetastoreReadReplica);
    if read_replica_enabled {
        ensure!(
            node_config.enabled_services.len() == 1,
            "the `metastore_read_replica` service must run standalone and cannot be combined with \
             any other service"
        );
        ensure!(
            node_config.metastore_read_replica_uri.is_some(),
            "the `metastore_read_replica` service requires `metastore_read_replica_uri` to be set"
        );
    }
    let searcher_uses_read_replica = node_config.is_service_enabled(QuickwitService::Searcher)
        && node_config.searcher_config.use_metastore_read_replica;
    // Avoid a deadlock where the searcher waits for a READY read replica, while the read
    // replica waits for this primary metastore to become READY.
    ensure!(
        !(searcher_uses_read_replica && node_config.is_service_enabled(QuickwitService::Metastore)),
        "`searcher.use_metastore_read_replica` cannot be enabled on a node running the \
         `metastore` service"
    );
    Ok(())
}

/// A list of all the known disk budgets
///
/// External disk usage and unbounded disk usages, e.g the indexing workbench
/// (indexing/) and the delete task workbench (delete_task_service/) are not included.
#[derive(Default, Debug)]
struct ExpectedDiskUsage {
    // indexer / ingester
    split_store_max_num_bytes: Option<ByteSize>,
    max_queue_disk_usage: Option<ByteSize>,
    // searcher
    split_cache: Option<ByteSize>,

View on GitHub (pinned to a39730c5cd)

Solutions

1. Set `searcher.use_metastore_read_replica: false` on nodes running the primary metastore service.
2. Or remove the `metastore` service from this node and run the primary metastore elsewhere, keeping the read-replica-backed searcher.
3. Deploy the searcher that uses the read replica as a standalone node pointed at the replica.

Example fix

// before (quickwit.yaml)
services: [searcher, metastore]
searcher:
  use_metastore_read_replica: true

// after (quickwit.yaml)
services: [searcher, metastore]
searcher:
  use_metastore_read_replica: false

if config.is_service_enabled(QuickwitService::Metastore)
    && config.searcher_config.use_metastore_read_replica {
    return Err("disable use_metastore_read_replica on nodes running the metastore service");
}

fn searcher_replica_allowed(cfg: &NodeConfig) -> bool {
    !(cfg.is_service_enabled(QuickwitService::Metastore) && cfg.searcher_config.use_metastore_read_replica)
}

if let Err(e) = load_node_config(path) {
    if e.to_string().contains("use_metastore_read_replica") {
        eprintln!("Use read-replica-backed searchers only on nodes without the metastore service");
    }
    return Err(e);
}

When it happens

Trigger: A node whose `services` includes `metastore` (and typically `searcher`) with `searcher.use_metastore_read_replica: true` set in the searcher config; validate fails during config load.

Common situations: Single-node setups where all services are enabled by default and an operator flips `use_metastore_read_replica` on for search scaling; copied searcher config snippets pasted into an all-in-one node config.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

• {left:?} metastore config is defined multiple times
• the `metastore_read_replica` service must run standalone and
• the `metastore_read_replica` service requires `metastore_rea
• index ID pattern `{pattern}` is invalid: patterns must not c
• index ID pattern `{pattern}` is invalid: an index ID must ha

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/88d45315cf023b94. Report an issue: GitHub.