quickwit-oss/quickwit · error

Prometheus metrics renderer is already installed

Error message

Prometheus metrics renderer is already installed

What it means

build_recorder creates the Prometheus recorder and stores its metrics render handle into the process-global PROMETHEUS_HANDLE OnceCell. If that cell already holds a handle, installation fails with this error, because the Prometheus exposition endpoint (`text_payload`) would otherwise read from the wrong/absent renderer. Same one-time-only constraint as the global metrics recorder.

Source

Thrown at quickwit/quickwit-telemetry-exporters/src/prometheus/metrics.rs:38

    Matcher, PrometheusBuilder, PrometheusHandle, PrometheusRecorder,
};

static PROMETHEUS_HANDLE: OnceLock<PrometheusHandle> = OnceLock::new();

pub(crate) fn build_recorder() -> anyhow::Result<PrometheusRecorder> {
    let mut prometheus_builder = PrometheusBuilder::new();
    for (name, buckets) in quickwit_metrics::histogram_buckets() {
        prometheus_builder = prometheus_builder
            .set_buckets_for_metric(Matcher::Full(name.to_string()), &buckets)
            .with_context(|| {
                format!("failed to configure Prometheus histogram buckets for `{name}`")
            })?;
    }
    let prometheus_recorder = prometheus_builder.build_recorder();
    let prometheus_handle = prometheus_recorder.handle();
    PROMETHEUS_HANDLE
        .set(prometheus_handle.clone())
        .map_err(|_| anyhow::anyhow!("Prometheus metrics renderer is already installed"))?;
    spawn_prometheus_upkeep(prometheus_handle).map_err(anyhow::Error::msg)?;
    Ok(prometheus_recorder)
}

pub fn text_payload() -> Result<String, String> {
    let handle = PROMETHEUS_HANDLE
        .get()
        .ok_or_else(|| "Prometheus metrics rendering is not installed yet".to_string())?;
    Ok(handle.render())
}

fn spawn_prometheus_upkeep(handle: PrometheusHandle) -> Result<(), String> {
    // Quickwit serves the existing `/metrics` route itself, so we build only the
    // Prometheus recorder instead of using the exporter's HTTP listener. That lower-level
    // API does not spawn the upkeep task that periodically drains histogram buffers.
    std::thread::Builder::new()
        .name("telemetry-exporter-prometheus-upkeep".to_string())
        .spawn(move || {

View on GitHub (pinned to a39730c5cd)

Solutions

1. Call build_recorder only once per process lifetime
2. Guard with std::sync::Once or check if the Prometheus endpoint already responds before rebuilding
3. In tests, share one telemetry setup across the test binary instead of per-test init

Example fix

// before
each_test(|| { build_recorder().unwrap(); })
// after
static ONCE: Once = Once::new();
fn setup_prometheus() { ONCE.call_once(|| build_recorder().unwrap()); }

// Rust: guard global handle installation
static PROM_INIT: Once = Once::new();
fn ensure_prometheus() { PROM_INIT.call_once(|| build_recorder().expect("prometheus init")); }

if let Err(e) = build_recorder() {
    if e.to_string().contains("already installed") { /* reuse existing renderer */ }
    else { return Err(e); }
}

When it happens

Trigger: Calling build_recorder twice in one process (double bootstrap of telemetry, repeated service startup within a process, or multiple tests each building a Prometheus recorder).

Common situations: Integration test suites where each test spins up a serve/telemetry stack; embedding Quickwit and initializing Prometheus metrics twice; accidental duplicate call in custom bootstrap code.

Related errors

• failed to install global metrics recorder
• {QW_METRICS_LABELS_ENV_VAR} must contain valid Unicode
• {} contains an empty label name: {}
• {} contains an empty label value: {}
• {} contains invalid label format: {}

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/21835858ae31f197. Report an issue: GitHub.