xai-org/x-algorithm · critical

{misconfiguration}

Error message

{misconfiguration}

What it means

The reference-compare harness is only built when a set of environment/config conditions line up (dual_call_harness_enabled plus the APP_ENV check inside should_build_harness). When should_build_harness returns Err — i.e. the configuration combination is contradictory or invalid — build_reference_compare_harness panics with the misconfiguration message. This is a fail-fast guard against running dual-call comparison in an unsupported environment.

Source

Thrown at visibility-filtering/server_deps.rs:238

        recommendations_rule_count,
        "VFServer initialized with prod clients"
    );

    VFServer::from_endpoints(
        FilterTweetsEndpoint::new(filter_tweets, reference_compare),
        GetSafetyLabelsEndpoint::new(safety_label_source),
    )
}

async fn build_reference_compare_harness(
    datacenter: &str,
    init_deadline: tokio::time::Instant,
) -> Option<Arc<ReferenceCompareHarness>> {
    let should_build = crate::reference_compare::should_build_harness(
        crate::config::dual_call_harness_enabled(),
        std::env::var("APP_ENV").ok().as_deref(),
    )
    .unwrap_or_else(|misconfiguration| panic!("{misconfiguration}"));
    if !should_build {
        return None;
    }

    let client_id = format!(
        "visibility-filtering-service.{}",
        std::env::var("APP_ENV").unwrap_or_else(|_| "staging".to_string())
    );
    let strato: Arc<dyn VfClient + Send + Sync> = Arc::new(
        init_client_with_retry("strato_vf", init_deadline, || {
            let client_id = client_id.clone();
            async move {
                StratoVfClient::new(
                    S2S_CHAIN_PATH.clone(),
                    S2S_CRT_PATH.clone(),
                    S2S_KEY_PATH.clone(),
                    client_id,
                    datacenter.to_string(),

View on GitHub (pinned to 24c60942c5)

Solutions

1. Check the message text: it states which combination (flag + APP_ENV) is invalid; align them — e.g. disable the dual-call harness flag in prod, or set a permitted APP_ENV.
2. If the harness is intended for this environment, fix the APP_ENV value to one should_build_harness accepts.
3. Audit config precedence (env vars vs flag files) to ensure the harness flag is not unintentionally on.

Example fix

// before
.unwrap_or_else(|misconfiguration| panic!("{misconfiguration}"));

// after: degrade gracefully instead of aborting server startup
let should_build = match crate::reference_compare::should_build_harness(
    crate::config::dual_call_harness_enabled(),
    std::env::var("APP_ENV").ok().as_deref(),
) {
    Ok(v) => v,
    Err(misconfiguration) => {
        tracing::error!("{misconfiguration}; skipping reference compare harness");
        return None;
    }
};

use visibility_filtering::reference_compare::should_build_harness;

fn harness_config_ok() -> Result<bool, String> {
    should_build_harness(
        dual_call_harness_enabled(),
        std::env::var("APP_ENV").ok().as_deref(),
    )
}
// call before build_prod_server(); log/abort on Err before the panic path

let harness = std::panic::catch_unwind(|| build_reference_compare_harness(/* deps */))
    .unwrap_or_else(|_| { tracing::error!("harness misconfigured; starting without it"); None });

When it happens

Trigger: Deploying with the dual-call harness flag enabled but an APP_ENV value that should_build_harness considers invalid or disallowed (e.g. prod, or an unrecognized env name); or enabling the flag in an environment where dual-calling is forbidden. Also hit by the test reference_compare_harness_not_built_without_flag exercising these paths.

Common situations: Enabling dual_call_harness in production by accident, setting APP_ENV to an unexpected string, stale flag toggles from a canary config leaking to prod pods.

Related errors

• {ENV_CACHE_WARM_SAMPLE_PCT}: {error}
• Failed to create consumer for thread {}: {:#}
• async_emb requires ${name}=${expected} before process startu
• API key is required: set EapiModelConfig.api_key or the XAI_
• Not implemented: to_thrift for KeywordMatch

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/7509209abce5343c. Report an issue: GitHub.