astrid-runtime/astrid · error · GatewayError::Internal

registry did not respond

Error message

registry did not respond

What it means

Inside `registry_round_trip`'s bounded wait loop, if the absolute deadline has already elapsed (or has zero remaining budget) before awaiting the reply, the function short-circuits with `GatewayError::Internal('registry did not respond')` instead of calling `recv(Some(ZERO))`. It means the caller's time budget for the registry reply ran out before any reply was received.

Source

Thrown at crates/astrid-gateway/src/routes/models.rs:269

    // foreign `corr_id` (another concurrent same-principal SET's reply). The
    // budget is an absolute DEADLINE, not a per-iteration timeout: each
    // `recv` is bounded by the time REMAINING, so a stream of skipped foreign
    // replies can never extend the total wait past the original budget.
    let timeout = state.registry_timeout.unwrap_or(REGISTRY_TIMEOUT);
    // `checked_add` over the bare `+` so an absurd timeout can't panic on
    // overflow; saturating to `now` (a zero remaining budget) on overflow is
    // a harmless immediate timeout that the production budget never hits.
    let deadline = tokio::time::Instant::now()
        .checked_add(timeout)
        .unwrap_or_else(tokio::time::Instant::now);
    loop {
        let remaining = deadline.saturating_duration_since(tokio::time::Instant::now());
        // No budget left (deadline already elapsed, or zero remaining): break
        // to the timeout path rather than calling `recv(Some(ZERO))`, whose
        // behaviour with a zero duration is implementation-defined. The
        // absolute deadline above keeps total wait bounded regardless.
        if remaining.is_zero() {
            return Err(GatewayError::Internal(anyhow::anyhow!(
                "registry did not respond"
            )));
        }
        let Some(event) = reply_rx.recv(Some(remaining)).await else {
            return Err(GatewayError::Internal(anyhow::anyhow!(
                "registry did not respond"
            )));
        };
        let AstridEvent::Ipc { message, .. } = &*event else {
            return Err(GatewayError::Internal(anyhow::anyhow!(
                "registry reply was not an IPC message"
            )));
        };
        if !expected_source_ids.contains(&message.source_id) {
            continue;
        }
        // Extract the guest-facing payload. Capsule `publish_json` arrives as
        // `Custom { data }` when the JSON has no known IPC `type`; using the

View on GitHub (pinned to affd8760f4)

Solutions

1. Increase the registry round-trip deadline/timeout if it is tighter than realistic registry latency.
2. Verify the registry capsule is loaded and subscribed to the request topic (see the related 'no loaded capsule handles the registry request' error).
3. Check registry load/health; a slow or wedged capsule delays replies past the deadline.
4. Confirm request/response topic names and correlation IDs match so replies are not discarded as unexpected.

Example fix

// before
if remaining.is_zero() {
    return Err(GatewayError::Internal(anyhow::anyhow!("registry did not respond")));
}
// after
if remaining.is_zero() {
    tracing::warn!(%request_topic, "registry reply deadline elapsed");
    return Err(GatewayError::Timeout("registry did not respond within deadline".into()));
}

let deadline_budget = registry_deadline_config();
if deadline_budget < MIN_REGISTRY_DEADLINE {
    return Err(configError("registry deadline too small"));
}

match list_models(&state, caller).await {
    Err(e) if e.message().contains("registry did not respond") => {
        tokio::time::sleep(BACKOFF).await;
        list_models(&state, caller).await // one retry before surfacing
    }
    r => r,
}

When it happens

Trigger: The registry reply did not arrive within the configured deadline, and by the time the loop re-checked, `deadline.saturating_duration_since(now)` was zero — typically after one or more unrelated/late events were consumed from `reply_rx` or the registry was simply slow.

Common situations: Registry capsule slow to load or busy; high gateway load starving the reply channel; too-short request timeout configured; wrong request topic so no subscriber ever answers and the deadline expires.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors

• registry reply not JSON: {e}
• registry reply was not an IPC message
• no loaded capsule handles the registry request for caller
• timed out waiting for capsule command result
• an Astrid daemon appears to be running but its uplink is unr

AI-assisted analysis of astrid-runtime/astrid@affd8760f4 (2026-09-09). Data as JSON: /api/errors/ede6d7aa915e2b53. Report an issue: GitHub.