influxdata/influxdb · error · Error

The service is currently unavailable

Error message

The service is currently unavailable: {0}

What it means

This Error variant maps a gRPC Status with code UNAVAILABLE into "The service is currently unavailable: {0}". It indicates the IOx service is temporarily unreachable or refusing work — the server-side analogue of HTTP 503. The From<Status> conversion in core/influxdb_iox_client/src/client/error.rs produces it whenever the channel or server reports an unavailable status.

Solutions

1. Retry the request with exponential backoff and jitter — UNAVAILABLE is explicitly retryable in gRPC semantics.
2. Check server/cluster health (pods ready, listeners up) and recent deploy events.
3. Recreate the gRPC channel if the failure persists, as stale connections can keep returning unavailable statuses.
4. If load-shedding is the cause, throttle client write/query rate or add server capacity.

Example fix

// before: single attempt, no distinction
let resp = client.query(req).await?;
// after: retry only on Unavailable with backoff
let mut delay = Duration::from_millis(100);
let resp = loop {
    match client.query(req.clone()).await {
        Err(Error::Unavailable(server_err)) => {
            tokio::time::sleep(delay).await;
            delay = (delay * 2).min(Duration::from_secs(10));
        }
        other => break other?,
    }
};

// health-check the endpoint before the real request
client.health().await.expect("service should be reachable");

fn is_unavailable(e: &client::Error) -> bool {
    matches!(e, client::Error::Unavailable(_))
}

let res = retry_with_backoff(max_retries, jitter, || async {
    match client.query(req.clone()).await {
        Err(Error::Unavailable(_)) => Err(Retryable),
        other => other.map_err(|e| e.into()).map_err(Fatal),
    }
}).await;

When it happens

Trigger: Server restarting or being drained during a deploy; gRPC channel to the IOx namespace failing to connect; server shedding load and returning UNAVAILABLE; network interruption between client and server.

Common situations: Rolling restarts of an IOx cluster; Kubernetes pod rescheduling while a write is in flight; overload/adaptive-throttling messages from the server; DNS or service mesh temporarily unable to route to the backend.

Related errors

• An unexpected error occurred in the client library
• Internal error
• Operation is not implemented or supported
• Received an invalid response from the server
• The request does not have valid authentication credentials

AI-assisted analysis of influxdata/influxdb@06200ef96b (2026-09-19). Data as JSON: /api/errors/a797bb625c098855. Report an issue: GitHub.

Appendix: source

Thrown at core/influxdb_iox_client/src/client/error.rs:105

    #[error("Some resource has been exhausted: {0}")]
    ResourceExhausted(ServerError<()>),

    #[error("The system is not in a state required for the operation's execution: {0}")]
    FailedPrecondition(Box<ServerError<PreconditionViolation>>),

    #[error("The operation was aborted: {0}")]
    Aborted(ServerError<()>),

    #[error("Operation was attempted past the valid range: {0}")]
    OutOfRange(ServerError<()>),

    #[error("Operation is not implemented or supported: {0}")]
    Unimplemented(ServerError<()>),

    #[error("Internal error: {0}")]
    Internal(ServerError<()>),

    #[error("The service is currently unavailable: {0}")]
    Unavailable(ServerError<()>),

    #[error("Unrecoverable data loss or corruption: {0}")]
    DataLoss(ServerError<()>),

    #[error("The request does not have valid authentication credentials: {0}")]
    Unauthenticated(ServerError<()>),

    #[error("Received an invalid response from the server: {0}")]
    InvalidResponse(#[from] FieldViolation),

    #[error("An unexpected error occurred in the client library: {0}")]
    Client(StdError),
}

impl From<Status> for Error {
    fn from(s: Status) -> Self {
        match s.code() {

View on GitHub (pinned to 06200ef96b)