influxdata/influxdb · critical · Error

lost admin token recovery service

Error message

lost admin token recovery service

What it means

Variant of the serve command's Error enum in influxdb3. It is returned when the admin token recovery service task, spawned during server startup, terminates unexpectedly. The server treats loss of this service as fatal because admin token recovery is required for token management without a pre-existing admin token.

Solutions

1. Inspect server logs immediately preceding the error for a panic or listener failure in the admin token recovery service.
2. Verify the admin token recovery HTTP port is not already in use or blocked by firewall settings.
3. Upgrade to a newer influxdb3 release where recovery-service startup failures are reported with more detail.
4. Restart the server; if reproducible, file an issue with the logs and startup flags.

null

null

match server_handle.await {
    Err(Error::LostAdminTokenRecovery) => {
        log::error!("admin token recovery service died; restart and inspect logs");
        // attempt restart with backoff
    }
    Ok(v) => v,
    Err(e) => return Err(e),
}

When it happens

Trigger: The tokio task running the admin token recovery HTTP service panics, returns an error, or its JoinHandle resolves with a failure while `influxdb3 serve` is running.

Common situations: Runtime panics inside the recovery service handler, port binding or listener failures for the recovery endpoint, or shutdown races where the service future is dropped/aborted early.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• table cache index initialization failed
• tls config error
• ' ' is a reserved column
• auto field family exists
• cache error

AI-assisted analysis of influxdata/influxdb@06200ef96b (2026-09-19). Data as JSON: /api/errors/84152127f943ab2e. Report an issue: GitHub.

Appendix: source

Thrown at influxdb3/src/commands/serve.rs:159

    #[error("failed to initialize write buffer: {0:?}")]
    WriteBufferInit(#[source] anyhow::Error),

    #[error("failed to initialize catalog: {0}")]
    InitializeCatalog(#[source] CatalogError),

    #[error("failed to initialize last cache: {0}")]
    InitializeLastCache(#[source] last_cache::Error),

    #[error("failed to initialize distinct cache: {0:#}")]
    InitializeDistinctCache(#[source] influxdb3_cache::distinct_cache::ProviderError),

    #[error("lost backend")]
    LostBackend,

    #[error("lost HTTP/gRPC service")]
    LostHttpGrpc,

    #[error("lost admin token recovery service")]
    LostAdminTokenRecovery,

    #[error("tls requires both a cert and a key file to be passed in to work")]
    NoCertOrKeyFile,

    #[error("table cache index initialization failed: {0}")]
    TableIndexCacheInitialization(
        #[source] influxdb3_write::table_index_cache::TableIndexCacheError,
    ),

    #[error(
        "environment variable {0} (named by --node-id-from-env / INFLUXDB3_NODE_ID_FROM_ENV) \
        must be set to a valid node id"
    )]
    NodeIdEnvVarMissing(String),

    #[error(
        "Python environment initialization failed: {0}\nPlease ensure Python and pip package manager is installed"

View on GitHub (pinned to 06200ef96b)