rustfs/rustfs · error · HeartbeatError

Connect returned an invalid heartbeat response

Error message

Connect returned an invalid heartbeat response

What it means

The server returned 200 OK but the body is not a valid heartbeat response (heartbeat.rs:137-148): JSON deserialization into HeartbeatResponse failed, accepted_version != "v1", more than 32 capability hints, any hint longer than 32 chars, or server_time is not an exact RFC3339 UTC-seconds timestamp (is_exact_utc_seconds, telemetry.rs:195-201 -- must end in Z with zero offset and second precision).

Source

Thrown at rustfs/src/connect/heartbeat.rs:406

    StateIo {
        path: PathBuf,
        #[source]
        source: io::Error,
    },
    #[error("Connect heartbeat state at {path} is invalid: {source}")]
    StateInvalid {
        path: PathBuf,
        #[source]
        source: serde_json::Error,
    },
    #[error("Connect heartbeat state at {path} violates the protocol invariants")]
    StateCorrupt { path: PathBuf },
    #[cfg(unix)]
    #[error("Connect heartbeat state at {path} has mode {mode:o}, expected {expected:o}")]
    StatePermissions { path: PathBuf, mode: u32, expected: u32 },
    #[error("Connect heartbeat response exceeded 64 KiB")]
    ResponseTooLarge,
    #[error("Connect returned an invalid heartbeat response")]
    Response,
    #[error(transparent)]
    Url(#[from] url::ParseError),
    #[error(transparent)]
    Transport(#[from] reqwest::Error),
    #[error(transparent)]
    Identity(#[from] IdentityError),
    #[error(transparent)]
    IdentityStore(#[from] StoreError),
    #[error(transparent)]
    CredentialStore(#[from] CredentialStoreError),
    #[error(transparent)]
    CredentialValidation(#[from] CredentialValidationError),
}

impl From<TelemetryError> for HeartbeatError {
    fn from(error: TelemetryError) -> Self {
        match error {

View on GitHub (pinned to 201c653dcd)

Solutions

1. Capture the raw response body (debug proxy or server-side logs) to see which of the five checks failed
2. Confirm the Connect backend protocol version matches the agent (v1)
3. Remove or bypass any middlebox that rewrites response bodies

fn is_invalid_response(err: &HeartbeatError) -> bool {
    matches!(err, HeartbeatError::Response)
}

Err(HeartbeatError::Response) => {
    // 200 with a bad envelope: capture the body via a debug proxy to see
    // which check failed (version, hints, or server_time format).
    capture_response_body(&endpoint).await;
    alert("Connect returned an invalid heartbeat response");
}

When it happens

Trigger: A proxy returning 200 with an HTML body; a Connect control plane speaking a different protocol version; a server bug emitting millisecond timestamps or non-UTC offsets.

Common situations: Version skew between rustfs-agent and the Connect backend; middleboxes synthesizing 200 responses; backend regressions in the response envelope.

Related errors

• state path has no parent
• state filename is invalid
• Connect heartbeat schedule is invalid
• RustFS is not registered with Connect
• the stored Connect device certificate is not currently valid

AI-assisted analysis of rustfs/rustfs@201c653dcd (2026-08-23). Data as JSON: /api/errors/1bae583084d0ae74. Report an issue: GitHub.