What causes "Unexpected message {:?} while waiting for handshake" in neondatabase/neon?

Use a real libpq client (psql, tokio-postgres) which sends the password message when AuthenticationCleartextPassword is received

neondatabase/neon · error · QueryError

Unexpected message {:?} while waiting for handshake

Error message

Unexpected message {:?} while waiting for handshake

What it means

During the postgres wire-protocol handshake, after the server has sent AuthenticationCleartextPassword (NeonJWT auth), the next message must be a PasswordMessage carrying the JWT. If some other FeMessage arrives (Query, Terminate, a second startup packet, ...), PostgresBackend raises QueryError::Other with this message; the connection is unusable afterwards.

Source

Thrown at libs/postgres_backend/src/lib.rs:635

                    let (_, jwt_response) = m.split_last().context("protocol violation")?;

                    if let Err(e) = handler.check_auth_jwt(self, jwt_response) {
                        self.write_message_noflush(&BeMessage::ErrorResponse(
                            &short_error(&e),
                            Some(e.pg_error_code()),
                        ))?;
                        return Err(e);
                    }

                    self.write_message_noflush(&BeMessage::AuthenticationOk)?
                        .write_message_noflush(&BeMessage::CLIENT_ENCODING)?
                        .write_message(&BeMessage::ReadyForQuery)
                        .await?;
                    self.state = ProtoState::Established;
                }
                Some(m) => {
                    return Err(QueryError::Other(anyhow::anyhow!(
                        "Unexpected message {:?} while waiting for handshake",
                        m
                    )));
                }
                None => {
                    trace!(
                        "postgres backend to {:?} received EOF during auth",
                        self.peer_addr
                    );
                    self.state = ProtoState::Closed;
                    return Err(QueryError::Disconnected(ConnectionError::Protocol(
                        ProtocolError::Protocol("EOF during auth".to_string()),
                    )));
                }
            }
        }

        Ok(())

View on GitHub (pinned to 8f60b04da4)

Solutions

Use a real libpq client (psql, tokio-postgres) which sends the password message when AuthenticationCleartextPassword is received
Verify you are connecting to the intended endpoint and auth mode (trust vs NeonJWT) for this listener
If writing a custom client, follow the flow: Startup -> AuthenticationCleartextPassword -> PasswordMessage(jwt)
Capture the traffic and check what message replaced the password response (the {:?} in the log names it)

Defensive patterns

Strategy: try-catch

Try / catch

match backend.handshake(&mut handler).await {
    Err(QueryError::Other(e))
        if e.to_string().contains("while waiting for handshake") =>
    {
        // Wrong message order from the client; connection is unusable.
        // The {:?} names the offending message -- log it, close, do not retry on this socket.
        tracing::warn!(peer = ?peer_addr, error = %e, "handshake protocol violation");
        // server already sent / should send an ErrorResponse before closing
    }
    other => other?,
}

Prevention

Use battle-tested clients (libpq, tokio-postgres) which follow AuthenticationCleartextPassword with exactly one PasswordMessage
In custom drivers, encode the auth state machine explicitly and forbid Query before AuthenticationOk
Have proxies pass messages through unmodified during handshake -- never re-inject startup packets
Include an auth-flow integration test (startup -> password -> query) for every new client wrapper

When it happens

Trigger: A client that skips the password exchange and sends a Query directly after startup; a client restarting the handshake (new SSLRequest/GssEncRequest) while in the Authentication state; a broken or non-libpq client speaking the wrong message order; protocol desync after partial writes.

Common situations: Custom drivers or test harnesses that assume trust auth on an endpoint requiring JWT; pgbouncers or proxies re-injecting startup packets mid-auth; version-mismatched mock clients in CI; security scanners sending malformed sequences.

Understand the failure class

SSL/TLS and certificate errors — how TLS handshakes and certificate validation fail.

Related errors

AI-assisted analysis of neondatabase/neon@8f60b04da4 (2026-08-16). Data as JSON: /api/errors/08410c95d4c3eab8. Report an issue: GitHub.