shadow1ng/fscan · critical

failed to get server public key

Error message

failed to get server public key

What it means

During the RDP secure handshake in sec/sec.go, the client extracts the RSA public key from the server's X.509 certificate (ServerSecurityData.ServerCertificate.CertData.GetPublicKey). If extraction fails or returns nil, the client emits 'failed to get server public key' and aborts the connection, since it cannot encrypt the client random with PKCS1v15.

Source

Thrown at libs/grdp/protocol/sec/sec.go:696

	if err != nil {
		glog.Error("generateKeys failed:", err)
		c.Emit("error", err)
		return false
	}

	//initialize keys
	c.currentDecrytKey = c.initialDecrytKey
	c.currentEncryptKey = c.initialEncryptKey

	//verify certificate
	if !c.ServerSecurityData().ServerCertificate.CertData.Verify() {
		glog.Warn("Cannot verify server identity")
	}

	serverPubKey, err := c.ServerSecurityData().ServerCertificate.CertData.GetPublicKey()
	if err != nil || serverPubKey == nil {
		glog.Error("GetPublicKey failed:", err)
		c.Emit("error", errors.New("failed to get server public key"))
		return false
	}
	ret, err := rsa.EncryptPKCS1v15(rand.Reader, serverPubKey, core.Reverse(clientRandom))
	if err != nil {
		glog.Error("EncryptPKCS1v15 err:", err)
		c.Emit("error", err)
		return false
	}
	message := ClientSecurityExchangePDU{}
	message.EncryptedClientRandom = core.Reverse(ret)
	message.Length = uint32(len(message.EncryptedClientRandom) + 8)
	message.Padding = make([]byte, 8)

	glog.Debug("message:", message)

	c.sendFlagged(EXCHANGE_PKT, message.serialize())
	return true
}

View on GitHub (pinned to 95cc12e753)

Solutions

1. Inspect the server certificate bytes: run GetPublicKey against the CertData manually and check the error to see whether it's a parse failure or a nil key.
2. Ensure the client negotiates standard TLS/SSL (PROTOCOL_SSL or HYBRID) so the server sends a full X.509 certificate rather than a proprietary one.
3. Verify the ServerSecurityData unpacked correctly — log CertData length and dwVersion; a short read upstream yields an empty cert.
4. If the cert uses an EC/non-RSA key, add key-type handling in GetPublicKey or connect with encryption negotiated to match (the EncryptPKCS1v15 path requires RSA).
5. Retry the connection once; intermittent truncation can produce empty CertData.

Example fix

// before
serverPubKey, err := c.ServerSecurityData().ServerCertificate.CertData.GetPublicKey()
if err != nil || serverPubKey == nil {
    glog.Error("GetPublicKey failed:", err)
    c.Emit("error", errors.New("failed to get server public key"))
    return false
}
// after
serverPubKey, err := c.ServerSecurityData().ServerCertificate.CertData.GetPublicKey()
if err != nil {
    glog.Error("GetPublicKey failed:", err)
    c.Emit("error", fmt.Errorf("failed to get server public key: %w", err))
    return false
}
if serverPubKey == nil {
    c.Emit("error", errors.New("failed to get server public key: empty cert data"))
    return false
}

// pre-connect sanity check on the certificate blob
cert := c.ServerSecurityData().ServerCertificate.CertData
if cert == nil || len(cert) == 0 {
    return errors.New("server returned empty certificate data; check security protocol negotiation")
}

func hasServerPubKey(c *grdpClient) bool {
    key, err := c.ServerSecurityData().ServerCertificate.CertData.GetPublicKey()
    return err == nil && key != nil
}

client.On("error", func(err error) {
    if strings.Contains(err.Error(), "failed to get server public key") {
        glog.Error("server cert unusable; check TLS/NLA negotiation and cert format: ", err)
        return
    }
    glog.Error("rdp error: ", err)
})

When it happens

Trigger: Server's cert blob (PROPRIARYSERVERCERT / X.509 CertData) is malformed, uses an unsupported certificate type (e.g. proprietary format without a parseable key), contains a non-RSA key, or GetPublicKey returns (nil, nil) for an empty/truncated cert body — all reached during the ClientInfoPDU/security exchange after receiving the server security data.

Common situations: Connecting to a server with an unusual or self-signed cert chain the parser mishandles; NLA/TLS negotiation mismatch leaving the security header fields wrong; older servers sending a proprietary server certificate (dwVersion 0x00000001) that GetPublicKey can't parse; truncated cert data over flaky networks.

Related errors

• empty certificate chain
• Not a valid license packet
• Unsupported version
• NODE_RDP_PROTOCOL_T125_MCS_WAIT_CHANNEL_JOIN_CONFIRM
• NODE_RDP_PROTOCOL_T125_MCS_INVALID_USER_ID

AI-assisted analysis of shadow1ng/fscan@95cc12e753 (2026-09-06). Data as JSON: /api/errors/3b635aaf9c2dea55. Report an issue: GitHub.