What causes "unsupported selected UDP packet codec: %s" in fatedier/frp?

Have the server use selectUDPPacketCodec() which only ever returns "binary" or "" — do not hand-pick codec names.

unsupported selected UDP packet codec: %s

Error message

unsupported selected UDP packet codec: %s

What it means

Thrown by ValidateServerHelloForClient when the ServerHello selects a UDP packet codec other than "" (none) or UDPPacketCodecBinary. Binary is currently the only UDP packet codec implemented, so any other value in Selected.Message.UDPPacketCodec is rejected before the session proceeds.

Source

Thrown at pkg/proto/wire/crypto.go:99

func ValidateCryptoCapabilities(c CryptoCapabilities) error {
	if len(c.ClientRandom) != CryptoRandomSize {
		return fmt.Errorf("invalid crypto client random length %d, want %d", len(c.ClientRandom), CryptoRandomSize)
	}
	if _, ok := SelectAEADAlgorithm(c.Algorithms); !ok {
		return fmt.Errorf("no supported crypto algorithm")
	}
	return nil
}

func ValidateServerHelloForClient(clientHello ClientHello, serverHello ServerHello) error {
	if serverHello.Selected.Message.Codec != MessageCodecJSON {
		return fmt.Errorf("unsupported selected message codec: %s", serverHello.Selected.Message.Codec)
	}
	udpPacketCodec := serverHello.Selected.Message.UDPPacketCodec
	if udpPacketCodec != "" {
		if udpPacketCodec != UDPPacketCodecBinary {
			return fmt.Errorf("unsupported selected UDP packet codec: %s", udpPacketCodec)
		}
		if !Supports(clientHello.Capabilities.Message.UDPPacketCodecs, udpPacketCodec) {
			return fmt.Errorf("selected UDP packet codec was not advertised by client: %s", udpPacketCodec)
		}
	}
	cryptoSelection := serverHello.Selected.Crypto
	if !IsSupportedAEADAlgorithm(cryptoSelection.Algorithm) {
		return fmt.Errorf("unknown selected crypto algorithm: %s", cryptoSelection.Algorithm)
	}
	if !Supports(clientHello.Capabilities.Crypto.Algorithms, cryptoSelection.Algorithm) {
		return fmt.Errorf("selected crypto algorithm was not advertised by client: %s", cryptoSelection.Algorithm)
	}
	if len(cryptoSelection.ServerRandom) != CryptoRandomSize {
		return fmt.Errorf("invalid crypto server random length %d, want %d", len(cryptoSelection.ServerRandom), CryptoRandomSize)
	}
	return nil
}

View on GitHub (pinned to 6c8a8d0a97)

Solutions

Have the server use selectUDPPacketCodec() which only ever returns "binary" or "" — do not hand-pick codec names.
Upgrade/downgrade both peers to the same frp release so the codec vocabulary matches.
Fix test fixtures to leave UDPPacketCodec empty or set it to wire.UDPPacketCodecBinary.

Example fix

// before
serverHello.Selected.Message.UDPPacketCodec = "raw"

// after
serverHello.Selected.Message.UDPPacketCodec = wire.UDPPacketCodecBinary // or "" for none

Defensive patterns

Strategy: validation

Validate before calling

codec := serverHello.Selected.Message.UDPPacketCodec
if codec != "" && codec != wire.UDPPacketCodecBinary {
    return fmt.Errorf("refusing handshake: UDP packet codec %q unsupported", codec)
}

Try / catch

if err := wire.NewClientCryptoContext(clientRaw, serverRaw); err != nil {
    if strings.Contains(err.Error(), "UDP packet codec") {
        // protocol mismatch — fail fast, close conn, no retry
    }
    return err
}

Prevention

Servers must select UDP packet codecs only through selectUDPPacketCodec semantics (binary or none).
Keep hello fixtures generated by shared helpers, not copied structs.

When it happens

Trigger: Server sends ServerHello with Selected.Message.UDPPacketCodec set to a non-empty value that is not "binary" (e.g. a future codec name or a typo), while validating the hello on the client via NewClientCryptoContext.

Common situations: Version skew where a newer server supports an additional UDP packet codec; mock servers in e2e tests that populate the field incorrectly; corrupted or tampered handshake payloads.

Related errors

AI-assisted analysis of fatedier/frp@6c8a8d0a97 (2026-08-15). Data as JSON: /api/errors/d9031193d187ee43. Report an issue: GitHub.