fatedier/frp · error

unsupported selected message codec: %s

Error message

unsupported selected message codec: %s

What it means

Thrown by ValidateServerHelloForClient when the ServerHello's selected message codec is not MessageCodecJSON ("json"). The wire protocol currently supports JSON as the only message codec, so any other selection is a protocol violation. It fires on the client side after receiving the server's hello during handshake validation.

Source

Thrown at pkg/proto/wire/crypto.go:94

				ServerRandom: serverRandom,
			},
		},
	}, nil
}

func ValidateCryptoCapabilities(c CryptoCapabilities) error {
	if len(c.ClientRandom) != CryptoRandomSize {
		return fmt.Errorf("invalid crypto client random length %d, want %d", len(c.ClientRandom), CryptoRandomSize)
	}
	if _, ok := SelectAEADAlgorithm(c.Algorithms); !ok {
		return fmt.Errorf("no supported crypto algorithm")
	}
	return nil
}

func ValidateServerHelloForClient(clientHello ClientHello, serverHello ServerHello) error {
	if serverHello.Selected.Message.Codec != MessageCodecJSON {
		return fmt.Errorf("unsupported selected message codec: %s", serverHello.Selected.Message.Codec)
	}
	udpPacketCodec := serverHello.Selected.Message.UDPPacketCodec
	if udpPacketCodec != "" {
		if udpPacketCodec != UDPPacketCodecBinary {
			return fmt.Errorf("unsupported selected UDP packet codec: %s", udpPacketCodec)
		}
		if !Supports(clientHello.Capabilities.Message.UDPPacketCodecs, udpPacketCodec) {
			return fmt.Errorf("selected UDP packet codec was not advertised by client: %s", udpPacketCodec)
		}
	}
	cryptoSelection := serverHello.Selected.Crypto
	if !IsSupportedAEADAlgorithm(cryptoSelection.Algorithm) {
		return fmt.Errorf("unknown selected crypto algorithm: %s", cryptoSelection.Algorithm)
	}
	if !Supports(clientHello.Capabilities.Crypto.Algorithms, cryptoSelection.Algorithm) {
		return fmt.Errorf("selected crypto algorithm was not advertised by client: %s", cryptoSelection.Algorithm)
	}
	if len(cryptoSelection.ServerRandom) != CryptoRandomSize {

View on GitHub (pinned to 6c8a8d0a97)

Solutions

  1. Ensure the server only selects codecs the client advertised; the server should echo MessageCodecJSON from the client's ClientHello.
  2. Fix hand-built ServerHello fixtures in tests to set Selected.Message.Codec = MessageCodecJSON.
  3. Run matching frp versions on both ends of the connection.

Example fix

// before (test/mock)
serverHello.Selected.Message.Codec = "msgpack"

// after
serverHello.Selected.Message.Codec = wire.MessageCodecJSON
Defensive patterns

Strategy: validation

Validate before calling

if serverHello.Selected.Message.Codec != wire.MessageCodecJSON {
    // reject before building crypto context
    return fmt.Errorf("bad codec %q", serverHello.Selected.Message.Codec)
}

Try / catch

if err := wire.ValidateServerHelloForClient(clientHello, serverHello); err != nil {
    // treat connection as untrusted/broken: close, do not retry on same conn
    conn.Close()
    return err
}

Prevention

When it happens

Trigger: A malicious, buggy, or version-mismatched server sends ServerHello with Selected.Message.Codec set to anything other than "json" (or a codec string the client does not know). NewClientCryptoContext decodes both hello payloads and calls ValidateServerHelloForClient, which raises this error.

Common situations: Connecting a client to a server from a different frp version that introduced a new message codec; test mocks that build ServerHello by hand and forget to set the codec field; a man-in-the-middle or corrupted connection producing garbage JSON that decodes to a zero/unknown codec.

Related errors

AI-assisted analysis of fatedier/frp@6c8a8d0a97 (2026-08-15). Data as JSON: /api/errors/d20b20ba1aaa0b28. Report an issue: GitHub.