OpenNHP/opennhp · error

extractInitiatorStaticPubKey: curve scheme expected

Error message

extractInitiatorStaticPubKey: curve scheme expected %d-byte pubkey, got %d

What it means

After successfully AEAD-opening the static field, extractInitiatorStaticPubKey validates that the plaintext length matches the public key size declared by the header's cipher scheme. For CIPHER_SCHEME_CURVE it must be exactly PublicKeySize (32 bytes). A mismatch means the decrypted plaintext is not a Curve25519 public key — the scheme field in the header does not match the actual key material the initiator encrypted.

Solutions

1. Ensure the agent sets the header's CipherScheme to the same constant it used for key generation/ECDH (CURVE keys → CIPHER_SCHEME_CURVE).
2. Log the received length and scheme at the sender side before sending to confirm they agree.
3. Align agent and server on the same opennhp version so PublicKeySize and the scheme table match.
4. If a new cipher was added, extend the switch in extractInitiatorStaticPubKey to its own case instead of reusing CURVE.

Example fix

// before (sender)
hdr.SetCipherScheme(common.CIPHER_SCHEME_CURVE) // but key is SM2 65-byte
// after
hdr.SetCipherScheme(common.CIPHER_SCHEME_GMSM) // matches key size

if header.CipherScheme() == common.CIPHER_SCHEME_CURVE && senderKeyType != keyTypeCurve {
	return fmt.Errorf("sender key type does not match CURVE scheme")
}

func schemeMatchesKey(scheme int, pk []byte) bool {
	if scheme == common.CIPHER_SCHEME_CURVE {
		return len(pk) == core.PublicKeySize
	}
	return len(pk) == core.PublicKeySizeEx
}

peerPk, err := extractInitiatorStaticPubKey(dev, ciphers, header)
if err != nil {
	if strings.Contains(err.Error(), "expected "+strconv.Itoa(core.PublicKeySize)+"-byte") {
		log.Error("scheme/keysize mismatch — check agent build vs server build")
	}
	return err
}

When it happens

Trigger: A sender encrypts a GMSM/SM2 public key but stamps the header with CIPHER_SCHEME_CURVE (sender/receiver scheme tables out of sync); a custom or future cipher writes a different plaintext length; header CipherScheme bits corrupted while the GCM tag still passed (unlikely) — realistically a scheme mismatch between agent build and server build.

Common situations: Mixed-version deployments where the agent and server disagree on the scheme→keysize mapping; custom forks that added a third cipher scheme and reused the CURVE constant; hand-crafted packets from tooling that builds headers manually.

Related errors

• extractInitiatorStaticPubKey: gmsm scheme expected
• missing connection data for server
• missing remote address
• keepalive packet size is incorrect
• packet header type does not match device

AI-assisted analysis of OpenNHP/opennhp@6e04ca5ff0 (2026-09-07). Data as JSON: /api/errors/2f9b6dbf0582eb68. Report an issue: GitHub.

Appendix: source

Thrown at nhp/core/responder.go:192

	// header.CipherScheme(); that worked because the only two ciphers
	// today happen to match the scheme→size table exactly, but a
	// future cipher whose plaintext length doesn't fit either fixed
	// size would silently mis-key the cookie HMAC (Open writes
	// however many bytes the AEAD decrypted, then the caller would
	// either truncate them or hash trailing zero-padding).
	//
	// Validate the length explicitly before returning so future
	// breakage manifests as an error here, not as cookie failures
	// further down. Pass nil for the dst so Open allocates exactly
	// the right size.
	peerPk, err := aead.Open(nil, header.NonceBytes(), header.StaticBytes(), chainHash.Sum(nil))
	if err != nil {
		return nil, fmt.Errorf("extractInitiatorStaticPubKey: open: %w", err)
	}
	switch header.CipherScheme() {
	case common.CIPHER_SCHEME_CURVE:
		if len(peerPk) != PublicKeySize {
			return nil, fmt.Errorf("extractInitiatorStaticPubKey: curve scheme expected %d-byte pubkey, got %d", PublicKeySize, len(peerPk))
		}
	case common.CIPHER_SCHEME_GMSM:
		if len(peerPk) != PublicKeySizeEx {
			return nil, fmt.Errorf("extractInitiatorStaticPubKey: gmsm scheme expected %d-byte pubkey, got %d", PublicKeySizeEx, len(peerPk))
		}
	default:
		return nil, fmt.Errorf("extractInitiatorStaticPubKey: unknown cipher scheme %d (pubkey length %d)", header.CipherScheme(), len(peerPk))
	}
	return peerPk, nil
}

type ResponderScheme interface {
	CreatePacketParserData(d *Device, pd *PacketData) (ppd *PacketParserData, err error)
	DerivePacketParserDataFromPrevAssemblerData(mad *MsgAssemblerData, pkt *Packet, initTime int64) (ppd *PacketParserData)
	validatePeer(d *Device, ppd *PacketParserData) (err error)
	decryptBody(d *Device, ppd *PacketParserData) (err error)
}

View on GitHub (pinned to 6e04ca5ff0)