slackhq/nebula · error

unable to unmarshal pubkey: %w

Error message

unable to unmarshal pubkey: %w

What it means

This error is returned by nistCurve.DH in noiseutil when the peer's public key bytes cannot be parsed as a valid public key on the configured NIST curve (P-256/P-384/P-521) via ecdh.Curve.NewPublicKey. It means the Noise handshake received a remote static/ephemeral public key that is not a valid, canonical point on the expected curve. The underlying ecdh error is wrapped with %w so the root cause is preserved.

Source

Thrown at noiseutil/nist.go:47

	}
}

func (c nistCurve) GenerateKeypair(rng io.Reader) (noise.DHKey, error) {
	if rng == nil {
		rng = rand.Reader
	}
	privkey, err := c.curve.GenerateKey(rng)
	if err != nil {
		return noise.DHKey{}, err
	}
	pubkey := privkey.PublicKey()
	return noise.DHKey{Private: privkey.Bytes(), Public: pubkey.Bytes()}, nil
}

func (c nistCurve) DH(privkey, pubkey []byte) ([]byte, error) {
	ecdhPubKey, err := c.curve.NewPublicKey(pubkey)
	if err != nil {
		return nil, fmt.Errorf("unable to unmarshal pubkey: %w", err)
	}
	ecdhPrivKey, err := c.curve.NewPrivateKey(privkey)
	if err != nil {
		return nil, fmt.Errorf("unable to unmarshal private key: %w", err)
	}

	return ecdhPrivKey.ECDH(ecdhPubKey)
}

func (c nistCurve) DHLen() int {
	// NOTE: Noise Protocol specifies "DHLen" to represent two things:
	// - The size of the public key
	// - The return size of the DH() function
	// But for standard NIST ECDH, the sizes of these are different.
	// Luckily, the flynn/noise library actually only uses this DHLen()
	// value to represent the public key size, so that is what we are
	// returning here. The length of the DH() return bytes are unaffected by
	// this value here.

View on GitHub (pinned to dd8f660c0a)

Solutions

  1. Verify both peers use the same NIST curve (e.g. 'P256' in the Noise config) so key lengths match
  2. Regenerate or re-export the offending key pair with the same library used for parsing (crypto/ecdh)
  3. Check that the public key bytes are the full uncompressed point size for the curve (65/97/133 bytes) and not a truncated or DER-encoded copy
  4. Inspect the wrapped %w cause with errors.Unwrap/errors.As to distinguish length vs point-validity failures

Example fix

// before: mixing curves across peers
key, _ := nistCurve{curve: ecdh.P256()}.DH(priv, peerP384Pub) // fails to unmarshal
// after: ensure both sides use the same curve
if len(pub) != expectedPubLen(curveName) {
    return nil, fmt.Errorf("peer pubkey wrong size for %s", curveName)
}
key, err := nistCurve{curve: ecdh.P256()}.DH(priv, pub)
Defensive patterns

Strategy: validation

Validate before calling

func validNistPub(curveName string, pub []byte) bool {
    var n int
    switch curveName {
    case "P256": n = 65
    case "P384": n = 97
    case "P521": n = 133
    default: return false
    }
    return len(pub) == n && pub[0] == 4
}

Type guard

func isEcdhUnmarshalError(err error) bool {
    var e *ecdh.PublicKey // NewPublicKey failure is generic error; check wrapped text
    _ = e
    return err != nil && strings.Contains(err.Error(), "unable to unmarshal pubkey")
}

Try / catch

shared, err := curve.DH(priv, pub)
if err != nil {
    var cause error
    errors.As(err, &cause)
    return fmt.Errorf("handshake DH failed (check peer curve/key bytes): %w", err)
}

Prevention

When it happens

Trigger: Calling nistCurve.DH(privkey, pubkey) with pubkey bytes that are empty, the wrong length for the curve, or not a valid EC point — e.g. a remote peer configured with a key pair generated on a different curve or with a non-NIST key format.

Common situations: Mismatched curve configuration between peers (one on P-256, other on P-384); corrupted or truncated keys stored in config/certs; keys generated by a different library with a different encoding; a peer sending garbage during a handshake attempt.

Related errors

AI-assisted analysis of slackhq/nebula@dd8f660c0a (2026-09-03). Data as JSON: /api/errors/b80b2d70829c2ced. Report an issue: GitHub.