slackhq/nebula · error

unable to unmarshal pubkey: %w

Error message

unable to unmarshal pubkey: %w

What it means

This error is returned by the pkcs11 curve's DH method when the peer public key fails c.curve.NewPublicKey parsing. Note the method first checks if the local private key is a 'pkcs11:' URI; if not it delegates to DHP256.DH, so this specific error is only reached when a pkcs11: key IS in use and the remote public key bytes are invalid for the curve. The underlying cause is wrapped with %w.

Source

Thrown at noiseutil/pkcs11.go:36

}

func newNISTP11Curve(name string, curve ecdh.Curve, byteLen int) nistP11Curve {
	return nistP11Curve{
		newNISTCurve(name, curve, byteLen),
	}
}

func (c nistP11Curve) DH(privkey, pubkey []byte) ([]byte, error) {
	//for this function "privkey" is actually a pkcs11 URI
	pkStr := string(privkey)

	//to set up a handshake, we need to also do non-pkcs11-DH. Handle that here.
	if !strings.HasPrefix(pkStr, "pkcs11:") {
		return DHP256.DH(privkey, pubkey)
	}
	ecdhPubKey, err := c.curve.NewPublicKey(pubkey)
	if err != nil {
		return nil, fmt.Errorf("unable to unmarshal pubkey: %w", err)
	}

	//this is not the most performant way to do this (a long-lived client would be better)
	//but, it works, and helps avoid problems with stale sessions and HSMs used by multiple users.
	client, err := pkclient.FromUrl(pkStr)
	if err != nil {
		return nil, err
	}
	defer func(client *pkclient.PKClient) {
		_ = client.Close()
	}(client)

	return client.DeriveNoise(ecdhPubKey.Bytes())
}

View on GitHub (pinned to dd8f660c0a)

Solutions

1. Verify the peer's public key was read correctly from its source (cert/slot) and matches the expected curve
2. Confirm both peers use the same curve despite the HSM path
3. Log the wrapped cause (errors.Unwrap) to distinguish length vs point-validity errors
4. If the pubkey came from a pkcs11 object listing, re-export it and compare byte length against the curve's expected point size

Example fix

// before: trusting pubkey bytes straight from an external source
res, err := pkcs11Curve.DH(pkcs11URI, remotePub)
// after: sanity-check before DH
if len(remotePub) != 65 { // P-256 uncompressed point
    return nil, fmt.Errorf("remote pubkey has %d bytes, want 65", len(remotePub))
}
res, err := pkcs11Curve.DH(pkcs11URI, remotePub)

func dhGuard(pkStr string, pub []byte, curveLen int) error {
    if len(pub) != curveLen {
        return fmt.Errorf("remote pubkey %d bytes, want %d", len(pub), curveLen)
    }
    if strings.HasPrefix(pkStr, "pkcs11:") {
        u, err := url.Parse(pkStr)
        if err != nil || u.Scheme != "pkcs11" {
            return fmt.Errorf("malformed pkcs11 URI")
        }
    }
    return nil
}

func isPkcs11PubError(err error) bool {
    return err != nil && strings.Contains(err.Error(), "unable to unmarshal pubkey")
}

res, err := pkcs11Curve.DH(pkStr, pub)
if err != nil {
    if isPkcs11PubError(err) {
        return fmt.Errorf("remote key invalid for pkcs11 DH (verify peer cert/slot export): %w", err)
    }
    return err
}

When it happens

Trigger: Performing a Noise handshake where the local private key is a 'pkcs11:' URI (HSM-backed) and the remote peer's public key is empty, wrong-length, or not a valid EC point on the configured curve.

Common situations: HSM-backed handshake where the peer's fetched public key was corrupted, fetched from the wrong slot/object, or the peer is on a different curve; stale certificate/key material deployed alongside an HSM migration.

Related errors

• got a key of %d bytes, expected %d
• unable to unmarshal pubkey: %w
• unable to unmarshal private key: %w
• got an empty secret key
• notImplemented

AI-assisted analysis of slackhq/nebula@dd8f660c0a (2026-09-03). Data as JSON: /api/errors/ed3e276a11bc6d90. Report an issue: GitHub.