golang/go · error

crypto/rsa: unsupported hash function

Error message

crypto/rsa: unsupported hash function

What it means

Raised in setupRSA (OAEP path) when hashToMD(h) returns nil — the hash.Hash concrete type is not one of the boring wrappers (sha1/sha224/sha256/sha384/sha512Hash). BoringCrypto can only drive OAEP with a hash it recognises, so an unrecognised hash implementation is rejected before the EVP context is configured.

Source

Thrown at src/crypto/internal/boring/rsa.go:149

	if withKey(func(key *C.GO_RSA) C.int {
		return C._goboringcrypto_EVP_PKEY_set1_RSA(pkey, key)
	}) == 0 {
		return pkey, ctx, fail("EVP_PKEY_set1_RSA")
	}
	ctx = C._goboringcrypto_EVP_PKEY_CTX_new(pkey, nil)
	if ctx == nil {
		return pkey, ctx, fail("EVP_PKEY_CTX_new")
	}
	if init(ctx) == 0 {
		return pkey, ctx, fail("EVP_PKEY_operation_init")
	}
	if C._goboringcrypto_EVP_PKEY_CTX_set_rsa_padding(ctx, padding) == 0 {
		return pkey, ctx, fail("EVP_PKEY_CTX_set_rsa_padding")
	}
	if padding == C.GO_RSA_PKCS1_OAEP_PADDING {
		md := hashToMD(h)
		if md == nil {
			return pkey, ctx, errors.New("crypto/rsa: unsupported hash function")
		}
		mgfMD := hashToMD(mgfHash)
		if mgfMD == nil {
			return pkey, ctx, errors.New("crypto/rsa: unsupported hash function")
		}
		if C._goboringcrypto_EVP_PKEY_CTX_set_rsa_oaep_md(ctx, md) == 0 {
			return pkey, ctx, fail("EVP_PKEY_set_rsa_oaep_md")
		}
		if C._goboringcrypto_EVP_PKEY_CTX_set_rsa_mgf1_md(ctx, mgfMD) == 0 {
			return pkey, ctx, fail("EVP_PKEY_set_rsa_mgf1_md")
		}
		// ctx takes ownership of label, so malloc a copy for BoringCrypto to free.
		clabel := (*C.uint8_t)(C._goboringcrypto_OPENSSL_malloc(C.size_t(len(label))))
		if clabel == nil {
			return pkey, ctx, fail("OPENSSL_malloc")
		}
		copy((*[1 << 30]byte)(unsafe.Pointer(clabel))[:len(label)], label)
		if C._goboringcrypto_EVP_PKEY_CTX_set0_rsa_oaep_label(ctx, clabel, C.size_t(len(label))) == 0 {

View on GitHub (pinned to b6b368adc5)

Solutions

1. Use one of SHA-1, SHA-224, SHA-256, SHA-384, or SHA-512 for OAEP under boringcrypto.
2. If a non-SHA-2 hash is required, use a non-boring Go build or a pure-Go RSA implementation.
3. Confirm the hash instance originates from the boring shim's expected constructors.

Example fix

// before (boringcrypto build)
err := rsa.EncryptOAEP(blake2b.New256, rand, pub, msg, label) // hashToMD -> nil

// after
err := rsa.EncryptOAEP(sha256.New, rand, pub, msg, label)

func isBoringOAEPHash(h hash.Hash) bool {
    switch h.(type) {
    case *sha1Hash, *sha224Hash, *sha256Hash, *sha384Hash, *sha512Hash:
        return true
    }
    return false
}

When it happens

Trigger: Calling RSA OAEP encrypt/decrypt through the boring backend with an h hash.Hash whose concrete type is not one of the five supported boring sha wrappers. This is an internal API; externally it surfaces when crypto/rsa OAEP is used with a hash the boring shim does not map.

Common situations: Using a third-party hash (BLAKE2, SHA-3, etc.) with RSA-OAEP under a boringcrypto Go build; passing a standard crypto/sha256 instance where the boring wrapper is expected internally.

Related errors

• crypto/rsa: invalid PSS salt length
• crypto/rsa: unsupported hash function: %d
• rsa: requested hash function unavailable: {hash}
• rsa: requested hash function unavailable: {mgfHash}
• crypto/ecdsa: use of hash functions other than SHA-2 or SHA-

AI-assisted analysis of golang/go@b6b368adc5 (2026-08-12). Data as JSON: /api/errors/b83bb7a1df2e2fe4. Report an issue: GitHub.