denoland/deno · error · Error

ERR_CRYPTO_CUSTOM_ENGINE_NOT_SUPPORTED

ERR_CRYPTO_CUSTOM_ENGINE_NOT_SUPPORTED

Error message

Custom engines not supported by this OpenSSL

What it means

Node.js can load private keys from an OpenSSL engine/PKCS#11 provider via options.privateKeyIdentifier (the key label) plus options.privateKeyEngine (the engine name). Deno's TLS stack is rustls and has no OpenSSL engine support, so after validating both strings the polyfill throws ERR_CRYPTO_CUSTOM_ENGINE_NOT_SUPPORTED when both are set, failing fast instead of silently ignoring the HSM key.

Solutions

1. Drop privateKeyEngine/privateKeyIdentifier and supply key material directly as PEM (string/Buffer or { pem, passphrase } in a key array)
2. If the key must stay in the HSM, run that workload on a runtime with OpenSSL engine support or terminate TLS at a proxy in front of Deno
3. Feature-detect the config: if both options are present, branch to an alternative key-provisioning path before calling createSecureContext

Example fix

// before
const ctx = tls.createSecureContext({
  privateKeyIdentifier: 'hsm-key-1',
  privateKeyEngine: 'pkcs11',
  cert,
}); // throws ERR_CRYPTO_CUSTOM_ENGINE_NOT_SUPPORTED

// after
const ctx = tls.createSecureContext({
  key: [{ pem: process.env.TLS_KEY_PEM!, passphrase: process.env.TLS_KEY_PASS }],
  cert,
});

if (cfg.privateKeyEngine != null && cfg.privateKeyIdentifier != null) {
  // Deno has no OpenSSL engine support; fail early with a clear message
  throw new Error('HSM engine keys (privateKeyEngine/privateKeyIdentifier) are not supported; provide PEM key material instead');
}
const ctx = tls.createSecureContext(cfg);

try {
  ctx = tls.createSecureContext(cfg);
} catch (err) {
  if ((err as NodeJS.ErrnoException).code === 'ERR_CRYPTO_CUSTOM_ENGINE_NOT_SUPPORTED') {
    ctx = tls.createSecureContext({ ...cfg, privateKeyEngine: undefined, privateKeyIdentifier: undefined, key: pemKey });
  } else throw err;
}

When it happens

Trigger: tls.createSecureContext({ privateKeyIdentifier: 'my-key', privateKeyEngine: 'pkcs11', ... }) — both options non-null at the same time. Any config written for Node with HSM/PKCS#11 keys reused under Deno.

Common situations: Porting Node services that use SoftHSM, cloud HSM, or cloud KMS via OpenSSL engines; enterprise mTLS where keys never leave hardware; migration to Deno without auditing TLS key loading.

Related errors

• Can not create private key from
• can not create secret key from
• Context parameter is unsupported
• ERR_CRYPTO_ECDH_INVALID_FORMAT
• ERR_CRYPTO_ECDH_INVALID_PUBLIC_KEY

AI-assisted analysis of denoland/deno@9ad36f7a2c (2026-08-20). Data as JSON: /api/errors/000de8ede495d963. Report an issue: GitHub.

Appendix: source

Thrown at ext/node/polyfills/_tls_common.ts:544

      err.function = "dlfcn_load";
      err.reason = "could not load the shared library";
      err.code = "ERR_OSSL_DSO_COULD_NOT_LOAD_THE_SHARED_LIBRARY";
      throw err;
    }
    if (options.privateKeyEngine != null) {
      validateString(options.privateKeyEngine, "options.privateKeyEngine");
    }
    if (options.privateKeyIdentifier != null) {
      validateString(
        options.privateKeyIdentifier,
        "options.privateKeyIdentifier",
      );
    }
    if (
      options.privateKeyEngine != null &&
      options.privateKeyIdentifier != null
    ) {
      throw new ERR_CRYPTO_CUSTOM_ENGINE_NOT_SUPPORTED();
    }
    if (options.ecdhCurve != null) {
      validateString(options.ecdhCurve, "options.ecdhCurve");
    }
    // Validate cert before key - Node.js processes cert first (SetCert before SetKey)
    validateKeyCertOption(options.cert, "options.cert", false);
    validateKeyCertOption(options.key, "options.key", true);
    validateKeyCertOption(options.ca, "options.ca", false);

    // Load PFX / PKCS#12 data: extract the cert + private key so they can
    // be used by the underlying TLS implementation. Any additional certs
    // present in the PFX are merged into `ca`. Caller-supplied `cert`/`key`
    // (and `ca`) take precedence, matching Node, which loads PFX first and
    // then layers explicit cert/key on top.
    //
    // Node accepts both a single <string>|<Buffer> and an
    // <Array<string|Buffer|{ buf, passphrase? }>>; an empty array (which
    // playwright passes when no PFX is configured) must be a no-op rather

View on GitHub (pinned to 9ad36f7a2c)