apache/hadoop · error · RuntimeException

The OpenSSL native library is built without SM4 CTR support

Error message

The OpenSSL native library is built without SM4 CTR support

What it means

OpensslSm4CtrCryptoCodec's constructor verifies at instantiation that the loaded native OpenSSL library actually implements SM4 in CTR mode (OpensslCipher.isSupported). If the JNI bridge loaded fine but the underlying OpenSSL build lacks SM4-CTR, the codec throws a RuntimeException immediately, so it never gets selected as a usable codec.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/OpensslSm4CtrCryptoCodec.java:46

import static org.apache.hadoop.fs.CommonConfigurationKeysPublic.HADOOP_SECURITY_OPENSSL_ENGINE_ID_KEY;

/**
 * Implement the SM4-CTR crypto codec using JNI into OpenSSL.
 */
@InterfaceAudience.Private
public class OpensslSm4CtrCryptoCodec extends OpensslCtrCryptoCodec {

  private static final Logger LOG =
          LoggerFactory.getLogger(OpensslSm4CtrCryptoCodec.class.getName());

  public OpensslSm4CtrCryptoCodec() {
    String loadingFailureReason = OpensslCipher.getLoadingFailureReason();
    if (loadingFailureReason != null) {
      throw new RuntimeException(loadingFailureReason);
    }

    if (!OpensslCipher.isSupported(CipherSuite.SM4_CTR_NOPADDING)) {
      throw new RuntimeException("The OpenSSL native library is built without SM4 CTR support");
    }
  }

  @Override
  public Logger getLogger() {
    return LOG;
  }

  @Override
  public void setConf(Configuration conf) {
    super.setConf(conf);
    setEngineId(conf.get(HADOOP_SECURITY_OPENSSL_ENGINE_ID_KEY));
  }

  @Override
  public CipherSuite getCipherSuite() {
    return CipherSuite.SM4_CTR_NOPADDING;
  }

View on GitHub (pinned to 2add963021)

Solutions

Upgrade the Hadoop native library and OpenSSL to a build with SM4-CTR support (OpenSSL 1.1.1+ typically) on every node
Fall back to AES/CTR/NoPadding, which nearly every OpenSSL build supports
Gate on OpensslCipher.isSupported(CipherSuite.SM4_CTR_NOPADDING) before configuring SM4 cluster-wide
Confirm which native library is actually loaded via the native library load logs at startup

Example fix

// before
conf.set("hadoop.security.crypto.cipher.suite", "SM4/CTR/NoPadding");
CryptoCodec codec = CryptoCodec.getInstance(conf); // RuntimeException: no SM4 CTR support

// after
if (OpensslCipher.isSupported(CipherSuite.SM4_CTR_NOPADDING)) {
  conf.set("hadoop.security.crypto.cipher.suite", "SM4/CTR/NoPadding");
} else {
  conf.set("hadoop.security.crypto.cipher.suite", "AES/CTR/NoPadding");
}
CryptoCodec codec = CryptoCodec.getInstance(conf);

Defensive patterns

Strategy: fallback

Validate before calling

// Probe native SM4 support before configuring it
boolean sm4Ok = OpensslCipher.isSupported(CipherSuite.SM4_CTR_NOPADDING);
String suite = sm4Ok ? "SM4/CTR/NoPadding" : "AES/CTR/NoPadding";
conf.set("hadoop.security.crypto.cipher.suite", suite);

Type guard

public boolean nodeSupportsSm4() {
  return OpensslCipher.isSupported(CipherSuite.SM4_CTR_NOPADDING);
}

Try / catch

try {
  codec = new OpensslSm4CtrCryptoCodec();
} catch (RuntimeException e) {
  // native lib lacks SM4 CTR — fall back to AES
  LOG.warn("SM4 unavailable ({}); falling back to AES/CTR", e.getMessage());
  conf.set("hadoop.security.crypto.cipher.suite", "AES/CTR/NoPadding");
  codec = CryptoCodec.getInstance(conf);
}

Prevention

Gate cluster-wide SM4 enablement on a node-level capability check across the fleet
Keep native libhadoop and OpenSSL versions uniform during rolling upgrades
Watch native-library load logs at startup to know exactly which build is active

When it happens

Trigger: Constructing OpensslSm4CtrCryptoCodec (directly, or via CryptoCodec.getInstance when hadoop.security.crypto.cipher.suite=SM4/CTR/NoPadding) on a node whose libhadoop/OpenSSL native library was built without SM4 support — e.g. OpenSSL 1.0.x, older distro builds, or FIPS builds that disable SM4.

Common situations: Enabling SM4 encryption zones after a cluster was built with older OpenSSL; rolling upgrades where some DataNodes have new native libs and some do not; container images built on older base distros

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/07d478919c1ae272. Report an issue: GitHub.