apache/iceberg · error · RuntimeException

Failed to create message digest needed for s3 checksum…

Error message

Failed to create message digest needed for s3 checksum checks

What it means

S3OutputStream's constructor eagerly creates a MessageDigest for full-object checksum verification when s3.checksum-enabled is true, and wraps NoSuchAlgorithmException in a RuntimeException. It means the configured digest algorithm is unavailable in the JVM.

Solutions

Restore the standard JCE security providers (check java.security config).
Use a full JDK instead of a stripped/minimal runtime image.
Disable s3.checksum-enabled if checksums are not required.
Inspect the NoSuchAlgorithmException cause to identify the missing algorithm name.

Example fix

// before
props.put("s3.checksum-enabled", "true"); // on minimal JRE
// after
// either use full JDK or disable checksums
props.put("s3.checksum-enabled", "false");

Defensive patterns

Strategy: validation

Validate before calling

// Java
try {
  MessageDigest.getInstance("SHA-256");
} catch (NoSuchAlgorithmException e) {
  throw new IllegalStateException("JVM lacks digest support; disable s3.checksum-enabled", e);
}

Try / catch

// Java
try {
  S3OutputStream s = new S3OutputStream(...);
} catch (RuntimeException e) {
  if (e.getMessage().contains("message digest")) {
    // fall back to checksum-disabled configuration
  }
}

Prevention

Run full JDK images; avoid stripped minimal runtimes for write-heavy jobs
Keep java.security provider configuration intact
Only enable s3.checksum-enabled when the runtime provably supports the digest

When it happens

Trigger: Setting s3.checksum-enabled=true in a JVM that lacks the DIGEST_ALGORITHM provider (rare, e.g. stripped JDK/security provider misconfiguration).

Common situations: See trigger scenarios.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/0416f772db16fe06. Report an issue: GitHub.

Appendix: source

Thrown at aws/src/main/java/org/apache/iceberg/aws/s3/S3OutputStream.java:143

    }

    this.s3 = s3;
    this.location = location;
    this.s3FileIOProperties = s3FileIOProperties;
    this.writeTags = s3FileIOProperties.writeTags();

    this.createStack = Thread.currentThread().getStackTrace();

    this.multiPartSize = s3FileIOProperties.multiPartSize();
    this.multiPartThresholdSize =
        (int) (multiPartSize * s3FileIOProperties.multipartThresholdFactor());
    this.stagingDirectory = new File(s3FileIOProperties.stagingDirectory());
    this.isChecksumEnabled = s3FileIOProperties.isChecksumEnabled();
    try {
      this.completeMessageDigest =
          isChecksumEnabled ? MessageDigest.getInstance(DIGEST_ALGORITHM) : null;
    } catch (NoSuchAlgorithmException e) {
      throw new RuntimeException(
          "Failed to create message digest needed for s3 checksum checks", e);
    }

    this.writeBytes = metrics.counter(FileIOMetricsContext.WRITE_BYTES, Unit.BYTES);
    this.writeOperations = metrics.counter(FileIOMetricsContext.WRITE_OPERATIONS);

    newStream();
  }

  @Override
  public long getPos() {
    return pos;
  }

  @Override
  public void flush() throws IOException {
    stream.flush();
  }

View on GitHub (pinned to 86d9c8fc54)