apache/beam · error · IllegalArgumentException

AUTO is not supported for writing

Error message

AUTO is not supported for writing

What it means

FileBasedSink.CompressionType.fromCanonical maps a canonical Compression enum to a write-side CompressionType. AUTO means 'detect from filename when reading' and is meaningless for sinks, so the method throws IllegalArgumentException immediately.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileBasedSink.java:183

    @Override
    public String getSuggestedFilenameSuffix() {
      return canonical.getSuggestedSuffix();
    }

    @Override
    public @Nullable String getMimeType() {
      return (canonical == Compression.UNCOMPRESSED) ? null : MimeTypes.BINARY;
    }

    @Override
    public WritableByteChannel create(WritableByteChannel channel) throws IOException {
      return canonical.writeCompressed(channel);
    }

    public static CompressionType fromCanonical(Compression canonical) {
      switch (canonical) {
        case AUTO:
          throw new IllegalArgumentException("AUTO is not supported for writing");

        case UNCOMPRESSED:
          return UNCOMPRESSED;

        case GZIP:
          return GZIP;

        case BZIP2:
          return BZIP2;

        case ZIP:
          throw new IllegalArgumentException("ZIP is unsupported");

        case ZSTD:
          return ZSTD;

        case LZO:
          return LZO;

View on GitHub (pinned to 12126d8942)

Solutions

1. Choose an explicit write CompressionType (GZIP, BZIP2, ZSTD, or UNCOMPRESSED) for the sink.
2. Substitute UNCOMPRESSED when the configured value is AUTO before constructing the sink.
3. Validate sink compression options at pipeline build time and reject AUTO with a clear message.

Example fix

// before
FileBasedSink.CompressionType.fromCanonical(Compression.AUTO);
// after
Compression c = config.getCompression();
FileBasedSink.CompressionType t = (c == Compression.AUTO)
    ? FileBasedSink.CompressionType.UNCOMPRESSED
    : FileBasedSink.CompressionType.fromCanonical(c);

Compression c = config.getCompression();
if (c == null || c == Compression.AUTO) c = Compression.UNCOMPRESSED;

try {
  type = FileBasedSink.CompressionType.fromCanonical(canonical);
} catch (IllegalArgumentException e) {
  type = FileBasedSink.CompressionType.UNCOMPRESSED;
}

When it happens

Trigger: Calling FileBasedSink.CompressionType.fromCanonical(Compression.AUTO); constructing a sink whose write compression is AUTO (e.g., via withCompression(AUTO) on FileBasedSink-based sinks).

Common situations: Shared pipeline options carrying compression=AUTO used for both input and output; users expecting AUTO to pick compression automatically on write; read/write config shared from a template.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Unsupported compression type: " + compression
• Failed to determine if the source is splittable
• Must resolve compression into a concrete value before callin
• AUTO is applicable only to reading files
• Writing ZIP files is currently unsupported

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f261dda798d59fe5. Report an issue: GitHub.