apache/beam · error · IllegalArgumentException

configuration with compression is not compatible with…

Error message

configuration with compression is not compatible with ParquetIO

What it means

validateConfiguration rejects a compression setting when the format is PARQUET, because the ParquetIO sink used by this transform does not accept user-supplied compression via this configuration. IllegalArgumentException at validation time.

Solutions

1. Remove/unset compression when format is PARQUET
2. Configure compression on ParquetIO directly if outside this transform's contract
3. Verify only format-supported fields are populated

Example fix

// before
FileWriteSchemaTransformConfiguration.builder().setFormat("PARQUET").setCompression("GZIP").build();
// after
FileWriteSchemaTransformConfiguration.builder().setFormat("PARQUET").build();

if ("PARQUET".equals(cfg.getFormat()) && cfg.getCompression() != null && !cfg.getCompression().isEmpty()) throw new IllegalArgumentException("compression not allowed for PARQUET in this transform");

try { ... } catch (IllegalArgumentException e) { if (e.getMessage().contains("ParquetIO")) { /* reconfigure without compression */ } else throw e; }

When it happens

Trigger: FileWriteSchemaTransformConfiguration with format=PARQUET and a non-empty getCompression() value.

Common situations: Config templates that always set compression; migrating Avro configs to Parquet expecting the same knob to work.

Related errors

• configuration with compression is not compatible with AvroIO
• Batch size is too large! It should be smaller or equal than
• Both clientCertPath and clientCertKeyPath must be specified…
• ChangeStreamName can't be empty
• Column delimiter should be set if headers are present.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/5f538b8e6ba211ba. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformProvider.java:197

      if (configuration.getParquetConfiguration() != null && !format.equals(PARQUET)) {
        throw new IllegalArgumentException(
            String.format(
                "configuration with %s is not compatible with a %s format",
                FileWriteSchemaTransformConfiguration.ParquetConfiguration.class.getName(),
                format));
      }
      if (configuration.getXmlConfiguration() != null && !format.equals(XML)) {
        throw new IllegalArgumentException(
            String.format(
                "configuration with %s is not compatible with a %s format",
                FileWriteSchemaTransformConfiguration.XmlConfiguration.class.getName(), format));
      }
      if (format.equals(AVRO) && !Strings.isNullOrEmpty(configuration.getCompression())) {
        throw new IllegalArgumentException(
            "configuration with compression is not compatible with AvroIO");
      }
      if (format.equals(PARQUET) && !Strings.isNullOrEmpty(configuration.getCompression())) {
        throw new IllegalArgumentException(
            "configuration with compression is not compatible with ParquetIO");
      }
    }
  }
}

View on GitHub (pinned to 12126d8942)