{"record":{"id":"30c8ceedd9bf7b1e","repo":"apache/beam","slug":"configuration-with-compression-is-not-compatible-with-avroio","errorCode":null,"errorMessage":"configuration with compression is not compatible with AvroIO","messagePattern":"configuration with compression is not compatible with AvroIO","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformProvider.java","lineNumber":193,"sourceCode":"            String.format(\n                \"configuration with %s is not compatible with a %s format\",\n                FileWriteSchemaTransformConfiguration.CsvConfiguration.class.getName(), format));\n      }\n      if (configuration.getParquetConfiguration() != null && !format.equals(PARQUET)) {\n        throw new IllegalArgumentException(\n            String.format(\n                \"configuration with %s is not compatible with a %s format\",\n                FileWriteSchemaTransformConfiguration.ParquetConfiguration.class.getName(),\n                format));\n      }\n      if (configuration.getXmlConfiguration() != null && !format.equals(XML)) {\n        throw new IllegalArgumentException(\n            String.format(\n                \"configuration with %s is not compatible with a %s format\",\n                FileWriteSchemaTransformConfiguration.XmlConfiguration.class.getName(), format));\n      }\n      if (format.equals(AVRO) && !Strings.isNullOrEmpty(configuration.getCompression())) {\n        throw new IllegalArgumentException(\n            \"configuration with compression is not compatible with AvroIO\");\n      }\n      if (format.equals(PARQUET) && !Strings.isNullOrEmpty(configuration.getCompression())) {\n        throw new IllegalArgumentException(\n            \"configuration with compression is not compatible with ParquetIO\");\n      }\n    }\n  }\n}\n","sourceCodeStart":175,"sourceCodeEnd":203,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformProvider.java#L175-L203","documentation":"validateConfiguration rejects a compression setting when the format is AVRO, because the AvroIO sink in this transform does not accept a user-supplied compression. It is an IllegalArgumentException raised at validation time before the pipeline runs.","triggerScenarios":"FileWriteSchemaTransformConfiguration with format=AVRO and a non-empty compression (e.g. setCompression(\"SNAPPY\") or \"GZIP\").","commonSituations":"Reusing the same configuration object across formats where compression was set for Parquet; assuming Avro supports the same compression knob as Parquet in this transform.","solutions":["Remove/unset compression when format is AVRO","Set compression only for PARQUET format","Use AvroIO-specific write settings outside this transform if compression is required"],"exampleFix":"// before\nbuilder.setFormat(\"AVRO\").setCompression(\"SNAPPY\");\n// after\nbuilder.setFormat(\"AVRO\").setCompression(null); // or setFormat(\"PARQUET\") to keep SNAPPY","handlingStrategy":"validation","validationCode":"if (\"AVRO\".equals(cfg.getFormat()) && cfg.getCompression() != null && !cfg.getCompression().isEmpty()) throw new IllegalArgumentException(\"compression not allowed for AVRO\");","typeGuard":null,"tryCatchPattern":"try { transform.expand(p.apply(...)); } catch (IllegalArgumentException e) { if (e.getMessage().contains(\"compression\")) { log.warn(\"strip compression for AVRO\"); } else throw e; }","preventionTips":["Never set compression globally on shared config objects","Only set compression when format==PARQUET"],"tags":["java","apache-beam","validation","compression"],"backgroundTag":"unsupported-config-value","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}