{"record":{"id":"b88869cf69b79871","repo":"apache/seatunnel","slug":"canaljson-file-does-not-support-this-compress-type","errorCode":null,"errorMessage":"CanalJson file does not support this compress type: {}","messagePattern":"CanalJson file does not support this compress type: (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/writer/CanalJsonWriteStrategy.java","lineNumber":141,"sourceCode":"\n    @Override\n    public FSDataOutputStream getOrCreateOutputStream(@NonNull String filePath) {\n        FSDataOutputStream fsDataOutputStream = beingWrittenOutputStream.get(filePath);\n        if (fsDataOutputStream == null) {\n            try {\n                switch (compressFormat) {\n                    case LZO:\n                        LzopCodec lzo = new LzopCodec();\n                        OutputStream out =\n                                lzo.createOutputStream(\n                                        hadoopFileSystemProxy.getOutputStream(filePath));\n                        fsDataOutputStream = new FSDataOutputStream(out, null);\n                        break;\n                    case NONE:\n                        fsDataOutputStream = hadoopFileSystemProxy.getOutputStream(filePath);\n                        break;\n                    default:\n                        log.warn(\n                                \"CanalJson file does not support this compress type: {}\",\n                                compressFormat.getCompressCodec());\n                        fsDataOutputStream = hadoopFileSystemProxy.getOutputStream(filePath);\n                        break;\n                }\n                beingWrittenOutputStream.put(filePath, fsDataOutputStream);\n                isFirstWrite.put(filePath, true);\n            } catch (IOException e) {\n                throw CommonError.fileOperationFailed(\"CanalJsonFile\", \"open\", filePath, e);\n            }\n        }\n        return fsDataOutputStream;\n    }\n}\n","sourceCodeStart":123,"sourceCodeEnd":156,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/writer/CanalJsonWriteStrategy.java#L123-L156","documentation":"This is a non-fatal warning logged by CanalJsonWriteStrategy.getOrCreateOutputStream when the configured file compress format has no dedicated codec branch in the switch statement. The writer falls back to writing an uncompressed output stream instead of failing. It signals that the chosen compress format (e.g. LZO, ZSTD for this format) is not wired up for Canal JSON text output.","triggerScenarios":"Configuring a Canal JSON file sink with file_format=canal_json and a compress option whose CompressFormat has a codec string but no case in the switch (anything other than LZO/GZIP/NONE handled branches), e.g. compress=ZSTD, then starting a job so getOrCreateOutputStream opens the per-file stream.","commonSituations":"Users copy a compress setting from an ORC/Parquet sink into a text-based canal_json sink; upgrade/regression where a new CompressFormat enum value was added without updating CanalJsonWriteStrategy; typos resolving to an unexpected default codec.","solutions":["Set compress=lz4/gzip/none (a compress format supported by the Canal JSON writer) in the sink options","Check SeaTunnel docs for compress formats supported by canal_json text output","If you need the unsupported codec, change file_format to one that supports it (e.g. ORC/Parquet) or pre-compress via an archive compress option","If the format should be supported, add a case in CanalJsonWriteStrategy.getOrCreateOutputStream wrapping the stream with the proper compression codec"],"exampleFix":"// before\nsink = {\n  file_format = \"canal_json\"\n  compress = \"zstd\"\n}\n// after\nsink = {\n  file_format = \"canal_json\"\n  compress = \"gzip\"\n}","handlingStrategy":"validation","validationCode":"// before submitting the job, check the compress value against supported text codecs\nSet<String> supported = Set.of(\"none\", \"lzo\", \"gzip\");\nif (!supported.contains(config.get(\"compress\").toLowerCase())) {\n    throw new IllegalArgumentException(\"compress=\" + config.get(\"compress\") + \" not supported for canal_json; use \" + supported);\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Only use compress values documented for the specific file_format","Keep per-format sink configs separate instead of templating one compress value across formats","Watch logs for this warning in test runs before production","After changing CompressFormat enums, grep all WriteStrategy classes for missing switch cases"],"tags":["hadoop","file-sink","compression","configuration"],"backgroundTag":"unsupported-config-value","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}