{"record":{"id":"f206269b8cb3530d","repo":"apache/flink","slug":"compaction-reader-not-support-datastructure-conver","errorCode":null,"errorMessage":"Compaction reader not support DataStructure converter.","messagePattern":"Compaction reader not support DataStructure converter\\.","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/FileSystemTableSink.java","lineNumber":370,"sourceCode":"\n    private DynamicTableSource.Context createSourceContext(Context context) {\n        return new DynamicTableSource.Context() {\n            @Override\n            public <T> TypeInformation<T> createTypeInformation(DataType producedDataType) {\n                return context.createTypeInformation(producedDataType);\n            }\n\n            @Override\n            public <T> TypeInformation<T> createTypeInformation(LogicalType producedLogicalType) {\n                return context.createTypeInformation(producedLogicalType);\n            }\n\n            @Override\n            public DynamicTableSource.DataStructureConverter createDataStructureConverter(\n                    DataType producedDataType) {\n                // This method cannot be implemented without changing the\n                // DynamicTableSink.DataStructureConverter interface\n                throw new UnsupportedOperationException(\n                        \"Compaction reader not support DataStructure converter.\");\n            }\n        };\n    }\n\n    @SuppressWarnings(\"unchecked\")\n    private OutputFormatFactory<RowData> createOutputFormatFactory(Context sinkContext) {\n        Object writer = createWriter(sinkContext);\n        return writer instanceof Encoder\n                ? path -> createEncoderOutputFormat((Encoder<RowData>) writer, path)\n                : path -> createBulkWriterOutputFormat((BulkWriter.Factory<RowData>) writer, path);\n    }\n\n    private Object createWriter(Context sinkContext) {\n        DataType physicalDataTypeWithoutPartitionColumns =\n                DataType.getFields(physicalRowDataType).stream()\n                        .filter(field -> !partitionKeys.contains(field.getName()))\n                        .collect(Collectors.collectingAndThen(Collectors.toList(), DataTypes::ROW));","sourceCodeStart":352,"sourceCodeEnd":388,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/FileSystemTableSink.java#L352-L388","documentation":"FileSystemTableSink.createSourceContext returns a DynamicTableSource.Context for the compaction reader. Its createDataStructureConverter method throws UnsupportedOperationException because the compaction reader operates on internal RowData and cannot meaningfully convert to/from external data structures without changes to the DynamicTableSink.DataStructureConverter interface. The comment in code explains this is a known limitation.","triggerScenarios":"The compaction reader's source context has createDataStructureConverter called on it. This happens when the BulkFormat used for compaction reading internally calls createDataStructureConverter on the provided context. In standard Flink usage, the CompactBulkReader format does not call this method, so the error is only hit by custom or future format integrations.","commonSituations":"A custom compaction BulkFormat that calls createDataStructureConverter on the context. Future Flink version changes where the compaction path requires external data structure conversion. Integration code that exercises the source context beyond what the compaction reader supports.","solutions":["Ensure the BulkFormat used for compaction reading does not call createDataStructureConverter on the context.","If external data structure conversion is needed during compaction, implement it outside the context or restructure the format to work with internal RowData directly.","Use the standard CompactBulkReader which is designed to work within this constraint."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    context.createDataStructureConverter(dataType);\n} catch (UnsupportedOperationException e) {\n    // Compaction reader context does not support external conversion\n    // Work with internal RowData directly\n    LOG.warn(\"DataStructureConverter not available in compaction context; using internal RowData\");\n}","preventionTips":["Do not call createDataStructureConverter on the compaction reader's source context.","Design custom compaction BulkFormats to work with internal RowData only.","Use the standard CompactBulkReader which respects this constraint.","Document this limitation when building custom compaction integrations."],"tags":["table-sink","compaction-reader","data-structure-converter","unsupported-operation","flink"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}