apache/flink · error · UnsupportedOperationException

Compaction reader not support DataStructure converter.

Error message

Compaction reader not support DataStructure converter.

What it means

FileSystemTableSink.createSourceContext returns a DynamicTableSource.Context for the compaction reader. Its createDataStructureConverter method throws UnsupportedOperationException because the compaction reader operates on internal RowData and cannot meaningfully convert to/from external data structures without changes to the DynamicTableSink.DataStructureConverter interface. The comment in code explains this is a known limitation.

Source

Thrown at flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/FileSystemTableSink.java:370

    private DynamicTableSource.Context createSourceContext(Context context) {
        return new DynamicTableSource.Context() {
            @Override
            public <T> TypeInformation<T> createTypeInformation(DataType producedDataType) {
                return context.createTypeInformation(producedDataType);
            }

            @Override
            public <T> TypeInformation<T> createTypeInformation(LogicalType producedLogicalType) {
                return context.createTypeInformation(producedLogicalType);
            }

            @Override
            public DynamicTableSource.DataStructureConverter createDataStructureConverter(
                    DataType producedDataType) {
                // This method cannot be implemented without changing the
                // DynamicTableSink.DataStructureConverter interface
                throw new UnsupportedOperationException(
                        "Compaction reader not support DataStructure converter.");
            }
        };
    }

    @SuppressWarnings("unchecked")
    private OutputFormatFactory<RowData> createOutputFormatFactory(Context sinkContext) {
        Object writer = createWriter(sinkContext);
        return writer instanceof Encoder
                ? path -> createEncoderOutputFormat((Encoder<RowData>) writer, path)
                : path -> createBulkWriterOutputFormat((BulkWriter.Factory<RowData>) writer, path);
    }

    private Object createWriter(Context sinkContext) {
        DataType physicalDataTypeWithoutPartitionColumns =
                DataType.getFields(physicalRowDataType).stream()
                        .filter(field -> !partitionKeys.contains(field.getName()))
                        .collect(Collectors.collectingAndThen(Collectors.toList(), DataTypes::ROW));

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Ensure the BulkFormat used for compaction reading does not call createDataStructureConverter on the context.
2. If external data structure conversion is needed during compaction, implement it outside the context or restructure the format to work with internal RowData directly.
3. Use the standard CompactBulkReader which is designed to work within this constraint.

try {
    context.createDataStructureConverter(dataType);
} catch (UnsupportedOperationException e) {
    // Compaction reader context does not support external conversion
    // Work with internal RowData directly
    LOG.warn("DataStructureConverter not available in compaction context; using internal RowData");
}

When it happens

Trigger: The compaction reader's source context has createDataStructureConverter called on it. This happens when the BulkFormat used for compaction reading internally calls createDataStructureConverter on the provided context. In standard Flink usage, the CompactBulkReader format does not call this method, so the error is only hit by custom or future format integrations.

Common situations: A custom compaction BulkFormat that calls createDataStructureConverter on the context. Future Flink version changes where the compaction path requires external data structure conversion. Integration code that exercises the source context beyond what the compaction reader supports.

Related errors

• The SplitChange type of %s is not supported.
• Could not find any format factory for identifier '%s' in the
• Streaming mode not support overwrite.
• Can not find format factory.
• Currently, filesystem sink doesn't support setting paralleli

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/f206269b8cb3530d. Report an issue: GitHub.