{"record":{"id":"a9820699e0571647","repo":"prestodb/presto","slug":"unexpected-internal-column-s","errorCode":null,"errorMessage":"unexpected internal column '%s'","messagePattern":"unexpected internal column '(.+?)'","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"presto-kafka/src/main/java/com/facebook/presto/kafka/KafkaPageSinkProvider.java","lineNumber":71,"sourceCode":"    }\n\n    @Override\n    public ConnectorPageSink createPageSink(ConnectorTransactionHandle transactionHandle, ConnectorSession session, ConnectorOutputTableHandle outputTableHandle, PageSinkContext pageSinkContext)\n    {\n        throw new UnsupportedOperationException(\"Table creation is not supported by the kafka connector\");\n    }\n\n    @Override\n    public ConnectorPageSink createPageSink(ConnectorTransactionHandle transactionHandle, ConnectorSession session, ConnectorInsertTableHandle insertTableHandle, PageSinkContext pageSinkContext)\n    {\n        requireNonNull(insertTableHandle, \"tableHandle is null\");\n        KafkaTableHandle handle = (KafkaTableHandle) insertTableHandle;\n\n        ImmutableList.Builder<EncoderColumnHandle> keyColumns = ImmutableList.builder();\n        ImmutableList.Builder<EncoderColumnHandle> messageColumns = ImmutableList.builder();\n        handle.getColumns().forEach(col -> {\n            if (col.isInternal()) {\n                throw new IllegalArgumentException(format(\"unexpected internal column '%s'\", col.getName()));\n            }\n            if (col.isKeyCodec()) {\n                keyColumns.add(col);\n            }\n            else {\n                messageColumns.add(col);\n            }\n        });\n\n        RowEncoder keyEncoder = encoderFactory.create(\n                session,\n                handle.getKeyDataFormat(),\n                getDataSchema(handle.getKeyDataSchemaLocation()),\n                keyColumns.build());\n\n        RowEncoder messageEncoder = encoderFactory.create(\n                session,\n                handle.getMessageDataFormat(),","sourceCodeStart":53,"sourceCodeEnd":89,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-kafka/src/main/java/com/facebook/presto/kafka/KafkaPageSinkProvider.java#L53-L89","documentation":"KafkaPageSinkProvider.createPageSink refuses to build a Kafka insert handle when any column in the table handle is flagged as internal. Internal columns are metadata-only pseudo columns; they have no encoder mapping and cannot be serialized to Kafka, so the connector fails fast with an IllegalArgumentException rather than silently dropping or mis-encoding data.","triggerScenarios":"Calling ConnectorPageSinkProvider.createPageSink with a KafkaTableHandle whose columns list contains a KafkaColumnHandle with isInternal()==true, i.e. an INSERT/CREATE TABLE AS against a Kafka table where internal column handles leaked into the insert projection.","commonSituations":"Metadata plugins or custom table descriptors expose hidden/internal columns; a query planner bug or hand-built KafkaTableHandle includes internal columns; version drift where a column that used to be plain is now marked internal.","solutions":["Remove internal columns from the insert column list (SELECT only non-internal columns in the INSERT)","Regenerate the table description so column handles are built by the connector, not manually","Check for an upgraded metadata plugin producing internal column handles and file/verify against the connector version"],"exampleFix":"// before\nINSERT INTO kafka_table SELECT rowkey, _partition_id, payload FROM staging;\n// after\nINSERT INTO kafka_table SELECT rowkey, payload FROM staging;","handlingStrategy":"validation","validationCode":"// before INSERT, ensure no internal columns are projected\nSELECT column_name FROM information_schema.columns\nWHERE table_name='kafka_table' AND column_name NOT LIKE '\\_%';","typeGuard":"boolean isWritable(KafkaColumnHandle c) { return !c.isInternal(); }","tryCatchPattern":"null","preventionTips":["Only select regular columns when inserting into Kafka tables","Let the connector generate column handles; don't hand-build KafkaTableHandle","Pin connector/plugin versions to avoid metadata-shape drift"],"tags":["kafka","insert","column-mapping","presto-connector"],"backgroundTag":"internal-column-not-writable","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}