{"record":{"id":"c2da93d3f2197f03","repo":"apache/seatunnel","slug":"schema-evolution-partition-column-col-i","errorCode":null,"errorMessage":"Schema evolution: partition column [\" + col + \"] is missing from the post-ALTER schema. Dropping or renaming a partition column is not supported. Available columns: \" + java.util.Arrays.toString(fieldNames)","messagePattern":"Schema evolution: partition column \\[\" \\+ col \\+ \"\\] is missing from the post-ALTER schema\\. Dropping or renaming a partition column is not supported\\. Available columns: \" \\+ java\\.util\\.Arrays\\.toString\\(fieldNames\\)","errorType":"validation","errorClass":"IllegalStateException","httpStatus":null,"severity":"critical","filePath":"seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/writer/AbstractWriteStrategy.java","lineNumber":441,"sourceCode":"     * <p>Throws if a configured partition column has been dropped from the schema — a dropped\n     * partition key would corrupt the partition tree, so we fail fast rather than silently writing\n     * to the wrong directory.\n     */\n    private List<Integer> rebuildPartitionFieldsIndex() {\n        List<String> partitionFieldList = fileSinkConfig.getPartitionFieldList();\n        if (partitionFieldList == null || partitionFieldList.isEmpty()) {\n            return new ArrayList<>();\n        }\n        String[] fieldNames = seaTunnelRowType.getFieldNames();\n        Map<String, Integer> nameToIndex = new HashMap<>(fieldNames.length);\n        for (int i = 0; i < fieldNames.length; i++) {\n            nameToIndex.put(fieldNames[i].toLowerCase(), i);\n        }\n        List<Integer> newIndex = new ArrayList<>(partitionFieldList.size());\n        for (String col : partitionFieldList) {\n            Integer idx = nameToIndex.get(col.toLowerCase());\n            if (idx == null) {\n                throw new IllegalStateException(\n                        \"Schema evolution: partition column [\"\n                                + col\n                                + \"] is missing from the post-ALTER schema. \"\n                                + \"Dropping or renaming a partition column is not supported. \"\n                                + \"Available columns: \"\n                                + java.util.Arrays.toString(fieldNames));\n            }\n            newIndex.add(idx);\n        }\n        return newIndex;\n    }\n\n    /**\n     * use seaTunnelRow generate partition directory\n     *\n     * @param seaTunnelRow seaTunnelRow\n     * @return the map of partition directory\n     */","sourceCodeStart":423,"sourceCodeEnd":459,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/writer/AbstractWriteStrategy.java#L423-L459","documentation":"After applying a schema change, the write strategy rebuilds the mapping from partition columns to their indices in the new schema. If a partition column no longer exists in the post-ALTER schema (it was dropped or renamed), the rebuild fails with IllegalStateException. Dropping or renaming a partition column would corrupt the output directory layout, so it is explicitly rejected.","triggerScenarios":"applySchemaChange on the file sink processes an ALTER that DROPPED or RENAMED a column which is listed in the sink's partition_by config; rebuildPartitionFieldsIndex then cannot find that column in the new fieldNames.","commonSituations":"A DBA renames or drops a partitioned column on the CDC source table while a schema-evolution-enabled file sink is running; the sink partition_by was configured against an older table schema that no longer matches.","solutions":["Restore the dropped/renamed partition column (re-add it or rename it back) on the source table","Update the sink's partition_by config to use a column that survives the ALTER, and restart the job","If the partition column must be dropped, stop the pipeline, remove it from partition_by, and resubmit the job"],"exampleFix":"// before\nsink {\n  LocalFile {\n    partition_by = [\"dept_id\"]   # dept_id gets dropped upstream\n    schema_evolution_enabled = true\n  }\n}\n// after\nsink {\n  LocalFile {\n    partition_by = [\"country\"]   # stable column that is not dropped\n    schema_evolution_enabled = true\n  }\n}","handlingStrategy":"validation","validationCode":"// before enabling schema evolution on a partitioned file sink\nList<String> partitionBy = sinkConfig.partitionBy;\nboolean allPresent = partitionBy.stream()\n    .allMatch(col -> Arrays.stream(catalogTable.getTableSchema().toPhysicalRowDataType().getFieldNames())\n        .anyMatch(f -> f.equalsIgnoreCase(col)));\nif (!allPresent) throw new IllegalStateException(\"partition_by columns must exist and must never be dropped/renamed upstream\");","typeGuard":null,"tryCatchPattern":"try { sink.applySchemaChange(event); } catch (IllegalStateException e) { log.error(\"Partition column removed by ALTER: {}\", e.getMessage()); /* restore column or repartition and restart */ }","preventionTips":["Choose partition columns that are stable and never dropped or renamed","Coordinate table ALTERs on the source with pipeline owners before applying them","Keep partition_by in sync with the current source schema after any schema migration"],"tags":["file-sink","schema-evolution","partition","cdc"],"backgroundTag":"schema-validation-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}