{"record":{"id":"f471d3ad406e74ea","repo":"alibaba/canal","slug":"column-size-is-not-match-for-table-vs","errorCode":null,"errorMessage":"column size is not match for table:{},{} vs {}","messagePattern":"column size is not match for table:(.+?),(.+?) vs (.+?)","errorType":"exception","errorClass":"CanalParseException","httpStatus":null,"severity":"error","filePath":"parse/src/main/java/com/alibaba/otter/canal/parse/inbound/mysql/dbsync/LogEventConvert.java","lineNumber":660,"sourceCode":"                // 1. 新增一张临时表，将需要做ddl表的数据全量导入\n                // 2. 在老表上建立I/U/D的trigger，增量的将数据插入到临时表\n                // 3. 锁住应用请求，将临时表rename为老表的名字，完成增加字段的操作\n                // 尝试做一次reload，可能因为ddl没有正确解析，或者使用了类似online ddl的操作\n                // 因为online ddl没有对应表名的alter语法，所以不会有clear cache的操作\n                tableMeta = getTableMeta(event.getTable().getDbName(), event.getTable().getTableName(), false, position);// 强制重新获取一次\n                if (tableMeta == null) {\n                    tableError = true;\n                    if (!filterTableError) {\n                        throw new CanalParseException(\"not found [\" + event.getTable().getDbName() + \".\"\n                                                      + event.getTable().getTableName() + \"] in db , pls check!\");\n                    }\n                }\n\n                // 在做一次判断\n                if (tableMeta != null && columnInfo.length > tableMeta.getFields().size()) {\n                    tableError = true;\n                    if (!filterTableError) {\n                        throw new CanalParseException(\"column size is not match for table:\" + tableMeta.getFullName()\n                                                      + \",\" + columnInfo.length + \" vs \" + tableMeta.getFields().size());\n                    }\n                }\n                // } else {\n                // logger.warn(\"[\" + event.getTable().getDbName() + \".\" +\n                // event.getTable().getTableName()\n                // + \"] is no primary key , skip alibaba_rds_row_id column\");\n            }\n        }\n\n        for (int i = 0; i < columnCnt; i++) {\n            ColumnInfo info = columnInfo[i];\n            // mysql 5.6开始支持nolob/mininal类型,并不一定记录所有的列,需要进行判断\n            if (!cols.get(i)) {\n                continue;\n            }\n\n            if (existRDSNoPrimaryKey && i == columnCnt - 1 && info.type == LogEvent.MYSQL_TYPE_LONGLONG) {","sourceCodeStart":642,"sourceCodeEnd":678,"githubUrl":"https://github.com/alibaba/canal/blob/87be50e87686a3e8af08c368d0e1ffd1f59eb04a/parse/src/main/java/com/alibaba/otter/canal/parse/inbound/mysql/dbsync/LogEventConvert.java#L642-L678","documentation":"Thrown from parseOneRow() when the binlog's column count (columnInfo.length) exceeds the cached table metadata's field count (tableMeta.getFields().size()). This means the binlog event references more columns than Canal's metadata believes the table has — a clear sign that a DDL (column addition) happened but the metadata cache was not invalidated or TSDB didn't track it.","triggerScenarios":"After the forced reload in parseOneRow(), tableMeta is non-null but columnInfo.length > tableMeta.getFields().size(). The binlog was written with N columns but the current metadata only knows about N-k columns. This happens when ALTER TABLE ADD COLUMN occurred but Canal's DDL cache wasn't updated (the DDL event was filtered, missed, or came from a different connection).","commonSituations":"An ALTER TABLE added columns but Canal filtered out the DDL event (filterQueryDdl=true). The DDL was executed on a different schema name or through a stored procedure that Canal's parser didn't capture. TSDB is disabled so no historical DDL tracking exists. A replication filter on the MySQL side excluded the DDL but not the subsequent DML.","solutions":["Enable TSDB: canal.instance.tsdb.enable=true and canal.instance.tsdb.dir=../conf/tsdb for persistent DDL history.","If filterQueryDdl=true, either disable it or manually trigger a metadata refresh after known schema changes.","Call tableMetaCache.clearTableMeta(schema, table) or restart Canal to force a full metadata reload.","Verify the canal user can see the DDL events in the binlog (no binlog filtering on the MySQL side)."],"exampleFix":"# before — no TSDB, DDL filtered\ncanal.instance.tsdb.enable=false\ncanal.instance.filter.query.ddl=true\n\n# after — TSDB enabled to track DDL history\ncanal.instance.tsdb.enable=true\ncanal.instance.tsdb.dir=../conf/tsdb\ncanal.instance.filter.query.ddl=false","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    // parseOneRow with column check\n    parseRowsEventForTableMeta(event);\n} catch (CanalParseException e) {\n    if (e.getMessage().contains(\"column size is not match\")) {\n        // Stale metadata — force a full cache invalidation and retry\n        tableMetaCache.clearTableMeta(schema, table);\n        logger.warn(\"Column mismatch detected, cache cleared for {}.{}, retrying\", schema, table);\n        // The next event will reload fresh metadata\n    }\n}","preventionTips":["Enable TSDB (canal.instance.tsdb.enable=true) to track DDL history and avoid stale metadata.","Do not set canal.instance.filter.query.ddl=true when DML events for the same tables are expected.","After manual schema changes, restart Canal or trigger a metadata cache flush."],"tags":["mysql","table-meta","ddl","column-mismatch","tsdb"],"backgroundTag":null,"analyzedSha":"87be50e87686a3e8af08c368d0e1ffd1f59eb04a","analyzedAt":"2026-08-14T04:30:11.918Z","schemaVersion":2},"datasetVersion":"2026-08-14T05:17:29.042Z"}