{"record":{"id":"9b2a07c745543482","repo":"alibaba/canal","slug":"batchid-d-is-not-the-firstly-d","errorCode":null,"errorMessage":"batchId:%d is not the firstly:%d","messagePattern":"batchId:(.+?) is not the firstly:(.+?)","errorType":"validation","errorClass":"CanalMetaManagerException","httpStatus":null,"severity":"error","filePath":"meta/src/main/java/com/alibaba/otter/canal/meta/MemoryMetaManager.java","lineNumber":152,"sourceCode":"        }\n\n        public synchronized void addPositionRange(PositionRange positionRange, Long batchId) {\n            updateMaxId(batchId);\n            batches.put(batchId, positionRange);\n        }\n\n        public synchronized Long addPositionRange(PositionRange positionRange) {\n            Long batchId = atomicMaxBatchId.getAndIncrement();\n            batches.put(batchId, positionRange);\n            return batchId;\n        }\n\n        public synchronized PositionRange removePositionRange(Long batchId) {\n            if (batches.containsKey(batchId)) {\n                Long minBatchId = Collections.min(batches.keySet());\n                if (!minBatchId.equals(batchId)) {\n                    // 检查一下提交的ack/rollback，必须按batchId分出去的顺序提交，否则容易出现丢数据\n                    throw new CanalMetaManagerException(String.format(\"batchId:%d is not the firstly:%d\",\n                        batchId,\n                        minBatchId));\n                }\n                return batches.remove(batchId);\n            } else {\n                return null;\n            }\n        }\n\n        public synchronized PositionRange getPositionRange(Long batchId) {\n            return batches.get(batchId);\n        }\n\n        public synchronized PositionRange getLastestPositionRange() {\n            if (batches.size() == 0) {\n                return null;\n            } else {\n                Long batchId = Collections.max(batches.keySet());","sourceCodeStart":134,"sourceCodeEnd":170,"githubUrl":"https://github.com/alibaba/canal/blob/87be50e87686a3e8af08c368d0e1ffd1f59eb04a/meta/src/main/java/com/alibaba/otter/canal/meta/MemoryMetaManager.java#L134-L170","documentation":"Thrown by MemoryMetaManager's MemoryClientIdentityBatch.removePositionRange when an ack/rollback is attempted for a batchId that exists but is not the smallest (oldest) outstanding batch. Canal requires batches to be acknowledged strictly in the order they were handed out, to prevent data gaps. Acking out of order is treated as a data-integrity hazard.","triggerScenarios":"A client calls ack(batchId) or rollback(batchId) via MemoryMetaManager where batches.containsKey(batchId) is true but Collections.min(batches.keySet()) != batchId, i.e. an earlier batch remains unacked.","commonSituations":"Client acknowledges a newer batch before an older one (concurrent/out-of-order consumers), a client crashed mid-batch and restarted picking a newer batch, or logic that picks the latest batch instead of processing in FIFO order. With in-memory meta this also means prior state is lost on restart so ordering assumptions break.","solutions":["Always ack the smallest outstanding batchId first; track and ack in ascending order.","If you must skip, use rollback on the older batch before acking newer ones, or reset the client identity.","Avoid sharing a clientId across concurrent consumers that ack independently; use distinct clientId per consumer.","Consider a persistent meta manager (ZooKeeper/file) so batch order survives restarts instead of in-memory loss."],"exampleFix":"// before: ack whatever batch arrived last\nclient.ack(receivedBatchId);\n\n// after: ack strictly the oldest outstanding\nlong oldest = client.getWithoutAck(...); // fetch in order\nclient.ack(oldest);","handlingStrategy":"validation","validationCode":"// before ack, ensure this batchId is the oldest outstanding\nLong oldest = memoryClientBatch.getFirstBatchId(); // expose/get min key\nif (oldest == null || !oldest.equals(batchId)) {\n    throw new IllegalStateException(\"refusing out-of-order ack; oldest=\" + oldest + \" acking=\" + batchId);\n}","typeGuard":null,"tryCatchPattern":"try { client.ack(batchId); }\ncatch (CanalMetaManagerException e) {\n    if (e.getMessage().startsWith(\"batchId:\")) { client.rollback(batchId); /* or reset cursor */ }\n    else throw e;\n}","preventionTips":["Always ack the smallest outstanding batchId first.","Use a unique clientId per consumer to avoid interleaved acks.","Process batches in FIFO order; do not skip ahead."],"tags":["meta","ack-ordering","data-integrity","memory"],"backgroundTag":null,"analyzedSha":"87be50e87686a3e8af08c368d0e1ffd1f59eb04a","analyzedAt":"2026-08-14T04:30:11.918Z","schemaVersion":2},"datasetVersion":"2026-08-14T05:17:29.042Z"}