{"record":{"id":"e8033d2722d7ae12","repo":"apache/shardingsphere","slug":"batch-is-too-big-accumulated-d-incoming-d-dat","errorCode":null,"errorMessage":"Batch is too big: accumulated %d + incoming %d data bytes exceeds buffer size limit %d bytes","messagePattern":"Batch is too big: accumulated (.+?) \\+ incoming (.+?) data bytes exceeds buffer size limit (.+?) bytes","errorType":"exception","errorClass":"BatchTooBigException","httpStatus":null,"severity":"error","filePath":"proxy/frontend/dialect/firebird/src/main/java/org/apache/shardingsphere/proxy/frontend/firebird/command/query/batch/FirebirdBatchMessageCommandExecutor.java","lineNumber":54,"sourceCode":"/**\n * Batch send message command executor for Firebird.\n */\n@RequiredArgsConstructor\npublic final class FirebirdBatchMessageCommandExecutor implements CommandExecutor {\n    \n    private final FirebirdBatchMessageCommandPacket packet;\n    \n    private final ConnectionSession connectionSession;\n    \n    @Override\n    public Collection<DatabasePacket> execute() throws SQLException {\n        int connectionId = connectionSession.getConnectionId();\n        FirebirdBatchStatement batchStatement = FirebirdBatchRegistry.getInstance().getBatchStatement(connectionId, packet.getStatementHandle());\n        if (null == batchStatement) {\n            throw new InvalidBatchHandleException(packet.getStatementHandle());\n        }\n        if (batchStatement.getAccumulatedSize() + packet.getDataLength() > batchStatement.getBufferSize()) {\n            throw new BatchTooBigException(packet.getStatementHandle(), batchStatement.getAccumulatedSize(), packet.getDataLength(), batchStatement.getBufferSize());\n        }\n        for (List<Object> each : packet.readParameterValues(batchStatement.getColumnDescriptors())) {\n            batchStatement.addParameterValues(each);\n        }\n        batchStatement.addSize(packet.getDataLength());\n        return Collections.singleton(new FirebirdGenericResponsePacket());\n    }\n}\n","sourceCodeStart":36,"sourceCodeEnd":63,"githubUrl":"https://github.com/apache/shardingsphere/blob/e952770a215630a3659c75d64369168cd3e26b82/proxy/frontend/dialect/firebird/src/main/java/org/apache/shardingsphere/proxy/frontend/firebird/command/query/batch/FirebirdBatchMessageCommandExecutor.java#L36-L63","documentation":"BatchTooBigException thrown by FirebirdBatchMessageCommandExecutor when accumulated batch bytes plus the incoming MESSAGE data exceed the batch buffer size negotiated at CREATE BATCH (from TAG_BUFFER_BYTES_SIZE, capped at MAX_BUFFER_SIZE, default DEFAULT_BUFFER_SIZE). This is Firebird's native 'batch is too big' condition surfaced by the proxy.","triggerScenarios":"batchStatement.getAccumulatedSize() + packet.getDataLength() > batchStatement.getBufferSize() inside execute(); clients queueing more rows than the negotiated buffer holds before executing, or negotiating a small TAG_BUFFER_BYTES_SIZE.","commonSituations":"bulk loaders queuing millions of rows without periodic executeBatch; client requesting a tiny buffer size; rows with large VARCHAR payloads filling the buffer quickly.","solutions":["Call executeBatch periodically (flush) so accumulated size stays below the buffer limit, then continue with a fresh batch.","Request a larger TAG_BUFFER_BYTES_SIZE at CREATE BATCH (up to the protocol maximum) if the default is too small.","Reduce per-row payload size (shorter strings, fewer columns) when batching huge rows."],"exampleFix":"// before\nfor (Row r : millionRows) { batch.add(r); }\nbatch.execute(); // BatchTooBigException\n\n// after\nint i = 0;\nfor (Row r : millionRows) {\n    batch.add(r);\n    if (++i % 10_000 == 0) { batch.execute(); batch.clear(); }\n}\nbatch.execute();","handlingStrategy":"validation","validationCode":"// Flush before the buffer overflows\nif (accumulatedBytes + rowBytes > negotiatedBufferSize) { sendExecuteBatch(stmtHandle); accumulatedBytes = 0; }","typeGuard":null,"tryCatchPattern":"catch (SQLException e) {\n    if (e.getMessage().contains(\"Batch is too big\")) { sendExecuteBatch(stmtHandle); resendLastMessage(stmtHandle); } else throw e;\n}","preventionTips":["Count accumulated bytes and flush at ~80% of the negotiated buffer size.","Request a larger TAG_BUFFER_BYTES_SIZE for bulk loads."],"tags":["firebird","batch","buffer-limit","bulk-load"],"backgroundTag":null,"analyzedSha":"e952770a215630a3659c75d64369168cd3e26b82","analyzedAt":"2026-08-14T13:54:53.392Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}