{"record":{"id":"ae15185c62d5a44c","repo":"apache/beam","slug":"batch-size-is-too-large-it-should-be-smaller-or-equal-than-d-ae1518","errorCode":null,"errorMessage":"Batch size is too large! It should be smaller or equal than %d.","messagePattern":"Batch size is too large! It should be smaller or equal than (.+?)\\.","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/ml/src/main/java/org/apache/beam/sdk/extensions/ml/DLPReidentifyText.java","lineNumber":172,"sourceCode":"\n    /**\n     * Sets ID of Google Cloud project to be used when deidentifying data.\n     *\n     * @param projectId ID of Google Cloud project to be used when deidentifying data.\n     */\n    public abstract Builder setProjectId(String projectId);\n\n    abstract DLPReidentifyText autoBuild();\n\n    public DLPReidentifyText build() {\n      DLPReidentifyText dlpReidentifyText = autoBuild();\n      if (dlpReidentifyText.getReidentifyConfig() == null\n          && dlpReidentifyText.getReidentifyTemplateName() == null) {\n        throw new IllegalArgumentException(\n            \"Either reidentifyConfig or reidentifyTemplateName need to be set!\");\n      }\n      if (dlpReidentifyText.getBatchSizeBytes() > DLP_PAYLOAD_LIMIT_BYTES) {\n        throw new IllegalArgumentException(\n            String.format(\n                \"Batch size is too large! It should be smaller or equal than %d.\",\n                DLP_PAYLOAD_LIMIT_BYTES));\n      }\n      if (dlpReidentifyText.getColumnDelimiter() == null\n          && dlpReidentifyText.getHeaderColumns() != null) {\n        throw new IllegalArgumentException(\n            \"Column delimiter should be set if headers are present.\");\n      }\n      if (dlpReidentifyText.getHeaderColumns() == null\n          && dlpReidentifyText.getColumnDelimiter() != null) {\n        throw new IllegalArgumentException(\n            \"Column headers should be supplied when delimiter is present.\");\n      }\n      return dlpReidentifyText;\n    }\n  }\n","sourceCodeStart":154,"sourceCodeEnd":190,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/ml/src/main/java/org/apache/beam/sdk/extensions/ml/DLPReidentifyText.java#L154-L190","documentation":"DLPReidentifyText.Builder.build() validates the configured byte batch size against DLP_PAYLOAD_LIMIT_BYTES, the maximum payload the Google DLP reidentify API accepts per request. If the user sets a batchSizeBytes larger than this limit, build() refuses to construct the transform. This fails fast at pipeline construction time so an oversized request is never sent to the DLP service.","triggerScenarios":"Calling DLPReidentifyText.newBuilder().setBatchSizeBytes(n).build() where n > DLP_PAYLOAD_LIMIT_BYTES. The message's %d is filled with DLP_PAYLOAD_LIMIT_BYTES.","commonSituations":"Users assuming DLP can accept arbitrarily large text batches, copying a batch size from another DLP transform with a different limit, or setting batch size in a config knob that allows values above the API cap.","solutions":["Lower batchSizeBytes to a value <= DLP_PAYLOAD_LIMIT_BYTES (or simply don't set it to use the default, which is already within the limit).","Split your input into smaller elements/PCollections so each DLP batch stays under the limit.","Consult the constant DLP_PAYLOAD_LIMIT_BYTES in DLPReidentifyText for the exact allowed maximum."],"exampleFix":"// before\nDLPReidentifyText.newBuilder().setBatchSizeBytes(2_000_000).build();\n// after\nDLPReidentifyText.newBuilder().setBatchSizeBytes(500_000).build(); // <= DLP_PAYLOAD_LIMIT_BYTES","handlingStrategy":"validation","validationCode":"long payloadLimit = DLP_PAYLOAD_LIMIT_BYTES; // e.g. read from DLPReidentifyText\nif (options.batchSizeBytes() > payloadLimit) {\n  throw new IllegalArgumentException(\"batchSizeBytes must be <= \" + payloadLimit);\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Don't set batchSizeBytes unless you have a reason; the default already respects the DLP limit.","Clamp configured batch size to DLP_PAYLOAD_LIMIT_BYTES when reading from config.","Unit-test builders with the max configured batch size."],"tags":["java","apache-beam","dlp","validation","configuration"],"backgroundTag":"value-out-of-range","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}