{"record":{"id":"b4e00f55622cf045","repo":"apache/beam","slug":"batch-size-is-too-large-it-should-be-smaller-or-equal-than-d","errorCode":null,"errorMessage":"Batch size is too large! It should be smaller or equal than %d.","messagePattern":"Batch size is too large! It should be smaller or equal than (.+?)\\.","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/ml/src/main/java/org/apache/beam/sdk/extensions/ml/DLPDeidentifyText.java","lineNumber":167,"sourceCode":"     * Sets configuration object for data deidentification. If present, supersedes the template\n     * settings.\n     *\n     * @param deidentifyConfig Configuration object for data deidentification. If present,\n     *     supersedes the template settings.\n     */\n    public abstract Builder setDeidentifyConfig(DeidentifyConfig deidentifyConfig);\n\n    abstract DLPDeidentifyText autoBuild();\n\n    public DLPDeidentifyText build() {\n      DLPDeidentifyText dlpDeidentifyText = autoBuild();\n      if (dlpDeidentifyText.getDeidentifyConfig() == null\n          && dlpDeidentifyText.getDeidentifyTemplateName() == null) {\n        throw new IllegalArgumentException(\n            \"Either deidentifyConfig or deidentifyTemplateName need to be set!\");\n      }\n      if (dlpDeidentifyText.getBatchSizeBytes() > DLP_PAYLOAD_LIMIT_BYTES) {\n        throw new IllegalArgumentException(\n            String.format(\n                \"Batch size is too large! It should be smaller or equal than %d.\",\n                DLP_PAYLOAD_LIMIT_BYTES));\n      }\n      if (dlpDeidentifyText.getColumnDelimiter() == null\n          && dlpDeidentifyText.getHeaderColumns() != null) {\n        throw new IllegalArgumentException(\n            \"Column delimiter should be set if headers are present.\");\n      }\n      if (dlpDeidentifyText.getHeaderColumns() == null\n          && dlpDeidentifyText.getColumnDelimiter() != null) {\n        throw new IllegalArgumentException(\n            \"Column headers should be supplied when delimiter is present.\");\n      }\n      return dlpDeidentifyText;\n    }\n  }\n","sourceCodeStart":149,"sourceCodeEnd":185,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/ml/src/main/java/org/apache/beam/sdk/extensions/ml/DLPDeidentifyText.java#L149-L185","documentation":"DLPDeidentifyText.Builder.build enforces the DLP payload limit (DLP_PAYLOAD_LIMIT_BYTES). If the configured batchSizeBytes exceeds it, build throws IllegalArgumentException because the DLP API would reject the request as too large.","triggerScenarios":"Calling .setBatchSizeBytes(n) with n greater than DLP_PAYLOAD_LIMIT_BYTES and then building the transform.","commonSituations":"Setting a very large batch to reduce API calls; misremembering the DLP content limit; converting record counts to bytes incorrectly.","solutions":["Lower batchSizeBytes to at or below DLP_PAYLOAD_LIMIT_BYTES (setBatchSizeBytes(DLP_PAYLOAD_LIMIT_BYTES)).","Compute batch sizes from average record size to stay under the limit.","Remove the explicit size and rely on the default safe batch size."],"exampleFix":"// before\n.setBatchSizeBytes(50_000_000)\n// after\n.setBatchSizeBytes(DLP_PAYLOAD_LIMIT_BYTES) // e.g. within the allowed max","handlingStrategy":"validation","validationCode":"if (batchSizeBytes > DLP_PAYLOAD_LIMIT_BYTES) batchSizeBytes = DLP_PAYLOAD_LIMIT_BYTES;","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Cap batchSizeBytes at DLP_PAYLOAD_LIMIT_BYTES","Derive batch size from measured record sizes","Prefer defaults over large custom sizes"],"tags":["java","dlp","payload-size","google-cloud"],"backgroundTag":"payload-too-large","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}