{"record":{"id":"0d742cce467e0d53","repo":"apache/beam","slug":"batch-size-is-too-large-it-should-be-smaller-or-equal-than-d-0d742c","errorCode":null,"errorMessage":"Batch size is too large! It should be smaller or equal than %d.","messagePattern":"Batch size is too large! It should be smaller or equal than (.+?)\\.","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/ml/src/main/java/org/apache/beam/sdk/extensions/ml/DLPInspectText.java","lineNumber":145,"sourceCode":"    public abstract Builder setColumnDelimiter(String delimiter);\n\n    /**\n     * Sets list of column names if the input KV value is a delimited row.\n     *\n     * @param headerColumns List of column names if the input KV value is a delimited row.\n     */\n    public abstract Builder setHeaderColumns(PCollectionView<List<String>> headerColumns);\n\n    abstract DLPInspectText autoBuild();\n\n    public DLPInspectText build() {\n      DLPInspectText inspectText = autoBuild();\n      if (inspectText.getInspectTemplateName() == null && inspectText.getInspectConfig() == null) {\n        throw new IllegalArgumentException(\n            \"Either inspectTemplateName or inspectConfig must be supplied!\");\n      }\n      if (inspectText.getBatchSizeBytes() > DLP_PAYLOAD_LIMIT_BYTES) {\n        throw new IllegalArgumentException(\n            String.format(\n                \"Batch size is too large! It should be smaller or equal than %d.\",\n                DLP_PAYLOAD_LIMIT_BYTES));\n      }\n      if (inspectText.getColumnDelimiter() == null && inspectText.getHeaderColumns() != null) {\n        throw new IllegalArgumentException(\n            \"Column delimiter should be set if headers are present.\");\n      }\n      if (inspectText.getHeaderColumns() == null && inspectText.getColumnDelimiter() != null) {\n        throw new IllegalArgumentException(\n            \"Column headers should be supplied when delimiter is present.\");\n      }\n      return inspectText;\n    }\n  }\n\n  public static Builder newBuilder() {\n    return new AutoValue_DLPInspectText.Builder();","sourceCodeStart":127,"sourceCodeEnd":163,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/ml/src/main/java/org/apache/beam/sdk/extensions/ml/DLPInspectText.java#L127-L163","documentation":"DLPInspectText batches input elements and sends them to the Cloud DLP API, which enforces a maximum payload size (DLP_PAYLOAD_LIMIT_BYTES). build() throws this IllegalArgumentException when the configured batchSizeBytes exceeds that limit, since the resulting requests would be rejected by the service.","triggerScenarios":"Calling DLPInspectText.newBuilder()...setBatchSizeBytes(n).build() where n is greater than DLP_PAYLOAD_LIMIT_BYTES (the DLP content API payload cap).","commonSituations":"Developers raise the batch size to improve throughput without checking the DLP API's content limit, or reuse a batch size tuned for a different API/service.","solutions":["Lower .setBatchSizeBytes(n) to a value less than or equal to DLP_PAYLOAD_LIMIT_BYTES.","Use the default batch size by omitting setBatchSizeBytes entirely.","If larger batches are needed for throughput, split the work across multiple transforms or reduce per-element size instead of raising the batch size."],"exampleFix":"// before\n.setBatchSizeBytes(20 * 1024 * 1024) // over the DLP payload limit\n// after\n.setBatchSizeBytes(5 * 1024 * 1024) // within DLP_PAYLOAD_LIMIT_BYTES","handlingStrategy":"validation","validationCode":"long limit = 524288; // DLP_PAYLOAD_LIMIT_BYTES constant in DLPInspectText\nif (batchSizeBytes > limit) {\n  batchSizeBytes = limit;\n}","typeGuard":null,"tryCatchPattern":"try {\n  DLPInspectText inspect = DLPInspectText.newBuilder()...setBatchSizeBytes(size).build();\n} catch (IllegalArgumentException e) {\n  if (e.getMessage().startsWith(\"Batch size is too large\")) {\n    // clamp size to DLP_PAYLOAD_LIMIT_BYTES and rebuild\n  }\n}","preventionTips":["Reference DLP_PAYLOAD_LIMIT_BYTES instead of hardcoding a batch size.","Keep the default batch size unless profiling shows a need to change it.","Add a config sanity check that clamps batch size to the documented limit."],"tags":["java","apache-beam","google-cloud-dlp","payload-size","builder-validation"],"backgroundTag":"value-out-of-range","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}