{"record":{"id":"a6c464e81d2887ac","repo":"apache/beam","slug":"offset-length-exceeds-the-buffer-limit-offset-length-limit","errorCode":null,"errorMessage":"offset + length exceeds the buffer limit. offset: , length: , limit: ","messagePattern":"offset \\+ length exceeds the buffer limit\\. offset: , length: , limit: ","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/core/src/main/java/org/apache/beam/sdk/io/TextSource.java","lineNumber":484,"sourceCode":"   * String s = new String(buffer, offset, length); // 2nd-copy\n   * }</pre>\n   */\n  static class SubstringByteArrayOutputStream extends ByteArrayOutputStream {\n    public String toString(int offset, int length, Charset charset) {\n      if (offset < 0) {\n        throw new IllegalArgumentException(\"offset is negative: \" + offset);\n      }\n      if (offset > count) {\n        throw new IllegalArgumentException(\n            \"offset exceeds the buffer limit. offset: \" + offset + \", limit: \" + count);\n      }\n\n      if (length < 0) {\n        throw new IllegalArgumentException(\"length is negative: \" + length);\n      }\n\n      if (offset + length > count) {\n        throw new IllegalArgumentException(\n            \"offset + length exceeds the buffer limit. offset: \"\n                + offset\n                + \", length: \"\n                + length\n                + \", limit: \"\n                + count);\n      }\n\n      return new String(buf, offset, length, charset);\n    }\n  }\n\n  /**\n   * See <a\n   * href=\"https://en.wikipedia.org/wiki/Knuth%E2%80%93Morris%E2%80%93Pratt_algorithm\">Knuth–Morris–Pratt\n   * algorithm</a>.\n   */\n  static class KMPDelimiterFinder {","sourceCodeStart":466,"sourceCodeEnd":502,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/core/src/main/java/org/apache/beam/sdk/io/TextSource.java#L466-L502","documentation":"SubstringByteArrayOutputStream.toString validates that offset + length does not exceed the total bytes written (count). This ensures the requested slice [offset, offset+length) is entirely within the buffer, preventing partial/garbage reads or ArrayIndexOutOfBoundsException from the underlying copy.","triggerScenarios":"Calling toString(offset, length, charset) where offset + length > count — e.g. length computed from an expected record size larger than what was actually buffered, or offsets from stale buffer state after more/less data was appended than assumed.","commonSituations":"TextSource custom record-finding logic that assumes a full record was read when the stream ended mid-record; buffer reuse across reads where offsets were computed against an old buffer length; reading past EOF because end-of-stream checks were skipped.","solutions":["Before slicing, assert offset + length <= buffer.count and handle the truncated-record case (wait for more data or error out).","Clamp length to Math.min(length, buffer.count - offset) if truncation is acceptable in your context.","Fix upstream EOF handling so records are never sliced before the full record has been buffered."],"exampleFix":"// before\nString record = buffer.toString(offset, expectedLen, UTF_8);\n\n// after\ncheckState(offset + expectedLen <= buffer.count, \"incomplete record buffered\");\nString record = buffer.toString(offset, expectedLen, UTF_8);","handlingStrategy":"validation","validationCode":"checkState(offset + len <= buffer.count, \"requested record [%s, %s) exceeds buffered %s bytes\", offset, offset + len, buffer.count);","typeGuard":null,"tryCatchPattern":"try {\n  String record = buffer.toString(offset, len, UTF_8);\n} catch (IllegalArgumentException e) {\n  // handle incomplete record: wait for more data or fail the read\n}","preventionTips":["Only slice after confirming the full record has been buffered (EOF handling).","Clamp length to buffer.count - offset when truncation is acceptable.","Track buffered bytes vs expected record sizes explicitly."],"tags":["java","apache-beam","io","argument-validation"],"backgroundTag":"argument-out-of-range","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}