What causes "Bad value buffer offset-length combination." in apache/hadoop?

Validate voff >= 0, vlen >= 0, and voff + vlen <= value.length (and the key pair) before calling append

apache/hadoop · error · IndexOutOfBoundsException

Bad value buffer offset-length combination.

Error message

Bad value buffer offset-length combination.

What it means

Thrown by TFile.Writer.append(byte[] key, int koff, int klen, byte[] value, int voff, int vlen) when the value offset/length pair is invalid: voff or vlen negative, voff+vlen overflows int, or voff+vlen exceeds value.length. It is the value-side twin of the key check and fires as IndexOutOfBoundsException before any data is written, leaving the writer untouched by the bad call.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java:401

     * @param voff
     *          offset in value buffer.
     * @param vlen
     *          length of value.
     * @throws IOException
     *           Upon IO errors.
     *           <p>
     *           If an exception is thrown, the TFile will be in an inconsistent
     *           state. The only legitimate call after that would be close
     */
    public void append(byte[] key, int koff, int klen, byte[] value, int voff,
        int vlen) throws IOException {
      if ((koff | klen | (koff + klen) | (key.length - (koff + klen))) < 0) {
        throw new IndexOutOfBoundsException(
            "Bad key buffer offset-length combination.");
      }

      if ((voff | vlen | (voff + vlen) | (value.length - (voff + vlen))) < 0) {
        throw new IndexOutOfBoundsException(
            "Bad value buffer offset-length combination.");
      }

      try {
        DataOutputStream dosKey = prepareAppendKey(klen);
        try {
          ++errorCount;
          dosKey.write(key, koff, klen);
          --errorCount;
        } finally {
          dosKey.close();
        }

        DataOutputStream dosValue = prepareAppendValue(vlen);
        try {
          ++errorCount;
          dosValue.write(value, voff, vlen);
          --errorCount;

View on GitHub (pinned to 2add963021)

Solutions

Validate voff >= 0, vlen >= 0, and voff + vlen <= value.length (and the key pair) before calling append
Use append(byte[] key, byte[] value) when the full arrays form the record
Validate external length fields at ingestion time and reject malformed records explicitly

Example fix

// before
writer.append(key, 0, key.length, value, valOff, valLen);

// after
if (valOff < 0 || valLen < 0 || valOff + valLen > value.length) {
  throw new IllegalArgumentException("Bad value offset/length: " + valOff + "/" + valLen);
}
writer.append(key, 0, key.length, value, valOff, valLen);

Defensive patterns

Strategy: validation

Validate before calling

static void checkValueBounds(byte[] value, int voff, int vlen) {
  if (value == null || voff < 0 || vlen < 0 || voff + vlen < 0 || voff + vlen > value.length) {
    throw new IllegalArgumentException("bad value bounds: off=" + voff + " len=" + vlen);
  }
}
checkValueBounds(value, voff, vlen);
writer.append(key, koff, klen, value, voff, vlen);

Try / catch

catch (IndexOutOfBoundsException e) {
  // pre-write check failed; nothing appended: skip or repair the malformed record
}

Prevention

Validate both key and value pairs in one helper before every offset-based append
Reject malformed external records explicitly instead of letting array bounds surface downstream
Use end-exclusive slicing consistently (off + len <= buffer.length)

When it happens

Trigger: Passing a value length derived from a remaining-bytes computation that went negative, a value offset past the end of the buffer, or lengths from an untrusted/external source without validation.

Common situations: Parsing records from a byte stream where the value length field is corrupt or truncated; slicing buffers with end-exclusive vs end-inclusive confusion; reusing one scratch buffer with offsets that drift after variable-length records.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/4dfef9d239df3bf1. Report an issue: GitHub.