apache/hadoop · error · UnsupportedOperationException

byte[] arrays are not supported for DirectDecompressor

Error message

byte[] arrays are not supported for DirectDecompressor

What it means

SnappyCodec can return a DirectDecompressor (via createDirectDecompressor()) whose implementation, SnappyDecompressor.DirectDecompressor, works exclusively on direct ByteBuffers (decompress(ByteBuffer src, ByteBuffer dst)). The DirectDecompressor interface intentionally does not support dictionary-based APIs, so setDictionary(byte[], int, int) always throws UnsupportedOperationException with this message. Calling it means generic Decompressor-style code is being run against a direct-buffer-only object.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/snappy/SnappyDecompressor.java:335

      super.reset();
      endOfInput = true;
    }

    private boolean endOfInput;

    @Override
    public void decompress(ByteBuffer src, ByteBuffer dst)
        throws IOException {
      assert dst.isDirect() : "dst.isDirect()";
      assert src.isDirect() : "src.isDirect()";
      assert dst.remaining() > 0 : "dst.remaining() > 0";
      this.decompressDirect(src, dst);
      endOfInput = !src.hasRemaining();
    }

    @Override
    public void setDictionary(byte[] b, int off, int len) {
      throw new UnsupportedOperationException(
          "byte[] arrays are not supported for DirectDecompressor");
    }

    @Override
    public int decompress(byte[] b, int off, int len) {
      throw new UnsupportedOperationException(
          "byte[] arrays are not supported for DirectDecompressor");
    }
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. Branch on type before calling: skip setDictionary when the decompressor implements DirectDecompressor.
2. If you need dictionary or byte[] APIs, use the normal path: codec.createDecompressor() (array-based SnappyDecompressor) instead of createDirectDecompressor().
3. Keep the two code paths separate: ByteBuffer-to-ByteBuffer for DirectDecompressor, byte[] for Decompressor.
4. If you own the interface design, feature-test capabilities once (instanceof DirectDecompressor) rather than catching UnsupportedOperationException in production flow.

Example fix

// before
Decompressor d = (Decompressor) snappyCodec.createDirectDecompressor();
d.setDictionary(dict, 0, dict.length); // UnsupportedOperationException

// after
if (snappyCodec instanceof DirectDecompressionCodec) {
  DirectDecompressor dd = snappyCodec.createDirectDecompressor();
  dd.decompress(srcBuf, dstBuf); // ByteBuffer path, no dictionary
} else {
  Decompressor d = snappyCodec.createDecompressor();
  d.setDictionary(dict, 0, dict.length);
}

if (decompressor instanceof DirectDecompressor) {
  // direct path: no dictionary API exists — do not call setDictionary
  ((DirectDecompressor) decompressor).decompress(srcBuf, dstBuf);
} else {
  decompressor.setDictionary(dict, 0, dict.length);
}

static boolean supportsArrayDictionary(Object codecResult) {
  return !(codecResult instanceof DirectDecompressor);
}

When it happens

Trigger: Obtaining a DirectDecompressor (codec.createDirectDecompressor(), or an allocator/pool handing one back) and invoking setDictionary(b, off, len) on it — e.g. generic codec utilities that call setDictionary when a dictionary is configured, or code casting DirectDecompressor back to Decompressor and driving the full array-based API.

Common situations: Shared compression helpers handling both regular and direct codecs; frameworks that apply preset dictionaries (mostly zlib-oriented) uniformly to all Decompressors; codec pools where the object's concrete type is erased; code migrated from zlib/snappy array paths to the direct path without removing dictionary setup.

Related errors

• {getClass().getSimpleName()} doesn't support truncate
• Append is not supported by ChecksumFileSystem
• Truncate is not supported by ChecksumFileSystem
• Concat is not supported by ChecksumFileSystem
• Truncate is not supported by ChecksumFs

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/3df0ca28518eae9e. Report an issue: GitHub.