apache/druid · error · RuntimeException

Unable to decompress zstd data

Error message

Unable to decompress zstd data: %s

What it means

The Zstd Decompressor's decompressHeap calls Zstd.decompress into a preallocated output array and checks Zstd.isError on the result. When the native zstd call returns an error (bad magic bytes, truncated input, wrong frame), a RuntimeException is thrown naming the zstd error code. It indicates the compressed bytes are not valid zstd data.

Solutions

1. Verify the segment's compression strategy metadata matches the data (re-ingest if the spec changed strategy mid-write)
2. Check segment file integrity/transfer completeness; re-copy or re-download the segment
3. Re-generate the corrupt segment from source data

if (input.length < 4 || !looksLikeZstdFrame(input)) { throw new IllegalArgumentException("not zstd data"); }

try {
  strategy.decompress(source, input, output);
} catch (RuntimeException e) {
  if (e.getMessage().startsWith("Unable to decompress zstd")) {
    // re-fetch or re-ingest the corrupt segment
  } else throw e;
}

When it happens

Trigger: Decompressing a byte array that is truncated, corrupted, or was never zstd-compressed (e.g. wrong strategy used at write time).

Common situations: Segments written with a different compression strategy than the reader assumes; disk corruption or incomplete segment transfer; mixing zstd-compressed blocks with uncompressed ones.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

• Output buffer too small, please allocate more space.
• Bad null-marker byte, delegate class
• Directory compression not supported for
• Directory decompression not supported for
• directory[ ] is not a directory

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/5153c85b931c7a7a. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/segment/data/CompressionStrategy.java:458

            in.position(),
            numBytes
        );
        out.limit(out.position() + decompressedBytes);
      }
    }

    private static void decompressHeap(byte[] inputBytes, int numBytes, byte[] outputBytes, ByteBuffer out)
    {
      final long decompressedBytes = Zstd.decompressByteArray(
          outputBytes,
          0,
          outputBytes.length,
          inputBytes,
          0,
          numBytes
      );
      if (Zstd.isError(decompressedBytes)) {
        throw new RuntimeException("Unable to decompress zstd data: " + Zstd.getErrorName(decompressedBytes));
      }
      out.put(outputBytes, 0, (int) decompressedBytes);
      out.flip();
    }
  }

  /**
   * Logs info relating to whether LZ4 is using native or pure Java implementations
   */
  private static void logLZ4State()
  {
    LOG.debug("java.library.path: " + System.getProperty("java.library.path"));
    LZ4Factory fastestInstance = LZ4Factory.fastestInstance();
    try {
      //noinspection ObjectEquality
      if (fastestInstance == LZ4Factory.nativeInstance()) {
        LOG.debug("LZ4 compression is using native instance.");
      }

View on GitHub (pinned to 9b90983fd2)