elastic/elasticsearch · error · IllegalArgumentException

{zstdLib.getErrorName(hint)}

Error message

{zstdLib.getErrorName(hint)}

What it means

Thrown inside DStream.decompress when libzstd's ZSTD_decompressStream returns an error code for the hint. This is the streaming-decompression path that stages src into a native inBuf and out into a native outBuf, then copies produced bytes back into the caller's array. An error here means the frame is corrupt mid-stream, the prior context (ZSTD_DCtx) was reset/mis-used, or the staged window is malformed. IllegalArgumentException carrying getErrorName(hint).

Source

Thrown at libs/native/src/main/java/org/elasticsearch/nativeaccess/Zstd.java:391

            // loop will keep calling us if they wanted more than one buffer worth. Capping here
            // means we never overrun outBuf on the libzstd-side write.
            int outRoom = Math.min(dstAvail, outBufSize);

            // Copy caller's input slice into the native staging buffer at offset 0; libzstd reads
            // from inBuf[0..srcAvail) on this call. We always feed from offset 0 (rather than
            // tracking partial consumption inside the staging buffer) because the wrapper above
            // re-supplies the leftover bytes on the next call.
            if (srcAvail > 0) {
                MemorySegment.copy(src, srcPos, inBuf, JAVA_BYTE, 0L, srcAvail);
            }
            SIZE_VH.set(inStruct, (long) srcAvail);
            POS_VH.set(inStruct, 0L);
            SIZE_VH.set(outStruct, (long) outRoom);
            POS_VH.set(outStruct, 0L);

            long hint = zstdLib.decompressStream(handle, outStruct, inStruct);
            if (zstdLib.isError(hint)) {
                throw new IllegalArgumentException(zstdLib.getErrorName(hint));
            }

            int srcConsumed = (int) (long) POS_VH.get(inStruct);
            int dstProduced = (int) (long) POS_VH.get(outStruct);
            // libzstd guarantees pos ≤ size on return — the size fields we stamped above are the
            // upper bounds here, both already int-typed and bounded by the staging buffer sizes.
            assert srcConsumed >= 0 && srcConsumed <= srcAvail : "srcConsumed " + srcConsumed + " out of [0, " + srcAvail + "]";
            assert dstProduced >= 0 && dstProduced <= outRoom : "dstProduced " + dstProduced + " out of [0, " + outRoom + "]";
            if (dstProduced > 0) {
                MemorySegment.copy(outBuf, JAVA_BYTE, 0L, dst, dstPos, dstProduced);
            }
            // Translate native-staging positions back into absolute caller-array offsets — keeps
            // the SPI contract identical to zstd-jni's "positions are absolute in your byte[]".
            this.lastSrcPosAbsolute = srcPos + srcConsumed;
            this.lastDstPosAbsolute = dstPos + dstProduced;
            return hint;
        }

View on GitHub (pinned to db6a809a66)

Solutions

1. On any exception from decompressStream, discard the current DStream (close it) and create a fresh one for the next frame — do not reuse a half-broken context.
2. Ensure the input byte range handed to the DStream corresponds to exactly one logical frame; re-buffer at frame boundaries.
3. Verify outRoom > 0 (>= dStreamOutSize) before each call.
4. If reading from storage, cross-check the page/region CRC before blaming zstd — the bytes were likely corrupted upstream.

Example fix

// before: reusing dstream after an earlier failure
long hint = zstdLib.decompressStream(handle, outStruct, inStruct);

// after: scope dstream per frame, fail fast on error
try (var ds = zstd.newDStream()) {
    ds.decompress(dst, dstPos, outRoom, src, srcPos, srcAvail);
} catch (IllegalArgumentException e) {
    throw new CorruptFrameException("zstd stream decode failed", e);
}

// Validate outRoom and src window before streaming decompress.
if (outRoom <= 0) throw new IllegalArgumentException("outRoom must be > 0");
if (srcAvail <= 0) throw new IllegalArgumentException("no input bytes");
// Ensure the DStream context is fresh for this frame.

try (var ds = zstd.newDStream()) {
    ds.decompress(dst, dstPos, outRoom, src, srcPos, srcAvail);
} catch (IllegalArgumentException e) {
    throw new CorruptFrameException("zstd stream decode failed", e);
}

When it happens

Trigger: Streaming a partially-read frame whose bytes were corrupted on disk or in transit; reusing a DStream across frames without proper flush/reset; feeding bytes that span two frames concatenated incorrectly; outRoom sized to 0; the in-buffer window being advanced past frame boundaries by a buggy caller.

Common situations: Lucene/parquet skip() operations reading into the middle of a partially-consumed frame; a network reader reusing the DStream after an earlier exception left it in a bad state; concurrent use of a single-threaded DStream context; checksum failures in the underlying storage surfacing as zstd decode errors.

Related errors

• {zstdLib.getErrorName(ret)}
• blockSize must be a power of 2 in [{}, {}], got: {}
• ColumNAR field type [{}] is not implemented yet
• No points have been consumed
• LinearRing cannot have less than 4 points

AI-assisted analysis of elastic/elasticsearch@db6a809a66 (2026-08-12). Data as JSON: /api/errors/ab32b7c0cddc43ef. Report an issue: GitHub.