What causes "nvImageCodec could not decode the JPEG image" in sgl-project/sglang?

Catch RuntimeError and fall back to a CPU decoder (PIL/torchvision) for that image

sgl-project/sglang · error · RuntimeError

nvImageCodec could not decode the JPEG image

Error message

nvImageCodec could not decode the JPEG image

What it means

The pooled nvImageCodec decoder returned None from decode() for a JPEG byte buffer, meaning the GPU JPEG decoder could not decode the payload. The wrapper converts this into a RuntimeError so callers can fall back to CPU decoding.

Solutions

Catch RuntimeError and fall back to a CPU decoder (PIL/torchvision) for that image
Validate/skip corrupt images upstream (SOI/EOI markers, PIL verify)
Check CUDA health if all decodes suddenly fail (reset pool, restart worker)
Upgrade nvImageCodec if a specific JPEG variant is unsupported

Example fix

# before
img = decode_jpeg_with_fancy_upsampling(data)
# after
try:
    img = decode_jpeg_with_fancy_upsampling(data)
except RuntimeError:
    img = cpu_decode_jpeg(data)  # PIL/torchvision fallback

Defensive patterns

Strategy: fallback

Validate before calling

def looks_like_jpeg(data: bytes) -> bool:
    return len(data) > 4 and data[0:2] == b'\xff\xd8' and b'\xff\xd9' in data[-64:]

Type guard

null

Try / catch

try:
    tensor = decode_jpeg_with_fancy_upsampling(data)
except RuntimeError:
    tensor = decode_with_pil(data)  # CPU fallback

Prevention

Pre-validate JPEGs (magic bytes, PIL .verify()) before GPU decode
Always wrap GPU decode in try/except with a CPU path
Monitor for sudden all-images-fail patterns indicating CUDA state corruption

When it happens

Trigger: Calling decode() on corrupted/truncated JPEG bytes, unsupported JPEG variants (e.g. CMYK, progressive with odd sampling), or when the NVJPEG handle is in a bad state after a prior CUDA error.

Common situations: Multimodal image pipelines feeding user-uploaded or URL-fetched images where some are truncated; GPU memory pressure or stale decoder state after CUDA errors.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ae96e9a5d26711a4. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/utils/nvjpeg_decoder.py:60

                    max_num_cpu_threads=1,
                    options=_DECODER_OPTIONS,
                )
                self._created += 1
                return decoder

        return self._decoders.get()

    def decode(self, image_bytes: bytes) -> torch.Tensor:
        decoder = self._acquire()
        try:
            stream = torch.cuda.current_stream(self._device_id)
            image = decoder.decode(
                image_bytes,
                params=self._decode_params,
                cuda_stream=stream.cuda_stream,
            )
            if image is None:
                raise RuntimeError("nvImageCodec could not decode the JPEG image")
            return torch.from_dlpack(image.to_dlpack(cuda_stream=stream.cuda_stream))
        finally:
            self._decoders.put(decoder)


@lru_cache(maxsize=None)
def _get_decoder_pool(device_id: int) -> _NvJpegDecoderPool:
    return _NvJpegDecoderPool(device_id)


def decode_jpeg_with_fancy_upsampling(image_bytes: bytes) -> torch.Tensor:
    """Decode a JPEG to contiguous CHW RGB uint8 on the current CUDA device.

    torchvision's CUDA JPEG decoder creates nvJPEG with its default flags,
    which use nearest-neighbor chroma upsampling. nvImageCodec exposes nvJPEG's
    interpolated ("fancy") upsampling and exports the result to PyTorch through
    DLPack without copying it.
    """

View on GitHub (pinned to 0132848349)