sgl-project/sglang · critical · RuntimeError

Grouped pipeline returned fewer outputs than requests.

Error message

Grouped pipeline returned fewer outputs than requests.

What it means

While draining the grouped multimodal pipeline during forward execution, an exception path detected a StopIteration — the generator/pipeline yielded fewer outputs than the number of submitted requests. The engine wraps it in a RuntimeError (and only re-raises the original when propagate_forward_errors and forward_failed are set).

Source

Thrown at python/sglang/multimodal_gen/runtime/managers/gpu_worker.py:596

                if not req.is_warmup:
                    PerformanceLogger.log_request_summary(metrics=output_batch.metrics)

            # dump per-request perf report to the server-mode file path.
            if (
                req.perf_dump_path is not None
                and not req.is_warmup
                and output_batch.metrics is not None
            ):
                PerformanceLogger.dump_benchmark_report(
                    file_path=req.perf_dump_path,
                    metrics=output_batch.metrics,
                    meta={"model": self.server_args.model_path},
                    tag="server_perf_dump",
                )
        except Exception as e:
            if propagate_forward_errors and forward_failed:
                if isinstance(e, StopIteration):
                    raise RuntimeError(
                        "Grouped pipeline returned fewer outputs than requests."
                    ) from e
                raise
            logger.error(
                f"Error executing {error_context}: {e}",
                exc_info=True,
            )
            if isinstance(e, _oom_exceptions()):
                logger.warning(OOM_MSG)
            if output_batch is None:
                output_batch = OutputBatch()
            output_batch.error = f"Error executing {error_context}: {e}"
            self._record_output_peak_memory(output_batch)
            # clean cache if OOM
            if not current_platform.is_cpu():
                torch.get_device_module().empty_cache()
        return output_batch

View on GitHub (pinned to 0132848349)

Solutions

Check server logs just above this error for the underlying exception that truncated the pipeline (it is logged with exc_info)
Report/upsert to a matching sglang issue including the model, batch composition, and the inner exception
As a workaround, run with sequential forward (execute_forward_sequentially path) or smaller batches to see if the drop is batch-shape dependent
Verify you are not mixing pipeline stage versions after a partial upgrade

Defensive patterns

Strategy: try-catch

Try / catch

try:
    out = worker.execute_forward(reqs)
except RuntimeError as e:
    if "fewer outputs than requests" in str(e):
        logger.error("grouped pipeline truncated; retrying sequentially")
        out = worker.execute_forward_sequentially(reqs)
    else:
        raise

Prevention

Keep sglang pipeline stages and worker versions in sync
Capture the logged inner exception — the StopIteration is only the wrapper
Test grouped batches in staging before prod rollouts

When it happens

Trigger: execute_forward / _execute_forward_batch on a grouped batch where the pipeline generator terminates early (e.g. an internal filter or nesting bug drops outputs) so output count < len(reqs), combined with propagate_forward_errors and forward_failed being true.

Common situations: Upgrading the runtime with a changed grouped-pipeline contract; a custom pipeline stage that returns without yielding for some requests; OOM-adjacent partial failures inside the pipeline.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6f363b31a1e38e1a. Report an issue: GitHub.