exo-explore/exo · error · RuntimeError

Prefill server error [{msg.code}]: {msg.message}

Error message

Prefill server error [{msg.code}]: {msg.message}

What it means

Error "Prefill server error [{msg.code}]: {msg.message}" thrown in exo-explore/exo.

Source

Thrown at src/exo/worker/engines/mlx/disaggregated/client.py:98

        kv_by_layer: dict[int, list[KVChunk]] = defaultdict(list)
        chunks_received = 0

        while True:
            msg = read_message(stream)
            if msg is None:
                break
            if isinstance(msg, KVChunk):
                kv_by_layer[msg.layer_idx].append(msg)
                chunks_received += 1
                if on_kv_chunk is not None:
                    on_kv_chunk(msg, chunks_received)
            elif isinstance(msg, ArraysState):
                result.arrays[msg.layer_idx] = msg.arrays
            elif isinstance(msg, Done):
                result.total_tokens = msg.total_tokens
                break
            else:
                raise RuntimeError(f"Prefill server error [{msg.code}]: {msg.message}")

        result.kv_chunks = dict(kv_by_layer)
        return result
    finally:
        sock.close()


def ingest_into_mlx_cache(
    result: PrefillResult,
    caches: list[KVCache | RotatingKVCache | ArraysCache],
    *,
    start_pos: int = 0,
) -> int:
    max_received = max(
        (sum(c.num_tokens for c in chunks) for chunks in result.kv_chunks.values()),
        default=0,
    )
    final_offset = start_pos + max_received

View on GitHub (pinned to 21a54c5ea0)

When it happens

Trigger: Thrown at src/exo/worker/engines/mlx/disaggregated/client.py:98 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of exo-explore/exo@21a54c5ea0 (2026-08-26). Data as JSON: /api/errors/0804835034b75759. Report an issue: GitHub.