xai-org/x-algorithm · warning · ValueError

Failed to get num_servers or num_processors from gRPC servic

Error message

Failed to get num_servers or num_processors from gRPC service

What it means

Defensive re-check in get_server_assignments after get_service_dimensions already validates the gRPC response. It fires only if dimensions came back as None without the earlier ValueError being raised (defensive dead-code guard in normal operation).

Source

Thrown at phoenix/xrex/data/streaming/kafkadispatcherloader.py:76

    async with get_grpc_channel(master_node_host, grpc_port) as channel:
        stub = KafkaDispatcherStub(channel)
        response = await stub.CheckState(CheckStateRequest())
        num_servers = response.num_shards
        num_processors = response.num_processors
        if num_servers is None or num_processors is None:
            raise ValueError(f"Invalid {num_servers=} or {num_processors=} in CheckStateResponse")
        return num_servers, num_processors


async def get_server_assignments(
    grpc_host_template: str,
    grpc_port: int,
    num_clients: int,
    client_ix: int,
) -> tuple[int, int, list[int]]:
    num_servers, num_processors = await get_service_dimensions(grpc_host_template, grpc_port)
    if num_servers is None or num_processors is None:
        raise ValueError("Failed to get num_servers or num_processors from gRPC service")

    assert num_clients % (num_servers * num_processors) == 0, (
        f"Num dataloaders {num_clients} must be a multiple of number of servers {num_servers=} * {num_processors=} = {num_servers * num_processors}"
    )

    global_num_processors = num_servers * num_processors
    num_clients_per_processor = num_clients // global_num_processors
    global_processor_ix = client_ix // num_clients_per_processor
    server_index = global_processor_ix // num_processors
    processor_index = global_processor_ix % num_processors
    clients_assigned_to_processor = [
        a for a in range(num_clients) if a // num_clients_per_processor == global_processor_ix
    ]
    assert len(clients_assigned_to_processor) == num_clients_per_processor, (
        f"Number of clients assigned to server {global_processor_ix} is not equal to num_clients_per_processor {num_clients_per_processor}"
    )
    return server_index, processor_index, clients_assigned_to_processor

View on GitHub (pinned to 24c60942c5)

Solutions

1. Fix the stub/mock in tests to return valid (num_servers, num_processors) tuples.
2. Keep validation in one place: rely on get_service_dimensions' ValueError and remove/ignore this redundant check in forks.
3. If hit in production, check for monkeypatches or duplicated modified code paths.

Example fix

# before (test stub)
monkeypatch.setattr(loader, "get_service_dimensions", async lambda h, p: (None, None))

# after
monkeypatch.setattr(loader, "get_service_dimensions", async lambda h, p: (2, 4))

num_servers, num_processors = await get_service_dimensions(host, port)
if num_servers is None or num_processors is None:
    raise RuntimeError("dispatcher returned invalid dimensions")

When it happens

Trigger: Effectively unreachable when get_service_dimensions raises first; could fire only if get_service_dimensions is monkeypatched or a modified version returns Nones without validating.

Common situations: Custom forks/tests stubbing get_service_dimensions to return (None, None); refactors that bypass the validation inside get_service_dimensions.

Related errors

• Invalid {num_servers=} or {num_processors=} in CheckStateRes
• Client registration failed: {register_response.error_message
• Container is not an InputContainer: {type(container).__name_
• enable_stale_post is unsupported on grpc_recsys, which does
• PhoenixGrpcDataset requires server_address: a gRPC address p

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/480219e4ee170f28. Report an issue: GitHub.