xai-org/x-algorithm · error · ValueError

num_shards must be > 0, got {num_shards}

Error message

num_shards must be > 0, got {num_shards}

What it means

Internal validation in _range_partitions: the sharding math requires at least one shard. num_shards <= 0 (often 0 from an uninitialized/derived value) makes partition range computation meaningless, so a ValueError is raised immediately.

Source

Thrown at phoenix/xrex/data/streaming/kafkaconsumer.py:473

            partitions = await force_kafka_metadata_update(consumer, topic)
        if not partitions:
            available = await consumer.topics()
            raise ValueError(
                f"No partitions found for topic '{topic}'. Available topics: {available}"
            )
        count = len(partitions)
        rank_logger.info(f"Discovered {count} partitions for topic '{topic}' from Kafka metadata.")
        return count
    finally:
        try:
            await consumer.stop()
        except asyncio.CancelledError:
            pass


def _range_partitions(total_partitions: int, shard_index: int, num_shards: int) -> list[int]:
    if num_shards <= 0:
        raise ValueError(f"num_shards must be > 0, got {num_shards}")
    if total_partitions < 0:
        raise ValueError(f"total_partitions must be >= 0, got {total_partitions}")
    start = shard_index * total_partitions // num_shards
    end = (shard_index + 1) * total_partitions // num_shards
    return list(range(start, end))


async def handle_topic_offset(
    consumer: AIOKafkaConsumer,
    topic: str,
    shard_index: int,
    num_shards: int,
    reset_to_latest: bool,
    seek_to_timestamp_ms: int | None = None,
    seek_to_offset: dict[int, int] | None = None,
) -> list[TopicPartition]:
    assert seek_to_timestamp_ms is None or seek_to_offset is None, (
        "seek_to_timestamp_ms and seek_to_offset cannot both be set"

View on GitHub (pinned to 24c60942c5)

Solutions

1. Trace where num_shards comes from and ensure it is a positive integer (typically world_size or number of server shards).
2. Add a config validation step at startup: assert num_shards >= 1 before entering the consume path.
3. If derived from gRPC CheckState (num_servers), check the dispatcher service returned sane dimensions.

Example fix

# before
partition_ids = _range_partitions(total_partitions, shard_index, num_shards=0)

# after
assert num_shards >= 1, f"num_shards must be >= 1, got {num_shards}"
partition_ids = _range_partitions(total_partitions, shard_index, num_shards)

if not isinstance(num_shards, int) or num_shards < 1:
    raise ConfigError(f"num_shards must be a positive int, got {num_shards!r}")

def is_valid_shard_config(shard_index: int, num_shards: int) -> bool:
    return isinstance(num_shards, int) and num_shards > 0 and 0 <= shard_index < num_shards

When it happens

Trigger: Calling handle_topic_offset or _consume_multi_consumer with num_shards=0 (e.g. num_servers or world_size computed as 0), or a negative shard count passed via config/CLI.

Common situations: Distributed launcher returned world_size 0 because the process group was not initialized; num_shards derived from gRPC num_servers=0; CLI flag parsed with a default of 0 and never set.

Related errors

• total_partitions must be >= 0, got {total_partitions}
• Invalid argument {arg!r}, not a key=value replacement and no
• Expected bool [True, true, False, false], got {val!r}
• Got {val} for union type {ty}
• Tuple {ty} has different number of arguments from given valu

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/1c37738b84e8267a. Report an issue: GitHub.