xai-org/x-algorithm · critical · ValueError

No partitions found for topic {topic}, available topics: {av

Error message

No partitions found for topic {topic}, available topics: {available_topics}

What it means

Raised in handle_topic_offset when, after partitions_for_topic plus two forced metadata refreshes, no partitions for the requested topic are known; the error lists topics the consumer can see. This blocks the consume path because there is nothing to assign or seek.

Source

Thrown at phoenix/xrex/data/streaming/kafkaconsumer.py:510

    )
    assert not (seek_to_timestamp_ms is not None and reset_to_latest), (
        "seek_to_timestamp_ms and reset_to_latest cannot both be set"
    )
    rank_logger.info(
        f"Try to get partition info for {topic}, shard_index: {shard_index}, num_shards: {num_shards}."
    )

    partitions: set[int] | None = consumer.partitions_for_topic(topic)
    if not partitions:
        partitions = await force_kafka_metadata_update(consumer, topic)
    if not partitions:
        partitions = await force_kafka_metadata_update(consumer, topic)
    if not partitions:
        available_topics = await consumer.topics()
        rank_logger.error(
            f"No partitions found for topic {topic}, available topics: {available_topics}"
        )
        raise ValueError(
            f"No partitions found for topic {topic}, available topics: {available_topics}"
        )

    total_partitions = len(partitions)

    partition_ids = _range_partitions(total_partitions, shard_index, num_shards)
    if not partition_ids:
        rank_logger.warning(
            f"Worker {shard_index} received 0 partitions (total_partitions={total_partitions}, "
            f"num_shards={num_shards}).  This worker will be idle."
        )
    assigned_partitions = [TopicPartition(topic, i) for i in partition_ids]
    for tp in assigned_partitions:
        rank_logger.info(
            f"Assigned partition {tp} to worker {shard_index}, num_shards: {num_shards}, total_partitions: {total_partitions}."
        )
    consumer.assign(assigned_partitions)
    rank_logger.info(

View on GitHub (pinned to 24c60942c5)

Solutions

1. Use the 'available topics' list in the message to confirm the topic name and cluster, fix config typos or environment mismatch.
2. Pre-create the topic and wait for metadata propagation (e.g. kafka-topics --describe) before starting the consumer.
3. Add a startup readiness check/retry loop that polls partitions_for_topic until non-empty or a timeout.
4. Verify network/ACL access to all brokers, not just the bootstrap one.

Example fix

# before
await consume_messages(topic="events-prod", ...)

# after
for _ in range(30):
    if await consumer.partitions_for_topic(topic):
        break
    await asyncio.sleep(2)
await consume_messages(topic="events-prod", ...)  # proceed only when partitions visible

async def wait_for_partitions(consumer, topic, timeout=60.0) -> list:
    deadline = time.monotonic() + timeout
    while time.monotonic() < deadline:
        parts = await consumer.partitions_for_topic(topic)
        if parts:
            return parts
        await asyncio.sleep(2)
    raise TimeoutError(f"topic {topic} never became visible")

try:
    await handle_topic_offset(...)
except ValueError as e:
    if "available topics" in str(e):
        # log available list, fix config or retry after provisioning
        ...

When it happens

Trigger: Calling consume_messages with a topic absent from the cluster metadata; brokers returning empty metadata because they are unreachable; topic created moments ago and metadata not yet propagated even after forced refresh.

Common situations: Race condition where the consumer starts before the topic is provisioned; wrong bootstrap servers for the environment; broker DNS resolving but brokers not accepting connections so topics() returns partial/empty results; ACL restrictions hiding the topic.

Related errors

• No partitions found for topic '{topic}'. Available topics: {
• No partitions found for topic {topic}, available topics: {av
• Kafka mTLS requires CA at {ca_file!r} (mount internal-ca Con
• `clusters` must contain at least one region
• Region {region!r} must list at least one broker

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/8908ab02f798e5c8. Report an issue: GitHub.