{"record":{"id":"f12b89512058a541","repo":"t8y2/dbx","slug":"kafka-topic-has-more-than-max-peek-scan-records","errorCode":null,"errorMessage":"Kafka topic has more than ${MAX_PEEK_SCAN_RECORDS} readable partitions; select a partition to browse latest messages","messagePattern":"Kafka topic has more than (.+?) readable partitions; select a partition to browse latest messages","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"agents/drivers/kafka/src/main/java/com/dbx/agent/kafka/KafkaAgent.java","lineNumber":2167,"sourceCode":"        boolean legacyOffsetRequest,\n        long beginningOffset,\n        long endOffset\n    ) {\n        return switch (startPosition) {\n            case LATEST -> endOffset > beginningOffset ? beginningOffset : null;\n            case OFFSET -> offset;\n            case EARLIEST -> legacyOffsetRequest ? offset : beginningOffset;\n        };\n    }\n\n    static int peekScanLimit(int count, int readablePartitionCount) {\n        return peekScanLimit(count, readablePartitionCount, PeekStartPosition.EARLIEST);\n    }\n\n    static int peekScanLimit(int count, int readablePartitionCount, PeekStartPosition startPosition) {\n        if (startPosition == PeekStartPosition.LATEST\n            && readablePartitionCount > MAX_PEEK_SCAN_RECORDS) {\n            throw new IllegalArgumentException(\n                \"Kafka topic has more than \" + MAX_PEEK_SCAN_RECORDS\n                    + \" readable partitions; select a partition to browse latest messages\"\n            );\n        }\n        return MAX_PEEK_SCAN_RECORDS;\n    }\n\n    /**\n     * Latest is a topic-level query. Below the scan budget, every partition contributes the\n     * requested count so the global merge is exact. Above it, the fixed budget is shared across\n     * partitions and the response is marked incomplete.\n     */\n    static int latestPeekMessagesPerPartition(int count, int readablePartitionCount) {\n        int safePartitionCount = Math.max(1, readablePartitionCount);\n        if (safePartitionCount > MAX_PEEK_SCAN_RECORDS) {\n            throw new IllegalArgumentException(\n                \"Kafka topic has more than \" + MAX_PEEK_SCAN_RECORDS\n                    + \" readable partitions; select a partition to browse latest messages\"","sourceCodeStart":2149,"sourceCodeEnd":2185,"githubUrl":"https://github.com/t8y2/dbx/blob/c0390bff16418b651f4728520d99adf8ce48829a/agents/drivers/kafka/src/main/java/com/dbx/agent/kafka/KafkaAgent.java#L2149-L2185","documentation":"This guard fires when a Kafka topic has more readable partitions than MAX_PEEK_SCAN_RECORDS while browsing latest messages across all partitions. Scanning every partition's latest offset requires one query per partition, so the agent caps the partition count to keep the peek operation bounded; past the cap it refuses and asks the caller to select a single partition.","triggerScenarios":"Calling the peek/latest-messages browse tool (peekScanLimit with PeekStartPosition.LATEST) on a topic whose readablePartitionCount exceeds MAX_PEEK_SCAN_RECORDS.","commonSituations":"High-partition topics (hundreds of partitions) created for throughput; auto-created topics with default partition counts; browsing 'latest' against a busy production topic instead of a small test topic.","solutions":["Browse a specific partition instead of all partitions (pass a partition selection in the browse request).","Reduce the topic's partition count (topic reassignment/recreation) if full-topic latest browsing is truly needed.","Read the latest messages via a different path (e.g. subscribe with seek-to-end) rather than the peek browse API.","Raise MAX_PEEK_SCAN_RECORDS in the agent build if your deployment can afford the scan cost."],"exampleFix":"// before\nagent.call(\"kafka.peek_latest\", {\"topic\": \"events\"})\n// after\nagent.call(\"kafka.peek_latest\", {\"topic\": \"events\", \"partition\": 3})","handlingStrategy":"validation","validationCode":"int readablePartitions = getReadablePartitionCount(topic);\nif (readablePartitions > MAX_PEEK_SCAN_RECORDS) {\n    // browse a single partition instead\n    request.setPartition(0);\n}","typeGuard":null,"tryCatchPattern":"try {\n    agent.browseLatest(topic);\n} catch (IllegalArgumentException e) {\n    if (e.getMessage().contains(\"readable partitions\")) {\n        agent.browseLatest(topic, partition); // fall back to single partition\n    } else { throw e; }\n}","preventionTips":["Track partition counts of topics you browse and prefer per-partition browsing on wide topics.","For latest-message checks on large topics, use a consumer with seekToEnd instead of the peek API.","Keep test topics small; avoid pointing browse tooling at high-partition production topics."],"tags":["kafka","partitions","limit-exceeded","peek"],"backgroundTag":"partition-limit-exceeded","analyzedSha":"c0390bff16418b651f4728520d99adf8ce48829a","analyzedAt":"2026-09-05T23:05:10.900Z","contentChangedAt":"2026-09-05T23:05:10.900Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}