{"record":{"id":"65e1297a245bf7ae","repo":"pathwaycom/pathway","slug":"max-backlog-size-must-be-positive-got-max-back","errorCode":null,"errorMessage":"'max_backlog_size' must be positive; got {max_backlog_size}. A non-positive value would prevent any entry from being processed and the reader would never make progress.","messagePattern":"'max_backlog_size' must be positive; got (.+?)\\. A non-positive value would prevent any entry from being processed and the reader would never make progress\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/pathway/io/kafka/__init__.py","lineNumber":265,"sourceCode":"    \"\"\"\n    # The data_storage is common to all kafka connectors\n\n    if not rdkafka_settings.get(\"bootstrap.servers\"):\n        raise ValueError(\n            \"rdkafka_settings must contain a non-empty 'bootstrap.servers' \"\n            \"entry so the consumer can locate a broker; got \"\n            f\"{rdkafka_settings.get('bootstrap.servers')!r}.\"\n        )\n    if not rdkafka_settings.get(\"group.id\"):\n        raise ValueError(\n            \"rdkafka_settings must contain a non-empty 'group.id' entry: \"\n            \"Pathway's Kafka reader uses 'subscribe' (not 'assign'), which \"\n            \"librdkafka refuses to perform without a configured consumer \"\n            f\"group id; got {rdkafka_settings.get('group.id')!r}.\"\n        )\n\n    if max_backlog_size is not None and max_backlog_size <= 0:\n        raise ValueError(\n            f\"'max_backlog_size' must be positive; got {max_backlog_size}. \"\n            f\"A non-positive value would prevent any entry from being \"\n            f\"processed and the reader would never make progress.\"\n        )\n    if parallel_readers is not None and parallel_readers <= 0:\n        raise ValueError(\n            f\"'parallel_readers' must be positive; got {parallel_readers}.\"\n        )\n    if start_from_timestamp_ms is not None and start_from_timestamp_ms < 0:\n        raise ValueError(\n            f\"'start_from_timestamp_ms' must be non-negative; got \"\n            f\"{start_from_timestamp_ms}. The value is a Unix timestamp in \"\n            f\"milliseconds — negative values are pre-epoch and not \"\n            f\"meaningful for Kafka.\"\n        )\n    if autocommit_duration_ms is not None and autocommit_duration_ms <= 0:\n        raise ValueError(\n            f\"'autocommit_duration_ms' must be positive; got \"","sourceCodeStart":247,"sourceCodeEnd":283,"githubUrl":"https://github.com/pathwaycom/pathway/blob/fa2f74a4649b7c5908690cf60137263d8d80de5f/python/pathway/io/kafka/__init__.py#L247-L283","documentation":"In pw.io.kafka.read, max_backlog_size bounds how many unread Kafka entries may accumulate before the connector slows down reading. A value of zero or below would mean the reader can never buffer anything, so no entry could ever be processed and the pipeline would stall forever — Pathway rejects it up front with ValueError.","triggerScenarios":"pw.io.kafka.read(..., max_backlog_size=0) or max_backlog_size=-5. Passing None is allowed (no limit) — only explicit non-positive integers raise.","commonSituations":"Setting max_backlog_size=0 intending 'no backlog' when the developer actually wants None (unbounded) or a small positive bound; computing the value from another config variable that can degenerate to 0.","solutions":["Use a positive value sized to your throughput, e.g. max_backlog_size=1000.","If you meant 'no limit', pass max_backlog_size=None (or omit the argument).","Validate the value at config-load time if it is computed dynamically."],"exampleFix":"# before\nt = pw.io.kafka.read(rdkafka_settings, topic=\"t\", max_backlog_size=0)\n# after\nt = pw.io.kafka.read(rdkafka_settings, topic=\"t\", max_backlog_size=1000)","handlingStrategy":"validation","validationCode":"if max_backlog_size is not None and max_backlog_size <= 0:\n    raise SystemExit(\"max_backlog_size must be a positive integer or None\")\n\nt = pw.io.kafka.read(rdkafka_settings, topic=\"t\", max_backlog_size=max_backlog_size)","typeGuard":"def valid_backlog(v) -> bool:\n    return v is None or (isinstance(v, int) and v > 0)","tryCatchPattern":null,"preventionTips":["Use None for 'unlimited', never 0.","Validate tuning knobs from config files once at startup.","Document each knob's unit and semantics next to its definition."],"tags":["kafka","validation","backpressure","pathway"],"backgroundTag":null,"analyzedSha":"fa2f74a4649b7c5908690cf60137263d8d80de5f","analyzedAt":"2026-08-15T01:48:17.006Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}