{"record":{"id":"3df87498b51f2f6d","repo":"apache/iceberg","slug":"interrupted-while-polling-queue-3df874","errorCode":null,"errorMessage":"Interrupted while polling queue","messagePattern":"Interrupted while polling queue","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"warning","filePath":"spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/source/AsyncSparkMicroBatchPlanner.java","lineNumber":183,"sourceCode":"        key -> {\n          LOG.info(\n              \"running planFiles for {}, startOffset: {}, endOffset: {}\",\n              table().name(),\n              startOffset,\n              endOffset);\n          List<FileScanTask> result = new LinkedList<>();\n          Pair<StreamingOffset, FileScanTask> elem;\n          StreamingOffset currentOffset;\n          boolean shouldTerminate = false;\n          long filesInPlan = 0;\n          long rowsInPlan = 0;\n\n          do {\n            try {\n              elem = queue.pollFirst(QUEUE_POLL_TIMEOUT_MS, TimeUnit.MILLISECONDS);\n            } catch (InterruptedException e) {\n              Thread.currentThread().interrupt();\n              throw new RuntimeException(\"Interrupted while polling queue\", e);\n            }\n\n            if (elem != null) {\n              currentOffset = elem.first();\n              LOG.debug(\"planFiles consumed: {}\", currentOffset);\n              FileScanTask currentTask = elem.second();\n              filesInPlan += 1;\n              long elemRows = currentTask.file().recordCount();\n              rowsInPlan += elemRows;\n              queuedFileCount.decrementAndGet();\n              queuedRowCount.addAndGet(-elemRows);\n              result.add(currentTask);\n\n              // try to peek at the next entry of the queue and see if we should stop\n              Pair<StreamingOffset, FileScanTask> nextElem = queue.peekFirst();\n              boolean endOffsetPeek = false;\n              if (nextElem != null) {\n                endOffsetPeek = endOffset.equals(nextElem.first());","sourceCodeStart":165,"sourceCodeEnd":201,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/source/AsyncSparkMicroBatchPlanner.java#L165-L201","documentation":"AsyncSparkMicroBatchPlanner.planFiles consumes planned file-scan tasks from a bounded blocking queue, polling with a timeout in a loop. If the polling thread is interrupted while waiting for the next element, it re-interrupts the thread and throws a RuntimeException('Interrupted while polling queue', e).","triggerScenarios":"A streaming query consuming an Iceberg table (micro-batch planning) whose planning thread is interrupted — typically by query.stop(), cancellation of the job, or Spark executor shutdown while a batch is being planned.","commonSituations":"Stopping a Spark structured streaming query mid-batch; executor kill / decommissioning; cancelJobGroup during long planning of a huge snapshot; application shutdown.","solutions":["If the interruption was intentional (query stop/shutdown), no fix needed — this is expected cancellation propagation; handle or ignore the resulting failure in streaming termination logic.","Retry the micro-batch after restart; streaming checkpoints make replanning safe.","Reduce batch planning cost (smaller snapshots, more frequent commits, adequate executor resources) so planning finishes before shutdown windows."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"// streaming side: let Spark retry the batch; wrap stop() calls\ntry {\n  query.awaitTermination()\n} catch {\n  case e: StreamingQueryException if e.getCause.getMessage.contains(\"Interrupted while polling\") =>\n    logger.info(\"Query stopped mid-plan; restart from checkpoint\")\n}","preventionTips":["Rely on streaming checkpoints for safe re-planning after restarts","Avoid killing executors during batch planning windows","Size resources so planning of large snapshots completes before shutdown"],"tags":["spark","streaming","interruption","concurrency"],"backgroundTag":"thread-interrupted","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}