{"record":{"id":"8b5a3f7b19d43988","repo":"apache/iceberg","slug":"table-refresh-failed-8b5a3f","errorCode":null,"errorMessage":"Table refresh failed","messagePattern":"Table refresh failed","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/source/AsyncSparkMicroBatchPlanner.java","lineNumber":216,"sourceCode":"              Pair<StreamingOffset, FileScanTask> nextElem = queue.peekFirst();\n              boolean endOffsetPeek = false;\n              if (nextElem != null) {\n                endOffsetPeek = endOffset.equals(nextElem.first());\n              }\n              // end offset may be synthetic and not exist in the queue\n              boolean endOffsetSynthetic =\n                  currentOffset.snapshotId() == endOffset.snapshotId()\n                      && (currentOffset.position() + 1) == endOffset.position();\n              shouldTerminate = endOffsetPeek || endOffsetSynthetic;\n            } else {\n              LOG.trace(\"planFiles hasn't reached {}, waiting\", endOffset);\n            }\n          } while (!shouldTerminate\n              && refreshFailedThrowable == null\n              && fillQueueFailedThrowable == null);\n\n          if (refreshFailedThrowable != null) {\n            throw new RuntimeException(\"Table refresh failed\", refreshFailedThrowable);\n          }\n\n          if (fillQueueFailedThrowable != null) {\n            throw new RuntimeException(\"Queue filling failed\", fillQueueFailedThrowable);\n          }\n\n          LOG.info(\n              \"completed planFiles for {}, startOffset: {}, endOffset: {}, files: {}, rows: {}\",\n              table().name(),\n              startOffset,\n              endOffset,\n              filesInPlan,\n              rowsInPlan);\n          return result;\n        });\n  }\n\n  /**","sourceCodeStart":198,"sourceCodeEnd":234,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/source/AsyncSparkMicroBatchPlanner.java#L198-L234","documentation":"This error wraps any Throwable caught by the background refresh thread inside AsyncSparkMicroBatchPlanner. The planner refreshes the table asynchronously in a background loop; exceptions from that loop are stored and re-raised as a RuntimeException when the Spark micro-batch thread next calls planFiles, so streaming query failures surface at the batch planning point.","triggerScenarios":"A background thread refreshing the Iceberg table (e.g. during ongoing micro-batch planning) throws; the stored refreshFailedThrowable is non-null when planFiles exits its do/while loop and rethrows it.","commonSituations":"Metastore or file-system outages during streaming reads; catalog authorization failures mid-stream; transient network errors refreshing table metadata from a long-running Spark structured streaming query.","solutions":["Inspect the refreshFailedThrowable cause in the stack trace to identify the underlying refresh failure","Check catalog/metastore connectivity and credentials used by the streaming query","Retry the query; refresh failures are often transient network/metastore issues","Pin table state or reduce refresh frequency if the table is being mutated concurrently by writers"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { planFiles(...) } catch (RuntimeException e) { Throwable cause = e.getCause(); if (cause is transient) restart query with checkpointing; else alert }","preventionTips":["Monitor catalog/metastore health for long-running streaming queries","Use Spark checkpointing so failed queries can resume","Avoid aggressive concurrent snapshot expiry while streaming"],"tags":["spark","streaming","table-refresh"],"backgroundTag":"api-error-response","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}