{"record":{"id":"f4c2677bbe947325","repo":"apache/iceberg","slug":"table-refresh-failed","errorCode":null,"errorMessage":"Table refresh failed","messagePattern":"Table refresh failed","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"critical","filePath":"spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/AsyncSparkMicroBatchPlanner.java","lineNumber":216,"sourceCode":"              Pair<StreamingOffset, FileScanTask> nextElem = queue.peekFirst();\n              boolean endOffsetPeek = false;\n              if (nextElem != null) {\n                endOffsetPeek = endOffset.equals(nextElem.first());\n              }\n              // end offset may be synthetic and not exist in the queue\n              boolean endOffsetSynthetic =\n                  currentOffset.snapshotId() == endOffset.snapshotId()\n                      && (currentOffset.position() + 1) == endOffset.position();\n              shouldTerminate = endOffsetPeek || endOffsetSynthetic;\n            } else {\n              LOG.trace(\"planFiles hasn't reached {}, waiting\", endOffset);\n            }\n          } while (!shouldTerminate\n              && refreshFailedThrowable == null\n              && fillQueueFailedThrowable == null);\n\n          if (refreshFailedThrowable != null) {\n            throw new RuntimeException(\"Table refresh failed\", refreshFailedThrowable);\n          }\n\n          if (fillQueueFailedThrowable != null) {\n            throw new RuntimeException(\"Queue filling failed\", fillQueueFailedThrowable);\n          }\n\n          LOG.info(\n              \"completed planFiles for {}, startOffset: {}, endOffset: {}, files: {}, rows: {}\",\n              table().name(),\n              startOffset,\n              endOffset,\n              filesInPlan,\n              rowsInPlan);\n          return result;\n        });\n  }\n\n  /**","sourceCodeStart":198,"sourceCodeEnd":234,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/AsyncSparkMicroBatchPlanner.java#L198-L234","documentation":"planFiles runs a background refresh loop whose failures are captured in refreshFailedThrowable and rethrown after the loop ends as \"Table refresh failed\" with the original exception as cause. It means the table refresh inside the async micro-batch planner failed, so planning cannot produce a valid end offset.","triggerScenarios":"The background refresh thread's table refresh throws (e.g. metadata file unreadable, catalog/network error, snapshot expiry race) while the batch planner loop is polling.","commonSituations":"Snapshot expiration deleting metadata mid-stream, catalog connectivity problems, object-store permission errors, or table concurrently overwritten by writers.","solutions":["Inspect the cause of this RuntimeException for the real refresh failure.","Check catalog and object-store connectivity/permissions for the table location.","Verify metadata files were not removed by concurrent expire_snapshots; adjust expiration retention.","Restart the streaming query after fixing the underlying cause; planning will re-plan from the last committed offset."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  query.awaitTermination();\n} catch (StreamingQueryException e) {\n  Throwable cause = e.getCause();\n  if (cause != null && cause.getMessage() != null && cause.getMessage().contains(\"Table refresh failed\")) {\n    // inspect cause.getCause() for the underlying refresh failure; restart after fixing\n    restartQueryWithBackoff();\n  } else {\n    throw e;\n  }\n}","preventionTips":["Keep expire_snapshots retention longer than max stream downtime.","Monitor catalog/object-store availability for streaming sources.","Implement automatic restart with backoff for streaming queries."],"tags":["spark","streaming","table-refresh","io"],"backgroundTag":"internal-invariant-violation","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}