{"record":{"id":"d2b042703675bde5","repo":"apache/beam","slug":"unable-to-read-file-s-after-retrying-d-times","errorCode":null,"errorMessage":"Unable to read file(s) after retrying %d times","messagePattern":"Unable to read file\\(s\\) after retrying (.+?) times","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"sdks/java/core/src/main/java/org/apache/beam/sdk/util/ExplicitShardedFile.java","lineNumber":88,"sourceCode":"      throws IOException, InterruptedException {\n    if (files.isEmpty()) {\n      return Collections.emptyList();\n    }\n\n    IOException lastException = null;\n\n    do {\n      try {\n        // Read data from file paths\n        return readLines(files);\n      } catch (IOException e) {\n        // Ignore and retry\n        lastException = e;\n        LOG.warn(\"Error in file reading. Ignore and retry.\");\n      }\n    } while (BackOffUtils.next(sleeper, backOff));\n    // Failed after max retries\n    throw new IOException(\n        String.format(\"Unable to read file(s) after retrying %d times\", MAX_READ_RETRIES),\n        lastException);\n  }\n\n  /**\n   * Discovers all shards of this file.\n   *\n   * <p>Because of eventual consistency, reads may discover no files or fewer files than the shard\n   * template implies. In this case, the read is considered to have failed.\n   */\n  public List<String> readFilesWithRetries() throws IOException, InterruptedException {\n    return readFilesWithRetries(Sleeper.DEFAULT, BACK_OFF_FACTORY.backoff());\n  }\n\n  @Override\n  public String toString() {\n    return String.format(\"explicit sharded file (%s)\", Joiner.on(\", \").join(files));\n  }","sourceCodeStart":70,"sourceCodeEnd":106,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/core/src/main/java/org/apache/beam/sdk/util/ExplicitShardedFile.java#L70-L106","documentation":"ExplicitShardedFile.readFilesWithRetries reads all shards of an explicitly-sharded file, retrying up to MAX_READ_RETRIES times with exponential backoff. If every attempt throws (missing files, transient filesystem/IO errors, permissions), it gives up and wraps the last IOException in this message. The causing exception is attached as the cause.","triggerScenarios":"Calling readFilesWithRetries() when the underlying FileSystem keeps throwing IOException for the full retry budget: file(s) deleted mid-run, GCS/S3 throttling or auth failures, wrong path, or network outage persisting longer than the backoff window.","commonSituations":"Streaming pipelines reading intermediate files from cloud storage during transient outages; typos in shard paths so matches are empty/404; GCS 429/503 rate limits exceeding the retry window; credentials expiring between retries.","solutions":["Inspect the chained cause (lastException) to see the real per-attempt error and fix that root cause first","Verify the shard paths exist and are readable from the pipeline's service account/credentials","For cloud storage, check for rate limiting or transient outages and retry the pipeline after backoff","Increase MAX_READ_RETRIES or the backoff configuration if the failure is genuinely transient"],"exampleFix":"// before\nList<String> lines = new ExplicitShardedFile(\"gs://bucket/wrong-prefix/shard-*\").readFilesWithRetries();\n// after\nString pattern = \"gs://bucket/correct-prefix/shard-*\"; // verified with FileSystems.match()\nList<String> lines = new ExplicitShardedFile(pattern).readFilesWithRetries();","handlingStrategy":"retry","validationCode":"MatchResult r = FileSystems.match(shardedFile.toString());\nif (r.status() != MatchResult.Status.OK || r.metadata().isEmpty()) {\n  throw new IllegalStateException(\"No shards found for pattern before read: \" + shardedFile);\n}","typeGuard":null,"tryCatchPattern":"try {\n  lines = shardedFile.readFilesWithRetries();\n} catch (IOException e) {\n  LOG.error(\"Shard read exhausted retries; cause: {}\", e.getCause(), e);\n  throw new RuntimeException(\"Unreadable shards for \" + shardedFile, e);\n}","preventionTips":["Pre-verify shard existence with FileSystems.match before reading","Check service-account read permissions on the storage bucket","Monitor for cloud-storage rate limits (429/503) in pipeline logs"],"tags":["java","io","file-read","retry-exhausted"],"backgroundTag":"file-read-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}