{"record":{"id":"30eba6861a3e1372","repo":"apache/beam","slug":"unable-to-read-file-s-after-retrying-d-times-30eba6","errorCode":null,"errorMessage":"Unable to read file(s) after retrying %d times","messagePattern":"Unable to read file\\(s\\) after retrying (.+?) times","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"sdks/java/core/src/main/java/org/apache/beam/sdk/util/NumberedShardedFile.java","lineNumber":138,"sourceCode":"            Iterables.getOnlyElement(FileSystems.match(Collections.singletonList(filePattern)))\n                .metadata();\n\n        LOG.debug(\"Found {} file(s) by matching the path: {}\", files.size(), filePattern);\n\n        if (files.isEmpty() || !checkTotalNumOfFiles(files)) {\n          continue;\n        }\n\n        // Read data from file paths\n        return readLines(files);\n      } catch (IOException e) {\n        // Ignore and retry\n        lastException = e;\n        LOG.warn(\"Error in file reading. Ignore and retry.\");\n      }\n    } while (BackOffUtils.next(sleeper, backOff));\n    // Failed after max retries\n    throw new IOException(\n        String.format(\"Unable to read file(s) after retrying %d times\", MAX_READ_RETRIES),\n        lastException);\n  }\n\n  /**\n   * Discovers all shards of this file.\n   *\n   * <p>Because of eventual consistency, reads may discover no files or fewer files than the shard\n   * template implies. In this case, the read is considered to have failed.\n   */\n  public List<String> readFilesWithRetries() throws IOException, InterruptedException {\n    return readFilesWithRetries(Sleeper.DEFAULT, BACK_OFF_FACTORY.backoff());\n  }\n\n  @Override\n  public String toString() {\n    return String.format(\"%s with shard template '%s'\", filePattern, shardTemplate);\n  }","sourceCodeStart":120,"sourceCodeEnd":156,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/core/src/main/java/org/apache/beam/sdk/util/NumberedShardedFile.java#L120-L156","documentation":"NumberedShardedFile.readFilesWithRetries retries reading a sharded file (e.g. a Dataflow job's output shards) using a bounded BackOff. If all retries fail — because shards are missing, the pattern matches nothing, or the filesystem keeps erroring — it throws an IOException stating the retry count was exhausted, with the last exception as cause.","triggerScenarios":"MatchResult/file pattern fails on every retry: output directory has no shards, shard template doesn't match actual files (e.g. wrong shard name template in tests), or the filesystem returns transient errors for MAX_READ_RETRIES attempts.","commonSituations":"Assertions on Dataflow job outputs where the template path doesn't match produced shard filenames; missing/empty output directory; GCS permission or throttling errors persisting through the backoff window.","solutions":["Check getCause()/logs for the underlying per-retry exceptions.","Verify the file pattern/shard template matches the actual output naming (e.g. shard-N-of-M format).","Confirm the output directory exists and contains at least one shard before calling.","Check filesystem permissions/quotas; wait for the writing job to finish before reading."],"exampleFix":"// before\nnew NumberedShardedFile(\"gs://bucket/out\", \"\").readFilesWithRetries(fs);\n// after\nnew NumberedShardedFile(\"gs://bucket/out\", \"SSS-of-NNN\").readFilesWithRetries(fs);","handlingStrategy":"retry","validationCode":"// before reading:\nMatchResult mr = fs.match(pattern);\nif (mr.status() != MatchResult.Status.OK || mr.metadata().isEmpty())\n  throw new IOException(\"no shards matched \" + pattern);","typeGuard":"null","tryCatchPattern":"try { files = shardedFile.readFilesWithRetries(fs); } catch (IOException e) { LOG.error(\"shard read failed after retries\", e.getCause()); }","preventionTips":["Ensure the writing job has completed before asserting/reading shards","Verify the shard template matches the actual naming (e.g. SSS-of-NNN)","Check bucket permissions and quota before long read loops"],"tags":["java","io","retry","gcs"],"backgroundTag":"file-read-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}