apache/beam · error · IOException

Unable to read file(s) after retrying

Error message

Unable to read file(s) after retrying %d times

What it means

FilePatternMatchingShardedFile.readFilesWithRetries discovers shards matching a file pattern and reads them, retrying up to MAX_READ_RETRIES times with exponential backoff. If all attempts fail (or the output directory stays empty), it throws this IOException with the last per-attempt exception attached as cause.

Solutions

1. Check the chained cause for the underlying per-attempt error and fix that
2. Verify the file pattern matches existing shards (use FileSystems.match(pattern) to test)
3. Confirm the upstream job actually completed and wrote the expected shards before reading
4. If failures are transient (throttling), rerun later or increase retry budget

Example fix

// before
FilePatternMatchingShardedFile f = new FilePatternMatchingShardedFile("gs://bucket/missing-output/*");
List<String> lines = f.readFilesWithRetries();
// after
MatchResult match = FileSystems.match("gs://bucket/actual-output/*");
if (match.status() == MatchResult.Status.OK && !match.metadata().isEmpty()) {
  List<String> lines = new FilePatternMatchingShardedFile("gs://bucket/actual-output/*").readFilesWithRetries();
}

MatchResult r = FileSystems.match(filePattern);
if (r.status() != MatchResult.Status.OK || r.metadata().isEmpty()) {
  throw new IllegalStateException("File pattern matched no shards: " + filePattern);
}

try {
  lines = shardedFile.readFilesWithRetries();
} catch (IOException e) {
  LOG.error("Read failed after retries. Root cause: {}", e.getCause(), e);
  throw e;
}

When it happens

Trigger: Calling readFilesWithRetries() when the file pattern matches nothing (empty output dir) or every read attempt throws IOException for the entire backoff window — e.g. writer job never produced shards, wrong pattern, or persistent storage errors.

Common situations: Runner-side reads of write-then-read intermediate data where the upstream write failed silently; file pattern typo (wrong stage/output directory); GCS/S3 throttling; eventual consistency delays where shards aren't visible yet.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Unable to read file(s) after retrying
• Error completing file copies with retries, sample: from
• AUTO is applicable only to reading files
• AUTO is not supported for writing
• Caller does not own the underlying input stream and should…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/5d81c7c7fa9cfa59. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/util/FilePatternMatchingShardedFile.java:108

            "Found file(s) {} by matching the path: {}",
            files.stream()
                .map(Metadata::resourceId)
                .map(ResourceId::getFilename)
                .collect(Collectors.joining(",")),
            filePattern);
        if (files.isEmpty()) {
          continue;
        }
        // Read data from file paths
        return readLines(files);
      } catch (IOException e) {
        // Ignore and retry
        lastException = e;
        LOG.warn("Error in file reading. Ignore and retry.");
      }
    } while (BackOffUtils.next(sleeper, backOff));
    // Failed after max retries
    throw new IOException(
        String.format("Unable to read file(s) after retrying %d times", MAX_READ_RETRIES),
        lastException);
  }

  /** Discovers all shards of this file. */
  public List<String> readFilesWithRetries() throws IOException, InterruptedException {
    return readFilesWithRetries(Sleeper.DEFAULT, BACK_OFF_FACTORY.backoff());
  }

  @Override
  public String toString() {
    return String.format("sharded file matching pattern: %s", filePattern);
  }

  /**
   * Reads all the lines of all the files.
   *
   * <p>Not suitable for use except in testing of small data, since the data size may be far more

View on GitHub (pinned to 12126d8942)