apache/beam · error · IOError

No files found based on the file pattern

Error message

No files found based on the file pattern %s

What it means

Raised as an IOError by FileBasedSource._validate when a FileSystems.match() call against the user-supplied glob pattern returns zero metadata entries. Beam validates at pipeline-construction time that the pattern actually matches at least one file before building split sources. It fails fast so a bad pattern is caught before the job runs.

Solutions

1. Verify the pattern matches by running apache_beam.io.filesystems.FileSystems.match([pattern]) locally and inspecting the result
2. List the actual location (gsutil ls / aws s3 ls) to confirm files exist and correct the prefix or glob
3. Check that credentials for the filesystem are set so listing is permitted
4. If zero files is legitimately possible, use the empty_match_treatment/allow_empty_match option of the fileio transforms instead of the classic IO

Example fix

// before
lines = p | 'read' >> beam.io.ReadFromText('gs://my-bucket/data/2026-09-1*.json')
// after
# verify pattern first
from apache_beam.io.filesystems import FileSystems
assert FileSystems.match(['gs://my-bucket/data/2026-09-1*.json'])[0].metadata_list
lines = p | 'read' >> beam.io.ReadFromText('gs://my-bucket/data/2026-09-1*.json')

from apache_beam.io.filesystems import FileSystems
result = FileSystems.match([pattern], limits=[1])[0]
if not result.metadata_list:
    raise FileNotFoundError(f'Pattern matches no files: {pattern}')

def pattern_matches_files(pattern: str) -> bool:
    from apache_beam.io.filesystems import FileSystems
    return len(FileSystems.match([pattern], limits=[1])[0].metadata_list) > 0

import errno
try:
    lines = p | beam.io.ReadFromText(pattern)
except IOError as e:
    if e.errno == errno.ENOENT or 'No files found' in str(e):
        logging.warning('No files for %s; using empty fallback', pattern)
    else:
        raise

When it happens

Trigger: Passing a glob (e.g. 'gs://bucket/data/*.json') or literal path to ReadFromText/ReadFromAvro/etc. when no file exists at that location, the bucket/prefix is misspelled, the files were deleted/moved, or credentials restrict listing so the match returns nothing.

Common situations: Typos in bucket names or prefixes; reading yesterday's partition files that haven't been written yet; wrong GCS/AWS credentials limiting list results; using a pattern without wildcards pointing at a single missing file; Windows path separators in a pattern.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• Checksum operation failed
• chunk write failed
• Copy operation failed
• Could not create a temporary directory for storing…
• Could not create a temporary directory for storing…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/020ecb581eff4c9b. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/filebasedsource.py:191

    return self._concat_source

  def open_file(self, file_name):
    return FileSystems.open(
        file_name,
        'application/octet-stream',
        compression_type=self._compression_type)

  @check_accessible(['_pattern'])
  def _validate(self):
    """Validate if there are actual files in the specified glob pattern
    """
    pattern = self._pattern.get()

    # Limit the responses as we only want to check if something exists
    match_result = FileSystems.match([pattern], limits=[1])[0]
    if len(match_result.metadata_list) <= 0:
      raise IOError('No files found based on the file pattern %s' % pattern)

  def split(
      self, desired_bundle_size=None, start_position=None, stop_position=None):
    return self._get_concat_source().split(
        desired_bundle_size=desired_bundle_size,
        start_position=start_position,
        stop_position=stop_position)

  def estimate_size(self):
    return self._get_concat_source().estimate_size()

  def read(self, range_tracker):
    return self._get_concat_source().read(range_tracker)

  def get_range_tracker(self, start_position, stop_position):
    return self._get_concat_source().get_range_tracker(
        start_position, stop_position)

View on GitHub (pinned to 12126d8942)