apache/beam · error · BeamIOError

Empty match for pattern

Error message

Empty match for pattern %s. Disallowed.

What it means

_MatchAllFn.process raises BeamIOError when the glob pattern matched zero files and the configured EmptyMatchTreatment disallows empty matches (default). Beam refuses to continue silently with no input files so the pipeline surfaces the problem instead of producing empty output.

Solutions

1. Fix the pattern or ensure files exist before the pipeline reads
2. Pass empty_match_treatment=EmptyMatchTreatment.ALLOW to tolerate zero matches
3. Use ALLOW_IF_WILDCARD and include a wildcard in the pattern
4. Guard with FileSystems.match first if you need custom handling

Example fix

// before
p | MatchAll('gs://b/inbox/*', empty_match_treatment=EmptyMatchTreatment.DISALLOW)
// after
from apache_beam.io.fileio import EmptyMatchTreatment
p | MatchAll('gs://b/inbox/*', empty_match_treatment=EmptyMatchTreatment.ALLOW)

from apache_beam.io.filesystems import FileSystems
if not FileSystems.match([pattern], limits=[1])[0].metadata_list:
    logging.warning('Pattern matches no files: %s', pattern)

from apache_beam.io import fileio
try:
    p | 'match' >> fileio.MatchAll(pattern, empty_match_treatment=EmptyMatchTreatment.ALLOW)
except Exception as e:
    if isinstance(e, fileio.BeamIOError) and 'Empty match' in str(e):
        logging.warning('No files matched %s', pattern)
    else:
        raise

When it happens

Trigger: Running MatchFiles/MatchAll with a pattern that matches nothing while empty_match_treatment is DISALLOW or ALLOW_IF_WILDCARD with no '*' in the pattern; files not yet written at runtime; wrong bucket/prefix.

Common situations: Streaming or scheduled pipelines whose input directory is momentarily empty; typo'd pattern; permissions limiting the listing; tests against empty temp dirs.

Understand the failure class

Background: EmptyResultError / "no results found": when an API or scraper succeeds but returns zero rows — this error's family across 9 libraries.

Related errors

• No files found based on the file pattern
• A schema is required to write non-schema'd data.
• A sink must inherit iobase.Sink, iobase.NativeSink, or be a…
• An explicit schema is required to write non-schema'd…
• AUTO is applicable only to reading files

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/2b622df4a7e41da8. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/fileio.py:178

    elif setting == EmptyMatchTreatment.DISALLOW:
      return False
    else:
      raise ValueError(setting)


class _MatchAllFn(beam.DoFn):
  def __init__(self, empty_match_treatment):
    self._empty_match_treatment = empty_match_treatment

  def process(self, file_pattern: str) -> list[filesystem.FileMetadata]:
    # TODO: Should we batch the lookups?
    match_results = filesystems.FileSystems.match([file_pattern])
    match_result = match_results[0]

    if (not match_result.metadata_list and
        not EmptyMatchTreatment.allow_empty_match(file_pattern,
                                                  self._empty_match_treatment)):
      raise BeamIOError(
          'Empty match for pattern %s. Disallowed.' % file_pattern)

    return match_result.metadata_list


class MatchFiles(beam.PTransform):
  """Matches a file pattern using ``FileSystems.match``.

  This ``PTransform`` returns a ``PCollection`` of matching files in the form
  of ``FileMetadata`` objects."""
  def __init__(
      self,
      file_pattern: str,
      empty_match_treatment=EmptyMatchTreatment.ALLOW_IF_WILDCARD):
    self._file_pattern = file_pattern
    self._empty_match_treatment = empty_match_treatment

  def expand(self, pcoll) -> beam.PCollection[filesystem.FileMetadata]:

View on GitHub (pinned to 12126d8942)