{"record":{"id":"46cf4eadcbbbf8f5","repo":"apache/beam","slug":"directories-are-not-allowed-in-readmatches-transform-found-s","errorCode":null,"errorMessage":"Directories are not allowed in ReadMatches transform.Found %s.","messagePattern":"Directories are not allowed in ReadMatches transform\\.Found (.+?)\\.","errorType":"exception","errorClass":"BeamIOError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/fileio.py","lineNumber":253,"sourceCode":"\nclass _ReadMatchesFn(beam.DoFn):\n  def __init__(self, compression, skip_directories):\n    self._compression = compression\n    self._skip_directories = skip_directories\n\n  def process(\n      self,\n      file_metadata: Union[str, filesystem.FileMetadata],\n  ) -> Iterable[ReadableFile]:\n    metadata = (\n        filesystem.FileMetadata(file_metadata, 0) if isinstance(\n            file_metadata, str) else file_metadata)\n\n    if ((metadata.path.endswith('/') or metadata.path.endswith('\\\\')) and\n        self._skip_directories):\n      return\n    elif metadata.path.endswith('/') or metadata.path.endswith('\\\\'):\n      raise BeamIOError(\n          'Directories are not allowed in ReadMatches transform.'\n          'Found %s.' % metadata.path)\n\n    # TODO: Mime type? Other arguments? Maybe arguments passed in to transform?\n    yield ReadableFile(metadata, self._compression)\n\n\nclass _PollClock(object):\n  \"\"\"Shares one clock reading per poll round, so the start gate and the poll\n  budget judge the ``start_timestamp`` boundary consistently.\"\"\"\n  def __init__(self):\n    self.last_poll_micros: Optional[int] = None\n\n\nclass _WatchWindowTermination(TerminationCondition):\n  \"\"\"Stops after the polls that fall in the ``[start, stop)`` window.\n\n  ``max_polls`` is the ``PeriodicImpulse`` tick count","sourceCodeStart":235,"sourceCodeEnd":271,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/fileio.py#L235-L271","documentation":"ReadMatches.process raises BeamIOError when a matched metadata path is a directory (ends with '/' or backslash) and skip_directories is False. ReadMatches can only turn file blobs into ReadableFile objects, so directories cannot be read. Matching patterns like 'dir/*' that match directories directly trigger this.","triggerScenarios":"Using a match pattern that includes directories (e.g. 'gs://b/data/*' with subdirectories present) while skip_directories=False; explicitly configuring skip_directories=False and feeding directory entries.","commonSituations":"Nested directory trees under a globbed prefix; cloud-storage listings that include directory placeholders; users setting skip_directories=False expecting nested traversal instead of an error.","solutions":["Enable skip_directories=True (default) on ReadMatches so directory entries are silently skipped","Refine the glob to exclude directories or match only the file extension (e.g. '*.json')","Pre-filter matches with a filter transform checking not path.endswith('/')"],"exampleFix":"// before\np | MatchAll('gs://b/data/*') | ReadMatches(skip_directories=False)\n// after\np | MatchAll('gs://b/data/*') | ReadMatches(skip_directories=True)  # default; skips dirs","handlingStrategy":"validation","validationCode":"bad = [m.path for m in matches if m.path.endswith('/') or m.path.endswith('\\\\')]\nif bad:\n    logging.warning('Directory entries will fail ReadMatches: %s', bad)","typeGuard":null,"tryCatchPattern":"from apache_beam.io.fileio import ReadMatches\ntry:\n    files = matches | ReadMatches()\nexcept Exception as e:\n    if 'Directories are not allowed' in str(e):\n        matches = matches | beam.Filter(lambda m: not (m.path.endswith('/') or m.path.endswith('\\\\')))\n        files = matches | ReadMatches()\n    else:\n        raise","preventionTips":["Keep skip_directories=True (the default)","Glob on file extensions ('data/*.json') rather than bare prefixes to exclude subdirectories","Filter out directory-looking paths before ReadMatches when the backend lists placeholders"],"tags":["io","directory","readmatches"],"backgroundTag":"incompatible-source-type","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}