{"record":{"id":"384bb53c00dd26cc","repo":"apache/beam","slug":"no-such-file-or-directory-s","errorCode":null,"errorMessage":"No such file or directory: %s","messagePattern":"No such file or directory: (.+?)","errorType":"exception","errorClass":"IOError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/testing/pipeline_verifiers.py","lineNumber":120,"sourceCode":"      else:\n        raise ValueError(\n            'Sleep seconds, if received, must be int. '\n            'But received: %r, %s' % (sleep_secs, type(sleep_secs)))\n    else:\n      self.sleep_secs = None\n\n    self.file_path = file_path\n    self.expected_checksum = expected_checksum\n\n  @retry.with_exponential_backoff(\n      num_retries=MAX_RETRIES, retry_filter=retry_on_io_error_and_server_error)\n  def _read_with_retry(self):\n    \"\"\"Read path with retry if I/O failed\"\"\"\n    read_lines = []\n    match_result = FileSystems.match([self.file_path])[0]\n    matched_path = [f.path for f in match_result.metadata_list]\n    if not matched_path:\n      raise IOError('No such file or directory: %s' % self.file_path)\n\n    _LOGGER.info(\n        'Find %d files in %s: \\n%s',\n        len(matched_path),\n        self.file_path,\n        '\\n'.join(matched_path))\n    for path in matched_path:\n      with FileSystems.open(path, 'r') as f:\n        for line in f:\n          read_lines.append(line)\n    return read_lines\n\n  def _matches(self, _):\n    if self.sleep_secs:\n      # Wait to have output file ready on FS\n      _LOGGER.info('Wait %d seconds...', self.sleep_secs)\n      time.sleep(self.sleep_secs)\n","sourceCodeStart":102,"sourceCodeEnd":138,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/testing/pipeline_verifiers.py#L102-L138","documentation":"FileBasedSourceVerifier._read_with_retry uses the Beam FileSystems API to match the given file path; if the match returns no metadata entries the path resolved to nothing, so it raises IOError 'No such file or directory'. This happens before any retry/read logic, meaning the file simply does not exist at the specified location.","triggerScenarios":"Calling _matches (during verify()) with a file_path glob that matched zero files — path misspelled, file not yet written, wrong bucket/directory, or wrong filesystem scheme.","commonSituations":"Verifier runs before the pipeline finished writing output; glob pattern wrong for the temp/output location; reading from GCS with a mistyped bucket name.","solutions":["Verify the file_path exists (ls / gsutil ls) and correct the path","Ensure the pipeline writing the output has completed and files are flushed before verifying","Check the glob pattern — a too-narrow pattern may match nothing","Increase sleep_secs on the verifier so it waits for output readiness"],"exampleFix":"# before\nFileBasedOutputVerifier('gs://wrong-bucket/out/*')\n// after\nFileBasedOutputVerifier('gs://my-bucket/correct/output-dir/*')","handlingStrategy":"try-catch","validationCode":"from apache_beam.io.filesystems import FileSystems\nassert FileSystems.match([file_path]) and FileSystems.match([file_path])[0].metadata_list","typeGuard":null,"tryCatchPattern":"try:\n    verifier.match()\nexcept IOError as e:\n    _LOGGER.warning('Output not found yet (%s); retrying after sleep', e)\n    time.sleep(sleep_secs)\n    verifier.match()","preventionTips":["Ensure the producing pipeline finishes before verification","Use sleep_secs to wait for output readiness","Test glob patterns against the actual output location (gsutil ls)"],"tags":["ioerror","file-not-found","beam-filesystem","python"],"backgroundTag":"file-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}