{"record":{"id":"101adb96254238c7","repo":"apache/beam","slug":"list-operation-failed-blobstoragefilesystem","errorCode":null,"errorMessage":"List operation failed","messagePattern":"List operation failed","errorType":"exception","errorClass":"BeamIOError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/azure/blobstoragefilesystem.py","lineNumber":130,"sourceCode":"    return False\n\n  def _list(self, dir_or_prefix):\n    \"\"\"List files in a location.\n    Listing is non-recursive (for filesystems that support directories).\n    Args:\n      dir_or_prefix: (string) A directory or location prefix (for filesystems\n        that don't have directories).\n    Returns:\n      Generator of ``FileMetadata`` objects.\n    Raises:\n      ``BeamIOError``: if listing fails, but not if no files were found.\n    \"\"\"\n    try:\n      for path, (size, updated) in self._blobstorageIO().list_files(\n          dir_or_prefix, with_metadata=True):\n        yield FileMetadata(path, size, updated)\n    except Exception as e:  # pylint: disable=broad-except\n      raise BeamIOError(\"List operation failed\", {dir_or_prefix: e})\n\n  def _blobstorageIO(self):\n    return blobstorageio.BlobStorageIO(pipeline_options=self._pipeline_options)\n\n  def _path_open(\n      self,\n      path,\n      mode,\n      mime_type='application/octet-stream',\n      compression_type=CompressionTypes.AUTO):\n    \"\"\"Helper functions to open a file in the provided mode.\n    \"\"\"\n    compression_type = FileSystem._get_compression_type(path, compression_type)\n    mime_type = CompressionTypes.mime_type(compression_type, mime_type)\n    raw_file = self._blobstorageIO().open(path, mode, mime_type=mime_type)\n    if compression_type == CompressionTypes.UNCOMPRESSED:\n      return raw_file\n    return CompressedFile(raw_file, compression_type=compression_type)","sourceCodeStart":112,"sourceCodeEnd":148,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/azure/blobstoragefilesystem.py#L112-L148","documentation":"BlobStorageFileSystem._list wraps the underlying BlobStorageIO.list_files iteration; any exception (auth failure, missing container, network error, bad prefix) is caught and re-raised as BeamIOError('List operation failed') with the failing path and original exception attached to exception_details.","triggerScenarios":"Calling match_files/list_prefix on a container that does not exist, with missing/misconfigured Azure credentials (AZURE_STORAGE_ACCOUNT / AZURE_STORAGE_KEY or token), or during a transient network failure.","commonSituations":"Wrong container name in the path; DefaultAzureCredential chain exhausted in CI; storage account firewall blocking the runner; typo in prefix producing 404 from the service.","solutions":["Inspect BeamIOError.exception_details[path] for the root cause exception.","Verify the container exists and the prefix is correct.","Check Azure credentials (connection string / account key / Entra token) are present and valid in the runner environment.","Retry on transient errors; configure storage firewall/network rules to allow the job's egress."],"exampleFix":"// before\nfor m in fsys.match(['az://wrong-container/*']): ...\n// after\ntry:\n    for m in fsys.match(['az://container/*']): ...\nexcept BeamIOError as e:\n    for path, err in e.exception_details.items():\n        logging.error('list failed for %s: %s', path, err)","handlingStrategy":"try-catch","validationCode":"from azure.storage.blob import BlobServiceClient\ntry:\n    BlobServiceClient(account_url=f\"https://{account}.blob.core.windows.net\", credential=cred).get_container_client(container).get_account_information()\nexcept Exception as e:\n    raise RuntimeError(f\"Azure list precheck failed: {e}\")","typeGuard":"def list_root_cause(exc):\n    return getattr(exc, 'exception_details', {}).get(path) if (exc := exc) else None","tryCatchPattern":"try:\n    metas = fsys.metadata(['az://container/prefix'])\nexcept BeamIOError as e:\n    for path, cause in e.exception_details.items():\n        logging.error(\"list failed for %s: %r\", path, cause)\n    if is_transient(cause):\n        retry(...)","preventionTips":["Verify container names and credentials in the runner environment before jobs start.","Unwrap exception_details to see the real Azure error.","Add retries for transient storage/network faults.","Check storage-account firewall/VNet rules for the compute environment."],"tags":["python","azure","blob-storage","list-operation","wrapped-exception"],"backgroundTag":"api-error-response","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}