What causes "Azure Blob Storage path must be in the form azfs:// / / ." in apache/beam?

Use the exact form azfs:// / / , all lowercase

apache/beam · error · ValueError

Azure Blob Storage path must be in the form…

Error message

Azure Blob Storage path must be in the form azfs://<storage-account>/<container>/<path>.

What it means

blobstorageio.parse_azfs_path() validates Azure Blob Storage URLs with a strict regex: azfs://<storage-account 3-24 chars lowercase alnum>/<valid container name>/<blob path>. It raises ValueError with this message when the path does not match, or when the blob path component is empty and blob_optional is False.

Solutions

Use the exact form azfs://<storage-account>/<container>/<blob-path>, all lowercase
Replace other schemes (wasb://, wasbs://, https://, gs://) with azfs://
Check the account name is 3-24 lowercase alphanumeric characters and the container is a valid Azure container name (lowercase, digits, hyphens, no leading/trailing/double hyphens)
Ensure a non-empty blob path unless calling a code path that explicitly allows blob-optional paths

Example fix

// before
path = 'https://myaccount.blob.core.windows.net/container/file.txt'
// after
path = 'azfs://myaccount/container/file.txt'

Defensive patterns

Strategy: validation

Validate before calling

import re
AZFS_RE = re.compile(r'^azfs://([a-z0-9]{3,24})/([a-z0-9](?![a-z0-9-]*--[a-z0-9-]*)[a-z0-9-]{1,61}[a-z0-9])/(.+)$')
def is_valid_azfs_path(p):
    return bool(AZFS_RE.match(p))

# before any filesystem call:
assert is_valid_azfs_path(path), f"bad azfs path: {path}"

Type guard

def is_azfs_path(p):
    return isinstance(p, str) and p.startswith('azfs://') and re.match(
        r'^azfs://[a-z0-9]{3,24}/[a-z0-9][a-z0-9-]{0,60}[a-z0-9]/.+$', p)

Try / catch

try:
    blobstorageio.parse_azfs_path(path)
except ValueError as e:
    logging.error("invalid Azure path %r: %s", path, e)
    raise

Prevention

Centralize path construction in a helper that always emits azfs:// URLs
Lowercase account/container names when building paths from config or the portal
Never pass https:// endpoint URLs where an azfs:// path is expected
Include a non-empty blob path unless the API explicitly tolerates blob-optional paths
Add a path-format unit test to CI for all configured storage paths

When it happens

Trigger: Calling any Beam Azure filesystem API (copy, delete, list_files, open via BlobStorageIO.__init__, etc.) with a path that is not a valid azfs:// URL: wrong scheme (wasb://, gs://, https://), uppercase or invalid characters in account/container, missing blob path when one is required, account names outside 3-24 lowercase alnum, or container names violating Azure naming rules (including the embedded '//' disambiguation rule in the regex).

Common situations: Migrating pipelines from GCS/S3 and reusing gs:// or s3:// prefixes; using a full https:// endpoint URL instead of azfs://; uppercase account names (Azure portal sometimes shows them capitalized); trailing slash with no blob path passed to an API that requires one.

Understand the failure class

Background: "Invalid URL" errors: why new URL(), URI.parse, and reqwest::Url reject your string — missing scheme, whitespace, and bad path format — this error's family across 39 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/625c578d89f6adad. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/azure/blobstorageio.py:65

  AZURE_DEPS_INSTALLED = True
except ImportError:
  AZURE_DEPS_INSTALLED = False

DEFAULT_READ_BUFFER_SIZE = 16 * 1024 * 1024

MAX_BATCH_OPERATION_SIZE = 100


def parse_azfs_path(azfs_path, blob_optional=False, get_account=False):
  """Return the storage account, the container and
  blob names of the given azfs:// path.
  """
  match = re.match(
      '^azfs://([a-z0-9]{3,24})/([a-z0-9](?![a-z0-9-]*--[a-z0-9-]*)'
      '[a-z0-9-]{1,61}[a-z0-9])/(.*)$',
      azfs_path)
  if match is None or (match.group(3) == '' and not blob_optional):
    raise ValueError(
        'Azure Blob Storage path must be in the form '
        'azfs://<storage-account>/<container>/<path>.')
  result = None
  if get_account:
    result = match.group(1), match.group(2), match.group(3)
  else:
    result = match.group(2), match.group(3)
  return result


def get_azfs_url(storage_account, container, blob=''):
  """Returns the url in the form of
   https://account.blob.core.windows.net/container/blob-name
  """
  return 'https://' + storage_account + '.blob.core.windows.net/' + \
          container + '/' + blob

View on GitHub (pinned to 12126d8942)