apache/beam · error · ValueError

Cannot create a temporary directory for root path prefix %s.

Error message

Cannot create a temporary directory for root path prefix %s. Please specify a file path prefix with at least two components.

What it means

FileBasedSink._create_temp_dir builds a temporary staging directory next to the output prefix ('beam-temp-<last>-<uuid>'). If file_path_prefix is a bare root (e.g. '/', 'gs://bucket') it has no last component to build the directory name from, so a ValueError is raised asking for a prefix with at least two path components.

Source

Thrown at sdks/python/apache_beam/io/filebasedsink.py:193

    """
    if file_handle is not None:
      file_handle.close()

  @check_accessible(['file_path_prefix', 'file_name_suffix'])
  def initialize_write(self):
    file_path_prefix = self.file_path_prefix.get()

    tmp_dir = self._create_temp_dir(file_path_prefix)
    FileSystems.mkdirs(tmp_dir)
    return tmp_dir

  def _create_temp_dir(self, file_path_prefix):
    base_path, last_component = FileSystems.split(file_path_prefix)
    if not last_component:
      # Trying to re-split the base_path to check if it's a root.
      new_base_path, _ = FileSystems.split(base_path)
      if base_path == new_base_path:
        raise ValueError(
            'Cannot create a temporary directory for root path '
            'prefix %s. Please specify a file path prefix with '
            'at least two components.' % file_path_prefix)
    path_components = [
        base_path, 'beam-temp-' + last_component + '-' + uuid.uuid1().hex
    ]
    return FileSystems.join(*path_components)

  @check_accessible(['file_path_prefix', 'file_name_suffix'])
  def open_writer(self, init_result, uid):
    # A proper suffix is needed for AUTO compression detection.
    # We also ensure there will be no collisions with uid and a
    # (possibly unsharded) file_path_prefix and a (possibly empty)
    # file_name_suffix.
    from apache_beam.pvalue import EmptySideInput

    # Handle case where init_result is EmptySideInput (empty collection)
    # TODO: https://github.com/apache/beam/issues/36563 for Prism

View on GitHub (pinned to 12126d8942)

Solutions

Append a file name prefix to the path, e.g. 'gs://my-bucket/output/data-'
Ensure the prefix has at least a directory and a file name component separated by '/'
Check the value passed to WriteToText/file_path_prefix for trailing-slash-only paths

Example fix

// before
WriteToText('gs://my-bucket/')
// after
WriteToText('gs://my-bucket/output/results')

Defensive patterns

Strategy: validation

Validate before calling

base, last = FileSystems.split(file_path_prefix)
if not last:
    raise ValueError(f'file_path_prefix needs a file name component: {file_path_prefix}')

Type guard

def has_file_component(prefix: str) -> bool:
    from apache_beam.io.filesystems import FileSystems
    return bool(FileSystems.split(prefix)[1])

Try / catch

try:
    _ = beam.io.WriteToText(file_path_prefix)
except ValueError as e:
    if 'root path prefix' in str(e):
        file_path_prefix = file_path_prefix.rstrip('/') + '/output/data'

Prevention

Always end output prefixes with a file name, not a directory or bucket root
Avoid trailing slashes on output paths
Test output prefixes against the target filesystem before launching jobs

When it happens

Trigger: Writing with a file_path_prefix like '/', 'gs://bucket/', or otherwise a single root component with no file name part; FileSystems.split yields an empty last_component and re-splitting the base path does not change it.

Common situations: Users point WriteToText at just a bucket/directory instead of a file prefix such as 'gs://bucket/output/data'.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d4889d92140fef8c. Report an issue: GitHub.