apache/beam · error · TypeError

%s: file_pattern must be of type string or ValueProvider; go

Error message

%s: file_pattern must be of type string or ValueProvider; got %r instead

What it means

FileBasedSource.__init__ requires file_pattern to be a str or a ValueProvider; anything else raises TypeError. ValueProvider support allows runtime-templated patterns (e.g. --input=gs://bucket/*.json) in Dataflow templates.

Source

Thrown at sdks/python/apache_beam/io/filebasedsource.py:108

        :data:`True` by the user, :class:`FileBasedSource` may choose to not
        split the file, for example, for compressed files where currently it is
        not possible to efficiently read a data range without decompressing the
        whole file.
      validate (bool): Boolean flag to verify that the files exist during the
        pipeline creation time.

    Raises:
      TypeError: when **compression_type** is not valid or if
        **file_pattern** is not a :class:`str` or a
        :class:`~apache_beam.options.value_provider.ValueProvider`.
      ValueError: when compression and splittable files are
        specified.
      IOError: when the file pattern specified yields an empty
        result.
    """

    if not isinstance(file_pattern, (str, ValueProvider)):
      raise TypeError(
          '%s: file_pattern must be of type string'
          ' or ValueProvider; got %r instead' %
          (self.__class__.__name__, file_pattern))

    if isinstance(file_pattern, str):
      file_pattern = StaticValueProvider(str, file_pattern)
    self._pattern = file_pattern

    self._concat_source = None
    self._min_bundle_size = min_bundle_size
    if not CompressionTypes.is_valid_compression_type(compression_type):
      raise TypeError(
          'compression_type must be CompressionType object but '
          'was %s' % type(compression_type))
    self._compression_type = compression_type
    self._splittable = splittable
    if validate and file_pattern.is_accessible():
      self._validate()

View on GitHub (pinned to 12126d8942)

Solutions

Pass a single glob string, e.g. 'gs://bucket/input/*.json'
Convert non-str values: str(path) for pathlib.Path, or wrap runtime values in StaticValueProvider/RuntimeValueProvider
To read multiple patterns, create one source per pattern or union the PCollection results

Example fix

// before
source = ReadFromText(['gs://b/a.json', 'gs://b/b.json'])
// after
source = ReadFromText('gs://b/*.json')  # or read each pattern and flatten

Defensive patterns

Strategy: type-guard

Validate before calling

if not isinstance(file_pattern, (str, ValueProvider)):
    file_pattern = str(file_pattern)  # or join the list into a glob

Type guard

def is_valid_file_pattern(p) -> bool:
    return isinstance(p, (str, ValueProvider))

Try / catch

try:
    _ = ReadFromText(file_pattern)
except TypeError as e:
    if 'file_pattern must be' in str(e):
        file_pattern = str(file_pattern)

Prevention

Pass glob strings, not lists or Path objects
Convert pathlib.Path with str() before constructing sources
Use ValueProvider for runtime-parameterized input paths

When it happens

Trigger: Passing a list of paths, a pathlib.Path, bytes, or None as file_pattern to ReadFromText/FileBasedSource subclasses.

Common situations: Users pass a list ['gs://.../a.json','gs://.../b.json'] expecting multi-path reads; migrating scripts that pass Path objects; template authors bypassing ValueProvider.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/3432377838ff1a0f. Report an issue: GitHub.