apache/beam · error · TypeError

compression_type must be CompressionType object but was %s

Error message

compression_type must be CompressionType object but was %s

What it means

FileBasedSource.__init__ validates compression_type against CompressionTypes.is_valid_compression_type; non-enum values raise TypeError. Like the sink-side check, it forces callers to use CompressionTypes constants (AUTO, UNCOMPRESSED, GZIP, BZIP2) rather than arbitrary values.

Source

Thrown at sdks/python/apache_beam/io/filebasedsource.py:120

        specified.
      IOError: when the file pattern specified yields an empty
        result.
    """

    if not isinstance(file_pattern, (str, ValueProvider)):
      raise TypeError(
          '%s: file_pattern must be of type string'
          ' or ValueProvider; got %r instead' %
          (self.__class__.__name__, file_pattern))

    if isinstance(file_pattern, str):
      file_pattern = StaticValueProvider(str, file_pattern)
    self._pattern = file_pattern

    self._concat_source = None
    self._min_bundle_size = min_bundle_size
    if not CompressionTypes.is_valid_compression_type(compression_type):
      raise TypeError(
          'compression_type must be CompressionType object but '
          'was %s' % type(compression_type))
    self._compression_type = compression_type
    self._splittable = splittable
    if validate and file_pattern.is_accessible():
      self._validate()

  def display_data(self):
    return {
        'file_pattern': DisplayDataItem(
            str(self._pattern), label="File Pattern"),
        'compression': DisplayDataItem(
            str(self._compression_type), label='Compression Type')
    }

  @check_accessible(['_pattern'])
  def _get_concat_source(self) -> concat_source.ConcatSource:
    if self._concat_source is None:

View on GitHub (pinned to 12126d8942)

Solutions

Pass a CompressionTypes member, e.g. CompressionTypes.GZIP
Use CompressionTypes.AUTO to infer from the file suffix
Convert strings via apache_beam.io.CompressionTypes mapping before constructing the source

Example fix

// before
FileBasedSource('gs://b/data.json.gz', coder, compression_type='gzip')
// after
from apache_beam.io import CompressionTypes
FileBasedSource('gs://b/data.json.gz', coder, compression_type=CompressionTypes.GZIP)

Defensive patterns

Strategy: validation

Validate before calling

from apache_beam.io import CompressionTypes
if not CompressionTypes.is_valid_compression_type(compression_type):
    raise ValueError(f'use a CompressionTypes member, got {compression_type!r}')

Type guard

def is_compression_type(v) -> bool:
    return CompressionTypes.is_valid_compression_type(v)

Prevention

Use enum members (CompressionTypes.GZIP/BZIP2/UNCOMPRESSED/AUTO) only
Prefer higher-level reader options that accept strings and normalize them
Map legacy string names to CompressionTypes at config load time

When it happens

Trigger: Constructing FileBasedSource or a subclass (e.g. ReadFromText's underlying source) with compression_type='gzip', None, or another non-member value.

Common situations: String vs enum confusion as in 2800; code migrated from libraries that accept string compression names.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c7b812fed1057ae5. Report an issue: GitHub.