apache/beam · error · TypeError

compression_type must be CompressionType object but was %s

Error message

compression_type must be CompressionType object but was %s

What it means

FileBasedSink.__init__ validates that compression_type is one of the recognized members of the CompressionTypes enum (via CompressionTypes.is_valid_compression_type). A raw string like 'gzip' or an arbitrary object is rejected. The library requires an actual CompressionTypes constant so compression handling is unambiguous.

Source

Thrown at sdks/python/apache_beam/io/filebasedsink.py:99

     Raises:
      TypeError: if file path parameters are not a :class:`str` or
        :class:`~apache_beam.options.value_provider.ValueProvider`, or if
        **compression_type** is not member of
        :class:`~apache_beam.io.filesystem.CompressionTypes`.
      ValueError: if **shard_name_template** is not of expected
        format.
    """
    if not isinstance(file_path_prefix, (str, ValueProvider)):
      raise TypeError(
          'file_path_prefix must be a string or ValueProvider;'
          'got %r instead' % file_path_prefix)
    if not isinstance(file_name_suffix, (str, ValueProvider)):
      raise TypeError(
          'file_name_suffix must be a string or ValueProvider;'
          'got %r instead' % file_name_suffix)

    if not CompressionTypes.is_valid_compression_type(compression_type):
      raise TypeError(
          'compression_type must be CompressionType object but '
          'was %s' % type(compression_type))
    if shard_name_template is None:
      shard_name_template = DEFAULT_SHARD_NAME_TEMPLATE
    elif shard_name_template == '':
      num_shards = 1
    if triggering_frequency is None:
      triggering_frequency = DEFAULT_TRIGGERING_FREQUENCY
    if isinstance(file_path_prefix, str):
      file_path_prefix = StaticValueProvider(str, file_path_prefix)
    if isinstance(file_name_suffix, str):
      file_name_suffix = StaticValueProvider(str, file_name_suffix)
    self.file_path_prefix = file_path_prefix
    self.file_name_suffix = file_name_suffix
    self.num_shards = num_shards
    self.coder = coder
    self.shard_name_template = shard_name_template
    self.shard_name_format = self._template_to_format(shard_name_template)

View on GitHub (pinned to 12126d8942)

Solutions

Import CompressionTypes from apache_beam.io.filebasedsink (or apache_beam.io) and pass e.g. CompressionTypes.GZIP instead of a string
Pass CompressionTypes.AUTO to let the sink infer compression from the file_name_suffix
If a string is unavoidable, map it first: CompressionTypes.validate_compression_type(compression_type)

Example fix

// before
sink = FileBasedSink(file_path_prefix, coder, compression_type='gzip')
// after
from apache_beam.io import CompressionTypes
sink = FileBasedSink(file_path_prefix, coder, compression_type=CompressionTypes.GZIP)

Defensive patterns

Strategy: validation

Validate before calling

from apache_beam.io import CompressionTypes
if not CompressionTypes.is_valid_compression_type(compression_type):
    raise ValueError(f'compression_type must be a CompressionTypes member, got {compression_type!r}')

Type guard

def is_compression_type(v) -> bool:
    from apache_beam.io import CompressionTypes
    return CompressionTypes.is_valid_compression_type(v)

Prevention

Always import and use CompressionTypes members, never raw strings
Prefer compression= on WriteToText, which accepts strings and validates them

When it happens

Trigger: Passing a plain string (e.g. 'gzip', 'bzip2'), None, or any non-CompressionTypes object as compression_type to a FileBasedSink subclass (e.g. WriteToText with a hand-constructed sink).

Common situations: Users confuse the string-based compression= parameter of WriteToText/WriteToFiles with the enum-based compression_type of the underlying sink; copied sample code uses raw strings.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/df63b10c329eb5e3. Report an issue: GitHub.