apache/beam · error · TypeError

compression_type must be CompressionType object but was %s

Error message

compression_type must be CompressionType object but was %s

What it means

CompressedFile only accepts compression types from the CompressionTypes enum that denote an actual codec (GZIP, BZIP2, etc.). Passing a non-member object (e.g. a string 'gzip') fails CompressionTypes.is_valid_compression_type and raises TypeError.

Source

Thrown at sdks/python/apache_beam/io/filesystem.py:153

class CompressedFile(object):
  """File wrapper for easier handling of compressed files."""
  # XXX: This class is not thread safe in the read path.

  # The bit mask to use for the wbits parameters of the zlib compressor and
  # decompressor objects.
  _gzip_mask = zlib.MAX_WBITS | 16  # Mask when using GZIP headers.

  def __init__(
      self,
      fileobj: BinaryIO,
      compression_type=CompressionTypes.GZIP,
      read_size=DEFAULT_READ_BUFFER_SIZE):
    if not fileobj:
      raise ValueError('File object must not be None')

    if not CompressionTypes.is_valid_compression_type(compression_type):
      raise TypeError(
          'compression_type must be CompressionType object but '
          'was %s' % type(compression_type))
    if compression_type in (CompressionTypes.AUTO,
                            CompressionTypes.UNCOMPRESSED):
      raise ValueError(
          'Cannot create object with unspecified or no compression')

    self._file = fileobj
    self._compression_type = compression_type

    if self._file.tell() != 0:
      raise ValueError(
          'File object must be at position 0 but was %d' % self._file.tell())
    self._uncompressed_position = 0
    self._uncompressed_size: Optional[int] = None

    if self.readable():
      self._read_size = read_size

View on GitHub (pinned to 12126d8942)

Solutions

Pass a CompressionTypes enum member, e.g. CompressionTypes.GZIP or CompressionTypes.BZIP2.
Convert string input via getattr(CompressionTypes, value.upper()) or a mapping dict.
Use filesystems.FileSystems.open() which handles compression types automatically instead of constructing CompressedFile manually.

Example fix

// before
CompressedFile(fh, compression_type='gzip')
# after
from apache_beam.io.filesystem import CompressionTypes
CompressedFile(fh, compression_type=CompressionTypes.GZIP)

Defensive patterns

Strategy: validation

Validate before calling

if not CompressionTypes.is_valid_compression_type(ctype):
    raise TypeError(f'unsupported compression_type: {ctype!r}')

Type guard

def is_compression_type(v) -> bool:
    return CompressionTypes.is_valid_compression_type(v)

Try / catch

try:
    reader = CompressedFile(fh, compression_type=ctype)
except TypeError:
    ctype = getattr(CompressionTypes, str(ctype).upper(), CompressionTypes.GZIP)
    reader = CompressedFile(fh, compression_type=ctype)

Prevention

Always use CompressionTypes enum members, never raw strings.
Centralize string-to-enum conversion in one helper.

When it happens

Trigger: filesystem.CompressedFile(fh, compression_type='gzip') or any value not in CompressionTypes (other than AUTO/UNCOMPRESSED, which fail with a different error).

Common situations: Reading the compression type from config/CLI as a raw string and passing it unconverted; confusing this API with gzip.open which takes strings.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0739b551daf163d4. Report an issue: GitHub.