apache/beam · error · ValueError

Cannot create object with unspecified or no compression

Error message

Cannot create object with unspecified or no compression

What it means

CompressedFile's whole purpose is to wrap a compressed stream; creating it with AUTO (deferred detection) or UNCOMPRESSED (no compression) leaves it with no codec, which is a contradictory request. The constructor raises ValueError for these sentinel values.

Solutions

1. Resolve AUTO to a concrete type first (e.g. via CompressionTypes.detect_compression_type(filename) or filesystems logic) before constructing CompressedFile.
2. For uncompressed data, use the raw file object directly instead of CompressedFile.
3. Route through FileSystems.open(), which resolves AUTO/UNCOMPRESSED appropriately.

Example fix

# before
CompressedFile(fh, compression_type=CompressionTypes.AUTO)
# after
type_ = CompressionTypes.detect_compression_type(file_name)
if type_ in (CompressionTypes.AUTO, CompressionTypes.UNCOMPRESSED):
    reader = fh  # plain file
else:
    reader = CompressedFile(fh, compression_type=type_)

if ctype in (CompressionTypes.AUTO, CompressionTypes.UNCOMPRESSED):
    ctype = CompressionTypes.detect_compression_type(file_name)
reader = CompressedFile(fh, compression_type=ctype) if ctype not in (CompressionTypes.AUTO, CompressionTypes.UNCOMPRESSED) else fh

def is_concrete_compression(v) -> bool:
    return v not in (CompressionTypes.AUTO, CompressionTypes.UNCOMPRESSED)

When it happens

Trigger: filesystem.CompressedFile(fh, compression_type=CompressionTypes.AUTO) or CompressionTypes.UNCOMPRESSED.

Common situations: Propagating a compression_type read from FileMetadata (often AUTO) directly into CompressedFile; passing UNCOMPRESSED thinking it behaves like a plain file wrapper.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• A schema is required to write non-schema'd data.
• A sink must inherit iobase.Sink, iobase.NativeSink, or be a…
• An explicit schema is required to write non-schema'd…
• Cannot call read after iterating.
• Cannot create a temporary directory for root path prefix

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/01b51773e640f759. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/filesystem.py:158

  # The bit mask to use for the wbits parameters of the zlib compressor and
  # decompressor objects.
  _gzip_mask = zlib.MAX_WBITS | 16  # Mask when using GZIP headers.

  def __init__(
      self,
      fileobj: BinaryIO,
      compression_type=CompressionTypes.GZIP,
      read_size=DEFAULT_READ_BUFFER_SIZE):
    if not fileobj:
      raise ValueError('File object must not be None')

    if not CompressionTypes.is_valid_compression_type(compression_type):
      raise TypeError(
          'compression_type must be CompressionType object but '
          'was %s' % type(compression_type))
    if compression_type in (CompressionTypes.AUTO,
                            CompressionTypes.UNCOMPRESSED):
      raise ValueError(
          'Cannot create object with unspecified or no compression')

    self._file = fileobj
    self._compression_type = compression_type

    if self._file.tell() != 0:
      raise ValueError(
          'File object must be at position 0 but was %d' % self._file.tell())
    self._uncompressed_position = 0
    self._uncompressed_size: Optional[int] = None

    if self.readable():
      self._read_size = read_size
      self._read_buffer = io.BytesIO()
      self._read_position = 0
      self._read_eof = False

      self._initialize_decompressor()

View on GitHub (pinned to 12126d8942)