What causes "File object must not be None" in apache/beam?

Pass a valid binary-mode file object (e.g. open(path, 'rb') or filesystem.open(...)).

apache/beam · error · ValueError

File object must not be None

Error message

File object must not be None

What it means

CompressedFile (the base for compressed file readers/writers in apache_beam.io.filesystem) requires a real binary file object to wrap. Its __init__ rejects falsy file objects (None, or objects whose __bool__ is False) because there is nothing to compress or decompress against.

Source

Thrown at sdks/python/apache_beam/io/filesystem.py:150

        return compression_type
    return cls.UNCOMPRESSED


class CompressedFile(object):
  """File wrapper for easier handling of compressed files."""
  # XXX: This class is not thread safe in the read path.

  # The bit mask to use for the wbits parameters of the zlib compressor and
  # decompressor objects.
  _gzip_mask = zlib.MAX_WBITS | 16  # Mask when using GZIP headers.

  def __init__(
      self,
      fileobj: BinaryIO,
      compression_type=CompressionTypes.GZIP,
      read_size=DEFAULT_READ_BUFFER_SIZE):
    if not fileobj:
      raise ValueError('File object must not be None')

    if not CompressionTypes.is_valid_compression_type(compression_type):
      raise TypeError(
          'compression_type must be CompressionType object but '
          'was %s' % type(compression_type))
    if compression_type in (CompressionTypes.AUTO,
                            CompressionTypes.UNCOMPRESSED):
      raise ValueError(
          'Cannot create object with unspecified or no compression')

    self._file = fileobj
    self._compression_type = compression_type

    if self._file.tell() != 0:
      raise ValueError(
          'File object must be at position 0 but was %d' % self._file.tell())
    self._uncompressed_position = 0
    self._uncompressed_size: Optional[int] = None

View on GitHub (pinned to 12126d8942)

Solutions

Pass a valid binary-mode file object (e.g. open(path, 'rb') or filesystem.open(...)).
Check why the producer returned None — fix the failed open or error handling upstream.
Assert the file object is not None before constructing CompressedFile.

Example fix

# before
fh = maybe_open(path)
reader = CompressedFile(fh, compression_type=CompressionTypes.GZIP)
# after
fh = maybe_open(path)
assert fh is not None, f'failed to open {path}'
reader = CompressedFile(fh, compression_type=CompressionTypes.GZIP)

Defensive patterns

Strategy: validation

Validate before calling

if fh is None:
    raise ValueError(f'failed to open file for {path}')
reader = CompressedFile(fh, compression_type=CompressionTypes.GZIP)

Type guard

def is_fileobj(obj) -> bool:
    return obj is not None and hasattr(obj, 'read') and hasattr(obj, 'tell')

Try / catch

try:
    reader = CompressedFile(fh, compression_type=ctype)
except ValueError as e:
    if 'File object must not be None' in str(e):
        fh = open(path, 'rb')
        reader = CompressedFile(fh, compression_type=ctype)
    else:
        raise

Prevention

Never let helper functions return None on open failure — raise instead.
Assert file objects are non-None at API boundaries.

When it happens

Trigger: Calling filesystem.CompressedFile(None) or passing the result of an open()/gcsio call that returned None (e.g. a failed open stubbed to return None).

Common situations: A helper function returns None instead of a file object on failure and the result is passed straight to CompressedFile; mocks in tests returning None.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/fefd62073dc90476. Report an issue: GitHub.