apache/beam · error · NotImplementedError

offset: , whence: , position: , last

Error message

offset: %s, whence: %s, position: %s, last: %s

What it means

The concatenated/block-oriented read stream only supports seeking to special positions it can satisfy: the start of the stream, the current position, or the previously buffered block boundary (tracked via last_block_position). Any other seek target raises NotImplementedError('offset: %s, whence: %s, position: %s, last: %s') because the underlying source cannot be rewound to an arbitrary byte offset.

Solutions

1. Only seek to offset 0 (rewind), the current position, or a block boundary the stream reports; otherwise reopen the file and skip bytes by reading.
2. Avoid random access on compressed streams: decompress to a local temp file first if your format requires seeks.
3. Restructure the reader to consume the stream sequentially instead of seeking.
4. For formats needing random access, read the full data into memory (BytesIO) and seek on that instead.

Example fix

// before
with FileSystems.open(path) as f:
    f.seek(512)  # NotImplementedError on compressed stream
    data = f.read()
// after
with FileSystems.open(path) as f:
    f.seek(0)  # rewind from start is supported
    data = f.read(512)  # read forward instead of seeking

def can_seek(f, offset, whence):
    # stream only supports start, current position, or last block boundary
    if whence not in (os.SEEK_SET, os.SEEK_CUR, os.SEEK_END):
        return False
    return offset in (0, f.tell(), getattr(f, 'last_block_position', None))

try:
    f.seek(offset)
except NotImplementedError:
    f.seek(0)
    shutil.copyfileobj(f, tmpfile)  # materialize for random access
    tmpfile.seek(offset)

When it happens

Trigger: Calling seek() with an arbitrary absolute offset on a compressed or sequentially-read stream (e.g. f.seek(1024) after reading partway through), or seeking relative to SEEK_END on a source that doesn't know its total size.

Common situations: Code that assumes all file-like objects support random access being pointed at Beam's compressed-stream readers; random-access reading of gzip/bzip2 files through the pipeline filesystem; parsing formats (e.g. zip, HDF5) that require arbitrary seeks.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Whence mode %r is invalid.
• Whence mode %r is invalid.
• A schema is required to write non-schema'd data.
• A sink must inherit iobase.Sink, iobase.NativeSink, or be a…
• An explicit schema is required to write non-schema'd…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0ad15c9938c51c69. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/filesystemio.py:299

      ``ValueError``: When this stream is closed.
    """
    self._check_open()
    return self.position

  def seek(self, offset, whence=os.SEEK_SET):
    # Certain upload stream implementations seek to the end of a stream to check
    # length before completing an upload, so we support a no-op seek(0, SEEK_END).
    if whence == os.SEEK_END and offset == 0:
      return
    elif whence == os.SEEK_SET:
      if offset == self.position:
        return
      elif offset == self.last_block_position and self.last_block:
        self.position = offset
        self.remaining = b''.join([self.last_block, self.remaining])
        self.last_block = b''
        return
    raise NotImplementedError(
        'offset: %s, whence: %s, position: %s, last: %s' %
        (offset, whence, self.position, self.last_block_position))

  def _check_open(self):
    if self.closed:
      raise IOError('Stream is closed.')

View on GitHub (pinned to 12126d8942)