{"record":{"id":"0ad15c9938c51c69","repo":"apache/beam","slug":"offset-s-whence-s-position-s-last-s","errorCode":null,"errorMessage":"offset: %s, whence: %s, position: %s, last: %s","messagePattern":"offset: (.+?), whence: (.+?), position: (.+?), last: (.+?)","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/filesystemio.py","lineNumber":299,"sourceCode":"      ``ValueError``: When this stream is closed.\n    \"\"\"\n    self._check_open()\n    return self.position\n\n  def seek(self, offset, whence=os.SEEK_SET):\n    # Certain upload stream implementations seek to the end of a stream to check\n    # length before completing an upload, so we support a no-op seek(0, SEEK_END).\n    if whence == os.SEEK_END and offset == 0:\n      return\n    elif whence == os.SEEK_SET:\n      if offset == self.position:\n        return\n      elif offset == self.last_block_position and self.last_block:\n        self.position = offset\n        self.remaining = b''.join([self.last_block, self.remaining])\n        self.last_block = b''\n        return\n    raise NotImplementedError(\n        'offset: %s, whence: %s, position: %s, last: %s' %\n        (offset, whence, self.position, self.last_block_position))\n\n  def _check_open(self):\n    if self.closed:\n      raise IOError('Stream is closed.')\n","sourceCodeStart":281,"sourceCodeEnd":306,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/filesystemio.py#L281-L306","documentation":"The concatenated/block-oriented read stream only supports seeking to special positions it can satisfy: the start of the stream, the current position, or the previously buffered block boundary (tracked via last_block_position). Any other seek target raises NotImplementedError('offset: %s, whence: %s, position: %s, last: %s') because the underlying source cannot be rewound to an arbitrary byte offset.","triggerScenarios":"Calling seek() with an arbitrary absolute offset on a compressed or sequentially-read stream (e.g. f.seek(1024) after reading partway through), or seeking relative to SEEK_END on a source that doesn't know its total size.","commonSituations":"Code that assumes all file-like objects support random access being pointed at Beam's compressed-stream readers; random-access reading of gzip/bzip2 files through the pipeline filesystem; parsing formats (e.g. zip, HDF5) that require arbitrary seeks.","solutions":["Only seek to offset 0 (rewind), the current position, or a block boundary the stream reports; otherwise reopen the file and skip bytes by reading.","Avoid random access on compressed streams: decompress to a local temp file first if your format requires seeks.","Restructure the reader to consume the stream sequentially instead of seeking.","For formats needing random access, read the full data into memory (BytesIO) and seek on that instead."],"exampleFix":"// before\nwith FileSystems.open(path) as f:\n    f.seek(512)  # NotImplementedError on compressed stream\n    data = f.read()\n// after\nwith FileSystems.open(path) as f:\n    f.seek(0)  # rewind from start is supported\n    data = f.read(512)  # read forward instead of seeking","handlingStrategy":"try-catch","validationCode":"def can_seek(f, offset, whence):\n    # stream only supports start, current position, or last block boundary\n    if whence not in (os.SEEK_SET, os.SEEK_CUR, os.SEEK_END):\n        return False\n    return offset in (0, f.tell(), getattr(f, 'last_block_position', None))","typeGuard":null,"tryCatchPattern":"try:\n    f.seek(offset)\nexcept NotImplementedError:\n    f.seek(0)\n    shutil.copyfileobj(f, tmpfile)  # materialize for random access\n    tmpfile.seek(offset)","preventionTips":["Assume Beam compressed/sequential streams only support rewind (seek(0)) and sequential reads.","Materialize to a local temp file or BytesIO when your parser needs arbitrary seeks.","Check the stream class before issuing non-trivial seeks; only uncompressed/local streams support them.","Prefer reading forward and discarding data over seeking."],"tags":["python","io","seek","compressed-stream","not-implemented"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}