{"record":{"id":"36bd59d07c797276","repo":"apache/beam","slug":"flush-on-closed-file","errorCode":null,"errorMessage":"flush on closed file","messagePattern":"flush on closed file","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/gcp/bigquery_tools.py","lineNumber":1530,"sourceCode":"      raise ValueError(\"Output stream must be writable\")\n\n    self._file_handle = file_handle\n    avro_schema = fastavro.parse_schema(\n        get_avro_schema_from_table_schema(schema))\n    self._avro_writer = fastavro.write.Writer(self._file_handle, avro_schema)\n\n  def close(self):\n    if not self._file_handle.closed:\n      self.flush()\n      self._file_handle.close()\n\n  @property\n  def closed(self):\n    return self._file_handle.closed\n\n  def flush(self):\n    if self._file_handle.closed:\n      raise ValueError(\"flush on closed file\")\n\n    self._avro_writer.flush()\n    self._file_handle.flush()\n\n  def read(self, size=-1):\n    raise io.UnsupportedOperation(\"AvroRowWriter is not readable\")\n\n  def tell(self):\n    # Flush the fastavro Writer to the underlying stream, otherwise there isn't\n    # a reliable way to determine how many bytes have been written.\n    self._avro_writer.flush()\n    return self._file_handle.tell()\n\n  def writable(self):\n    return self._file_handle.writable()\n\n  def write(self, row):\n    try:","sourceCodeStart":1512,"sourceCodeEnd":1548,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/gcp/bigquery_tools.py#L1512-L1548","documentation":"AvroRowWriter.flush() raises ValueError when the underlying file handle is already closed. The wrapper refuses to flush an Avro writer into a closed file, since flushing after close would fail or corrupt output. This guards against double-close or write-after-close misuse in BigQuery file sinks.","triggerScenarios":"Calling flush() explicitly after close() was already called on the AvroRowWriter; calling close() twice, where the second close() internally flushes via the first path; using the writer in a context where the file handle was closed externally before flush.","commonSituations":"Writing Avro files for a BigQuery load job and manually closing the writer then calling flush in a finally block; a sink pipeline that closes the file and a caller that also flushes on teardown; refactored code that lost track of writer lifecycle.","solutions":["Check writer.closed before calling flush(): only flush when the file handle is still open.","Do not call flush() after close(); rely on close() to perform the final flush.","If managing manually, restructure to a single owner of the writer lifecycle so flush/close happen exactly once."],"exampleFix":"// before\nwriter.close()\nwriter.flush()  # ValueError: flush on closed file\n\n// after\nwriter.flush()\nwriter.close()\n# or guard:\nif not writer.closed:\n    writer.flush()","handlingStrategy":"type-guard","validationCode":"if writer.closed:\n    raise RuntimeError('cannot flush: writer already closed')\nwriter.flush()","typeGuard":"def can_flush(writer):\n    return not writer.closed","tryCatchPattern":"try:\n    writer.flush()\nexcept ValueError as e:\n    if 'flush on closed file' not in str(e):\n        raise\n    # writer already closed; nothing to do","preventionTips":["Give a single code path ownership of flush/close for each writer.","Never call flush() after close(); call flush() before close() if needed.","Use try/finally so close() runs exactly once.","Check the .closed property before any lifecycle operation."],"tags":["python","apache-beam","bigquery","file-io","avro"],"backgroundTag":"invalid-state-transition","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}