{"record":{"id":"aa908678e63113d6","repo":"apache/beam","slug":"cannot-pickle-closed-files","errorCode":null,"errorMessage":"Cannot pickle closed files","messagePattern":"Cannot pickle closed files","errorType":"exception","errorClass":"PicklingError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/internal/cloudpickle/cloudpickle.py","lineNumber":1080,"sourceCode":"  orig_func = obj.__func__\n  return type(obj), (orig_func, )\n\n\ndef _file_reduce(obj):\n  \"\"\"Save a file.\"\"\"\n  import io\n\n  if not hasattr(obj, \"name\") or not hasattr(obj, \"mode\"):\n    raise pickle.PicklingError(\n        \"Cannot pickle files that do not map to an actual file\")\n  if obj is sys.stdout:\n    return getattr, (sys, \"stdout\")\n  if obj is sys.stderr:\n    return getattr, (sys, \"stderr\")\n  if obj is sys.stdin:\n    raise pickle.PicklingError(\"Cannot pickle standard input\")\n  if obj.closed:\n    raise pickle.PicklingError(\"Cannot pickle closed files\")\n  if hasattr(obj, \"isatty\") and obj.isatty():\n    raise pickle.PicklingError(\"Cannot pickle files that map to tty objects\")\n  if \"r\" not in obj.mode and \"+\" not in obj.mode:\n    raise pickle.PicklingError(\n        \"Cannot pickle files that are not opened for reading: %s\" % obj.mode)\n\n  name = obj.name\n\n  retval = io.StringIO()\n\n  try:\n    # Read the whole file\n    curloc = obj.tell()\n    obj.seek(0)\n    contents = obj.read()\n    obj.seek(curloc)\n  except OSError as e:\n    raise pickle.PicklingError(","sourceCodeStart":1062,"sourceCodeEnd":1098,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/internal/cloudpickle/cloudpickle.py#L1062-L1098","documentation":"_file_reduce in Beam's cloudpickle refuses closed file objects: a closed file has no meaningful readable state to reconstruct remotely, so pickling raises pickle.PicklingError('Cannot pickle closed files').","triggerScenarios":"Cloudpickling a closure or object holding a file handle after f.close() was called; context-manager-exited files retained in module state; objects caching an exhausted/closed handle.","commonSituations":"Using 'with open(...)' and then storing the handle in a global or class attribute that later gets serialized; long-lived workers holding stale handles; retry logic re-serializing objects whose files were closed between attempts.","solutions":["Keep the file open (or reopen it) before the object is pickled.","Store the file path string instead of the handle and open it lazily at use time.","Remove the closed handle from the pickled object (set attribute to None or del).","Use a class with __getstate__/__setstate__ that serializes the path and reopens on load."],"exampleFix":"// before\nwith open('f.txt') as f:\n  job.state = f  # closed when pickled\n// after\njob.path = 'f.txt'  # reopen inside worker when needed","handlingStrategy":"validation","validationCode":"if hasattr(f, 'closed') and f.closed:\n    raise ValueError('file is closed and cannot be pickled')","typeGuard":"def is_open_file(obj):\n    return hasattr(obj, 'closed') and not obj.closed","tryCatchPattern":"try:\n    cloudpickle.dump(state)\nexcept pickle.PicklingError as e:\n    if 'closed files' in str(e): state = reopen_files(state)\n    else: raise","preventionTips":["Store paths, not handles, in long-lived objects","Reopen files lazily where used","Avoid keeping handles alive past with-blocks"],"tags":["python","pickle","serialization","file-handle"],"backgroundTag":"json-serialization-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}