{"record":{"id":"e120945d4ebd9f5e","repo":"apache/beam","slug":"cannot-pickle-files-that-do-not-map-to-an-actual-file","errorCode":null,"errorMessage":"Cannot pickle files that do not map to an actual file","messagePattern":"Cannot pickle files that do not map to an actual file","errorType":"exception","errorClass":"PicklingError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/internal/cloudpickle/cloudpickle.py","lineNumber":1071,"sourceCode":"  try:\n    obj.cell_contents\n  except ValueError:  # cell is empty\n    return _make_empty_cell, ()\n  else:\n    return _make_cell, (obj.cell_contents, )\n\n\ndef _classmethod_reduce(obj):\n  orig_func = obj.__func__\n  return type(obj), (orig_func, )\n\n\ndef _file_reduce(obj):\n  \"\"\"Save a file.\"\"\"\n  import io\n\n  if not hasattr(obj, \"name\") or not hasattr(obj, \"mode\"):\n    raise pickle.PicklingError(\n        \"Cannot pickle files that do not map to an actual file\")\n  if obj is sys.stdout:\n    return getattr, (sys, \"stdout\")\n  if obj is sys.stderr:\n    return getattr, (sys, \"stderr\")\n  if obj is sys.stdin:\n    raise pickle.PicklingError(\"Cannot pickle standard input\")\n  if obj.closed:\n    raise pickle.PicklingError(\"Cannot pickle closed files\")\n  if hasattr(obj, \"isatty\") and obj.isatty():\n    raise pickle.PicklingError(\"Cannot pickle files that map to tty objects\")\n  if \"r\" not in obj.mode and \"+\" not in obj.mode:\n    raise pickle.PicklingError(\n        \"Cannot pickle files that are not opened for reading: %s\" % obj.mode)\n\n  name = obj.name\n\n  retval = io.StringIO()","sourceCodeStart":1053,"sourceCodeEnd":1089,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/internal/cloudpickle/cloudpickle.py#L1053-L1089","documentation":"Beam's vendored cloudpickle supports pickling open file objects via _file_reduce, but only files that expose both .name and .mode attributes. Objects lacking them (e.g. io.StringIO, io.BytesIO, temporary buffers) cannot be mapped back to a real file, so pickling raises pickle.PicklingError.","triggerScenarios":"Cloudpickling a task graph / function closure that references an open file-like object without name and mode attributes, such as io.StringIO(), io.BytesIO(), or a wrapped stream.","commonSituations":"Capturing an in-memory buffer in a lambda or DoFn closure that Beam serializes; passing StringIO fixtures into remote-executed code; wrapping files in adapter classes that hide name/mode.","solutions":["Replace the StringIO/BytesIO reference with a real file opened by path (it will have name and mode).","Remove the file object from the pickled closure; open it inside the DoFn/function at runtime.","Pass the file path string through closure and reopen it on the worker.","Write buffer contents to a temp file first: tempfile.NamedTemporaryFile(delete=False)."],"exampleFix":"// before\nbuf = io.StringIO('data')\ndo_fn_capturing(buf)  # pickling fails\n// after\ndef load():\n  return io.StringIO('data')  # construct at runtime inside the function","handlingStrategy":"type-guard","validationCode":"if not (hasattr(obj, 'name') and hasattr(obj, 'mode')):\n    raise TypeError('object cannot be cloudpickled as a file')","typeGuard":"def is_picklable_file(obj):\n    return hasattr(obj, 'name') and hasattr(obj, 'mode') and not isinstance(obj, (io.StringIO, io.BytesIO))","tryCatchPattern":"try:\n    cloudpickle.dump(payload)\nexcept pickle.PicklingError:\n    payload = strip_file_objects(payload)","preventionTips":["Never capture StringIO/BytesIO in serialized closures","Open files by path inside worker functions","Audit closure variables with cloudpickle before submitting"],"tags":["python","pickle","serialization","cloudpickle"],"backgroundTag":"json-serialization-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}