{"record":{"id":"26252d8e7671706f","repo":"apache/beam","slug":"cannot-pickle-file-s-as-it-cannot-be-read","errorCode":null,"errorMessage":"Cannot pickle file %s as it cannot be read","messagePattern":"Cannot pickle file (.+?) as it cannot be read","errorType":"exception","errorClass":"PicklingError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/internal/cloudpickle/cloudpickle.py","lineNumber":1098,"sourceCode":"    raise pickle.PicklingError(\"Cannot pickle closed files\")\n  if hasattr(obj, \"isatty\") and obj.isatty():\n    raise pickle.PicklingError(\"Cannot pickle files that map to tty objects\")\n  if \"r\" not in obj.mode and \"+\" not in obj.mode:\n    raise pickle.PicklingError(\n        \"Cannot pickle files that are not opened for reading: %s\" % obj.mode)\n\n  name = obj.name\n\n  retval = io.StringIO()\n\n  try:\n    # Read the whole file\n    curloc = obj.tell()\n    obj.seek(0)\n    contents = obj.read()\n    obj.seek(curloc)\n  except OSError as e:\n    raise pickle.PicklingError(\n        \"Cannot pickle file %s as it cannot be read\" % name) from e\n  retval.write(contents)\n  retval.seek(curloc)\n\n  retval.name = name\n  return _file_reconstructor, (retval, )\n\n\ndef _getset_descriptor_reduce(obj):\n  return getattr, (obj.__objclass__, obj.__name__)\n\n\ndef _mappingproxy_reduce(obj):\n  return types.MappingProxyType, (dict(obj), )\n\n\ndef _memoryview_reduce(obj):\n  return bytes, (obj.tobytes(), )","sourceCodeStart":1080,"sourceCodeEnd":1116,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/internal/cloudpickle/cloudpickle.py#L1080-L1116","documentation":"When pickling a file, cloudpickle tries to read its entire contents (seek(0), read(), seek back). If that raises OSError, the file cannot be captured, and _file_reduce re-raises as pickle.PicklingError('Cannot pickle file <name> as it cannot be read'), chaining the original OSError.","triggerScenarios":"Pickling a file handle whose backing file is unreadable: deleted-but-open files, permission changes, special/device files that fail on seek or read, files on unmounted media, or sockets/ fifos opened with a name and read mode.","commonSituations":"Rotated/deleted log files still held open; files on NFS/volumes that went away between open and pickle; attempting to serialize handles to pipes or character devices; container filesystem changes between pipeline submission and serialization.","solutions":["Verify the file still exists and is readable (os.access(path, os.R_OK)); reopen it if deleted.","Copy the file to a stable local temp location and pickle a handle/path to that.","Avoid pickling handles to special files (pipes, devices); read the needed data eagerly instead.","Check file permissions and mount status; fix access before running the pipeline."],"exampleFix":"// before\nf = open(path)  # file deleted before serialization\ndefer(fn_capturing(f))\n// after\nshutil.copy(path, tmp_path)\nf = open(tmp_path)\ndefer(fn_capturing(f))","handlingStrategy":"validation","validationCode":"import os\nif not os.access(f.name, os.R_OK) or not os.path.exists(f.name):\n    raise IOError(f'file {f.name} not readable at serialization time')","typeGuard":"def is_file_readable(obj):\n    import os\n    return hasattr(obj, 'name') and os.path.exists(obj.name) and os.access(obj.name, os.R_OK)","tryCatchPattern":"try:\n    cloudpickle.dump(fn)\nexcept pickle.PicklingError as e:\n    if 'cannot be read' in str(e): fn = copy_and_reopen(fn)\n    else: raise","preventionTips":["Ensure files remain present and readable until submission","Copy to stable local storage before serialization","Avoid handles to special/rotated files"],"tags":["python","pickle","serialization","oserror"],"backgroundTag":"file-read-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}