juicedata/juicefs · error · ValueError

Cannot serialise open write-mode local file

Error message

Cannot serialise open write-mode local file

What it means

LocalFile.__getstate__ (used for pickling/serializing the handle) refuses to serialize a write-mode local file whose underlying temp file is still open, raising ValueError('Cannot serialise open write-mode local file'). Write-mode files carry uncommitted temp state that cannot be transferred across processes.

Source

Thrown at sdk/python/juicefs/juicefs/spec.py:256

        return self.f.read(end - start)

    def __setstate__(self, state):
        self.f = None
        loc = state.pop("loc", None)
        self.__dict__.update(state)
        if "r" in state["mode"]:
            self.f = None
            self._open()
            self.f.seek(loc)

    def __getstate__(self):
        d = self.__dict__.copy()
        d.pop("f")
        if "r" in self.mode:
            d["loc"] = self.f.tell()
        else:
            if not self.f.closed:
                raise ValueError("Cannot serialise open write-mode local file")
        return d

    def commit(self):
        if self.autocommit:
            raise RuntimeError("Can only commit if not already set to autocommit")
        self.fs.fs.rename(self.temp, self.path)

    def discard(self):
        if self.autocommit:
            raise RuntimeError("Can only commit if not already set to autocommit")
        self.fs.fs.remove(self.temp)

    def tell(self):
        return self.f.tell()

    def seek(self, loc, whence=0):
        return self.f.seek(loc, whence)

View on GitHub (pinned to c9a67b23e8)

Solutions

1. Close or commit the file before pickling
2. Open the file again inside the target process instead of transferring the handle
3. Open in read mode ('r') if the handle must be pickled, since read-mode files serialize their location
4. Subclass and override __getstate__ only if you can guarantee the temp file semantics yourself

Example fix

// before
f = fs.open('/mnt/jfs/out', 'wb')
pickle.dumps(f)  # ValueError
// after
f = fs.open('/mnt/jfs/out', 'wb')
f.commit()   # or f.close()
pickle.dumps(f)
# or reopen in worker:
worker_fn(path='/mnt/jfs/out')

if 'w' in f.mode and not f.f.closed:
    f.commit()  # or f.close() before pickling
pickle.dumps(f)

def picklable_file(f) -> bool:
    return 'r' in f.mode or f.f.closed

try:
    state = pickle.dumps(f)
except ValueError as e:
    if 'serialise open write-mode' in str(e):
        f.close(); state = pickle.dumps(f)

When it happens

Trigger: Pickling a LocalFile opened for writing ('w'/'wb') before close()/commit(); using multiprocessing or distributed frameworks (e.g. Dask) that pickle open write handles; copy.copy/deep_copy of such a handle.

Common situations: Passing open write handles to worker processes; serializing file objects in task queues; checkpointing code that stores file objects in pickled state.

Understand the failure class

Background: "JSON serialization failed", "not JSON serializable", "Failed to serialize": why JSON marshaling errors happen and how to fix them — this error's family across 46 libraries.

Related errors

• binary mode doesn't take an errors argument
• invalid whence ({whence}, should be {os.SEEK_SET}, {os.SEEK_
• I/O operation on closed file.
• Can only commit if not already set to autocommit
• invalid mode: {mode}

AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06). Data as JSON: /api/errors/ad95464f5a17f47d. Report an issue: GitHub.