xai-org/x-algorithm · error · ValueError

Execution devices belong to a client other than `backend`. G

Error message

Execution devices belong to a client other than `backend`. Got backend client: {(backend.platform, backend.platform_version)} and execution devices client: {(device_backend.platform, device_backend.platform_version)}

What it means

The pickler checks that the first execution device's client (backend) equals the backend passed in, because serialized device references are only valid within one PJRT client. Devices obtained from a different jax Backend (e.g. a CPU backend vs GPU backend, or a second client created manually) trigger this error, with both platforms printed for diagnosis.

Source

Thrown at phoenix/xrex/utils/aot.py:252

        if isinstance(obj, xc.CompileOptions):
            return ("compile_options", obj.SerializeAsString())
        return None


class _JaxPjrtUnpickler(pickle.Unpickler):
    def __init__(
        self,
        file,
        backend: xc.Client,
        execution_devices: Sequence[xc.Device],
        host_callbacks: Any,
        compile_options: xc.CompileOptions | None,
    ):
        super().__init__(file)
        self.backend = backend
        device_backend = execution_devices[0].client
        if device_backend != backend:
            raise ValueError(
                "Execution devices belong to a client other than `backend`. Got "
                f"backend client: {(backend.platform, backend.platform_version)} "
                "and execution devices client: "
                f"{(device_backend.platform, device_backend.platform_version)}"
            )
        self.execution_devices = xc.DeviceList(tuple(execution_devices))
        self.host_callbacks = host_callbacks
        self.compile_options = compile_options

    def persistent_load(self, pid):
        if pid[0] == "exec":
            compile_options = self.compile_options
            if self.host_callbacks:
                return self.backend.deserialize_executable(
                    pid[1],
                    executable_devices=self.execution_devices,
                    compile_options=compile_options,
                    host_callbacks=self.host_callbacks,

View on GitHub (pinned to 24c60942c5)

Solutions

Derive both backend and devices from the same source: backend = jax.devices()[0].client and execution_devices = jax.devices()
Do not create extra Backends/clients; use the process-default jax backend
Compare the printed platform tuples to spot which backend is mismatched

Example fix

# before
backend = jax.backend_by_platform('cpu')
devices = jax.devices()  # gpu devices
# after
backend = jax.devices()[0].client
devices = jax.devices()

Defensive patterns

Strategy: validation

Validate before calling

devices = jax.devices()
backend = devices[0].client
assert devices[0].client == backend

Prevention

Derive backend from jax.devices()[0].client, never a separately constructed backend
Avoid multi-client initialization in single-process jobs
Log (backend.platform, devices[0].client.platform) at startup as a sanity check

When it happens

Trigger: Constructing the pickler with backend = jax.backend_by_platform('cpu') but devices from jax.devices() on GPU; creating multiple backends/clients (common with jax.extend or custom PJRT initialization) and mixing them.

Common situations: Multi-backend setups (CPU host backend + GPU compute backend); plugin runtimes where a separate client is created; refactors that cache a backend across processes.

Related errors

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/4a563c7fafcefbf7. Report an issue: GitHub.