jax-ml/jax · error · RuntimeError

mTLS for the JAX distributed service requires jaxlib 0.11.2

Error message

mTLS for the JAX distributed service requires jaxlib 0.11.2 or newer.

What it means

The distributed service's mTLS options (cert/key/CA files, verify_secure_credentials) only exist in jaxlib >= 0.11.2 (extension version >= 483). Requesting any of them on an older jaxlib raises this RuntimeError from _get_mtls_kwargs during jax.distributed.initialize.

Source

Thrown at jax/_src/distributed.py:84

  kwargs: dict[str, Any] = dict(
      mtls_cert_file=value_or_config(
          mtls_cert_file, config.mtls_cert_file),
      mtls_key_file=value_or_config(
          mtls_key_file, config.mtls_key_file),
      mtls_ca_file=value_or_config(
          mtls_ca_file, config.mtls_ca_file),
      mtls_peer_uri_prefix=value_or_config(
          mtls_peer_uri_prefix, config.mtls_peer_uri_prefix),
      verify_secure_credentials=value_or_config(
          verify_secure_credentials,
          config.distributed_verify_secure_credentials),
  )
  if jaxlib_extension_version < 483:
    verify_secure_credentials = kwargs.pop('verify_secure_credentials')
    if (verify_secure_credentials
        or any(v is not None for v in kwargs.values())):
      raise RuntimeError('mTLS for the JAX distributed service requires '
                         'jaxlib 0.11.2 or newer.')
    return {}
  return kwargs


class State:
  process_id: int = 0
  num_processes: int = 1
  service: _jax.DistributedRuntimeService | Any | None = None
  client: _jax.DistributedRuntimeClient | Any | None = None
  preemption_sync_manager: Any | None = None
  coordinator_address: str | None = None
  partition_index: int | None = None

  def initialize(self,
                 coordinator_address: str | None = None,
                 num_processes: int | None = None,
                 process_id: int | None = None,

View on GitHub (pinned to 1e1c6a8fc0)

Solutions

1. Upgrade jaxlib (and jax) to >= 0.11.2: pip install -U jax jaxlib
2. If you cannot upgrade, omit all mTLS kwargs and verify_secure_credentials and use plain TCP for the coordinator
3. Verify versions: python -c "import jaxlib; print(jaxlib.__version__)" and check extension version >= 483

Example fix

# before (old jaxlib)
jax.distributed.initialize(..., mtls_cert_file='c.pem', mtls_key_file='k.pem')
# after
pip install -U "jax[cpu]>=0.11.2" "jaxlib>=0.11.2"
jax.distributed.initialize(..., mtls_cert_file='c.pem', mtls_key_file='k.pem')

from jaxlib import extension_version if False else None
import jax._src.lib as jlib
MTLS_OK = jlib.jaxlib_extension_version >= 483
assert MTLS_OK or not any([cert, key, ca, verify]), 'jaxlib too old for mTLS'

When it happens

Trigger: Calling jax.distributed.initialize with mtls_cert_file/mtls_key_file/mtls_ca_file or verify_secure_credentials=True while the installed jaxlib extension version is below 483.

Common situations: Pinning old jaxlib in requirements while using new jax; installing jax via a stale pip environment or older CUDA wheel; security-hardened clusters requiring mTLS with an outdated toolchain.

Related errors

• stop_gradient only works on valid JAX arrays, but input argu
• Mapped away dimension of inputs passed to vmap should be sha
• Unknown GPU platform for __dlpack__: {platform_version}
• The 'sharding' attribute is not available on {self._error_re
• The is_fully_addressable property was called on {self._error

AI-assisted analysis of jax-ml/jax@1e1c6a8fc0 (2026-08-27). Data as JSON: /api/errors/4e128737e0037c05. Report an issue: GitHub.