mudler/LocalAI · error · ValueError

Unknown backend: {backend}

Error message

Unknown backend: {backend}

What it means

Raised by the distributed-init helper in the mlx-distributed backend when the backend string is neither 'ring' nor 'jaccl'. Only those two MLX distributed backends are wired up; anything else falls to the else branch and raises immediately.

Source

Thrown at backend/python/mlx-distributed/backend.py:66

    JACCL: MLX_IBV_DEVICES points to a JSON 2D matrix of RDMA device names.
    MLX_JACCL_COORDINATOR is rank 0's ip:port where it runs a TCP service that
    helps all ranks establish RDMA connections.
    """
    import mlx.core as mx

    if backend == "ring":
        os.environ["MLX_HOSTFILE"] = hostfile
        os.environ["MLX_RANK"] = str(rank)
        os.environ["MLX_RING_VERBOSE"] = "1"
        return mx.distributed.init(backend="ring", strict=True)
    elif backend == "jaccl":
        os.environ["MLX_IBV_DEVICES"] = hostfile
        os.environ["MLX_RANK"] = str(rank)
        if coordinator:
            os.environ["MLX_JACCL_COORDINATOR"] = coordinator
        return mx.distributed.init(backend="jaccl", strict=True)
    else:
        raise ValueError(f"Unknown backend: {backend}")


# Re-export the shared helper under the local name for back-compat with
# any callers (and the existing distributed worker tests) that imported
# parse_options directly from this module.
parse_options = _shared_parse_options


class BackendServicer(backend_pb2_grpc.BackendServicer):
    """gRPC servicer for distributed MLX inference (runs on rank 0).

    When started by LocalAI (server mode), distributed init happens at
    LoadModel time using config from model options or environment variables.
    """

    def __init__(self):
        self.group = None
        self.dist_backend = None

View on GitHub (pinned to 44413a9d06)

Solutions

1. Use exactly 'ring' or 'jaccl'
2. Check case and whitespace in the config value (strip() it)
3. For MPI-style clusters use 'ring' with a hostfile; for IB/ROCm clusters use 'jaccl'

Example fix

# before
init_distributed('NCCL', hostfile, rank)  # ValueError

# after
init_distributed('ring', hostfile, rank)

VALID = {'ring', 'jaccl'}
backend = (cfg.get('distributed', {}).get('backend') or 'ring').strip().lower()
if backend not in VALID:
    raise ValueError(f'backend must be one of {sorted(VALID)}, got {backend!r}')

def is_known_backend(name) -> bool:
    return isinstance(name, str) and name.strip().lower() in {'ring', 'jaccl'}

try:
    init_distributed(backend, hostfile, rank)
except ValueError as err:
    sys.exit(f'config error: {err}')  # fail fast, no fallback

When it happens

Trigger: Calling the init function with backend='nccl', 'gloo', or a typo like 'Ring'/'JACCL' (case-sensitive). Values typically come from CLI options or the model config's distributed section.

Common situations: Porting configs from PyTorch distributed setups that use nccl/gloo, case mismatches, or referencing a backend removed/renamed between MLX versions.

Related errors

• unknown engine: {name!r}
• dataset_source is required (path to a preprocessed dataset)
• resolution must be 480p or 720p
• start_image is not a readable staged file
• num_frames must not be negative

AI-assisted analysis of mudler/LocalAI@44413a9d06 (2026-08-15). Data as JSON: /api/errors/0e79237ed5a8bcb1. Report an issue: GitHub.