exo-explore/exo · error · ValueError

CfgShardMetadata is not supported for text model loading - t

Error message

CfgShardMetadata is not supported for text model loading - this metadata type is only for image generation models

What it means

Error "CfgShardMetadata is not supported for text model loading - this metadata type is only for image generation models" thrown in exo-explore/exo.

Source

Thrown at src/exo/worker/engines/mlx/utils_mlx.py:272

        #        model, config, group_size=KV_GROUP_SIZE, bits=ATTENTION_KV_BITS, quant_predicate=quant_predicate, mode=QUANTIZE_MODEL_MODE
        #    )

    assert isinstance(model, nn.Module)

    tokenizer = get_tokenizer(model_path, shard_metadata)

    logger.info(f"Group size: {group.size()}, group rank: {group.rank()}")

    match shard_metadata:
        case TensorShardMetadata():
            logger.info(f"loading model from {model_path} with tensor parallelism")
            model = yield from tensor_auto_parallel(model, group)
        case PipelineShardMetadata():
            logger.info(f"loading model from {model_path} with pipeline parallelism")
            model = yield from pipeline_auto_parallel(model, group, shard_metadata)
            mx.eval(model.parameters())
        case CfgShardMetadata():
            raise ValueError(
                "CfgShardMetadata is not supported for text model loading - "
                "this metadata type is only for image generation models"
            )

    # TODO: Do we need this?
    mx.eval(model)

    logger.debug("SHARDED")
    logger.debug(model)

    # Synchronize processes before generation to avoid timeout
    mx_barrier(group)

    return model, tokenizer


def get_tokenizer(model_path: Path, shard_metadata: ShardMetadata) -> TokenizerWrapper:
    """Load tokenizer for a model shard. Delegates to load_tokenizer_for_model_id."""

View on GitHub (pinned to 21a54c5ea0)

When it happens

Trigger: Thrown at src/exo/worker/engines/mlx/utils_mlx.py:272 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of exo-explore/exo@21a54c5ea0 (2026-08-26). Data as JSON: /api/errors/92629f255ec93475. Report an issue: GitHub.