jamiepine/voicebox · warning · HTTPException

Invalid LLM size '{model_size}'. Must be one of: {sorted(val

Error message

Invalid LLM size '{model_size}'. Must be one of: {sorted(valid_sizes)}

What it means

Returned by POST /llm/generate when the resolved model_size is not one of the registered Qwen3 LLM sizes. valid_sizes is built at runtime from get_llm_model_configs(), which yields exactly {'0.6B','1.7B','4B'}. model_size comes from request.model_size (defaulting via the Pydantic field) or falls back to backend.model_size, so a drifted backend default can also trip it even when the client omits the field.

Source

Thrown at backend/routes/llm.py:27

from ..backends import get_llm_model_configs
from ..services import llm
from ..services.task_queue import create_background_task
from ..utils.tasks import get_task_manager

logger = logging.getLogger(__name__)

router = APIRouter()


@router.post("/llm/generate", response_model=models.LLMGenerateResponse)
async def llm_generate(request: models.LLMGenerateRequest):
    """Run a single-turn Qwen3 completion."""
    backend = llm.get_llm_model()
    model_size = request.model_size or backend.model_size

    valid_sizes = {cfg.model_size for cfg in get_llm_model_configs()}
    if model_size not in valid_sizes:
        raise HTTPException(
            status_code=400,
            detail=f"Invalid LLM size '{model_size}'. Must be one of: {sorted(valid_sizes)}",
        )

    already_loaded = backend.is_loaded() and backend.model_size == model_size
    if not already_loaded and not backend._is_model_cached(model_size):
        progress_model_name = f"qwen3-{model_size.lower()}"
        task_manager = get_task_manager()

        async def download_llm_background():
            try:
                await backend.load_model(model_size)
                task_manager.complete_download(progress_model_name)
            except Exception as e:
                task_manager.error_download(progress_model_name, str(e))

        task_manager.start_download(progress_model_name)
        create_background_task(download_llm_background())

View on GitHub (pinned to 51f49dea19)

Solutions

  1. Send one of the documented sizes: "0.6B", "1.7B", or "4B" (exact case, capital B).
  2. Omit model_size entirely to use the backend's configured default for that request.
  3. If you maintain the client, fetch the valid set dynamically rather than hard-coding — call GET /models/status or read the LLMGenerateRequest schema and filter to qwen_llm engine configs.
  4. If you hit this while omitting model_size, check that the LLM backend instance has not had its model_size reassigned to a non-registry value at startup.

Example fix

// before
fetch('/llm/generate', {method:'POST', body: JSON.stringify({prompt: text, model_size: '7B'})})
// after
fetch('/llm/generate', {method:'POST', body: JSON.stringify({prompt: text, model_size: '4B'})})
Defensive patterns

Strategy: validation

Validate before calling

const VALID_LLM_SIZES = ['0.6B','1.7B','4B'];
function pickSize(req) {
  const size = req.model_size || '0.6B';
  if (!VALID_LLM_SIZES.includes(size)) {
    throw new Error(`model_size must be one of ${VALID_LLM_SIZES.join(', ')}`);
  }
  return size;
}
// before fetch:
const model_size = pickSize(body);
fetch('/llm/generate', {method:'POST', body: JSON.stringify({...body, model_size})});

Type guard

function isLlmSize(x: unknown): x is '0.6B' | '1.7B' | '4B' {
  return typeof x === 'string' && ['0.6B','1.7B','4B'].includes(x);
}

Try / catch

try {
  const res = await fetch('/llm/generate', {method:'POST', body: JSON.stringify(payload)});
  if (res.status === 400) {
    const err = await res.json();
    // err.detail lists valid sizes — refresh client's size list and surface to user
    throw new UserInputError(err.detail);
  }
  return await res.json();
} catch (e) { /* network/retry logic */ }

Prevention

When it happens

Trigger: POST /llm/generate with body {"prompt":"...","model_size":"7B"} (or "2B", "8b", "0.6b" with wrong case, etc.); also reachable if request.model_size is omitted/null and the live LLM backend's model_size attribute has been set to a value outside the registry (e.g. monkey-patched, or a config refactor that changed size strings).

Common situations: Client hard-coded an outdated size after an upgrade that renamed sizes; frontend dropdown populated from a stale list; casing mismatch ('0.6b' vs '0.6B'); backend default drift after manually reassigning backend.model_size for testing.

Related errors

AI-assisted analysis of jamiepine/voicebox@51f49dea19 (2026-08-12). Data as JSON: /api/errors/fb98984f4b755f80. Report an issue: GitHub.