jingyaogong/minimind · error · HTTPException

str(e)

Error message

str(e)

What it means

This is a catch-all except inside the FastAPI /v1/chat/completions handler: any exception raised while running generation (tokenization, model forward pass, sampling, tool-call formatting) is re-raised as HTTPException(500, detail=str(e)). The 'str(e)' message is a placeholder artifact — the actual text is the string of whatever inner exception occurred, so the 500 carries no stable identifier. Diagnosing it requires reproducing the inner exception server-side.

Source

Thrown at scripts/serve_openai_api.py:234

            if reasoning_content:
                message["reasoning_content"] = reasoning_content
            if tool_calls:
                message["tool_calls"] = tool_calls
            return {
                "id": f"chatcmpl-{int(time.time())}",
                "object": "chat.completion",
                "created": int(time.time()),
                "model": "minimind",
                "choices": [
                    {
                        "index": 0,
                        "message": message,
                        "finish_reason": "tool_calls" if tool_calls else "stop"
                    }
                ]
            }
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))


if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Server for MiniMind")
    parser.add_argument('--load_from', default='../model', type=str, help="模型加载路径（model=原生torch权重，其他路径=transformers格式）")
    parser.add_argument('--save_dir', default='out', type=str, help="模型权重目录")
    parser.add_argument('--weight', default='full_sft', type=str, help="权重名称前缀（pretrain, full_sft, dpo, reason, ppo_actor, grpo, spo）")
    parser.add_argument('--lora_weight', default='None', type=str, help="LoRA权重名称（None表示不使用，可选：lora_identity, lora_medical）")
    parser.add_argument('--hidden_size', default=768, type=int, help="隐藏层维度")
    parser.add_argument('--num_hidden_layers', default=8, type=int, help="隐藏层数量")
    parser.add_argument('--max_seq_len', default=8192, type=int, help="最大序列长度")
    parser.add_argument('--use_moe', default=0, type=int, choices=[0, 1], help="是否使用MoE架构（0=否，1=是）")
    parser.add_argument('--inference_rope_scaling', default=False, action='store_true', help="启用RoPE位置编码外推（4倍，仅解决位置编码问题）")
    parser.add_argument('--device', default='cuda' if torch.cuda.is_available() else 'cpu', type=str, help="运行设备")
    args = parser.parse_args()
    device = args.device
    model, tokenizer = init_model(args)
    uvicorn.run(app, host="0.0.0.0", port=8998)

View on GitHub (pinned to 393e387e9a)

Solutions

1. Run the server in the foreground and read the traceback printed before the HTTPException is raised — the 500 detail is only str(e), the real stack is server-side.
2. Verify --load_from path exists and matches the checkpoint format (torch weights vs transformers dir) and that --weight prefix names an existing file.
3. Check --hidden_size/--num_hidden_layers/--use_moe match the checkpoint config; a mismatch typically raises a tensor-shape error that becomes this 500.
4. Shorten the input below --max_seq_len (8192 default) or raise the flag if the GPU allows.
5. If it is CUDA OOM, reduce batch/seq length or move to a smaller quantization/beam setting.
6. Improve the handler: log the full traceback (logging.exception) and return a typed detail so clients can distinguish causes.

Example fix

# before
except Exception as e:
    raise HTTPException(status_code=500, detail=str(e))

# after
except Exception as e:
    logging.exception("chat completion failed")
    raise HTTPException(status_code=500, detail={"error": type(e).__name__, "message": str(e)})

# Before sending the real request, validate the server can generate
import requests
health = requests.get(f"{base_url}/health", timeout=5)
health.raise_for_status()
# optional: probe with a 1-token request to surface model-loading errors cheaply
probe = requests.post(f"{base_url}/v1/chat/completions", json={
    "model": "minimind", "messages": [{"role": "user", "content": "hi"}], "max_tokens": 1
}, timeout=30)
probe.raise_for_status()

from requests import HTTPError, RequestException
try:
    r = requests.post(f"{base_url}/v1/chat/completions", json=payload, timeout=120)
    r.raise_for_status()
    result = r.json()
except HTTPError as e:
    if e.response.status_code == 500:
        # detail is str(e) of an unknown inner exception: safe to retry once for transient
        # causes (OOM recovery, racey load); if it persists, inspect server logs
        time.sleep(2)
        r = requests.post(f"{base_url}/v1/chat/completions", json=payload, timeout=120)
        r.raise_for_status()
        result = r.json()
    else:
        raise

When it happens

Trigger: POSTing to the MiniMind OpenAI-compatible endpoint (serve_openai_api.py) with a request whose prompt exceeds max_seq_len (default 8192), a tool-call payload the formatting code cannot serialize, CUDA OOM on the forward pass, a missing/invalid weight path (--load_from default '../model' not present), or a tokenizer/model config mismatch (hidden_size=768/num_hidden_layers=8 not matching the checkpoint). Any of these raises inside the try block and surfaces as detail=str(e) with status 500.

Common situations: Running the server from a different working directory so '../model' resolves wrong; loading a full_sft/DPO checkpoint with wrong --hidden_size or --num_hidden_layers; sending messages longer than --max_seq_len; hitting a device mismatch (model on cuda, request path assumes cpu or vice versa); version drift between transformers and the saved checkpoint format.

AI-assisted analysis of jingyaogong/minimind@393e387e9a (2026-08-15). Data as JSON: /api/errors/c6295c8e5f5c5f91. Report an issue: GitHub.