binary-husky/gpt_academic · critical · RuntimeError

不能正常加载ChatGLMFT的参数！

Error message

不能正常加载ChatGLMFT的参数！

What it means

GetModelHolderLoadThread for ChatGLMFT retries loading the fine-tuned model/parameters up to 3 times; on the 4th failure (retry > 3) it raises RuntimeError('不能正常加载ChatGLMFT的参数！') and also notifies the child process. The underlying exception from model loading (missing checkpoint files, GPU OOM, config mismatch) is discarded — only the attempt counter survives.

Source

Thrown at request_llms/bridge_chatglmft.py:107

                    model.transformer.prefix_encoder.load_state_dict(new_prefix_state_dict)

                    if model_args['quantization_bit'] is not None and model_args['quantization_bit'] != 0:
                        logger.info(f"Quantized to {model_args['quantization_bit']} bit")
                        model = model.quantize(model_args['quantization_bit'])
                    model = model.cuda()
                    if model_args['pre_seq_len'] is not None:
                        # P-tuning v2
                        model.transformer.prefix_encoder.float()
                    self.chatglmft_model = model.eval()

                    break
                else:
                    break
            except Exception as e:
                retry += 1
                if retry > 3:
                    self.child.send('[Local Message] Call ChatGLMFT fail 不能正常加载ChatGLMFT的参数。')
                    raise RuntimeError("不能正常加载ChatGLMFT的参数！")

        while True:
            # 进入任务等待状态
            kwargs = self.child.recv()
            # 收到消息，开始请求
            try:
                for response, history in self.chatglmft_model.stream_chat(self.chatglmft_tokenizer, **kwargs):
                    self.child.send(response)
                    # # 中途接收可能的终止指令（如果有的话）
                    # if self.child.poll():
                    #     command = self.child.recv()
                    #     if command == '[Terminate]': break
            except:
                from toolbox import trimmed_format_exc
                self.child.send('[Local Message] Call ChatGLMFT fail.' + '\n```\n' + trimmed_format_exc() + '\n```\n')
            # 请求处理结束，开始下一个循环
            self.child.send('[Finish]')

View on GitHub (pinned to d6bde0fa54)

Solutions

  1. Point CHATGLMFT_CHECKPOINT at a complete checkpoint dir (contains config.json, tokenizer files, and the model weights) and verify the files exist.
  2. Match versions: checkpoint, transformers, and the ChatGLM base model version must agree; re-export the checkpoint if not.
  3. Check GPU memory (nvidia-smi) and free it / use a smaller device_map or CPU test-load first to see the real error.
  4. Reproduce the load outside the thread (plain AutoModel.from_pretrained on the same path) to surface the actual exception that the retry loop swallows.

Example fix

# before
MODEL_PATH = conf_singleton['CHATGLMFT_CHECKPOINT']

# after (surface the real error once instead of 3 blind retries)
try:
    model = AutoModel.from_pretrained(MODEL_PATH, trust_remote_code=True)
except Exception as e:
    logger.exception(f'ChatGLMFT checkpoint load failed: {e}')
    raise
Defensive patterns

Strategy: validation

Validate before calling

from pathlib import Path

def checkpoint_complete(path: str) -> bool:
    p = Path(path)
    required = ['config.json']
    weights = ['pytorch_model.bin', 'model.safetensors']
    return p.is_dir() and all((p / f).exists() for f in required) and any((p / w).exists() for w in weights)

if not checkpoint_complete(conf['CHATGLMFT_CHECKPOINT']):
    raise SystemExit('CHATGLMFT_CHECKPOINT incomplete: need config.json + weights')

Try / catch

try:
    run_chatglmft_request(...)
except RuntimeError as e:
    if 'ChatGLMFT' in str(e):
        logger.error('model load failed — check checkpoint path, GPU memory, version match')
        switch_to_remote_model()  # fallback provider
    raise

Prevention

When it happens

Trigger: CHATGLMFT_CHECKPOINT path points to a directory without the expected model files (missing pytorch_model.bin / config.json / tokenizer files); the fine-tuned checkpoint's architecture/parameter names do not match the ChatGLM version installed; CUDA out-of-memory or missing GPU when instantiating the model; P-tuning v2 checkpoint loaded without matching pre_seq_len settings.

Common situations: Wrong or partially-uploaded checkpoint directory; checkpoint trained on ChatGLM2 loaded into a ChatGLM3 runtime (or vice versa); shared GPU already occupied so every load attempt OOMs; config keys (pre_seq_len, dtype) not matching the checkpoint, making load_state_dict raise on each retry.

Related errors

AI-assisted analysis of binary-husky/gpt_academic@d6bde0fa54 (2026-08-14). Data as JSON: /api/errors/95a42eddd25d0083. Report an issue: GitHub.