binary-husky/gpt_academic · error · RuntimeError

程序终止。

Error message

程序终止。

What it means

Watchdog abort in llama_predict_no_ui_long_connection: while iterating llama_glm_handle.stream_chat, if more than watch_dog_patience (5s) elapse since observe_window[1] without a new yield, RuntimeError('程序终止。') kills the generator. JittorLLMs generation can pause far longer than 5s between tokens (long prefill, model swapping, CPU inference).

Solutions

1. Confirm the llama subprocess is alive and check its stderr — if it died (OOM, crash), fix that first.
2. Raise watch_dog_patience well past worst-case first-token latency (60s+ for CPU inference).
3. Set LOCAL_MODEL_DEVICE to a real GPU or use a smaller model/quantization to cut per-step time.
4. Refresh observe_window[1] on every yield in the caller so only genuine stalls trip the dog.

Example fix

// before
watch_dog_patience = 5
if (time.time()-observe_window[1]) > watch_dog_patience:
    raise RuntimeError("程序终止。")

# after
watch_dog_patience = 60  # CPU/prefill can far exceed 5s between yields
if (time.time()-observe_window[1]) > watch_dog_patience:
    raise RuntimeError("llama stream stalled >60s; check child process/OOM.")

import time
window = ['', time.time()]
# ensure caller refreshes window[1] before each poll so only real stalls trip the dog

try:
    resp = llama_predict_no_ui_long_connection(..., observe_window=window)
except RuntimeError as e:
    if '程序终止' in str(e) and llama_glm_handle is not None:
        window[1] = time.time()
        resp = llama_predict_no_ui_long_connection(..., observe_window=window)  # single retry

When it happens

Trigger: Local llama inference where a single forward/prefill pass takes >5s (CPU device, long context, first-token latency), or the loader subprocess died/hung so stream_chat stops yielding.

Common situations: LOCAL_MODEL_DEVICE=cpu with a large model, cold start downloading/loading weights mid-stream, GPU OOM in the child process silently stopping the stream, machine under heavy load.

Related errors

• 程序终止。
• 程序终止。
• 程序终止。
• 不能正常加载jittorllms的参数！
• 程序终止。

AI-assisted analysis of binary-husky/gpt_academic@d6bde0fa54 (2026-08-14). Data as JSON: /api/errors/30adafe2ae3a3948. Report an issue: GitHub.

Appendix: source

Thrown at request_llms/bridge_jittorllms_llama.py:136

        if len(observe_window) >= 1: observe_window[0] = load_message + "\n\n" + llama_glm_handle.info
        if not llama_glm_handle.success:
            error = llama_glm_handle.info
            llama_glm_handle = None
            raise RuntimeError(error)

    # jittorllms 没有 sys_prompt 接口，因此把prompt加入 history
    history_feedin = []
    for i in range(len(history)//2):
        history_feedin.append([history[2*i], history[2*i+1]] )

    watch_dog_patience = 5 # 看门狗 (watchdog) 的耐心, 设置5秒即可
    response = ""
    for response in llama_glm_handle.stream_chat(query=inputs, history=history_feedin, system_prompt=sys_prompt, max_length=llm_kwargs['max_length'], top_p=llm_kwargs['top_p'], temperature=llm_kwargs['temperature']):
        print(response)
        if len(observe_window) >= 1:  observe_window[0] = response
        if len(observe_window) >= 2:
            if (time.time()-observe_window[1]) > watch_dog_patience:
                raise RuntimeError("程序终止。")
    return response



def predict(inputs, llm_kwargs, plugin_kwargs, chatbot, history=[], system_prompt='', stream = True, additional_fn=None):
    """
        单线程方法
        函数的说明请见 request_llms/bridge_all.py
    """
    chatbot.append((inputs, ""))

    global llama_glm_handle
    if llama_glm_handle is None:
        llama_glm_handle = GetGLMHandle()
        chatbot[-1] = (inputs, load_message + "\n\n" + llama_glm_handle.info)
        yield from update_ui(chatbot=chatbot, history=[])
        if not llama_glm_handle.success:
            llama_glm_handle = None

View on GitHub (pinned to d6bde0fa54)