binary-husky/gpt_academic · error · RuntimeError

程序终止。

Error message

程序终止。

What it means

Watchdog abort in llama_predict_no_ui_long_connection: while iterating llama_glm_handle.stream_chat, if more than watch_dog_patience (5s) elapse since observe_window[1] without a new yield, RuntimeError('程序终止。') kills the generator. JittorLLMs generation can pause far longer than 5s between tokens (long prefill, model swapping, CPU inference).

Source

Thrown at request_llms/bridge_jittorllms_llama.py:136

        if len(observe_window) >= 1: observe_window[0] = load_message + "\n\n" + llama_glm_handle.info
        if not llama_glm_handle.success:
            error = llama_glm_handle.info
            llama_glm_handle = None
            raise RuntimeError(error)

    # jittorllms 没有 sys_prompt 接口，因此把prompt加入 history
    history_feedin = []
    for i in range(len(history)//2):
        history_feedin.append([history[2*i], history[2*i+1]] )

    watch_dog_patience = 5 # 看门狗 (watchdog) 的耐心, 设置5秒即可
    response = ""
    for response in llama_glm_handle.stream_chat(query=inputs, history=history_feedin, system_prompt=sys_prompt, max_length=llm_kwargs['max_length'], top_p=llm_kwargs['top_p'], temperature=llm_kwargs['temperature']):
        print(response)
        if len(observe_window) >= 1:  observe_window[0] = response
        if len(observe_window) >= 2:
            if (time.time()-observe_window[1]) > watch_dog_patience:
                raise RuntimeError("程序终止。")
    return response



def predict(inputs, llm_kwargs, plugin_kwargs, chatbot, history=[], system_prompt='', stream = True, additional_fn=None):
    """
        单线程方法
        函数的说明请见 request_llms/bridge_all.py
    """
    chatbot.append((inputs, ""))

    global llama_glm_handle
    if llama_glm_handle is None:
        llama_glm_handle = GetGLMHandle()
        chatbot[-1] = (inputs, load_message + "\n\n" + llama_glm_handle.info)
        yield from update_ui(chatbot=chatbot, history=[])
        if not llama_glm_handle.success:
            llama_glm_handle = None

View on GitHub (pinned to d6bde0fa54)

Solutions

1. Confirm the llama subprocess is alive and check its stderr — if it died (OOM, crash), fix that first.
2. Raise watch_dog_patience well past worst-case first-token latency (60s+ for CPU inference).
3. Set LOCAL_MODEL_DEVICE to a real GPU or use a smaller model/quantization to cut per-step time.
4. Refresh observe_window[1] on every yield in the caller so only genuine stalls trip the dog.

Example fix

// before
watch_dog_patience = 5
if (time.time()-observe_window[1]) > watch_dog_patience:
    raise RuntimeError("程序终止。")

# after
watch_dog_patience = 60  # CPU/prefill can far exceed 5s between yields
if (time.time()-observe_window[1]) > watch_dog_patience:
    raise RuntimeError("llama stream stalled >60s; check child process/OOM.")

import time
window = ['', time.time()]
# ensure caller refreshes window[1] before each poll so only real stalls trip the dog

try:
    resp = llama_predict_no_ui_long_connection(..., observe_window=window)
except RuntimeError as e:
    if '程序终止' in str(e) and llama_glm_handle is not None:
        window[1] = time.time()
        resp = llama_predict_no_ui_long_connection(..., observe_window=window)  # single retry

When it happens

Trigger: Local llama inference where a single forward/prefill pass takes >5s (CPU device, long context, first-token latency), or the loader subprocess died/hung so stream_chat stops yielding.

Common situations: LOCAL_MODEL_DEVICE=cpu with a large model, cold start downloading/loading weights mid-stream, GPU OOM in the child process silently stopping the stream, machine under heavy load.

Related errors

• 程序终止。
• 程序终止。
• 不能正常加载jittorllms的参数！
• 程序终止。
• 程序终止。

AI-assisted analysis of binary-husky/gpt_academic@d6bde0fa54 (2026-08-14). Data as JSON: /api/errors/30adafe2ae3a3948. Report an issue: GitHub.