{"record":{"id":"30adafe2ae3a3948","repo":"binary-husky/gpt_academic","slug":"error-30adaf","errorCode":null,"errorMessage":"程序终止。","messagePattern":"程序终止。","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"request_llms/bridge_jittorllms_llama.py","lineNumber":136,"sourceCode":"        if len(observe_window) >= 1: observe_window[0] = load_message + \"\\n\\n\" + llama_glm_handle.info\n        if not llama_glm_handle.success:\n            error = llama_glm_handle.info\n            llama_glm_handle = None\n            raise RuntimeError(error)\n\n    # jittorllms 没有 sys_prompt 接口，因此把prompt加入 history\n    history_feedin = []\n    for i in range(len(history)//2):\n        history_feedin.append([history[2*i], history[2*i+1]] )\n\n    watch_dog_patience = 5 # 看门狗 (watchdog) 的耐心, 设置5秒即可\n    response = \"\"\n    for response in llama_glm_handle.stream_chat(query=inputs, history=history_feedin, system_prompt=sys_prompt, max_length=llm_kwargs['max_length'], top_p=llm_kwargs['top_p'], temperature=llm_kwargs['temperature']):\n        print(response)\n        if len(observe_window) >= 1:  observe_window[0] = response\n        if len(observe_window) >= 2:\n            if (time.time()-observe_window[1]) > watch_dog_patience:\n                raise RuntimeError(\"程序终止。\")\n    return response\n\n\n\ndef predict(inputs, llm_kwargs, plugin_kwargs, chatbot, history=[], system_prompt='', stream = True, additional_fn=None):\n    \"\"\"\n        单线程方法\n        函数的说明请见 request_llms/bridge_all.py\n    \"\"\"\n    chatbot.append((inputs, \"\"))\n\n    global llama_glm_handle\n    if llama_glm_handle is None:\n        llama_glm_handle = GetGLMHandle()\n        chatbot[-1] = (inputs, load_message + \"\\n\\n\" + llama_glm_handle.info)\n        yield from update_ui(chatbot=chatbot, history=[])\n        if not llama_glm_handle.success:\n            llama_glm_handle = None","sourceCodeStart":118,"sourceCodeEnd":154,"githubUrl":"https://github.com/binary-husky/gpt_academic/blob/d6bde0fa54373309bd05823a49bda8da019d2c77/request_llms/bridge_jittorllms_llama.py#L118-L154","documentation":"Watchdog abort in llama_predict_no_ui_long_connection: while iterating llama_glm_handle.stream_chat, if more than watch_dog_patience (5s) elapse since observe_window[1] without a new yield, RuntimeError('程序终止。') kills the generator. JittorLLMs generation can pause far longer than 5s between tokens (long prefill, model swapping, CPU inference).","triggerScenarios":"Local llama inference where a single forward/prefill pass takes >5s (CPU device, long context, first-token latency), or the loader subprocess died/hung so stream_chat stops yielding.","commonSituations":"LOCAL_MODEL_DEVICE=cpu with a large model, cold start downloading/loading weights mid-stream, GPU OOM in the child process silently stopping the stream, machine under heavy load.","solutions":["Confirm the llama subprocess is alive and check its stderr — if it died (OOM, crash), fix that first.","Raise watch_dog_patience well past worst-case first-token latency (60s+ for CPU inference).","Set LOCAL_MODEL_DEVICE to a real GPU or use a smaller model/quantization to cut per-step time.","Refresh observe_window[1] on every yield in the caller so only genuine stalls trip the dog."],"exampleFix":"// before\nwatch_dog_patience = 5\nif (time.time()-observe_window[1]) > watch_dog_patience:\n    raise RuntimeError(\"程序终止。\")\n\n# after\nwatch_dog_patience = 60  # CPU/prefill can far exceed 5s between yields\nif (time.time()-observe_window[1]) > watch_dog_patience:\n    raise RuntimeError(\"llama stream stalled >60s; check child process/OOM.\")","handlingStrategy":"retry","validationCode":"import time\nwindow = ['', time.time()]\n# ensure caller refreshes window[1] before each poll so only real stalls trip the dog","typeGuard":null,"tryCatchPattern":"try:\n    resp = llama_predict_no_ui_long_connection(..., observe_window=window)\nexcept RuntimeError as e:\n    if '程序终止' in str(e) and llama_glm_handle is not None:\n        window[1] = time.time()\n        resp = llama_predict_no_ui_long_connection(..., observe_window=window)  # single retry","preventionTips":["Set watch_dog_patience >= 60s for CPU local inference","Monitor the child process for OOM/crash before blaming the watchdog","Warm up the model with a tiny prompt before long generations"],"tags":["jittorllms","llama","watchdog","timeout","local-model"],"backgroundTag":null,"analyzedSha":"d6bde0fa54373309bd05823a49bda8da019d2c77","analyzedAt":"2026-08-14T22:48:35.038Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}