{"record":{"id":"726b05af2f580690","repo":"binary-husky/gpt_academic","slug":"llm-handle-get-state","errorCode":null,"errorMessage":"_llm_handle.get_state()","messagePattern":"_llm_handle\\.get_state\\(\\)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"request_llms/local_llm_class.py","lineNumber":227,"sourceCode":"                    self.running = False\n                    self.corrupted = True\n                    break\n                else:\n                    std_out = \"\"\n                    yield res\n\ndef get_local_llm_predict_fns(LLMSingletonClass, model_name, history_format='classic'):\n    load_message = f\"{model_name}尚未加载，加载需要一段时间。注意，取决于`config.py`的配置，{model_name}消耗大量的内存（CPU）或显存（GPU），也许会导致低配计算机卡死 ……\"\n\n    def predict_no_ui_long_connection(inputs:str, llm_kwargs:dict, history:list=[], sys_prompt:str=\"\", observe_window:list=[], console_silence:bool=False):\n        \"\"\"\n            refer to request_llms/bridge_all.py\n        \"\"\"\n        _llm_handle = GetSingletonHandle().get_llm_model_instance(LLMSingletonClass)\n        if len(observe_window) >= 1:\n            observe_window[0] = load_message + \"\\n\\n\" + _llm_handle.get_state()\n        if not _llm_handle.running:\n            raise RuntimeError(_llm_handle.get_state())\n\n        if history_format == 'classic':\n            # 没有 sys_prompt 接口，因此把prompt加入 history\n            history_feedin = []\n            history_feedin.append([sys_prompt, \"Certainly!\"])\n            for i in range(len(history)//2):\n                history_feedin.append([history[2*i], history[2*i+1]])\n        elif history_format == 'chatglm3':\n            # 有 sys_prompt 接口\n            conversation_cnt = len(history) // 2\n            history_feedin = [{\"role\": \"system\", \"content\": sys_prompt}]\n            if conversation_cnt:\n                for index in range(0, 2*conversation_cnt, 2):\n                    what_i_have_asked = {}\n                    what_i_have_asked[\"role\"] = \"user\"\n                    what_i_have_asked[\"content\"] = history[index]\n                    what_gpt_answer = {}\n                    what_gpt_answer[\"role\"] = \"assistant\"","sourceCodeStart":209,"sourceCodeEnd":245,"githubUrl":"https://github.com/binary-husky/gpt_academic/blob/d6bde0fa54373309bd05823a49bda8da019d2c77/request_llms/local_llm_class.py#L209-L245","documentation":"In predict_no_ui_long_connection (the non-UI plugin pathway), after obtaining the singleton local-LLM handle it checks _llm_handle.running; if the loader subprocess died (e.g. it raised error 182's RuntimeError), running is False and this raises RuntimeError with the handle's last state string (e.g. '`加载模型失败`'). The message is the state label, not a stack trace.","triggerScenarios":"Calling predict_no_ui_long_connection for a local model whose background load failed or crashed: OOM kill of the loader process, exception during load_model_and_tokenizer, or the singleton was constructed moments ago and died before this call.","commonSituations":"Plugins/crazy_functions that use the no-ui pipeline against a local model on a machine with insufficient memory; first request after configuring a local model that fails to download; handle poisoned by a previous failed run.","solutions":["Reproduce with the interactive predict() path first: its chatbot output shows the full '[Local Message]' traceback from the loader.","Fix the underlying load failure (model path, memory, dependencies) — see error 182.","Restart the app to rebuild the singleton handle after fixing config; a dead handle never recovers in-process.","If memory-bound, choose a smaller model or add swap/quantization."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"_llm_handle = GetSingletonHandle().get_llm_model_instance(LLMSingletonClass)\nif not getattr(_llm_handle, 'running', False):\n    # fail fast with the loader's last state instead of proceeding\n    raise ModelNotReady(_llm_handle.get_state())","typeGuard":null,"tryCatchPattern":"try:\n    result = predict_no_ui_long_connection(...)\nexcept RuntimeError as e:\n    if '加载模型失败' in str(e) or '失败' in str(e):\n        restart_model_subprocess_with_diagnostics()","preventionTips":["Warm-load local models at startup and gate the UI on running state.","Restart the app after any local-model config change.","Monitor the loader subprocess liveness in long-running deployments."],"tags":["local-llm","singleton","subprocess","state"],"backgroundTag":null,"analyzedSha":"d6bde0fa54373309bd05823a49bda8da019d2c77","analyzedAt":"2026-08-14T22:48:35.038Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}