{"record":{"id":"39cbfc06eb3deede","repo":"BerriAI/litellm","slug":"importing-torch-transformers-petals-failed-try-p","errorCode":null,"errorMessage":"Importing torch, transformers, petals failed\nTry pip installing petals \npip install git+https://github.com/bigscience-workshop/petals","messagePattern":"Importing torch, transformers, petals failed\nTry pip installing petals \npip install git\\+https://github\\.com/bigscience-workshop/petals","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"litellm/llms/petals/completion/handler.py","lineNumber":95,"sourceCode":"            additional_args={\"complete_input_dict\": optional_params},\n        )\n\n        ## RESPONSE OBJECT\n        try:\n            output_text = response.json()[\"outputs\"]\n        except Exception as e:\n            PetalsError(\n                status_code=response.status_code,\n                message=str(e),\n                headers=response.headers,\n            )\n\n    else:\n        try:\n            from petals import AutoDistributedModelForCausalLM\n            from transformers import AutoTokenizer\n        except Exception:\n            raise Exception(\n                \"Importing torch, transformers, petals failed\\nTry pip installing petals \\npip install git+https://github.com/bigscience-workshop/petals\"\n            )\n\n        model = model\n\n        tokenizer: Final = AutoTokenizer.from_pretrained(model, use_fast=False, add_bos_token=False)\n        model_obj: Final = AutoDistributedModelForCausalLM.from_pretrained(model)\n\n        ## LOGGING\n        logging_obj.pre_call(\n            input=prompt,\n            api_key=\"\",\n            additional_args={\"complete_input_dict\": optional_params},\n        )\n\n        ## COMPLETION CALL\n        inputs: Final = tokenizer(prompt, return_tensors=\"pt\")[\"input_ids\"]\n","sourceCodeStart":77,"sourceCodeEnd":113,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/llms/petals/completion/handler.py#L77-L113","documentation":"When no api_base is configured, LiteLLM's Petals backend runs inference locally by importing petals (AutoDistributedModelForCausalLM) and transformers (AutoTokenizer), which in turn require torch. If any of those imports fail, it raises this generic Exception with pip install instructions pointing at the bigscience-workshop petals GitHub repo. This is a client-side dependency failure, not an API error.","triggerScenarios":"Calling litellm.completion with a petals/* model while petals, transformers, or torch is not installed (or fails to import) in the active Python interpreter, and no api_base was provided to route to a remote Petals server.","commonSituations":"Fresh virtualenv missing the heavy ML stack; Python version incompatible with available torch wheels; petals being unmaintained so plain 'pip install petals' no longer resolves - hence the git+https instruction; developers unaware petals models run locally via swarm inference.","solutions":["pip install git+https://github.com/bigscience-workshop/petals plus transformers and torch","Verify in the same interpreter: python -c 'import petals, transformers, torch'","If the stack will not install (petals is largely unmaintained), point api_base at a hosted Petals server or switch to an API-backed model","Ensure the host has enough RAM (and optionally GPU) for local swarm inference"],"exampleFix":"# before\nresp = litellm.completion(model=\"petals/petals-team/StableBeluga2\", messages=[...])  # ImportError\n\n# after\n# shell:\n#   pip install git+https://github.com/bigscience-workshop/petals transformers torch\nresp = litellm.completion(model=\"petals/petals-team/StableBeluga2\", messages=[...])","handlingStrategy":"validation","validationCode":"import importlib.util\n\ndef petals_stack_available() -> bool:\n    return all(importlib.util.find_spec(m) is not None for m in (\"petals\", \"transformers\", \"torch\"))\n\n# fail fast before routing traffic to petals models\nif not petals_stack_available():\n    raise RuntimeError(\"Install petals/transformers/torch or route petals traffic via api_base\")","typeGuard":null,"tryCatchPattern":"Catch Exception around litellm.completion for petals/* models and inspect the message for the 'Importing torch, transformers, petals failed' text; convert it into a clear dependency-missing error rather than letting the generic message leak to users.","preventionTips":["If you route to petals models, pin petals/transformers/torch in requirements","Health-check optional-backend imports at startup","Prefer api_base-backed Petals servers or API-hosted models unless you specifically need local swarm inference"],"tags":["petals","import-error","dependency","local-inference","litellm"],"backgroundTag":"missing-dependency","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","schemaVersion":2},"datasetVersion":"2026-08-30T08:17:16.595Z"}