{"record":{"id":"7cc516ea5bb2ef28","repo":"BerriAI/litellm","slug":"api-base-is-required-for-triton-please-pass-api","errorCode":null,"errorMessage":"api_base is required for triton. Please pass `api_base`","messagePattern":"api_base is required for triton\\. Please pass `api_base`","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"litellm/main.py","lineNumber":6488,"sourceCode":"            response = bedrock_embedding.embeddings(\n                model=model,\n                input=transformed_input,\n                encoding=_get_encoding(),\n                logging_obj=logging,\n                optional_params=optional_params,\n                model_response=EmbeddingResponse(),\n                client=client,\n                timeout=timeout,\n                aembedding=aembedding,\n                litellm_params={},\n                api_base=api_base,\n                print_verbose=print_verbose,\n                extra_headers=headers,\n                api_key=api_key,\n            )\n        elif custom_llm_provider == \"triton\":\n            if api_base is None:\n                raise ValueError(\"api_base is required for triton. Please pass `api_base`\")\n            response = base_llm_http_handler.embedding(\n                model=model,\n                input=input,\n                custom_llm_provider=custom_llm_provider,\n                api_base=api_base,\n                api_key=api_key,\n                logging_obj=logging,\n                timeout=timeout,\n                model_response=EmbeddingResponse(),\n                optional_params=optional_params,\n                client=client,\n                aembedding=aembedding,\n                litellm_params={},\n            )\n        elif custom_llm_provider == \"gemini\":\n            gemini_api_key: Final = api_key or get_api_key_from_env() or litellm.api_key\n\n            api_base = api_base or litellm.api_base or get_secret_str(\"GEMINI_API_BASE\")","sourceCodeStart":6470,"sourceCodeEnd":6506,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/main.py#L6470-L6506","documentation":"Triton embeddings are served by a self-hosted Triton Inference Server, so unlike cloud providers there is no default URL to fall back to. litellm hard-requires the server address via api_base and raises immediately when it is None.","triggerScenarios":"litellm.embedding(model='triton/<served-model>', input=[...]) without an api_base kwarg and without litellm.api_base set.","commonSituations":"Copying Triton examples that omit the endpoint; the Triton pod's DNS name or port changed after redeploy; assuming an env var like TRITON_API_BASE is read automatically when only the kwarg is checked.","solutions":["Pass the Triton HTTP endpoint explicitly: litellm.embedding(model='triton/E5', input=[...], api_base='http://triton-host:8000')","Or set litellm.api_base for the whole process if all calls hit the same server","Verify the server is up: curl http://triton-host:8000/v2/health/ready"],"exampleFix":"# before\nresp = litellm.embedding(model=\"triton/E5\", input=[\"hi\"])\n\n# after\nresp = litellm.embedding(model=\"triton/E5\", input=[\"hi\"], api_base=\"http://triton-host:8000\")","handlingStrategy":"validation","validationCode":"TRITON_URL = \"http://triton-host:8000\"\nif not TRITON_URL:\n    raise SystemExit(\"triton api_base missing\")\nresp = litellm.embedding(model=\"triton/E5\", input=[\"hi\"], api_base=TRITON_URL)","typeGuard":null,"tryCatchPattern":"try:\n    resp = litellm.embedding(model=\"triton/E5\", input=[\"hi\"], api_base=TRITON_URL)\nexcept ValueError as e:\n    if \"api_base is required for triton\" in str(e):\n        raise RuntimeError(\"Triton endpoint not configured\") from e","preventionTips":["Keep the Triton endpoint in settings/env and pass it explicitly on every call","Health-check the Triton server (/v2/health/ready) as part of app startup"],"tags":["triton","embedding","api-base","self-hosted","litellm"],"backgroundTag":"missing-config-parameter","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","schemaVersion":2},"datasetVersion":"2026-08-21T18:17:14.833Z"}