{"record":{"id":"8e2c2d32565c536b","repo":"BerriAI/litellm","slug":"error-str","errorCode":null,"errorMessage":"{error_str}","messagePattern":"\\{error_str\\}","errorType":"exception","errorClass":"ContextWindowExceededError","httpStatus":400,"severity":"error","filePath":"litellm/litellm_core_utils/exception_mapping_utils.py","lineNumber":1428,"sourceCode":"                llm_provider=\"cohere\",\n                model=model,\n                request=getattr(original_exception, \"request\", None),\n            )\n        raise cast(Exception, original_exception)\n\n\ndef _map_huggingface_exception(\n    *,\n    model: str,\n    original_exception: _ProviderHTTPException,\n    custom_llm_provider: str,\n    error_str: str,\n    exception_type: str,\n    exception_provider: str,\n    extra_information: str,\n) -> None:\n    if \"length limit exceeded\" in error_str:\n        raise ContextWindowExceededError(\n            message=error_str,\n            model=model,\n            llm_provider=\"huggingface\",\n            response=getattr(original_exception, \"response\", None),\n        )\n    elif \"A valid user token is required\" in error_str:\n        raise BadRequestError(\n            message=error_str,\n            llm_provider=\"huggingface\",\n            model=model,\n            response=getattr(original_exception, \"response\", None),\n        )\n    elif \"Rate limit reached\" in error_str:\n        raise RateLimitError(\n            message=error_str,\n            llm_provider=\"huggingface\",\n            model=model,\n            response=getattr(original_exception, \"response\", None),","sourceCodeStart":1410,"sourceCodeEnd":1446,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/litellm_core_utils/exception_mapping_utils.py#L1410-L1446","documentation":"_map_huggingface_exception raises litellm.ContextWindowExceededError with the raw error string when a HuggingFace Inference Endpoints error contains 'length limit exceeded'. The model behind the endpoint rejected the prompt because it exceeds the model's max sequence length (often far smaller than commercial APIs — 4k/8k/32k depending on the deployed model). litellm maps it so context-truncation and fallback strategies can key off the exception type.","triggerScenarios":"Sending prompts longer than the deployed model's max_seq_len to model='huggingface/<model>' via inference endpoints/embeddings; RAG contexts overflowing small models (e.g. bert-based embedders with 512-token limits); chat histories accumulated without trimming.","commonSituations":"Default embedding models with 512-token windows receiving whole paragraphs per chunk, text-generation endpoints running 8k models fed 20k-token transcripts, developers assuming commercial-API context sizes (128k) apply to HuggingFace deployments.","solutions":["Chunk inputs to the model's actual limit (check the model card for max sequence length, e.g. 512 for many embedders).","Trim chat history / summarize older turns before sending.","Switch to a long-context model on the endpoint (e.g. a 32k model) or a commercial provider via litellm fallback.","Pre-count tokens client-side with the model's tokenizer to fail before spending a request."],"exampleFix":"# before\nresp = litellm.embedding(model=\"huggingface/BAAI/bge-large-en-v1.5\", input=[long_document_text])  # ContextWindowExceededError: length limit exceeded\n\n# after\nchunks = [long_document_text[i:i+1500] for i in range(0, len(long_document_text), 1500)]  # ~<512 tokens each\nresp = litellm.embedding(model=\"huggingface/BAAI/bge-large-en-v1.5\", input=chunks)","handlingStrategy":"validation","validationCode":"HF_SEQ_LIMITS = {\"BAAI/bge-large-en-v1.5\": 512, \"sentence-transformers/all-MiniLM-L6-v2\": 512}  # per model cards\n\ndef chunk_for_hf(text: str, model: str, chars_per_token: float = 4.0) -> list[str]:\n    limit = HF_SEQ_LIMITS.get(model.split(\"/\", 1)[-1], 512)\n    max_chars = int(limit * chars_per_token * 0.9)\n    return [text[i:i + max_chars] for i in range(0, len(text), max_chars)] or [\"\"]","typeGuard":"import litellm\n\ndef is_context_window_exceeded(e: BaseException) -> bool:\n    return isinstance(e, litellm.ContextWindowExceededError)","tryCatchPattern":"try:\n    resp = litellm.embedding(model=\"huggingface/BAAI/bge-large-en-v1.5\", input=[doc])\nexcept litellm.ContextWindowExceededError as e:\n    logger.warning(\"doc over HF seq limit, chunking\")\n    resp = litellm.embedding(model=\"huggingface/BAAI/bge-large-en-v1.5\", input=chunk_for_hf(doc, \"BAAI/bge-large-en-v1.5\"))","preventionTips":["Look up each deployed model's max sequence length on its model card and chunk to ~90% of it.","Don't assume commercial-API context sizes (128k) for HuggingFace deployments.","Pre-split long documents with a token-aware splitter (e.g. langchain text splitter with the model tokenizer).","Catch ContextWindowExceededError specifically and fall back to chunking instead of failing the job."],"tags":["litellm","huggingface","context-window","length-limit","embeddings"],"backgroundTag":"context-window-exceeded","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}