BerriAI/litellm · error · ContextWindowExceededError

NLPCloudException - {error_str}

Error message

NLPCloudException - {error_str}

What it means

This is a litellm ContextWindowExceededError raised by the NLP Cloud exception mapper. When the NLP Cloud error body contains 'detail' and the substring 'Input text length should not exceed', litellm maps it to this class, meaning the prompt (plus expected output) exceeds the model's token limit on nlp_cloud. It enables uniform context-window handling across providers.

Source

Thrown at litellm/litellm_core_utils/exception_mapping_utils.py:1568

                llm_provider="ai21",
                model=model,
                request=getattr(original_exception, "request", None),
            )


def _map_nlp_cloud_exception(
    *,
    model: str,
    original_exception: _ProviderHTTPException,
    custom_llm_provider: str,
    error_str: str,
    exception_type: str,
    exception_provider: str,
    extra_information: str,
) -> None:
    if "detail" in error_str:
        if "Input text length should not exceed" in error_str:
            raise ContextWindowExceededError(
                message=f"NLPCloudException - {error_str}",
                model=model,
                llm_provider="nlp_cloud",
                response=getattr(original_exception, "response", None),
            )
        elif "value is not a valid" in error_str:
            raise BadRequestError(
                message=f"NLPCloudException - {error_str}",
                model=model,
                llm_provider="nlp_cloud",
                response=getattr(original_exception, "response", None),
            )
        else:
            raise APIError(
                status_code=500,
                message=f"NLPCloudException - {error_str}",
                model=model,
                llm_provider="nlp_cloud",

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Truncate or chunk the prompt so input tokens fit the model's limit
2. Switch to a larger-context model on NLP Cloud if available
3. Use litellm's get_max_tokens(model) to check the limit before sending
4. Add summarization/compaction of prior conversation turns

Example fix

# before
litellm.completion(model="finetuned-gpt-neox-20b", messages=full_history)

# after
max_tok = litellm.get_max_tokens("finetuned-gpt-neox-20b")
litellm.completion(
    model="finetuned-gpt-neox-20b",
    messages=trim_to_budget(full_history, max_tok - 256),
)

model = "finetuned-gpt-neox-20b"
budget = litellm.get_max_tokens(model) or 4096
prompt_tokens = len(litellm.encode(model=model, text="".join(m["content"] for m in messages)))
if prompt_tokens >= budget - 256:
    messages = truncate_messages(messages, budget - 256)

def is_context_window_error(e: Exception) -> bool:
    return isinstance(e, litellm.ContextWindowExceededError)

try:
    resp = litellm.completion(model=model, messages=messages)
except litellm.ContextWindowExceededError:
    messages = truncate_messages(messages, half=True)
    resp = litellm.completion(model=model, messages=messages)

When it happens

Trigger: Calling completion(model="finetuned-gpt-neox-20b", ...) or other NLP Cloud models with a prompt longer than the model's max input tokens; stuffing whole documents into messages.

Common situations: RAG pipelines that overload context instead of chunking; long chat histories never truncated; switching from a large-context model to a smaller NLP Cloud model without re-tuning prompt size.

Related errors

• TogetherAIException - {error_response['error']}
• {custom_llm_provider.capitalize()}Exception: Context Window
• BedrockException: Context Window Error - {error_str}
• {custom_llm_provider.capitalize()}Exception - {error_str}
• ContextWindowExceededError: {custom_llm_provider.capitalize(

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/712092313e317ed9. Report an issue: GitHub.