oobabooga/textgen · error · InvalidRequestError

max_tokens must be greater than or equal to 0.

Error message

max_tokens must be greater than or equal to 0.

What it means

In the text completions path, max_tokens (mapped to max_new_tokens) must be >= 0; negative values raise InvalidRequestError (400, param='max_tokens'). Unlike the chat path, 0 is allowed here but only together with 'logprobs' (see error 14); None means auto (512 + auto_max_new_tokens).

Source

Thrown at modules/api/completions.py:833

                    elif isinstance(content, list):
                        for item in content:
                            if isinstance(item, dict) and item.get('type') == 'text':
                                prompt_text += item.get('text', '')

            # Allow empty prompts for image-only requests
            body[prompt_str] = prompt_text
        else:
            raise InvalidRequestError("Missing required input", param=prompt_str)

    # common params
    generate_params = process_parameters(body, is_legacy=is_legacy)
    max_tokens = generate_params['max_new_tokens']
    if max_tokens is None:
        generate_params['max_new_tokens'] = 512
        generate_params['auto_max_new_tokens'] = True
        max_tokens = 512
    elif max_tokens < 0:
        raise InvalidRequestError(message="max_tokens must be greater than or equal to 0.", param="max_tokens")
    elif max_tokens == 0 and body.get('logprobs') is None:
        raise InvalidRequestError(message="max_tokens is 0 but no logprobs parameter was specified.", param="max_tokens")

    generate_params['stream'] = stream
    if stop_event is not None:
        generate_params['stop_event'] = stop_event
    requested_model = generate_params.pop('model')
    logprob_proc = generate_params.pop('logprob_proc', None)
    if logprob_proc:
        logprob_proc.token_alternatives_history.clear()
    suffix = body['suffix'] if body['suffix'] else ''
    echo = body['echo']

    # Add messages to generate_params if present for multimodal processing
    if body.get('messages'):
        generate_params['messages'] = body['messages']
        raw_images = convert_openai_messages_to_images(generate_params['messages'])
        if raw_images:

View on GitHub (pinned to ed888c71f2)

Solutions

  1. Send a non-negative integer, or omit max_tokens for auto sizing.
  2. Clamp computed values: max(0, n) client-side.
  3. Use 0 only when also requesting logprobs.

Example fix

# before
{"model": "x", "prompt": p, "max_tokens": -1}

# after
{"model": "x", "prompt": p}  # omit for auto, or set e.g. "max_tokens": 256
Defensive patterns

Strategy: validation

Validate before calling

def safe_max_tokens(v):
    if v is None or v >= 0:
        return v
    return None  # negative -> omit for auto sizing

Type guard

def is_valid_completion_max_tokens(v) -> bool:
    return v is None or (isinstance(v, int) and not isinstance(v, bool) and v >= 0)

Try / catch

try:
    resp = client.completions.create(model=m, prompt=p, max_tokens=mt)
except openai.BadRequestError as e:
    if 'greater than or equal to 0' in str(e):
        resp = client.completions.create(model=m, prompt=p)  # omit: auto
    else:
        raise

Prevention

When it happens

Trigger: POST /v1/completions with {"max_tokens": -1} or any negative value, e.g. computed as remaining_budget where the budget is overshot.

Common situations: Using -1 as an 'unlimited' convention (this server wants None/omission for auto); arithmetic bugs in sliding-window token managers; porting configs from backends that accept negative max_tokens.

Related errors

AI-assisted analysis of oobabooga/textgen@ed888c71f2 (2026-08-15). Data as JSON: /api/errors/e7228a4b7772be89. Report an issue: GitHub.