oobabooga/textgen · error · InvalidRequestError

max_tokens is 0 but no logprobs parameter was specified.

Error message

max_tokens is 0 but no logprobs parameter was specified.

What it means

In the text completions path, max_tokens=0 is a special mode for returning only prompt logprobs without generating anything. Setting 0 without also setting the 'logprobs' parameter is rejected with InvalidRequestError (400, param='max_tokens'), since a 0-token completion with no logprobs would return nothing useful.

Source

Thrown at modules/api/completions.py:835

                            if isinstance(item, dict) and item.get('type') == 'text':
                                prompt_text += item.get('text', '')

            # Allow empty prompts for image-only requests
            body[prompt_str] = prompt_text
        else:
            raise InvalidRequestError("Missing required input", param=prompt_str)

    # common params
    generate_params = process_parameters(body, is_legacy=is_legacy)
    max_tokens = generate_params['max_new_tokens']
    if max_tokens is None:
        generate_params['max_new_tokens'] = 512
        generate_params['auto_max_new_tokens'] = True
        max_tokens = 512
    elif max_tokens < 0:
        raise InvalidRequestError(message="max_tokens must be greater than or equal to 0.", param="max_tokens")
    elif max_tokens == 0 and body.get('logprobs') is None:
        raise InvalidRequestError(message="max_tokens is 0 but no logprobs parameter was specified.", param="max_tokens")

    generate_params['stream'] = stream
    if stop_event is not None:
        generate_params['stop_event'] = stop_event
    requested_model = generate_params.pop('model')
    logprob_proc = generate_params.pop('logprob_proc', None)
    if logprob_proc:
        logprob_proc.token_alternatives_history.clear()
    suffix = body['suffix'] if body['suffix'] else ''
    echo = body['echo']

    # Add messages to generate_params if present for multimodal processing
    if body.get('messages'):
        generate_params['messages'] = body['messages']
        raw_images = convert_openai_messages_to_images(generate_params['messages'])
        if raw_images:
            logger.info(f"Found {len(raw_images)} image(s) in request.")
            generate_params['raw_images'] = raw_images

View on GitHub (pinned to ed888c71f2)

Solutions

1. If you truly want zero generation, add a logprobs parameter: {"max_tokens": 0, "logprobs": 5}.
2. Otherwise clamp to at least 1, or omit max_tokens to let the server auto-size.
3. For pure token counting, use the /v1/token-count or token encoding endpoints instead of a 0-token completion.

Example fix

# before
{"model": "x", "prompt": p, "max_tokens": 0}

# after
{"model": "x", "prompt": p, "max_tokens": 0, "logprobs": 5}

def normalize_completion_params(body: dict) -> dict:
    mt = body.get('max_tokens')
    if mt == 0 and body.get('logprobs') is None:
        body.pop('max_tokens', None)  # or set logprobs
    return body

def zero_tokens_ok(body: dict) -> bool:
    return body.get('max_tokens') != 0 or body.get('logprobs') is not None

try:
    resp = client.completions.create(model=m, prompt=p, max_tokens=0)
except openai.BadRequestError as e:
    if 'no logprobs parameter' in str(e):
        resp = client.completions.create(model=m, prompt=p, max_tokens=0, logprobs=5)
    else:
        raise

When it happens

Trigger: POST /v1/completions with {"max_tokens": 0} and no 'logprobs' key, or with {"max_tokens": 0, "logprobs": null}.

Common situations: Token-budget managers that legitimately compute 0 remaining tokens; test payloads with 0; users trying to 'just count tokens' without knowing the logprobs convention.

Related errors

• max_tokens must be greater than or equal to 0.
• max_tokens must be greater than 0.
• Missing required input
• API Batched generation not yet supported.
• functions is not supported.

AI-assisted analysis of oobabooga/textgen@ed888c71f2 (2026-08-15). Data as JSON: /api/errors/3634ecdb32a0300b. Report an issue: GitHub.