{"record":{"id":"6428d334caa0295c","repo":"oobabooga/textgen","slug":"max-tokens-must-be-greater-than-0","errorCode":null,"errorMessage":"max_tokens must be greater than 0.","messagePattern":"max_tokens must be greater than 0\\.","errorType":"exception","errorClass":"InvalidRequestError","httpStatus":400,"severity":"error","filePath":"modules/api/completions.py","lineNumber":581,"sourceCode":"    generate_params.update({\n        'mode': body['mode'],\n        'name1': name1,\n        'name2': name2,\n        'context': context,\n        'greeting': greeting,\n        'user_bio': user_bio,\n        'instruction_template_str': instruction_template_str,\n        'custom_system_message': custom_system_message,\n        'chat_template_str': chat_template_str,\n        'chat-instruct_command': chat_instruct_command,\n        'tools': tools,\n        'history': history,\n        'stream': stream\n    })\n\n    max_tokens = generate_params['max_new_tokens']\n    if max_tokens is not None and max_tokens <= 0:\n        raise InvalidRequestError(message=\"max_tokens must be greater than 0.\", param=\"max_tokens\")\n\n    if max_tokens is None:\n        generate_params['max_new_tokens'] = 512\n        generate_params['auto_max_new_tokens'] = True\n\n    requested_model = generate_params.pop('model')\n    logprob_proc = generate_params.pop('logprob_proc', None)\n    if logprob_proc:\n        logprob_proc.token_alternatives_history.clear()\n    chat_logprobs_offset = [0]  # mutable for closure access in streaming\n\n    def chat_streaming_chunk(content=None, chunk_tool_calls=None, include_role=False, reasoning_content=None):\n        # begin streaming\n        delta = {}\n        if include_role:\n            delta['role'] = 'assistant'\n            delta['refusal'] = None\n        if content is not None:","sourceCodeStart":563,"sourceCodeEnd":599,"githubUrl":"https://github.com/oobabooga/textgen/blob/ed888c71f221df552750e1834b3654abab8ae345/modules/api/completions.py#L563-L599","documentation":"In the chat completions path, after process_parameters maps 'max_tokens' to 'max_new_tokens', a value that is set and <= 0 raises InvalidRequestError (400, param='max_tokens'). Note this is stricter than the text-completions path (errors 13/14): the chat endpoint does not allow 0 even for logprob-only requests; None means auto (512 + auto_max_new_tokens).","triggerScenarios":"POST /v1/chat/completions with {\"max_tokens\": 0} or a negative value. Compute-bound code doing max_tokens = budget - used and sending it when the budget is exhausted (0 or negative) is a classic trigger.","commonSituations":"Token-budget managers passing 0 when the context is full; sending -1 as an 'unlimited' convention from other inference servers (this backend treats None, not -1, as auto); test payloads with 0.","solutions":["Send a positive integer: {\"max_tokens\": 512}.","To request auto sizing, omit 'max_tokens' entirely (server then defaults to 512 + auto_max_new_tokens).","Clamp computed budgets client-side: max(1, budget - used) or drop the field when <= 0.","If you wanted a 0-token logprob-only call, use the /v1/completions endpoint with logprobs instead."],"exampleFix":"# before\nbody = {\"model\": m, \"messages\": msgs, \"max_tokens\": max(0, budget - used)}\n\n# after\nbody = {\"model\": m, \"messages\": msgs}\nif budget - used > 0:\n    body[\"max_tokens\"] = budget - used","handlingStrategy":"validation","validationCode":"def resolve_max_tokens(remaining: int | None):\n    if remaining is None or remaining <= 0:\n        return None  # omit -> server auto sizing (512 + auto_max_new_tokens)\n    return remaining\n\nbody = {'model': m, 'messages': msgs}\nmt = resolve_max_tokens(budget - used)\nif mt is not None:\n    body['max_tokens'] = mt","typeGuard":"def is_valid_chat_max_tokens(v) -> bool:\n    return v is None or (isinstance(v, int) and not isinstance(v, bool) and v > 0)","tryCatchPattern":"try:\n    resp = client.chat.completions.create(model=m, messages=msgs, max_tokens=mt)\nexcept openai.BadRequestError as e:\n    if 'max_tokens must be greater than 0' in str(e):\n        resp = client.chat.completions.create(model=m, messages=msgs)  # omit: auto\n    else:\n        raise","preventionTips":["Clamp computed budgets with max(1, n) or drop the field when <= 0.","Remember chat endpoint rejects 0 even with logprobs; only /v1/completions allows the 0+logprobs mode.","Use None/omission for auto sizing, never -1."],"tags":["openai-api","chat-completions","max-tokens","validation"],"backgroundTag":null,"analyzedSha":"ed888c71f221df552750e1834b3654abab8ae345","analyzedAt":"2026-08-15T05:24:21.000Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}