{"record":{"id":"983ae337e8c7d8b9","repo":"huggingface/transformers","slug":"prefix-allowed-tokens-fn-is-not-supported-for-cont","errorCode":null,"errorMessage":"prefix_allowed_tokens_fn is not supported for continuous batching. Got {prefix_allowed_tokens_fn = }","messagePattern":"prefix_allowed_tokens_fn is not supported for continuous batching\\. Got (.+?)","errorType":"validation","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/utils.py","lineNumber":2416,"sourceCode":"            # generate_batch expects a list of lists of ints, so we create it from the inputs or input_ids\n            inputs = inputs if inputs is not None else kwargs.get(\"input_ids\")\n            if inputs is None:\n                raise ValueError(\"inputs or input_ids must be provided for CB generation.\")\n\n            if inputs.dim() == 1:\n                inputs = inputs.unsqueeze(0).tolist()\n            elif inputs.dim() == 2:\n                inputs = inputs.tolist()\n            else:\n                raise ValueError(f\"inputs must be a 1D or 2D tensor, got {inputs.dim() = }\")\n\n            # some arguments are not supported for continuous batching\n            if stopping_criteria is not None:\n                raise NotImplementedError(\n                    f\"stopping_criteria is not supported for continuous batching. Got {stopping_criteria = }\"\n                )\n            if prefix_allowed_tokens_fn is not None:\n                raise NotImplementedError(\n                    f\"prefix_allowed_tokens_fn is not supported for continuous batching. Got {prefix_allowed_tokens_fn = }\"\n                )\n            if assistant_model is not None:\n                raise NotImplementedError(\n                    f\"assistant_model is not supported for continuous batching. Got {assistant_model = }\"\n                )\n            if streamer is not None:  # TODO: actually this could be supported\n                raise NotImplementedError(f\"streaming is not supported for continuous batching. Got {streamer = }\")\n            if negative_prompt_ids is not None:\n                raise NotImplementedError(\n                    f\"negative_prompt_ids is not supported for continuous batching. Got {negative_prompt_ids = }\"\n                )\n            if negative_prompt_attention_mask is not None:\n                raise NotImplementedError(\n                    f\"negative_prompt_attention_mask is not supported for continuous batching. Got {negative_prompt_attention_mask = }\"\n                )\n\n            # others are ignored","sourceCodeStart":2398,"sourceCodeEnd":2434,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/utils.py#L2398-L2434","documentation":"Error \"prefix_allowed_tokens_fn is not supported for continuous batching. Got {prefix_allowed_tokens_fn = }\" thrown in huggingface/transformers.","triggerScenarios":"Raised in generate() when prefix_allowed_tokens_fn is passed while using continuous batching mode.","commonSituations":"Structured/constrained decoding arguments combined with the continuous batching generate path, which does not support them.","solutions":["Remove `prefix_allowed_tokens_fn` when generating with continuous batching (do_cb=True).","Run sequential generation instead if you need prefix constraints."],"exampleFix":null,"handlingStrategy":null,"validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}