{"record":{"id":"81022472a36e2f88","repo":"huggingface/transformers","slug":"inputs-must-be-a-1d-or-2d-tensor-got-inputs-dim","errorCode":null,"errorMessage":"inputs must be a 1D or 2D tensor, got {inputs.dim() = }","messagePattern":"inputs must be a 1D or 2D tensor, got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/utils.py","lineNumber":2408,"sourceCode":"\n        # 0.b. If requested, switched to continuous batching generation\n        if kwargs.get(\"cache_implementation\") == \"paged\":\n            logger.warning(\n                \"Detected cache_implementation=paged: switching to continuous batching. You should consider using \"\n                \"generate_batch directly instead.\"\n            )\n\n            # generate_batch expects a list of lists of ints, so we create it from the inputs or input_ids\n            inputs = inputs if inputs is not None else kwargs.get(\"input_ids\")\n            if inputs is None:\n                raise ValueError(\"inputs or input_ids must be provided for CB generation.\")\n\n            if inputs.dim() == 1:\n                inputs = inputs.unsqueeze(0).tolist()\n            elif inputs.dim() == 2:\n                inputs = inputs.tolist()\n            else:\n                raise ValueError(f\"inputs must be a 1D or 2D tensor, got {inputs.dim() = }\")\n\n            # some arguments are not supported for continuous batching\n            if stopping_criteria is not None:\n                raise NotImplementedError(\n                    f\"stopping_criteria is not supported for continuous batching. Got {stopping_criteria = }\"\n                )\n            if prefix_allowed_tokens_fn is not None:\n                raise NotImplementedError(\n                    f\"prefix_allowed_tokens_fn is not supported for continuous batching. Got {prefix_allowed_tokens_fn = }\"\n                )\n            if assistant_model is not None:\n                raise NotImplementedError(\n                    f\"assistant_model is not supported for continuous batching. Got {assistant_model = }\"\n                )\n            if streamer is not None:  # TODO: actually this could be supported\n                raise NotImplementedError(f\"streaming is not supported for continuous batching. Got {streamer = }\")\n            if negative_prompt_ids is not None:\n                raise NotImplementedError(","sourceCodeStart":2390,"sourceCodeEnd":2426,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/utils.py#L2390-L2426","documentation":"When switching to continuous batching, generate converts the input tensor into a Python list of per-request token lists and only understands 1D (single unpadded sequence) or 2D (batch of sequences) tensors. A tensor with any other rank cannot be mapped to requests and raises, echoing the offending dimension count.","triggerScenarios":"`model.generate(inputs=input_ids, cache_implementation=\"paged\")` where `input_ids.dim()` is 3+ — e.g. tensors shaped [batch, num_beams, seq] from beam prep, extra leading axes from an encoder pass, or mistakenly passed encoder hidden states instead of token ids.","commonSituations":"Reusing tensors produced by earlier beam-search or shaping code (extra beam/axis dimension); passing `decoder_input_ids` reshaped for multi-beam; passing embeddings or hidden states where token ids are expected; pre-processing pipelines that add a spurious axis.","solutions":["Squeeze spurious axes: `input_ids = input_ids.squeeze(1)` (verify shape is [batch, seq] or [seq]) before the call.","If the extra dim is beams, drop beam shaping — continuous batching manages its own scheduling.","Pass what the backend expects: token ids only, not embeddings/hidden states.","Log `inputs.shape` right before generate to catch rank drift early."],"exampleFix":"# before\ninput_ids = input_ids.unsqueeze(0)  # -> shape [1, 1, seq], dim()==3\nout = model.generate(inputs=input_ids, cache_implementation=\"paged\")  # ValueError: got dim = 3\n\n# after\ninput_ids = input_ids.squeeze(0)  # ensure [batch, seq] or [seq]\nout = model.generate(inputs=input_ids, cache_implementation=\"paged\")","handlingStrategy":"type-guard","validationCode":"if inputs.dim() > 2:\n    raise ValueError(f\"continuous batching needs 1D/2D input_ids, got shape {tuple(inputs.shape)}\")","typeGuard":"def is_cb_ready(t) -> bool:\n    return t.dim() in (1, 2)","tryCatchPattern":null,"preventionTips":["Assert inputs.dim() in (1, 2) before paged-cache calls.","Squeeze beam/extra axes left over from earlier decoding stages.","Pass token ids, not embeddings or hidden states, to continuous batching."],"tags":["generation","continuous-batching","paged-cache","tensor-shape"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}