{"record":{"id":"1c73c28feb743948","repo":"huggingface/transformers","slug":"input-ids-should-be-of-shape-batch-size-input-le","errorCode":null,"errorMessage":"Input ids should be of shape (batch_size, input_len), but is {input_ids.shape}","messagePattern":"Input ids should be of shape \\(batch_size, input_len\\), but is (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/logits_process.py","lineNumber":2896,"sourceCode":"        we pre-compute a random sampling table, and use apply modulo table size to\n        map from ngram keys (int64) to g values.\n\n        Args:\n            ngram_keys (`torch.LongTensor`):\n                Random keys (batch_size, num_ngrams, depth).\n\n        Returns:\n            G values (batch_size, num_ngrams, depth).\n        \"\"\"\n        (sampling_table_size,) = self.sampling_table.shape\n        sampling_table = self.sampling_table.reshape((1, 1, sampling_table_size))\n        ngram_keys = ngram_keys % sampling_table_size\n        return torch.take_along_dim(sampling_table, indices=ngram_keys, dim=2)\n\n    def _check_input_ids_shape(self, input_ids: torch.LongTensor):\n        \"\"\"Checks the shape of input ids.\"\"\"\n        if len(input_ids.shape) != 2:\n            raise ValueError(f\"Input ids should be of shape (batch_size, input_len), but is {input_ids.shape}\")\n\n    def compute_g_values(self, input_ids: torch.LongTensor) -> torch.LongTensor:\n        \"\"\"\n        Computes g values for each ngram from the given sequence of tokens.\n\n        Args:\n            input_ids (`torch.LongTensor`):\n                Input token ids (batch_size, input_len).\n\n        Returns:\n            G values (batch_size, input_len - (ngram_len - 1), depth).\n        \"\"\"\n        self._check_input_ids_shape(input_ids)\n        ngrams = input_ids.unfold(dimension=1, size=self.ngram_len, step=1)\n        ngram_keys = self.compute_ngram_keys(ngrams)\n        return self.sample_g_values(ngram_keys)\n\n    def compute_context_repetition_mask(self, input_ids: torch.LongTensor) -> torch.LongTensor:","sourceCodeStart":2878,"sourceCodeEnd":2914,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/logits_process.py#L2878-L2914","documentation":"Raised by the n-gram-based logits processor in transformers generation (the one exposing _check_input_ids_shape / compute_g_values) when the input_ids tensor passed to the processor is not 2-D. The processor builds n-gram keys per (batch, sequence) position, so it requires input_ids of shape (batch_size, input_len). Any extra or missing dimension (e.g. a single unbatched sequence of shape (input_len,) or a (batch, seq, 1) tensor) triggers it.","triggerScenarios":"Calling compute_g_values (or a generate() path that routes through this processor) with input_ids of rank != 2; passing unbatched token ids like torch.tensor([1,2,3]) instead of [[1,2,3]]; passing input_ids with a trailing dimension from a custom forward hook.","commonSituations":"Custom generation loops or watermarking/n-gram sampling experiments where the user slices or unsqueezes input_ids manually; multimodal pipelines where input_ids come shaped (batch, seq, extra).","solutions":["Reshape input_ids to 2-D before calling the API: input_ids = input_ids.reshape(-1, input_ids.shape[-1]) or input_ids[None, :] for a single sequence.","If you already have (batch, seq, X), squeeze the last dim only if it is size 1 and inspect where the extra dim was introduced.","Check upstream code that produced input_ids (tokenizer output is always 2-D; something after tokenization altered the shape)."],"exampleFix":"// before\nseq = tokenizer(text, return_tensors=\"pt\").input_ids\nprocessor.compute_g_values(seq[0])  # rank-1, raises\n\n// after\ng_values = processor.compute_g_values(seq)  # keep (batch, seq) 2-D","handlingStrategy":"validation","validationCode":"def as_2d_input_ids(input_ids: torch.Tensor) -> torch.Tensor:\n    if input_ids.dim() == 1:\n        return input_ids.unsqueeze(0)\n    if input_ids.dim() == 3 and input_ids.size(-1) == 1:\n        return input_ids.squeeze(-1)\n    assert input_ids.dim() == 2, f\"expected (batch, seq), got {tuple(input_ids.shape)}\"\n    return input_ids","typeGuard":"def is_valid_input_ids(t: torch.Tensor) -> bool:\n    return isinstance(t, torch.Tensor) and t.dim() == 2 and t.dtype in (torch.long, torch.int)","tryCatchPattern":null,"preventionTips":["Always feed tokenizer(...) output (already 2-D) directly into generation APIs.","Assert input_ids.dim() == 2 in custom generation loops before calling logits processors.","Log tensor shapes at the boundary where input_ids enter your pipeline."],"tags":["pytorch","shape-validation","generation","logits-processor"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}