{"record":{"id":"984c0e274b4b510f","repo":"oobabooga/textgen","slug":"input-id-s-length-is-inconsistent-with-the-current","errorCode":null,"errorMessage":"Input ID's length is inconsistent with the current state of the GrammarConstrainedLogitsProcessor. If you want to process another input sequence, please instantiate a new GrammarConstrainedLogitsProcessor.","messagePattern":"Input ID's length is inconsistent with the current state of the GrammarConstrainedLogitsProcessor\\. If you want to process another input sequence, please instantiate a new GrammarConstrainedLogitsProcessor\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"modules/grammar/logits_process.py","lineNumber":90,"sourceCode":"        elif len(input_ids[0]) == self.last_size + 1:\n            # self.stacks = self.grammar_acceptor.accept_token_id(input_ids[0][-1], self.stacks)\n            self.batch_stacks = [\n                self.grammar_constraint.accept_token_id(single_input_ids[-1], stack)\n                for single_input_ids, stack in zip(input_ids, self.batch_stacks)\n            ]\n        #  ensure that the input size is consistent with the expected incremental processing\n        #  (i.e., one token at a time).\n        else:\n            # here we check if the input_ids are one token longer than the last time we processed\n            # but we don't check if input_ids are actually valid.\n            # Imagine a scenario where we generate 10 tokens, then we replace the 10 generated tokens with 10 new tokens.\n            # In this case, the input_ids will be consistent with the last_size, but the input_ids are not valid.\n            # However, should we really check if the input_ids are valid here?\n            # If we do, then we need to reparse the whole input_ids at each call, which is not efficient.\n            # Maybe we should just trust the user to provide valid input_ids?\n            # The conclusion is that, we assume the input_ids are valid, and our generation will be correct.\n            # If the input_ids are not valid, then the generation result will be wrong and we don't take responsibility for that.\n            raise RuntimeError(\n                \"Input ID's length is inconsistent with the current state of \"\n                \"the GrammarConstrainedLogitsProcessor. If you want to process \"\n                \"another input sequence, please instantiate a new \"\n                \"GrammarConstrainedLogitsProcessor.\"\n            )\n\n        self.filter_logits(scores, scores.device)\n\n        self.last_size = len(input_ids[0])\n        return scores\n\n    @add_start_docstrings(LOGITS_PROCESSOR_INPUTS_DOCSTRING)\n    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor) -> torch.FloatTensor:\n        return self.process_logits(input_ids, scores)\n","sourceCodeStart":72,"sourceCodeEnd":105,"githubUrl":"https://github.com/oobabooga/textgen/blob/ed888c71f221df552750e1834b3654abab8ae345/modules/grammar/logits_process.py#L72-L105","documentation":"Error \"Input ID's length is inconsistent with the current state of the GrammarConstrainedLogitsProcessor. If you want to process another input sequence, please instantiate a new GrammarConstrainedLogitsProcessor.\" thrown in oobabooga/textgen.","triggerScenarios":"Raised when a single GrammarConstrainedLogitsProcessor instance is reused for a second input sequence. The processor keeps per-sequence state (stacks, offsets), so passing an input whose length is inconsistent with the current state triggers this error. Instantiate a new processor per input sequence.","commonSituations":"See trigger scenarios.","solutions":["Instantiate a new GrammarConstrainedLogitsProcessor for each new input sequence instead of reusing one.","Pass input IDs whose length matches the processor's current state."],"exampleFix":null,"handlingStrategy":null,"validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"ed888c71f221df552750e1834b3654abab8ae345","analyzedAt":"2026-08-15T05:24:21.000Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}