{"record":{"id":"3ba5409b2e2f0b31","repo":"antlr/antlr4","slug":"replace-range-invalid-size","errorCode":null,"errorMessage":"replace: range invalid: {}..{}(size={})","messagePattern":"replace: range invalid: (.+?)\\.\\.(.+?)\\(size=(.+?)\\)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"runtime/Python3/src/antlr4/TokenStreamRewriter.py","lineNumber":77,"sourceCode":"        rewrites = self.getProgram(program_name)\n        op.instructionIndex = len(rewrites)\n        rewrites.append(op)\n\n    def replaceIndex(self, index, text):\n        self.replace(self.DEFAULT_PROGRAM_NAME, index, index, text)\n\n    def replaceRange(self, from_idx, to_idx, text):\n        self.replace(self.DEFAULT_PROGRAM_NAME, from_idx, to_idx, text)\n\n    def replaceSingleToken(self, token, text):\n        self.replace(self.DEFAULT_PROGRAM_NAME, token.tokenIndex, token.tokenIndex, text)\n\n    def replaceRangeTokens(self, from_token, to_token, text, program_name=DEFAULT_PROGRAM_NAME):\n        self.replace(program_name, from_token.tokenIndex, to_token.tokenIndex, text)\n\n    def replace(self, program_name, from_idx, to_idx, text):\n        if any((from_idx > to_idx, from_idx < 0, to_idx < 0, to_idx >= len(self.tokens.tokens))):\n            raise ValueError(\n                'replace: range invalid: {}..{}(size={})'.format(from_idx, to_idx, len(self.tokens.tokens)))\n        op = self.ReplaceOp(from_idx, to_idx, self.tokens, text)\n        rewrites = self.getProgram(program_name)\n        op.instructionIndex = len(rewrites)\n        rewrites.append(op)\n\n    def deleteToken(self, token):\n        self.delete(self.DEFAULT_PROGRAM_NAME, token, token)\n\n    def deleteIndex(self, index):\n        self.delete(self.DEFAULT_PROGRAM_NAME, index, index)\n\n    def delete(self, program_name, from_idx, to_idx):\n        if isinstance(from_idx, Token):\n            self.replace(program_name, from_idx.tokenIndex, to_idx.tokenIndex, \"\")\n        else:\n            self.replace(program_name, from_idx, to_idx, \"\")\n","sourceCodeStart":59,"sourceCodeEnd":95,"githubUrl":"https://github.com/antlr/antlr4/blob/7d5770395bb7b02eb56e7c62662cb1d7c08f42a3/runtime/Python3/src/antlr4/TokenStreamRewriter.py#L59-L95","documentation":"Thrown by TokenStreamRewriter.replace() when the supplied token index range is invalid for the underlying token stream. The rewrite API validates that 0 <= from_idx <= to_idx < len(tokens.tokens) before creating a ReplaceOp; any violation raises this ValueError. It almost always means the caller computed token indices from a different token stream, used stale indices after re-lexing, or passed a Token object from another parser.","triggerScenarios":"Calling rewriter.replace(prog, from_idx, to_idx, text), replaceRange, replaceSingleToken, deleteIndex, insertAfter, etc. with from_idx > to_idx, a negative index, or to_idx >= size of the token stream (e.g. using token.tokenIndex values from a second parse of different input, or indexes off the end of the stream).","commonSituations":"Keeping Token references from one parse and applying them to a rewriter built on another CommonTokenStream; off-by-one when computing the last token index (using len(tokens) instead of len(tokens)-1); deleting/replacing the EOF token; index arithmetic done on character offsets instead of token indices.","solutions":["Recompute from_idx/to_idx directly from the same CommonTokenStream that the TokenStreamRewriter wraps, e.g. stream.getTokens(from, to) or token.tokenIndex from that stream.","Clamp/validate before calling: assert 0 <= from_idx <= to_idx < len(stream.tokens).","Prefer passing Token objects (replaceRangeTokens / deleteToken) instead of raw ints so indices come from the stream itself.","Remember EOF has tokenIndex == size-1; do not use to_idx == size."],"exampleFix":"// before\nrewriter.replace('default', 0, len(stream.tokens), 'x')  // to_idx past EOF -> ValueError\n\n// after\nlast = len(stream.tokens) - 1  # EOF token index\nrewriter.replace('default', 0, last, 'x')","handlingStrategy":"validation","validationCode":"def valid_replace_range(stream, from_idx, to_idx):\n    return 0 <= from_idx <= to_idx < len(stream.tokens)\n\nif valid_replace_range(stream, from_idx, to_idx):\n    rewriter.replace('default', from_idx, to_idx, text)","typeGuard":null,"tryCatchPattern":"try:\n    rewriter.replace(prog, from_idx, to_idx, text)\nexcept ValueError as e:\n    if 'range invalid' in str(e):\n        raise IndexError('rewrite range %d..%d outside token stream (size %d)' % (from_idx, to_idx, len(stream.tokens)))\n    raise","preventionTips":["Derive all token indices from the same CommonTokenStream the rewriter wraps","Prefer token-object APIs (deleteToken, replaceRangeTokens) over raw ints","Remember EOF occupies the last index; never use to_idx == len(stream.tokens)"],"tags":["antlr4","python","token-stream-rewriter","index-out-of-range","validation"],"backgroundTag":null,"analyzedSha":"7d5770395bb7b02eb56e7c62662cb1d7c08f42a3","analyzedAt":"2026-08-14T14:47:56.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}