{"record":{"id":"61d0631dc8bed868","repo":"antlr/antlr4","slug":"tokens-cannot-be-null","errorCode":null,"errorMessage":"tokens cannot be null","messagePattern":"tokens cannot be null","errorType":"exception","errorClass":"NullPointerException","httpStatus":null,"severity":"error","filePath":"runtime/Java/src/org/antlr/v4/runtime/ListTokenSource.java","lineNumber":80,"sourceCode":"\t\tthis(tokens, null);\n\t}\n\n\t/**\n\t * Constructs a new {@link ListTokenSource} instance from the specified\n\t * collection of {@link Token} objects and source name.\n\t *\n\t * @param tokens The collection of {@link Token} objects to provide as a\n\t * {@link TokenSource}.\n\t * @param sourceName The name of the {@link TokenSource}. If this value is\n\t * {@code null}, {@link #getSourceName} will attempt to infer the name from\n\t * the next {@link Token} (or the previous token if the end of the input has\n\t * been reached).\n\t *\n\t * @exception NullPointerException if {@code tokens} is {@code null}\n\t */\n\tpublic ListTokenSource(List<? extends Token> tokens, String sourceName) {\n\t\tif (tokens == null) {\n\t\t\tthrow new NullPointerException(\"tokens cannot be null\");\n\t\t}\n\n\t\tthis.tokens = tokens;\n\t\tthis.sourceName = sourceName;\n\t}\n\n\t/**\n\t * {@inheritDoc}\n\t */\n\t@Override\n\tpublic int getCharPositionInLine() {\n\t\tif (i < tokens.size()) {\n\t\t\treturn tokens.get(i).getCharPositionInLine();\n\t\t}\n\t\telse if (eofToken != null) {\n\t\t\treturn eofToken.getCharPositionInLine();\n\t\t}\n\t\telse if (tokens.size() > 0) {","sourceCodeStart":62,"sourceCodeEnd":98,"githubUrl":"https://github.com/antlr/antlr4/blob/7d5770395bb7b02eb56e7c62662cb1d7c08f42a3/runtime/Java/src/org/antlr/v4/runtime/ListTokenSource.java#L62-L98","documentation":"ListTokenSource's constructor throws NullPointerException when the token list is null — the source's entire purpose is to replay that list (it holds no lexer or stream). It is documented via @exception NullPointerException and is the only precondition; an empty list is legal and yields immediate EOF. Typical use is re-lexing a token subset or feeding synthesized tokens into a parser.","triggerScenarios":"new ListTokenSource(null, name) when an upstream filtering step returned null instead of an empty list; building token sources from optional collections without defaults; map/filter pipelines that propagate null.","commonSituations":"Token-rewriting pipelines (e.g. dropping comment tokens then reparsing); code using getTokens(...) overloads that return null for empty results and passing that straight to ListTokenSource.","solutions":["Coalesce nulls to empty lists: new ListTokenSource(tokens != null ? tokens : Collections.emptyList(), name)","Fix the upstream getTokens(...) call to return an empty list (e.g. check the null return of get(start, stop) before use)","When synthesizing tokens in tests, initialize the list before constructing the source"],"exampleFix":"// before\nList<Token> ts = tokens.get(0, 5); // returns null when empty\nnew ListTokenSource(ts, \"src\"); // NPE\n\n// after\nList<Token> ts = Objects.requireNonNullElseGet(tokens.get(0, 5), ArrayList::new);\nnew ListTokenSource(ts, \"src\");","handlingStrategy":"validation","validationCode":"List<Token> safe = (tokens != null) ? tokens : new ArrayList<>();\nListTokenSource src = new ListTokenSource(safe, name);","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never pass null token lists — coalesce to an empty list","Remember get(start, stop) can return null; convert before feeding ListTokenSource","Initialize token lists at declaration in synthesizing code"],"tags":["antlr","token-source","null-check","constructor"],"backgroundTag":null,"analyzedSha":"7d5770395bb7b02eb56e7c62662cb1d7c08f42a3","analyzedAt":"2026-08-14T14:47:56.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}