{"record":{"id":"3140f07732e718a1","repo":"stanfordnlp/CoreNLP","slug":"you-cannot-ask-for-the-probability-of-null","errorCode":null,"errorMessage":"You cannot ask for the probability of null.","messagePattern":"You cannot ask for the probability of null\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/stats/DirichletProcess.java","lineNumber":41,"sourceCode":"  public E drawSample(Random random) {\n    E drawn = Counters.sample(sampled);\n    if (drawn == null) {\n      drawn = baseMeasure.drawSample(random);\n    }\n    sampled.incrementCount(drawn);\n    return drawn;\n  }\n\n  public double numOccurances(E object) {\n    if (object == null) {\n      throw new RuntimeException(\"You cannot ask for the number of occurances of null.\");\n    }\n    return sampled.getCount(object);\n  }\n  \n  public double probabilityOf(E object) {\n    if (object == null) {\n      throw new RuntimeException(\"You cannot ask for the probability of null.\");\n    }\n\n    if (sampled.keySet().contains(object)) {\n      return sampled.getCount(object) / sampled.totalCount();\n    } else {\n      return 0.0;\n    }\n  }\n\n  public double logProbabilityOf(E object) {\n    return Math.log(probabilityOf(object));\n  }\n\n  public double probabilityOfNewObject() {\n    return alpha / sampled.totalCount();\n  }\n  \n}","sourceCodeStart":23,"sourceCodeEnd":59,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/stats/DirichletProcess.java#L23-L59","documentation":"DirichletProcess.probabilityOf(object) computes the probability of an object under the sampled base distribution, but null is not a valid random variable value, so the method throws RuntimeException for null input. It is also called by logProbabilityOf, so a null propagates there too.","triggerScenarios":"Calling dp.probabilityOf(null) or dp.logProbabilityOf(null), typically with a null produced by a failed lookup or parse upstream.","commonSituations":"Scoring unknown/unparsed tokens where the token variable is null; scoring results of optional extraction code that returned null.","solutions":["Return a default probability (0.0, or log prob Double.NEGATIVE_INFINITY) for null before calling.","Fix the upstream pipeline so scoring only receives non-null objects.","Wrap scoring in a null check helper used by both probabilityOf and logProbabilityOf call sites."],"exampleFix":"// before\ndouble lp = dp.logProbabilityOf(token); // token may be null -> throws\n// after\ndouble lp = (token == null) ? Double.NEGATIVE_INFINITY : dp.logProbabilityOf(token);","handlingStrategy":"type-guard","validationCode":"if (object != null) {\n  double p = dp.probabilityOf(object);\n}","typeGuard":"boolean isScorable(E o) { return o != null; }\n// use: if (isScorable(token)) dp.logProbabilityOf(token); else NEGATIVE_INFINITY;","tryCatchPattern":"double lp;\ntry {\n  lp = dp.logProbabilityOf(object);\n} catch (RuntimeException e) {\n  if (e.getMessage().contains(\"probability of null\")) lp = Double.NEGATIVE_INFINITY;\n  else throw e;\n}","preventionTips":["Guard both probabilityOf and logProbabilityOf call sites with null checks.","Assign null objects a probability of 0 (log: -infinity) by convention.","Ensure tokenization/extraction code never produces null objects for scoring."],"tags":["null-argument","statistics","validation"],"backgroundTag":"null-argument","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}