stanfordnlp/CoreNLP · error · RuntimeException

Got NaN for prob in…

Error message

Got NaN for prob in CRFLogConditionalObjectiveFunction.calculate() - this may well indicate numeric underflow due to overly long documents.

What it means

CRFLogConditionalObjectiveFunction.calculate() computes the log-probability of the labeled sequences given the model. If that value comes back as NaN, the function throws a RuntimeException because training cannot proceed on a non-finite objective. The message explicitly warns that overly long documents can cause numeric underflow (log of 0) in the forward/backward computations.

Solutions

Shorten training documents (split into sentences/segments) so sequence probabilities do not underflow.
Inspect the parameter vector x for NaN/Inf before training (print/validate weights; reduce learning rate or use a more stable optimizer like L-BFGS with line search).
Scale/normalize feature values so log-linear scores stay in a sane range.
Enable VERBOSE and log intermediate values to find the offending document, then fix or drop it.
As a last resort, catch the RuntimeException and skip/re-initialize the offending training batch.

Example fix

// before: one huge training document
DocumentReader dr = new DocumentReader("longdoc.txt");
// after: split into sentence-level windows
List<List<CoreLabel>> sentences = DocumentPreprocessor.split("longdoc.txt");
// train on sentences instead of the full document

Defensive patterns

Strategy: try-catch

Validate before calling

for (double w : x) { if (Double.isNaN(w) || Double.isInfinite(w)) throw new IllegalArgumentException("NaN/Inf weight before CRF calculate"); }
if (docLength > MAX_SEQUENCE_LENGTH) doc = chunk(doc, MAX_SEQUENCE_LENGTH);

Type guard

static boolean isFinite(double[] v) { for (double d : v) if (Double.isNaN(d) || Double.isInfinite(d)) return false; return true; }

Try / catch

try {
  crf.calculate(x, batch, E);
} catch (RuntimeException e) {
  if (e.getMessage().contains("NaN for prob")) {
    // shrink documents, reduce learning rate, restore checkpoint
    x = lastGoodCheckpoint;
    learningRate *= 0.5;
  } else throw e;
}

Prevention

Split long documents into sentence-level segments before CRF training.
Validate weight vectors for NaN/Inf each optimizer iteration.
Normalize feature values to keep log-linear scores bounded.
Use a stable optimizer (L-BFGS) with line search instead of fixed aggressive steps.

When it happens

Trigger: Calling calculate(double[] x, double[] batch, double[] E) in the single-threaded path when regularGradientAndValue() returns NaN — typically when the log-probability of a sequence underflows (very long documents, extreme feature weights, or weights x containing NaN/Inf).

Common situations: Training a CRF on very long documents (underflow in the log-sum of sequence probabilities), a learning-rate blowup producing Inf/NaN weights, or pathological feature scaling before calling CRFClassifier training.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/e2574e752f5e7de9. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/ie/crf/CRFLogConditionalObjectiveFunction.java:425

  /**
   * Calculates both value and partial derivatives at the point x, and save them internally.
   */
  @Override
  public void calculate(double[] x) {

    // final double[][] weights = to2D(x);
    to2D(x, weights);
    setWeights(weights);

    // the expectations over counts
    // first index is feature index, second index is of possible labeling
    // double[][] E = empty2D();
    clear2D(E);

    double prob = regularGradientAndValue(); // the log prob of the sequence given the model, which is the negation of value at this point

    if (Double.isNaN(prob)) { // shouldn't be the case
      throw new RuntimeException("Got NaN for prob in CRFLogConditionalObjectiveFunction.calculate()" +
              " - this may well indicate numeric underflow due to overly long documents.");
    }

    // because we minimize -L(\theta)
    value = -prob;
    if (VERBOSE) {
      log.info("value is " + Math.exp(-value));
    }

    // compute the partial derivative for each feature by comparing expected counts to empirical counts
    int index = 0;
    for (int i = 0; i < E.length; i++) {
      double[] E_i = E[i], Ehat_i = Ehat[i];
      for (int j = 0; j < E_i.length; j++) {
        // because we minimize -L(\theta)
        derivative[index] = (E_i[j] - Ehat_i[j]);
        if (VERBOSE) {
          log.info("deriv(" + i + "," + j + ") = " + E_i[j] + " - " + Ehat_i[j] + " = " + derivative[index]);

View on GitHub (pinned to 1b7edd19c4)