stanfordnlp/CoreNLP · error · RuntimeException

Got NaN for prob in…

Error message

Got NaN for prob in CRFNonLinearSecondOrderLogConditionalObjectiveFunction.calculate()

What it means

CRFNonLinearSecondOrderLogConditionalObjectiveFunction.calculate() computes the sequence log-probability; if it comes out NaN, the function throws rather than feeding a broken value to the optimizer. NaN arises from numerical instability in the second-order (higher-order) CRF computations: overflow/underflow in forward-backward, zero softmax denominators, or non-finite weights.

Solutions

  1. Validate the parameter vector for NaN/Inf before each calculate() call and restart optimization from clean initial weights.
  2. Scale input features and consider reducing hidden-layer size or adding regularization to tame activation magnitudes.
  3. Fix output-layer flag combinations (softmaxOutputLayer with sparseOutputLayer/tieOutputLayer) to a supported configuration.
  4. Lower the learning rate / iteration count, or verify forward-backward scaling for long sequences.

Example fix

// before
minimizer.minimize(fn, tolerance, initialWeights);
// after
double[] w = initialWeights;
for (double v : w) if (!(Double.isFinite(v))) throw new IllegalArgumentException("non-finite initial weight");
double[] result = minimizer.minimize(fn, tolerance, w);
Defensive patterns

Strategy: validation

Validate before calling

// guard before each optimizer iteration is impractical; guard inputs instead
for (double v : initialWeights)
  if (!Double.isFinite(v)) throw new IllegalArgumentException("non-finite initial weight");
// and bound features
for (double[] doc : flatFeatures)
  for (double v : doc)
    if (!Double.isFinite(v) || Math.abs(v) > 1e6) throw new IllegalArgumentException("bad feature value");

Try / catch

try {
  minimizer.minimize(fn, tol, x0);
} catch (RuntimeException e) {
  if (e.getMessage().contains("Got NaN for prob in CRFNonLinearSecondOrder")) {
    // halve learning rate, rescale features, and restart from clean weights
  } else throw e;
}

Prevention

When it happens

Trigger: Calling calculate() during training of a second-order non-linear CRF (useNonLinearCRF=true with second-order window) when x contains NaN/Inf, activations overflow, or expected-count terms diverge to Inf-Inf.

Common situations: Divergent optimization runs (learning rate too high, too many iterations), unscaled feature values, degenerate softmax output-layer settings, or very long sequences whose probabilities underflow.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/52372572f3d06176. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/ie/crf/CRFNonLinearSecondOrderLogConditionalObjectiveFunction.java:722

              if (useHiddenLayer)
                deltaK *= fDeriv[k];
              double[] eWK = null;
              if (j == 0) {
                eWK = eW[k];
              } else {
                eWK = eW4Edge[k];
              }
              for (int cliqueFeature : cliqueFeatures) {
                eWK[cliqueFeature] += deltaK * p;
              }
            }
          }
        }
      }
    }

    if (Double.isNaN(prob)) { // shouldn't be the case
      throw new RuntimeException("Got NaN for prob in CRFNonLinearSecondOrderLogConditionalObjectiveFunction.calculate()");
    }

    value = -prob;
    if(VERBOSE){
      log.info("value is " + value);
    }

    // compute the partial derivative for each feature by comparing expected counts to empirical counts
    int index = 0;
    for (int i = 0; i < eW4Edge.length; i++) {
      for (int j = 0; j < eW4Edge[i].length; j++) {
        derivative[index++] = (eW4Edge[i][j] - What4Edge[i][j]);
        if (VERBOSE) {
          log.info("inputLayerWeights4Edge deriv(" + i + "," + j + ") = " + eW4Edge[i][j] + " - " + What4Edge[i][j] + " = " + derivative[index - 1]);
        }
      }
    }

View on GitHub (pinned to 1b7edd19c4)