What causes "Gradient is numerically zero, stopped on machine epsilon." in stanfordnlp/CoreNLP?

No action needed — training terminated because the gradient is effectively zero; use the returned parameters.

Gradient is numerically zero, stopped on machine epsilon.

Error message

Gradient is numerically zero, stopped on machine epsilon.

What it means

QNMinimizer's Record monitor checks convergence each iteration. When the gradient 1-, 2-, and max-norms all fall below machine epsilon relative to the parameter scale, the optimizer concludes the gradient is numerically zero and terminates with TERMINATE_GRADNORM. This is a successful (though possibly early) convergence, not a failure.

Solutions

No action needed — training terminated because the gradient is effectively zero; use the returned parameters.
If termination is too early, adjust tolerances passed to minimize / the Record tolerance so stopping criteria fit your needs.
Enable minimizer progress output to inspect the convergence history before deciding.
Check feature/objective scaling if you suspect premature gradient underflow.

Example fix

// before
QNMinimizer qn = new QNMinimizer();
double[] x = qn.minimize(f, 1e-4, initial, maxIters);
// after — tighten the function tolerance so epsilon-level stops happen only when truly converged
double[] x = qn.minimize(f, 1e-6, initial, maxIters);

Defensive patterns

Strategy: retry

Validate before calling

// sanity-check gradient magnitude before minimizing
double[] g = f.derivativeAt(initial);
if (ArrayMath.norm(g) < 1e-12) log.warn("initial gradient ~0; check objective scaling");

Try / catch

QNMinimizer.TerminationEvent ev = record.waitForTermination();
if (ev.getState() == eState.TERMINATE_GRADNORM) {
  // gradient numerically zero — accept parameters or restart with looser tol
  restartWithAdjustedTolerance();
}

Prevention

Inspect convergence logs to judge if stopping was premature.
Scale features/loss so gradient magnitudes are meaningful.
Avoid excessive iterations on tiny datasets.

When it happens

Trigger: Any QNMinimizer.minimize run (e.g. training a CRF or classifier) where the gradient norm decays below EPS * max(1, ||x||) — typically on flat objective regions, tiny datasets, or after many iterations as the gradient underflows.

Common situations: Normal convergence on small/clean data; repeated objective evaluations producing identical values; near-optimum points where the true gradient is ~0.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/11a32e33f4382653. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/optimization/QNMinimizer.java:488

          && (size > 5 && Math.abs(averageImprovement / newestVal) < TOL)) {
        return eState.TERMINATE_AVERAGEIMPROVE;
      }

      // Check to see if the gradient is sufficiently small
      if (useRelativeNorm && relNorm <= relativeTOL) {
        return eState.TERMINATE_RELATIVENORM;
      }

      if (useNumericalZero) {
        // This checks if the gradient is sufficiently small compared to x that
        // it is treated as zero.
        if (gNormLast < EPS * Math.max(1.0, ArrayMath.norm_1(xLast))) {
          // |g| < |x|_1
          // First we do the one norm, because that's easiest, and always bigger.
          if (gNormLast < EPS * Math.max(1.0, ArrayMath.norm(xLast))) {
            // |g| < max(1,|x|)
            // Now actually compare with the two norm if we have to.
            log.warn("Gradient is numerically zero, stopped on machine epsilon.");
            return eState.TERMINATE_GRADNORM;
          }
        }
        // give user information about the norms.
      }

      sb.append(" |").append(nf.format(gNormLast)).append("| {").append(nf.format(relNorm)).append("} ");
      sb.append(nf.format(Math.abs(averageImprovement / newestVal))).append(' ');
      sb.append(evalsSize > 0 ? evals.get(evalsSize - 1).toString() : "-").append(' ');
      return eState.CONTINUE;
    }

    /**
     *  Return the time in seconds since this class was created.
     *  @return The time in seconds since this class was created.
     */
    double howLong() {
      return (System.currentTimeMillis() - startTime) / 1000.0;

View on GitHub (pinned to 1b7edd19c4)