{"record":{"id":"11a32e33f4382653","repo":"stanfordnlp/CoreNLP","slug":"gradient-is-numerically-zero-stopped-on-machine-e","errorCode":null,"errorMessage":"Gradient is numerically zero, stopped on machine epsilon.","messagePattern":"Gradient is numerically zero, stopped on machine epsilon\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"src/edu/stanford/nlp/optimization/QNMinimizer.java","lineNumber":488,"sourceCode":"          && (size > 5 && Math.abs(averageImprovement / newestVal) < TOL)) {\n        return eState.TERMINATE_AVERAGEIMPROVE;\n      }\n\n      // Check to see if the gradient is sufficiently small\n      if (useRelativeNorm && relNorm <= relativeTOL) {\n        return eState.TERMINATE_RELATIVENORM;\n      }\n\n      if (useNumericalZero) {\n        // This checks if the gradient is sufficiently small compared to x that\n        // it is treated as zero.\n        if (gNormLast < EPS * Math.max(1.0, ArrayMath.norm_1(xLast))) {\n          // |g| < |x|_1\n          // First we do the one norm, because that's easiest, and always bigger.\n          if (gNormLast < EPS * Math.max(1.0, ArrayMath.norm(xLast))) {\n            // |g| < max(1,|x|)\n            // Now actually compare with the two norm if we have to.\n            log.warn(\"Gradient is numerically zero, stopped on machine epsilon.\");\n            return eState.TERMINATE_GRADNORM;\n          }\n        }\n        // give user information about the norms.\n      }\n\n      sb.append(\" |\").append(nf.format(gNormLast)).append(\"| {\").append(nf.format(relNorm)).append(\"} \");\n      sb.append(nf.format(Math.abs(averageImprovement / newestVal))).append(' ');\n      sb.append(evalsSize > 0 ? evals.get(evalsSize - 1).toString() : \"-\").append(' ');\n      return eState.CONTINUE;\n    }\n\n    /**\n     *  Return the time in seconds since this class was created.\n     *  @return The time in seconds since this class was created.\n     */\n    double howLong() {\n      return (System.currentTimeMillis() - startTime) / 1000.0;","sourceCodeStart":470,"sourceCodeEnd":506,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/optimization/QNMinimizer.java#L470-L506","documentation":"QNMinimizer's Record monitor checks convergence each iteration. When the gradient 1-, 2-, and max-norms all fall below machine epsilon relative to the parameter scale, the optimizer concludes the gradient is numerically zero and terminates with TERMINATE_GRADNORM. This is a successful (though possibly early) convergence, not a failure.","triggerScenarios":"Any QNMinimizer.minimize run (e.g. training a CRF or classifier) where the gradient norm decays below EPS * max(1, ||x||) — typically on flat objective regions, tiny datasets, or after many iterations as the gradient underflows.","commonSituations":"Normal convergence on small/clean data; repeated objective evaluations producing identical values; near-optimum points where the true gradient is ~0.","solutions":["No action needed — training terminated because the gradient is effectively zero; use the returned parameters.","If termination is too early, adjust tolerances passed to minimize / the Record tolerance so stopping criteria fit your needs.","Enable minimizer progress output to inspect the convergence history before deciding.","Check feature/objective scaling if you suspect premature gradient underflow."],"exampleFix":"// before\nQNMinimizer qn = new QNMinimizer();\ndouble[] x = qn.minimize(f, 1e-4, initial, maxIters);\n// after — tighten the function tolerance so epsilon-level stops happen only when truly converged\ndouble[] x = qn.minimize(f, 1e-6, initial, maxIters);","handlingStrategy":"retry","validationCode":"// sanity-check gradient magnitude before minimizing\ndouble[] g = f.derivativeAt(initial);\nif (ArrayMath.norm(g) < 1e-12) log.warn(\"initial gradient ~0; check objective scaling\");","typeGuard":null,"tryCatchPattern":"QNMinimizer.TerminationEvent ev = record.waitForTermination();\nif (ev.getState() == eState.TERMINATE_GRADNORM) {\n  // gradient numerically zero — accept parameters or restart with looser tol\n  restartWithAdjustedTolerance();\n}","preventionTips":["Inspect convergence logs to judge if stopping was premature.","Scale features/loss so gradient magnitudes are meaningful.","Avoid excessive iterations on tiny datasets."],"tags":["java","optimization","convergence","numerical"],"backgroundTag":"value-out-of-range","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}