{"record":{"id":"cd4f5250363b53f0","repo":"stanfordnlp/CoreNLP","slug":"got-nan-for-prob-in-crflogconditionalobjectivefunc-cd4f52","errorCode":null,"errorMessage":"Got NaN for prob in CRFLogConditionalObjectiveFunctionWithDropout.calculate() - this may well indicate numeric underflow due to overly long documents.","messagePattern":"Got NaN for prob in CRFLogConditionalObjectiveFunctionWithDropout\\.calculate\\(\\) - this may well indicate numeric underflow due to overly long documents\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/ie/crf/CRFLogConditionalObjectiveFunctionWithDropout.java","lineNumber":799,"sourceCode":"      Map<Integer, double[]> partialDropout = result.fourth();\n      if (partialDropout != null) {\n        if (isUnsup) {\n          combine2DArr(dropoutPriorGradTotal, partialDropout, unsupDropoutScale);\n        } else {\n          combine2DArr(dropoutPriorGradTotal, partialDropout);\n        }\n      }\n\n      if (!isUnsup) {\n        Map<Integer, double[]> partialE = result.third();\n        if (partialE != null)\n          combine2DArr(E, partialE);\n      }\n    }\n\n\n    if (Double.isNaN(prob)) { // shouldn't be the case\n      throw new RuntimeException(\"Got NaN for prob in CRFLogConditionalObjectiveFunctionWithDropout.calculate()\" +\n              \" - this may well indicate numeric underflow due to overly long documents.\");\n    }\n\n    // because we minimize -L(\\theta)\n    value = -prob;\n    if (VERBOSE) {\n      log.info(\"value is \" + Math.exp(-value));\n    }\n\n    // compute the partial derivative for each feature by comparing expected counts to empirical counts\n    int index = 0;\n    for (int i = 0; i < E.length; i++) {\n      for (int j = 0; j < E[i].length; j++) {\n        // because we minimize -L(\\theta)\n        derivative[index] = (E[i][j] - Ehat[i][j]);\n        derivative[index] += dropoutScale * dropoutPriorGradTotal[i][j];\n        if (VERBOSE) {\n          log.info(\"deriv(\" + i + ',' + j + \") = \" + E[i][j] + \" - \" + Ehat[i][j] + \" = \" + derivative[index]);","sourceCodeStart":781,"sourceCodeEnd":817,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/ie/crf/CRFLogConditionalObjectiveFunctionWithDropout.java#L781-L817","documentation":"CRFLogConditionalObjectiveFunctionWithDropout adds dropout regularization to the CRF objective; after computing prob (aggregated with combine2DArr over partial gradients) it throws a RuntimeException if prob is NaN, explicitly warning that overly long documents can cause numeric underflow. Training is aborted because a NaN objective cannot be optimized.","triggerScenarios":"Calling calculate() on the dropout CRF objective when the accumulated prob is NaN — dropout training on long sequences underflows, or parameters/feature values are non-finite.","commonSituations":"Dropout-regularized CRF training (SeqClassifierFlags.dropoutRate > 0) on long documents or with aggressive dropout schedules that push probabilities to zero.","solutions":["Reduce document length by splitting into sentences/segments before training.","Lower the dropout rate or adjust dropout schedule parameters (dropoutScale, multiTouchGrad) in SeqClassifierFlags.","Validate weights x for NaN/Inf before the optimizer step and reduce the learning rate if needed.","Check feature values for zeros that cause log(0) in the objective.","Catch the exception and restart from a checkpoint with smaller learning rate."],"exampleFix":"// before\nflags.dropoutRate = 0.5;\ntrain(fullLongDocs);\n// after\nflags.dropoutRate = 0.2;\nList<List<CoreLabel>> chunks = splitBySentence(fullLongDocs);\ntrain(chunks);","handlingStrategy":"validation","validationCode":"if (flags.dropoutRate > 0.5) throw new IllegalArgumentException(\"dropoutRate too high; underflow risk\");\nfor (List<CoreLabel> doc : trainingData) if (doc.size() > MAX_LEN) splitDocument(doc);","typeGuard":"static boolean validDropoutFlags(SeqClassifierFlags f) { return f.dropoutRate >= 0 && f.dropoutRate <= 0.5; }","tryCatchPattern":"try {\n  dropoutCrf.calculate(x, batch, E);\n} catch (RuntimeException e) {\n  if (e.getMessage().contains(\"NaN for prob\")) {\n    flags.dropoutRate *= 0.5;\n    restartTraining(lastGoodCheckpoint);\n  } else throw e;\n}","preventionTips":["Use moderate dropout rates; very high dropout pushes sequence probabilities to zero.","Chunk long documents into sentences before dropout training.","Tune dropoutScale and schedule parameters per SeqClassifierFlags docs.","Monitor the objective value per iteration and stop on the first non-finite value."],"tags":["crf","dropout","nan","underflow"],"backgroundTag":"value-out-of-range","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}