{"record":{"id":"bb234924a2410274","repo":"stanfordnlp/CoreNLP","slug":"after-edge-derivative-index-edgeparamcount","errorCode":null,"errorMessage":"after edge derivative, index() != edgeParamCount()","messagePattern":"after edge derivative, index\\(\\) != edgeParamCount\\(\\)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/ie/crf/CRFNonLinearLogConditionalObjectiveFunction.java","lineNumber":690,"sourceCode":"\n    value = -prob;\n    if(VERBOSE){\n      log.info(\"value is \" + value);\n    }\n\n    if (DEBUG) log.info(\"calculating derivative \");\n    // compute the partial derivative for each feature by comparing expected counts to empirical counts\n    int index = 0;\n    for (int i = 0; i < E.length; i++) {\n      for (int j = 0; j < E[i].length; j++) {\n        derivative[index++] = (E[i][j] - Ehat[i][j]);\n        if (VERBOSE) {\n          log.info(\"linearWeights deriv(\" + i + \",\" + j + \") = \" + E[i][j] + \" - \" + Ehat[i][j] + \" = \" + derivative[index - 1]);\n        }\n      }\n    }\n    if (index != edgeParamCount)\n      throw new RuntimeException(\"after edge derivative, index(\"+index+\") != edgeParamCount(\"+edgeParamCount+\")\");\n\n    for (int i = 0; i < eW.length; i++) {\n      for (int j = 0; j < eW[i].length; j++) {\n        derivative[index++] = (eW[i][j] - What[i][j]);\n        if (VERBOSE) {\n          log.info(\"inputLayerWeights deriv(\" + i + \",\" + j + \") = \" + eW[i][j] + \" - \" + What[i][j] + \" = \" + derivative[index - 1]);\n        }\n      }\n    }\n\n    if (index != beforeOutputWeights)\n      throw new RuntimeException(\"after W derivative, index(\"+index+\") != beforeOutputWeights(\"+beforeOutputWeights+\")\");\n\n    if (useOutputLayer) {\n      for (int i = 0; i < eU.length; i++) {\n        for (int j = 0; j < eU[i].length; j++) {\n          if (flags.hardcodeSoftmaxOutputWeights)\n            derivative[index++] = 0;","sourceCodeStart":672,"sourceCodeEnd":708,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/ie/crf/CRFNonLinearLogConditionalObjectiveFunction.java#L672-L708","documentation":"While filling the gradient array in CRFNonLinearLogConditionalObjectiveFunction.calculate(), the running write index did not land exactly on edgeParamCount after writing all edge (first-order) derivative terms. This is an internal layout invariant: each derivative slot must be written exactly once, so a mismatch means the parameter layout constants disagree with the loop bounds.","triggerScenarios":"A mismatch between numEdgeFeatures/edgeParamCount and the loops filling E/Ehat derivatives -- e.g. custom code that altered the 'map', labelIndices, or data dimensions passed to the constructor so the edge loops write more or fewer entries than edgeParamCount.","commonSituations":"Developers subclassing or modifying the non-linear CRF code (changing window size, label indices, or feature map) hit the inconsistency during CRF training with useNonLinearCRF=true; also possible in NER training pipelines with non-standard feature maps.","solutions":["Verify that the data/labels/map/labelIndices passed to the constructor are the ones produced by the standard CRFLogConditionalObjectiveFunction feature indexing.","Check for local modifications to the derivative-filling loops and restore them so exactly edgeParamCount entries are written for edges.","Update to an official Stanford CoreNLP release if working from a patched copy.","Ensure window size and numEdgeFeatures are consistent with second-order (window) labeling assumptions."],"exampleFix":"// before (patched loop writes fewer entries)\nfor (int i = 0; i < E.length - 1; i++) {\n  for (int j = 0; j < E[i].length; j++) derivative[index++] = E[i][j] - Ehat[i][j];\n}\n// after (write the full edge block)\nfor (int i = 0; i < E.length; i++) {\n  for (int j = 0; j < E[i].length; j++) derivative[index++] = E[i][j] - Ehat[i][j];\n}","handlingStrategy":"validation","validationCode":"// assert layout before training\nint expected = fn.domainDimension();\nif (derivative.length != expected)\n  throw new IllegalStateException(\"derivative length \" + derivative.length + \" != domainDimension \" + expected);","typeGuard":null,"tryCatchPattern":"try {\n  minimizer.minimize(fn, tol, x);\n} catch (RuntimeException e) {\n  if (e.getMessage().startsWith(\"after edge derivative\")) {\n    // fall back to standard (linear) CRF training\n  } else throw e;\n}","preventionTips":["Do not modify the edge-derivative loops or the map/labelIndices without recomputing edgeParamCount","Use standard CRF feature-indexing output as constructor input","Test non-linear CRF changes on a tiny dataset where the invariant fails fast"],"tags":["crf","gradient","invariant","indexing"],"backgroundTag":"internal-invariant-violation","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}