stanfordnlp/CoreNLP · error · RuntimeException

after W derivative, index() != beforeOutputWeights()

Error message

after W derivative, index() != beforeOutputWeights()

What it means

In CRFNonLinearSecondOrderLogConditionalObjectiveFunction.calculate(), after writing the input-layer W derivative block the write index must equal beforeOutputWeights (the start of output-layer parameters). A mismatch means the W-derivative loops wrote entries inconsistent with the declared layer layout, corrupting the gradient/parameter alignment.

Solutions

1. Ensure inputLayerSize and numClasses produce eW exactly of size inputLayerSize x numClasses and that beforeOutputWeights is computed from the same values.
2. Verify the W-derivative loop covers the full eW matrix exactly once.
3. Align output-layer flags (softmaxOutputLayer/sparseOutputLayer/tieOutputLayer) with the code path so offsets are computed identically.
4. Rebuild from official CoreNLP sources if local patches touched the layout constants.

Example fix

// before (hidden layer resized without updating offsets)
eW = new double[newHiddenSize][numClasses];
// after
inputLayerSize = newHiddenSize; // recomputes beforeOutputWeights consistently
eW = new double[inputLayerSize][numClasses];

// verify sizes line up before training
if (inputLayerSize * numClasses + edgeParamCount != beforeOutputWeights)
  throw new IllegalStateException("layer offsets inconsistent with weight dimensions");

try {
  minimizer.minimize(fn, tol, x);
} catch (RuntimeException e) {
  if (e.getMessage().contains("after W derivative")) {
    // re-derive layer sizes from flags and rebuild the objective function
  } else throw e;
}

When it happens

Trigger: calculate() invoked when eW/What dimensions (derived from inputLayerSize and numClasses) disagree with beforeOutputWeights -- e.g. non-linear second-order CRF training with useOutputLayer configurations or custom hidden-layer sizes that break the offset computation.

Common situations: Patched or extended second-order CRF training code; inconsistent constructor parameters (numNodeFeatures, numEdgeFeatures, window) relative to weight matrix sizes during NER model experiments.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• after edge derivative, index() != edgeParamCount()
• after W derivative, index() != beforeOutputWeights()
• after W derivative, index() != x.length()
• after blockInitialize, param Index ( ) not equal to…
• gradient check failed

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/392537b2468505e9. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/ie/crf/CRFNonLinearSecondOrderLogConditionalObjectiveFunction.java:751

      for (int j = 0; j < eW4Edge[i].length; j++) {
        derivative[index++] = (eW4Edge[i][j] - What4Edge[i][j]);
        if (VERBOSE) {
          log.info("inputLayerWeights4Edge deriv(" + i + "," + j + ") = " + eW4Edge[i][j] + " - " + What4Edge[i][j] + " = " + derivative[index - 1]);
        }
      }
    }

    for (int i = 0; i < eW.length; i++) {
      for (int j = 0; j < eW[i].length; j++) {
        derivative[index++] = (eW[i][j] - What[i][j]);
        if (VERBOSE) {
          log.info("inputLayerWeights deriv(" + i + "," + j + ") = " + eW[i][j] + " - " + What[i][j] + " = " + derivative[index - 1]);
        }
      }
    }

    if (index != beforeOutputWeights)
      throw new RuntimeException("after W derivative, index("+index+") != beforeOutputWeights("+beforeOutputWeights+")");

    if (useOutputLayer) {
      for (int i = 0; i < eU4Edge.length; i++) {
        for (int j = 0; j < eU4Edge[i].length; j++) {
          derivative[index++] = (eU4Edge[i][j] - Uhat4Edge[i][j]);
          if (VERBOSE) {
            log.info("outputLayerWeights4Edge deriv(" + i + "," + j + ") = " + eU4Edge[i][j] + " - " + Uhat4Edge[i][j] + " = " + derivative[index - 1]);
          }
        }
      }
      for (int i = 0; i < eU.length; i++) {
        for (int j = 0; j < eU[i].length; j++) {
          derivative[index++] = (eU[i][j] - Uhat[i][j]);
          if (VERBOSE) {
            log.info("outputLayerWeights deriv(" + i + "," + j + ") = " + eU[i][j] + " - " + Uhat[i][j] + " = " + derivative[index - 1]);
          }
        }
      }

View on GitHub (pinned to 1b7edd19c4)