stanfordnlp/CoreNLP · error · RuntimeException

after W derivative, index() != beforeOutputWeights()

Error message

after W derivative, index() != beforeOutputWeights()

What it means

After writing the input-layer (hidden W matrix) derivative block in CRFNonLinearLogConditionalObjectiveFunction.calculate(), the write index must equal beforeOutputWeights (the offset at which output-layer parameters start). A mismatch means the eW/What loops wrote a different number of entries than inputLayerSize * numClasses predicts, breaking the parameter-vector layout.

Solutions

1. Confirm inputLayerSize in the flags matches the eW array's first dimension, and that sparseOutputLayer/tieOutputLayer settings are the intended ones.
2. Check that the W-derivative loops iterate over the full eW matrix (inputLayerSize x numClasses) exactly once.
3. Rebuild against an unmodified official CoreNLP version to rule out local patches.
4. Log inputLayerSize, numClasses, and beforeOutputWeights before training to verify they line up.

Example fix

// before (custom hidden layer resized but flags not updated)
this.eW = new double[customHiddenSize][numClasses];
// after (keep flags and array in sync)
this.inputLayerSize = customHiddenSize;
this.eW = new double[this.inputLayerSize][numClasses];

// verify W block size matches offset math
if (inputLayerSize * numClasses != beforeOutputWeights - edgeParamCount)
  throw new IllegalStateException("W block size mismatch");

try {
  minimizer.minimize(fn, tol, x);
} catch (RuntimeException e) {
  if (e.getMessage().contains("!= beforeOutputWeights")) {
    // print inputLayerSize/numClasses and reconfigure flags
  } else throw e;
}

When it happens

Trigger: Calling calculate() when inputLayerSize was set inconsistently with the actual eW array dimensions -- e.g. flags.sparseOutputLayer or non-default hidden-layer sizing interacting with modified code, or constructor arguments (numNodeFeatures, numClasses) that disagree with the weight arrays.

Common situations: Custom extensions of the non-linear CRF (changing hidden layer size or sparsity flags) during CRFClassifier training with useNonLinearCRF=true; merges of upstream patches that desynchronized dimension constants.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• after edge derivative, index() != edgeParamCount()
• after W derivative, index() != x.length()
• after W derivative, index() != beforeOutputWeights()
• after blockInitialize, param Index ( ) not equal to…
• gradient check failed

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/360a99ea750cbe5a. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/ie/crf/CRFNonLinearLogConditionalObjectiveFunction.java:702

        if (VERBOSE) {
          log.info("linearWeights deriv(" + i + "," + j + ") = " + E[i][j] + " - " + Ehat[i][j] + " = " + derivative[index - 1]);
        }
      }
    }
    if (index != edgeParamCount)
      throw new RuntimeException("after edge derivative, index("+index+") != edgeParamCount("+edgeParamCount+")");

    for (int i = 0; i < eW.length; i++) {
      for (int j = 0; j < eW[i].length; j++) {
        derivative[index++] = (eW[i][j] - What[i][j]);
        if (VERBOSE) {
          log.info("inputLayerWeights deriv(" + i + "," + j + ") = " + eW[i][j] + " - " + What[i][j] + " = " + derivative[index - 1]);
        }
      }
    }

    if (index != beforeOutputWeights)
      throw new RuntimeException("after W derivative, index("+index+") != beforeOutputWeights("+beforeOutputWeights+")");

    if (useOutputLayer) {
      for (int i = 0; i < eU.length; i++) {
        for (int j = 0; j < eU[i].length; j++) {
          if (flags.hardcodeSoftmaxOutputWeights)
            derivative[index++] = 0;
          else
            derivative[index++] = (eU[i][j] - Uhat[i][j]);
          if (VERBOSE) {
            log.info("outputLayerWeights deriv(" + i + "," + j + ") = " + eU[i][j] + " - " + Uhat[i][j] + " = " + derivative[index - 1]);
          }
        }
      }
    }

    if (index != x.length)
      throw new RuntimeException("after W derivative, index("+index+") != x.length("+x.length+")");

View on GitHub (pinned to 1b7edd19c4)