{"record":{"id":"bd09bf68d0e40d6f","repo":"stanfordnlp/CoreNLP","slug":"attempting-to-remove-features-based-on-weight-from","errorCode":null,"errorMessage":"Attempting to remove features based on weight from a non-linear classifier","messagePattern":"Attempting to remove features based on weight from a non-linear classifier","errorType":"exception","errorClass":"java.lang.RuntimeException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/ie/ner/CMMClassifier.java","lineNumber":589,"sourceCode":"    }\n\n    if (flags.doAdaptation && flags.adaptFile != null) {\n      adapt(flags.adaptFile,train,readerAndWriter);\n    }\n\n    log.info(\"Built this classifier: \");\n    if (classifier instanceof LinearClassifier) {\n      String classString = ((LinearClassifier<String, String>)classifier).toString(flags.printClassifier, flags.printClassifierParam);\n      log.info(classString);\n    } else {\n      String classString = classifier.toString();\n      log.info(classString);\n    }\n  }\n\n  private Index<String> getFeaturesAboveThreshold(Dataset<String, String> dataset, double thresh) {\n    if (!(classifier instanceof LinearClassifier)) {\n      throw new RuntimeException(\"Attempting to remove features based on weight from a non-linear classifier\");\n    }\n    Index<String> featureIndex = dataset.featureIndex;\n    Index<String> labelIndex = dataset.labelIndex;\n\n    Index<String> features = new HashIndex<>();\n    Iterator<String> featureIt = featureIndex.iterator();\n    LinearClassifier<String, String> lc = (LinearClassifier<String, String>)classifier;\n    LOOP:\n    while (featureIt.hasNext()) {\n      String f = featureIt.next();\n      double smallest = Double.POSITIVE_INFINITY;\n      double biggest = Double.NEGATIVE_INFINITY;\n      for (String l : labelIndex) {\n        double weight = lc.weight(f, l);\n        if (weight < smallest) {\n          smallest = weight;\n        }\n        if (weight > biggest) {","sourceCodeStart":571,"sourceCodeEnd":607,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/ie/ner/CMMClassifier.java#L571-L607","documentation":"getFeaturesAboveThreshold removes dataset features by inspecting classifier weights, which only exist for LinearClassifier. It throws RuntimeException if the loaded classifier is not a LinearClassifier, since non-linear classifiers have no per-feature weights to threshold.","triggerScenarios":"Feature-removal/threshold pruning invoked after the classifier field was assigned a non-linear classifier (e.g. a different model type loaded from a serialized file).","commonSituations":"Loading a wrong or older model file that deserializes to a non-linear classifier; mixing classifier implementations in a training pipeline; configuration pointing at an incompatible serialized model.","solutions":["Verify the serialized model being loaded is a CRF/linear classifier before enabling weight-based feature removal","Disable or bypass the threshold-pruning step when using a non-linear classifier","Add an instanceof check with a friendlier error/log before entering the pruning path","Re-train and serialize the expected linear classifier so the correct type is loaded"],"exampleFix":"// before\nremoveFeatures(dataset, 0.1); // classifier is non-linear -> RuntimeException\n// after\nif (classifier instanceof LinearClassifier) { removeFeatures(dataset, 0.1); } else { log.warning(\"Skipping feature removal: non-linear classifier\"); }","handlingStrategy":"type-guard","validationCode":"if (!(classifier instanceof LinearClassifier)) { skipPruning(); }","typeGuard":"boolean isLinear(Object c) { return c instanceof LinearClassifier; }","tryCatchPattern":"try { pruneFeatures(dataset, thresh); } catch (RuntimeException e) { log.warning(\"Pruning requires a linear classifier: \" + e.getMessage()); }","preventionTips":["Confirm the serialized model type before enabling weight-based pruning","Only pair threshold pruning with linear CRF classifiers","Log the loaded classifier class after deserialization"],"tags":["java","ner","classifier","type-mismatch"],"backgroundTag":"incompatible-source-type","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}