{"record":{"id":"00e680bd5e82e613","repo":"stanfordnlp/CoreNLP","slug":"could-not-open-temporary-feature-index-file-for-re","errorCode":null,"errorMessage":"Could not open temporary feature index file for reading.","messagePattern":"Could not open temporary feature index file for reading\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"critical","filePath":"src/edu/stanford/nlp/ie/crf/CRFClassifier.java","lineNumber":1703,"sourceCode":"      if (oneDimWeights != null) {\n        this.weights = to2D(oneDimWeights, labelIndices, map);\n      }\n\n      // if (flags.useFloat) {\n      //   oneDimWeights = trainWeightsUsingFloatCRF(data, labels, evaluators, i, featureVals);\n      // } else if (flags.numLopExpert > 1) {\n      //   oneDimWeights = trainWeightsUsingLopCRF(data, labels, evaluators, i, featureVals);\n      // } else {\n      //   oneDimWeights = trainWeightsUsingDoubleCRF(data, labels, evaluators, i, featureVals);\n      // }\n\n      // save feature index to disk and read in later\n      if (flags.saveFeatureIndexToDisk) {\n        try {\n          log.info(\"Reading temporary feature index file.\");\n          featureIndex = IOUtils.readObjectFromFile(featIndexFile);\n        } catch (Exception e) {\n          throw new RuntimeException(\"Could not open temporary feature index file for reading.\");\n        }\n      }\n\n      if (i != flags.numTimesPruneFeatures) {\n        dropFeaturesBelowThreshold(flags.featureDiffThresh);\n        log.info(\"Removing features with weight below \" + flags.featureDiffThresh + \" and retraining...\");\n      }\n    }\n  }\n\n  public static float[][] to2D(double[] weights, List<Index<CRFLabel>> labelIndices, int[] map) {\n    float[][] newWeights = new float[map.length][];\n    int index = 0;\n    for (int i = 0; i < map.length; i++) {\n      newWeights[i] = new float[labelIndices.get(map[i]).size()];\n      final int arrLength = labelIndices.get(map[i]).size();\n      for (int j = 0; j < arrLength; j++) {\n        newWeights[i][j] = (float) weights[index++];","sourceCodeStart":1685,"sourceCodeEnd":1721,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/ie/crf/CRFClassifier.java#L1685-L1721","documentation":"On subsequent training iterations with flags.saveFeatureIndexToDisk, CRFClassifier reloads the feature index written earlier from the temp file via IOUtils.readObjectFromFile. If reading or deserializing fails (file deleted, truncated, or class mismatch), the Exception is wrapped in this RuntimeException since the iteration cannot continue without the index.","triggerScenarios":"flags.saveFeatureIndexToDisk=true and the featIndexN.tmp file is missing, unreadable, or corrupt when the next training iteration starts — e.g. a tmp cleaner wiped it, a concurrent job overwrote it, or the earlier write silently failed.","commonSituations":"Long training on shared machines where /tmp is periodically cleaned; multi-node jobs without shared storage; disk pressure truncating the file; deserialization classpath mismatches.","solutions":["Point java.io.tmpdir to a stable private directory (-Djava.io.tmpdir=/your/dir) that auto-cleaners don't touch.","Disable saveFeatureIndexToDisk to keep the index in memory and restart training.","Fix the root cause shown in the wrapped exception (permissions, corrupt file, serialization classpath)."],"exampleFix":"// before\njava -Djava.io.tmpdir=/tmp -cp ... edu.stanford.nlp.ie.crf.CRFClassifier ...\n// after (on a machine where /tmp is auto-cleaned)\njava -Djava.io.tmpdir=/data/crf-tmp -cp ... edu.stanford.nlp.ie.crf.CRFClassifier ...","handlingStrategy":"try-catch","validationCode":"File tmpDir = new File(System.getProperty(\"java.io.tmpdir\"));\nif (!tmpDir.canRead()) {\n  throw new IllegalStateException(\"tmpdir unreadable; temp feature index files may disappear between iterations\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  classifier.train(files);\n} catch (RuntimeException e) {\n  if (String.valueOf(e.getMessage()).contains(\"temporary feature index file for reading\")) {\n    // temp file vanished or is corrupt: restart with in-memory index\n    props.remove(\"saveFeatureIndexToDisk\");\n  } else throw e;\n}","preventionTips":["Use a private, stable tmpdir (-Djava.io.tmpdir) that auto-cleaners don't touch.","Avoid sharing tmpdir between concurrent training jobs (colliding featIndexN.tmp names).","Prefer the in-memory index unless memory constraints force disk usage."],"tags":["java","io","filesystem","serialization"],"backgroundTag":"file-read-failed","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}