{"record":{"id":"60a7870065840f64","repo":"stanfordnlp/CoreNLP","slug":"could-not-read-from-double-initial-lop-scales-file","errorCode":null,"errorMessage":"Could not read from double initial LOP scales file ","messagePattern":"Could not read from double initial LOP scales file ","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/ie/crf/CRFClassifierWithLOP.java","lineNumber":209,"sourceCode":"    // Arrays.fill(lopScales, 1.0);\n    CRFLogConditionalObjectiveFunctionForLOP func = new CRFLogConditionalObjectiveFunctionForLOP(data, labels, lopExpertWeights,\n        windowSize, classIndex, labelIndices, map, flags.backgroundSymbol, numLopExpert, featureIndicesSetArray, featureIndicesListArray,\n        flags.backpropLopTraining);\n    cliquePotentialFunctionHelper = func;\n\n    Minimizer<DiffFunction> minimizer = getMinimizer(0, evaluators);\n\n    double[] initialScales;\n    //TODO(mengqiu) clean this part up when backpropLogTraining == true\n    if (flags.initialLopScales == null) {\n      initialScales = func.initial();\n    } else {\n      log.info(\"Reading initial LOP scales from file \" + flags.initialLopScales);\n      try (DataInputStream dis = new DataInputStream(new BufferedInputStream(new GZIPInputStream(new FileInputStream(\n            flags.initialLopScales))))) {\n        initialScales = ConvertByteArray.readDoubleArr(dis);\n      } catch (IOException e) {\n        throw new RuntimeException(\"Could not read from double initial LOP scales file \" + flags.initialLopScales);\n      }\n    }\n\n    double[] learnedParams = minimizer.minimize(func, flags.tolerance, initialScales);\n    double[] rawScales = func.separateLopScales(learnedParams);\n    double[] lopScales = ArrayMath.softmax(rawScales);\n    log.info(\"After SoftMax Transformation, learned scales are:\");\n    for (int lopIter = 0; lopIter < numLopExpert; lopIter++) {\n      log.info(\"lopScales[\" + lopIter + \"] = \" + lopScales[lopIter]);\n    }\n    double[][] learnedLopExpertWeights = lopExpertWeights;\n    if (flags.backpropLopTraining) {\n      learnedLopExpertWeights = func.separateLopExpertWeights(learnedParams);\n    }\n    return CRFLogConditionalObjectiveFunctionForLOP.combineAndScaleLopWeights(numLopExpert, learnedLopExpertWeights, lopScales);\n  }\n\n} // end class CRFClassifierWithLOP","sourceCodeStart":191,"sourceCodeEnd":227,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/ie/crf/CRFClassifierWithLOP.java#L191-L227","documentation":"trainWeights reads initial LOP scales from flags.initialLopScales through a gzipped DataInputStream; an IOException there is wrapped in this RuntimeException. The configured scales file exists but could not be read or decoded as the expected double array format.","triggerScenarios":"flags.initialLopScales set to a missing, unreadable, or corrupt (not valid gzip / wrong binary layout) file; stream fails partway so readDoubleArr throws IOException.","commonSituations":"Scales file written by a different NLP Core version or different Convert/ConvertByteArray binary layout, corrupt download, wrong file supplied, or path/permission problems.","solutions":["Verify flags.initialLopScales points to a valid gzip file produced by the same library version (gunzip -t to test).","Regenerate the scales file with the same Stanford NLP Core version you are training with.","Check read permissions and that the path resolves from the training working directory.","Unset initialLopScales to let training initialize scales itself if no prior scales are required."],"exampleFix":"// before\ninitialLopScales=scales.gz   // file from an older core version\n// after\ninitialLopScales=/abs/path/scales_v4.gz  // regenerated with current version\n// verify: gunzip -t scales_v4.gz","handlingStrategy":"validation","validationCode":"File f = new File(flags.initialLopScales);\nif (!f.isFile() || !f.canRead() || !isGzip(f))\n  throw new IllegalArgumentException(\"initialLopScales missing or not gzip: \" + f.getAbsolutePath());\n// isGzip: first two bytes == 0x1f 0x8b","typeGuard":"static boolean isGzip(File f) throws IOException {\n  try (InputStream in = new BufferedInputStream(new FileInputStream(f))) {\n    int b0 = in.read(), b1 = in.read();\n    return b0 == 0x1f && b1 == (byte)0x8b;\n  }\n}","tryCatchPattern":"try {\n  trainer.train();\n} catch (RuntimeException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"Could not read from double initial LOP scales file\")) {\n    props.remove(\"initialLopScales\"); // fall back to default scale init\n    trainer.train();\n  } else throw e;\n}","preventionTips":["Verify gzip integrity (gunzip -t) before training.","Generate scales files with the same library version used for training.","Make initialLopScales optional in your pipeline so bad files can be dropped."],"tags":["io","file-read","gzip","training","configuration","crf"],"backgroundTag":"file-read-failed","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}