{"record":{"id":"786393984a244381","repo":"stanfordnlp/CoreNLP","slug":"datum","errorCode":null,"errorMessage":"datum ","messagePattern":"datum ","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/classify/RVFDataset.java","lineNumber":235,"sourceCode":"    }\n    */\n  }\n\n  /**\n   * Checks if the dataset has any unbounded values. Always good to use this\n   * before training a model on the dataset. This way, one can avoid seeing the\n   * infamous 4's that get printed by the QuasiNewton Method when NaNs exist in\n   * the data! -Ramesh\n   */\n  public void ensureRealValues() {\n    double[][] values = getValuesArray();\n    int[][] data = getDataArray();\n    for (int i = 0; i < size(); i++) {\n      for (int j = 0; j < values[i].length; j++) {\n        if (Double.isNaN(values[i][j])) {\n          int fID = data[i][j];\n          F feature = featureIndex.get(fID);\n          throw new RuntimeException(\"datum \" + i + \" has a NaN value for feature:\" + feature);\n        }\n        if (Double.isInfinite(values[i][j])) {\n          int fID = data[i][j];\n          F feature = featureIndex.get(fID);\n          throw new RuntimeException(\"datum \" + i + \" has infinite value for feature:\" + feature);\n        }\n      }\n    }\n  }\n\n  /**\n   * Scales the values of each feature in each linearly using the min and max\n   * values found in the training set. NOTE1: Not guaranteed to be between 0 and\n   * 1 for a test datum. NOTE2: Also filters out features from each datum that\n   * are not seen at training time.\n   *\n   * @param dataset\n   * @return a new dataset","sourceCodeStart":217,"sourceCodeEnd":253,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/classify/RVFDataset.java#L217-L253","documentation":"ensureRealValues validates that all stored feature values are finite real numbers. If datum i has a NaN for some feature it throws, naming the datum and the feature (resolved through featureIndex), because NaN poisons training math and scaling. NaN values usually indicate upstream arithmetic such as 0/0, log(0), or bad parsed input.","triggerScenarios":"Calling ensureRealValues (directly or indirectly before training/scaling) on an RVFDataset whose values contain Double.NaN, typically produced by user-supplied feature computation code or parsing that yields NaN.","commonSituations":"Feature weight functions computing ratios or logs that yield NaN on zero counts; loading numeric data where missing values were encoded as NaN; serialization round-trips that corrupted values.","solutions":["Scan the dataset and fix the producing code so NaN never enters values (guard division/logs, replace NaN with 0 or a sentinel)","Filter out datums with non-finite values before training: check each RVFDatum feature count with Double.isNaN","If NaN means 'missing', impute a value (mean/zero) during datum construction","Run ensureRealValues early in your pipeline to fail fast at the data source"],"exampleFix":"// before\ncounter.incrementCount(\"f1\", Math.log(0.0)); // NaN\n// after\ndouble v = rawCount == 0.0 ? 0.0 : Math.log(rawCount);\nif (Double.isNaN(v) || Double.isInfinite(v)) v = 0.0;\ncounter.incrementCount(\"f1\", v);","handlingStrategy":"validation","validationCode":"for (RVFDatum<L,F> d : ds) {\n  for (Map.Entry<F,Double> e : d.asFeaturesCounter().entrySet()) {\n    if (Double.isNaN(e.getValue())) throw new IllegalStateException(\"NaN for feature \" + e.getKey());\n  }\n}","typeGuard":"boolean isFiniteValue(double v) { return !Double.isNaN(v) && !Double.isInfinite(v); }","tryCatchPattern":"try {\n  ds.ensureRealValues();\n} catch (RuntimeException e) {\n  // message: datum i has a NaN value for feature:F\n  logger.severe(\"Bad value: \" + e.getMessage());\n  throw e;\n}","preventionTips":["Guard any log/divide/exp used to compute feature values","Replace NaN with 0 or a sentinel before incrementing counters","Call ensureRealValues early, at data-load time"],"tags":["java","stanford-nlp","machine-learning","nan"],"backgroundTag":"invalid-argument-value","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}