{"record":{"id":"8eb94e3bf739a1d7","repo":"stanfordnlp/CoreNLP","slug":"shufflewithsideinformation-sideinformation-not-of-8eb94e","errorCode":null,"errorMessage":"shuffleWithSideInformation: sideInformation not of same size as Dataset","messagePattern":"shuffleWithSideInformation: sideInformation not of same size as Dataset","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/classify/RVFDataset.java","lineNumber":1034,"sourceCode":"\n      int tmpl = labels[randIndex];\n      labels[randIndex] = labels[j];\n      labels[j] = tmpl;\n\n      double[] tmpv = values[randIndex];\n      values[randIndex] = values[j];\n      values[j] = tmpv;\n    }\n  }\n\n  /**\n   * Randomizes the data array in place. Needs to be redefined here because we\n   * need to randomize the values as well.\n   */\n  @Override\n  public <E> void shuffleWithSideInformation(long randomSeed, List<E> sideInformation) {\n    if (size != sideInformation.size()) {\n      throw new IllegalArgumentException(\"shuffleWithSideInformation: sideInformation not of same size as Dataset\");\n    }\n    Random rand = new Random(randomSeed);\n    for (int j = size - 1; j > 0; j--) {\n      int randIndex = rand.nextInt(j);\n\n      int[] tmp = data[randIndex];\n      data[randIndex] = data[j];\n      data[j] = tmp;\n\n      int tmpl = labels[randIndex];\n      labels[randIndex] = labels[j];\n      labels[j] = tmpl;\n\n      double[] tmpv = values[randIndex];\n      values[randIndex] = values[j];\n      values[j] = tmpv;\n\n      E tmpE = sideInformation.get(randIndex);","sourceCodeStart":1016,"sourceCodeEnd":1052,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/classify/RVFDataset.java#L1016-L1052","documentation":"shuffleWithSideInformation randomly permutes the dataset in place and applies the same permutation to a parallel list of side information (e.g. raw text per datum). If the list's size differs from the dataset size, the permutation can't be applied consistently, so it throws IllegalArgumentException immediately before shuffling.","triggerScenarios":"Calling dataset.shuffleWithSideInformation(seed, sideInfo) where sideInfo.size() != dataset.size — e.g. after filtering datums but not the side list, or building the side list from a different file than the dataset.","commonSituations":"Filtering or subsampling the RVFDataset (removing datums) while keeping the original full side-information list; accidentally shuffling twice; loading dataset and side info from files with different line counts.","solutions":["Make the side information list exactly parallel to the dataset: apply the same filtering/subsampling to both before shuffling","Check sizes first with an assert/exception of your own that also logs both sizes for diagnosis","Derive side information from the dataset itself (one entry per datum) rather than maintaining a separate list","If only the data should be shuffled, call shuffle() instead of shuffleWithSideInformation"],"exampleFix":"// before\nList<String> docs = readAllLines(\"docs.txt\");\nds = ds.subDataset(0, 1000); // dataset filtered\nds.shuffleWithSideInformation(42, docs); // size mismatch\n// after\nList<String> docs = readAllLines(\"docs.txt\").subList(0, 1000);\nassert docs.size() == ds.size();\nds.shuffleWithSideInformation(42, docs);","handlingStrategy":"validation","validationCode":"if (sideInformation.size() != dataset.size()) {\n  throw new IllegalArgumentException(\"side info size \" + sideInformation.size() + \" != dataset size \" + dataset.size());\n}\ndataset.shuffleWithSideInformation(seed, sideInformation);","typeGuard":null,"tryCatchPattern":"try {\n  ds.shuffleWithSideInformation(seed, sideInfo);\n} catch (IllegalArgumentException e) {\n  logger.severe(\"Size mismatch: dataset=\" + ds.size() + \" side=\" + sideInfo.size());\n  throw e;\n}","preventionTips":["Apply any filtering/subsampling to the side-information list and dataset together","Keep side info and datums in one structure (list of pairs) so they can never diverge","Assert size equality before every shuffle"],"tags":["java","stanford-nlp","machine-learning","argument-validation"],"backgroundTag":"invalid-argument-value","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}