stanfordnlp/CoreNLP · error · IllegalArgumentException

shuffleWithSideInformation: sideInformation not of same…

Error message

shuffleWithSideInformation: sideInformation not of same size as Dataset

What it means

shuffleWithSideInformation randomly permutes the dataset in place and applies the same permutation to a parallel list of side information (e.g. raw text per datum). If the list's size differs from the dataset size, the permutation can't be applied consistently, so it throws IllegalArgumentException immediately before shuffling.

Solutions

1. Make the side information list exactly parallel to the dataset: apply the same filtering/subsampling to both before shuffling
2. Check sizes first with an assert/exception of your own that also logs both sizes for diagnosis
3. Derive side information from the dataset itself (one entry per datum) rather than maintaining a separate list
4. If only the data should be shuffled, call shuffle() instead of shuffleWithSideInformation

Example fix

// before
List<String> docs = readAllLines("docs.txt");
ds = ds.subDataset(0, 1000); // dataset filtered
ds.shuffleWithSideInformation(42, docs); // size mismatch
// after
List<String> docs = readAllLines("docs.txt").subList(0, 1000);
assert docs.size() == ds.size();
ds.shuffleWithSideInformation(42, docs);

if (sideInformation.size() != dataset.size()) {
  throw new IllegalArgumentException("side info size " + sideInformation.size() + " != dataset size " + dataset.size());
}
dataset.shuffleWithSideInformation(seed, sideInformation);

try {
  ds.shuffleWithSideInformation(seed, sideInfo);
} catch (IllegalArgumentException e) {
  logger.severe("Size mismatch: dataset=" + ds.size() + " side=" + sideInfo.size());
  throw e;
}

When it happens

Trigger: Calling dataset.shuffleWithSideInformation(seed, sideInfo) where sideInfo.size() != dataset.size — e.g. after filtering datums but not the side list, or building the side list from a different file than the dataset.

Common situations: Filtering or subsampling the RVFDataset (removing datums) while keeping the original full side-information list; accidentally shuffling twice; loading dataset and side info from files with different line counts.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Not sure if RVFDataset runs correctly in this method…
• minValue for feature
• maxValue for feature
• datum
• Bad data format:

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/8eb94e3bf739a1d7. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/classify/RVFDataset.java:1034

      int tmpl = labels[randIndex];
      labels[randIndex] = labels[j];
      labels[j] = tmpl;

      double[] tmpv = values[randIndex];
      values[randIndex] = values[j];
      values[j] = tmpv;
    }
  }

  /**
   * Randomizes the data array in place. Needs to be redefined here because we
   * need to randomize the values as well.
   */
  @Override
  public <E> void shuffleWithSideInformation(long randomSeed, List<E> sideInformation) {
    if (size != sideInformation.size()) {
      throw new IllegalArgumentException("shuffleWithSideInformation: sideInformation not of same size as Dataset");
    }
    Random rand = new Random(randomSeed);
    for (int j = size - 1; j > 0; j--) {
      int randIndex = rand.nextInt(j);

      int[] tmp = data[randIndex];
      data[randIndex] = data[j];
      data[j] = tmp;

      int tmpl = labels[randIndex];
      labels[randIndex] = labels[j];
      labels[j] = tmpl;

      double[] tmpv = values[randIndex];
      values[randIndex] = values[j];
      values[j] = tmpv;

      E tmpE = sideInformation.get(randIndex);

View on GitHub (pinned to 1b7edd19c4)