tensorflow/models · error · ValueError

epochs_between_evals > 1 not supported for file based datase

Error message

epochs_between_evals > 1 not supported for file based dataset.

What it means

Error "epochs_between_evals > 1 not supported for file based dataset." thrown in tensorflow/models.

Source

Thrown at official/recommendation/data_pipeline.py:289

  def increment_request_epoch(self):
    self._epochs_requested += 1

  def get_dataset(self, batch_size, epochs_between_evals):
    """Construct the dataset to be used for training and eval.

    For local training, data is provided through Dataset.from_generator. For
    remote training (TPUs) the data is first serialized to files and then sent
    to the TPU through a StreamingFilesDataset.

    Args:
      batch_size: The per-replica batch size of the dataset.
      epochs_between_evals: How many epochs worth of data to yield. (Generator
        mode only.)
    """
    self.increment_request_epoch()
    if self._stream_files:
      if epochs_between_evals > 1:
        raise ValueError("epochs_between_evals > 1 not supported for file "
                         "based dataset.")
      epoch_data_dir = self._result_queue.get(timeout=300)
      if not self._is_training:
        self._result_queue.put(epoch_data_dir)  # Eval data is reused.

      file_pattern = os.path.join(epoch_data_dir,
                                  rconst.SHARD_TEMPLATE.format("*"))
      dataset = StreamingFilesDataset(
          files=file_pattern,
          worker_job=popen_helper.worker_job(),
          num_parallel_reads=rconst.NUM_FILE_SHARDS,
          num_epochs=1,
          sloppy=not self._deterministic)
      map_fn = functools.partial(
          self.deserialize,
          batch_size=batch_size,
          is_training=self._is_training)
      dataset = dataset.map(map_fn, num_parallel_calls=16)

View on GitHub (pinned to e006f5f0d5)

When it happens

Trigger: Thrown at official/recommendation/data_pipeline.py:289 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of tensorflow/models@e006f5f0d5 (2026-08-24). Data as JSON: /api/errors/05456732d60f15b5. Report an issue: GitHub.