tensorflow/models · error · ValueError

Can't evaluate using annotation file when TFDS is used.

Error message

Can't evaluate using annotation file when TFDS is used.

What it means

Config-conflict guard in RetinaNetTask.build_metrics: evaluation can load ground truth either from the TFDS source or from a separate annotation file; when both validation_data.tfds_name and annotation_file are set the two ground-truth paths would race, so eval-mode metric construction refuses the combination.

Solutions

  1. Disable the annotation-file-based COCO evaluator when training from TFDS.
  2. Use a file-based input (tfrecord) if you need evaluation against an annotation file.
Defensive patterns

Strategy: validation

When it happens

Trigger: Thrown at official/vision/tasks/retinanet.py:281 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of tensorflow/models@e006f5f0d5 (2026-08-24). Data as JSON: /api/errors/3b94af306eab8c85. Report an issue: GitHub.

Appendix: source

Thrown at official/vision/tasks/retinanet.py:281

      total_loss = model_loss + reg_loss

    total_loss = params.losses.loss_weight * total_loss

    return total_loss, cls_loss, box_loss, model_loss

  def build_metrics(self, training: bool = True):
    """Build detection metrics."""
    metrics = []
    metric_names = ['total_loss', 'cls_loss', 'box_loss', 'model_loss']
    for name in metric_names:
      metrics.append(tf_keras.metrics.Mean(name, dtype=tf.float32))

    if not training:
      if (
          self.task_config.validation_data.tfds_name
          and self.task_config.annotation_file
      ):
        raise ValueError(
            "Can't evaluate using annotation file when TFDS is used."
        )
      if self._task_config.use_coco_metrics:
        self.coco_metric = coco_evaluator.COCOEvaluator(
            annotation_file=self.task_config.annotation_file,
            include_mask=False,
            per_category_metrics=self.task_config.per_category_metrics,
            max_num_eval_detections=self.task_config.max_num_eval_detections,
        )
      if self._task_config.use_wod_metrics:
        # To use Waymo open dataset metrics, please install one of the pip
        # package `waymo-open-dataset-tf-*` from
        # https://github.com/waymo-research/waymo-open-dataset/blob/master/docs/quick_start.md#use-pre-compiled-pippip3-packages-for-linux
        # Note that the package is built with specific tensorflow version and
        # will produce error if it does not match the tf version that is
        # currently used.
        try:
          from official.vision.evaluation import wod_detection_evaluator  # pylint: disable=g-import-not-at-top

View on GitHub (pinned to e006f5f0d5)