HumanSignal/label-studio · error · ValidationError

batch_validation_errors

Error message

batch_validation_errors

What it means

At the end of each batch in _create_memory_efficient, accumulated per-batch prediction validation errors (invalid task references, custom interface errors, etc.) are raised together as ValidationError(batch_validation_errors). No predictions in the batch are created if any item failed validation.

Source

Thrown at label_studio/data_import/api.py:576

                custom_interface_errors = self._validate_custom_interface_prediction(project, item, prediction_index)
                if custom_interface_errors:
                    batch_validation_errors.extend(custom_interface_errors)
                    continue

                batch_predictions.append(
                    Prediction(
                        task_id=task_id,
                        project_id=project.id,
                        result=Prediction.prepare_prediction_result(item.get('result'), project),
                        score=item.get('score'),
                        model_version=item.get('model_version', 'undefined'),
                    )
                )
                all_task_ids.add(task_id)

            if batch_validation_errors:
                raise ValidationError(batch_validation_errors)

            # Bulk create this batch with the configured batch size
            batch_created = Prediction.objects.bulk_create(batch_predictions, batch_size=settings.BATCH_SIZE)
            total_created += len(batch_created)

            logger.debug(
                f'Processed batch {batch_start}-{batch_end - 1}: created {len(batch_created)} predictions '
                f'(total so far: {total_created})'
            )

        # Update task counters for all affected tasks
        # Only pass the unique task IDs that were actually processed
        if all_task_ids:
            start_job_async_or_sync(update_tasks_counters, Task.objects.filter(id__in=all_task_ids))

        return Response({'created': total_created}, status=status.HTTP_201_CREATED)

    def _create_legacy(self, project):

View on GitHub (pinned to 0b49e9b539)

Solutions

  1. Iterate the returned error array; each message identifies the failing prediction index/item
  2. Fix or drop the offending items and resubmit the batch
  3. Split large payloads into smaller chunks to isolate which items fail
  4. Validate one item first (dry run) before sending the full batch
  5. Cross-check all 'task' IDs against GET /api/projects/{id}/tasks before importing

Example fix

# before: fire full batch, get all-or-nothing failure
requests.post(url, json=predictions)
# after: pre-filter to predictions whose tasks exist
task_ids = {t['id'] for t in get_all_tasks(project_id)}
valid = [p for p in predictions if p.get('task') in task_ids]
requests.post(url, json=valid)
Defensive patterns

Strategy: validation

Validate before calling

existing_ids = {t['id'] for t in paginate(f'{LS}/api/projects/{pid}/tasks')}
valid = [p for p in batch if isinstance(p.get('task'), int) and p['task'] in existing_ids]
assert len(valid) == len(batch), f'{len(batch)-len(valid)} predictions would fail batch validation'

Type guard

def batch_is_clean(batch: list, valid_ids: set) -> bool:
    return all(isinstance(p.get('task'), int) and p['task'] in valid_ids for p in batch)

Try / catch

try:
    resp = requests.post(import_url, headers=H, json=batch)
    resp.raise_for_status()
except requests.HTTPError as e:
    errors = e.response.json()
    if isinstance(errors, list):
        log.error('%d/%d items rejected, resubmitting valid ones', len(errors), len(batch))

Prevention

When it happens

Trigger: POSTing a batch of predictions to the import/predictions endpoint where one or more items fail validation (bad 'task' ID, invalid result for the project's custom interface). The whole batch is rejected with an array of error messages.

Common situations: Bulk imports mixing valid and invalid prediction items — the entire batch fails even though most items are fine; label-config mismatch introduced mid-export; partial migrations where some tasks were deleted before predictions were re-imported.

Related errors

AI-assisted analysis of HumanSignal/label-studio@0b49e9b539 (2026-08-29). Data as JSON: /api/errors/3820b38bf00afea0. Report an issue: GitHub.