HumanSignal/label-studio · error · ValidationError

validation_errors

Error message

validation_errors

What it means

At the end of _create_legacy, all validation errors collected while iterating prediction items are raised together as ValidationError(validation_errors) when the prediction-validation feature flag is enabled; otherwise they are only logged. This blocks prediction creation so invalid items cannot be silently skipped.

Source

Thrown at label_studio/data_import/api.py:665

            # If prediction is valid, add it to predictions list to be created
            try:
                predictions.append(
                    Prediction(
                        task_id=item['task'],
                        project_id=project.id,
                        result=Prediction.prepare_prediction_result(item.get('result'), project),
                        score=item.get('score'),
                        model_version=item.get('model_version', 'undefined'),
                    )
                )
            except Exception as e:
                validation_errors.append(f'Prediction {i}: Failed to create prediction - {extract_message(e)}')
                continue

        # If there are validation errors, raise them before creating any predictions
        if validation_errors:
            if flag_set('fflag_feat_utc_210_prediction_validation_15082025', user='auto'):
                raise ValidationError(validation_errors)
            else:
                logger.error(f'Prediction validation failed ({len(validation_errors)} errors):\n{validation_errors}')

        predictions_obj = Prediction.objects.bulk_create(predictions, batch_size=settings.BATCH_SIZE)
        start_job_async_or_sync(update_tasks_counters, Task.objects.filter(id__in=tasks_ids))
        return Response({'created': len(predictions_obj)}, status=status.HTTP_201_CREATED)


@extend_schema(exclude=True)
class TasksBulkCreateAPI(ImportAPI):
    # just for compatibility - can be safely removed
    pass


class ReImportAPI(ImportAPI):
    permission_required = all_permissions.projects_change

    def sync_reimport(self, project, file_upload_ids, files_as_tasks_list):

View on GitHub (pinned to 0b49e9b539)

Solutions

1. Parse the returned array; each entry is prefixed with 'Prediction {i}:' identifying the failing item
2. Fix the listed items and resubmit
3. Pre-validate task IDs and label config compatibility before sending the payload
4. If behavior change is unacceptable, disable the feature flag (legacy: errors logged, bad items skipped) — not recommended
5. Filter predictions to only those passing validation client-side before import

Example fix

# before: send everything, get array failure
resp = requests.post(url, json=predictions)
# after: validate per-item against task existence first
existing = {t['id'] for t in get_tasks(project_id)}
clean = [p for p in predictions if p.get('task') in existing]
resp = requests.post(url, json=clean)

existing_ids = {t['id'] for t in paginate(f'{LS}/api/projects/{pid}/tasks')}
clean = [p for p in predictions if p.get('task') in existing_ids]
assert clean == predictions, 'some predictions reference missing tasks and will fail validation'

try:
    resp = requests.post(import_url, headers=H, json=predictions)
    resp.raise_for_status()
except requests.HTTPError as e:
    errors = e.response.json()
    if isinstance(errors, list):
        bad_idx = {int(m.split(':')[1].strip().replace('Prediction ', '')) for m in errors if m.startswith('Prediction ')}
        log.error('Failing prediction indices: %s', sorted(bad_idx))

When it happens

Trigger: POSTing multiple predictions where at least one fails validation (invalid task ID, failed creation, label-config mismatch). Aggregated errors are raised pre-create when fflag_feat_utc_210_prediction_validation_15082025 is set.

Common situations: Bulk prediction re-imports after label config changes; exports containing items whose tasks were deleted; behavior change after enabling the flag — previously logged-and-skipped items now abort the whole request with an error array.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Prediction validation failed ({len(validation_errors)} error
• {item} contains invalid "task" field: task ID {task_id} not
• batch_validation_errors
• "url" must be 2048 characters or fewer
• {item} contains invalid "task" field: corresponding task ID

AI-assisted analysis of HumanSignal/label-studio@0b49e9b539 (2026-08-29). Data as JSON: /api/errors/a7a8400310894dbb. Report an issue: GitHub.