HumanSignal/label-studio · error · ValidationError

Prediction validation failed ({len(validation_errors)} error

Error message

Prediction validation failed ({len(validation_errors)} errors):\n- {error}\n

What it means

Label Studio's sync_import (POST /api/projects/{id}/import) validates prediction payloads against the project's label config when predictions are included with imported tasks. When any prediction fails validation and the feature flag 'fflag_feat_utc_210_prediction_validation_15082025' is enabled, it aggregates all validation errors into a single DRF ValidationError listing each problem on its own line. Without the flag the errors are only logged and the import proceeds.

Source

Thrown at label_studio/data_import/api.py:317

            for i, task in enumerate(parsed_data):
                if 'predictions' in task:
                    for j, prediction in enumerate(task['predictions']):
                        try:
                            validation_errors_list = li.validate_prediction(prediction, return_errors=True)
                            if validation_errors_list:
                                for error in validation_errors_list:
                                    validation_errors.append(f'Task {i}, prediction {j}: {error}')
                        except Exception as e:
                            error_msg = f'Task {i}, prediction {j}: Error validating prediction - {extract_message(e)}'
                            validation_errors.append(error_msg)

            if validation_errors:
                error_message = f'Prediction validation failed ({len(validation_errors)} errors):\n'
                for error in validation_errors:
                    error_message += f'- {error}\n'

                if flag_set('fflag_feat_utc_210_prediction_validation_15082025', user='auto'):
                    raise ValidationError({'predictions': [error_message]})
                else:
                    logger.error(
                        f'Prediction validation failed, not raising error - ({len(validation_errors)} errors):\n{error_message}'
                    )

        if commit_to_project:
            # Immediately create project tasks and update project states and counters
            tasks, serializer = self._save(parsed_data)
            task_count = len(tasks)
            annotation_count = len(serializer.db_annotations)
            prediction_count = len(serializer.db_predictions)

            recalculate_stats_counts = {
                'task_count': task_count,
                'annotation_count': annotation_count,
                'prediction_count': prediction_count,
            }

View on GitHub (pinned to 0b49e9b539)

Solutions

  1. Read the per-line error messages in the response; each '- {error}' names the offending prediction field
  2. Compare the prediction result keys (from_name, to_name, labels) against the project's labeling config via GET /api/projects/{id}/
  3. Fix or remove the invalid predictions from the import payload and retry
  4. If legacy behavior is needed, ask an admin to disable fflag_feat_utc_210_prediction_validation_15082025 (not recommended)
  5. Use POST /api/projects/{id}/validate or the label-config validation endpoint to pre-check payload compatibility

Example fix

// before: prediction labels don't match config
{"data": {"text": "hi"}, "predictions": [{"result": [{"from_name": "sentiment", "to_name": "text", "type": "choices", "value": {"choices": ["happy"]}}]}]}
// after: labels match <Choices name="sentiment"> <Choice value="positive"/> ...
{"data": {"text": "hi"}, "predictions": [{"result": [{"from_name": "sentiment", "to_name": "text", "type": "choices", "value": {"choices": ["positive"]}}]}]}
Defensive patterns

Strategy: try-catch

Validate before calling

import re, requests
cfg = requests.get(f'{LS}/api/projects/{pid}/', headers=H).json()['label_config']
labels = set(re.findall(r'<Choice\s+value="([^"]+)"', cfg))
for t in tasks:
    for p in t.get('predictions', []):
        for r in p.get('result', []):
            for ch in r.get('value', {}).get('choices', []):
                assert ch in labels, f'label {ch!r} not in config'

Try / catch

try:
    requests.post(f'{LS}/api/projects/{pid}/import', headers=H, json=tasks).raise_for_status()
except requests.HTTPError as e:
    detail = e.response.json()
    msg = detail.get('predictions', detail.get('detail', ''))
    lines = [l[2:] for l in str(msg).splitlines() if l.startswith('- ')]
    log.error('Prediction validation failed: %s', lines)

Prevention

When it happens

Trigger: POSTing tasks with a 'predictions' array to the import endpoint where a prediction references a result whose from_name/to_name/labels do not match the project labeling config, or has a malformed result structure. Only raised when commit_to_project is true and the prediction-validation feature flag is on.

Common situations: Re-importing annotations exported from a project whose label config was changed since export; programmatic pre-annotation generation with wrong label names; migrating projects between instances with different label configs; an org recently enabling the prediction-validation flag so imports that previously 'worked' (silently skipping bad predictions) start failing.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of HumanSignal/label-studio@0b49e9b539 (2026-08-29). Data as JSON: /api/errors/21da433614299c6a. Report an issue: GitHub.