HumanSignal/label-studio · error · ValidationError

Created annotations are incompatible with provided labeling…

Error message

Created annotations are incompatible with provided labeling schema, we found:\n{diff_str}

What it means

Label Studio validates that annotations already created on tasks are still compatible with a newly submitted labeling config. When the new config changes a control tag (name, to_name, or type) so that existing annotation results no longer match the schema, this ValidationError is raised listing the incompatible results via diff_str. It prevents configs from silently invalidating existing annotations.

Solutions

1. Keep the original control tag names and types in the new config so existing annotation results stay valid
2. Delete or export-and-reset the incompatible annotations before applying the new config
3. If labels were merely renamed intentionally, migrate existing annotation results to the new names first (update result JSON from_name values)
4. Add the old tag back alongside the new one (keep both tags in config) so old results remain schema-valid

Example fix

// before
<View>
  <Text name='transcription' value='text'/>
  <Choices name='sentiment' toName='transcription'>...</Choices>
</View>
// after (keep 'sentiment' name/type; only change labels inside)
<View>
  <Text name='transcription' value='text'/>
  <Choices name='sentiment' toName='transcription'>
    <Choice value='Positive'/><Choice value='Negative'/><Choice value='Neutral'/>
  </Choices>
</View>

// Before saving config, diff control tags against existing annotations
const results = await api.get(`/api/projects/${projectId}/tasks`, {params:{page_size:1}});
if (results.count > 0) {
  // ensure all from_name/to_name/type combos used in annotations are unchanged
  const annotations = await api.get('/api/annotations', {params:{project: projectId}});
  const used = new Set(annotations.results.flatMap(a => a.result.map(r => `${r.from_name}|${r.to_name}|${r.type}`)));
  // parse new config and verify each tag in `used` still exists with same name/type
}

function keepsExistingControlTags(oldConfig, newConfig, usedTags) {
  return usedTags.every(t => newConfig.includes(`name="${t.fromName}"`));
}

try {
  await api.post(`/api/projects/${projectId}/validate-label-config`, {label_config: newConfig});
} catch (e) {
  if (e.response?.status === 400 && /incompatible with provided labeling schema/.test(e.response.data?.config ?? '')) {
    // surface diff to user; do not save until tags preserved or annotations reset
  } else throw e;
}

When it happens

Trigger: Calling POST /api/projects/<id>/validate-label-config or saving a project config (Project.validate_config) after annotations exist while renaming a from_name, changing to_name, or changing the control type (e.g. 'choices' to 'textarea') for results already stored.

Common situations: Editing an XML label config to rename a <Choices name='label'> tag after annotators have already submitted results; switching a control type mid-project; importing a project config from a template without checking existing annotations.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• There are annotation(s) created with tag " ", you can't…
• Error validating annotation
• These labels still exist in annotations or drafts:\n
• This task cannot be skipped.
• Wrong old label name, it is not from labeling config

AI-assisted analysis of HumanSignal/label-studio@0b49e9b539 (2026-08-29). Data as JSON: /api/errors/a985c190664fa547. Report an issue: GitHub.

Appendix: source

Thrown at label_studio/projects/models.py:717

            for ann_tuple in different_annotations:
                from_name, to_name, t = ann_tuple.split('|')
                # TODO tags that operate as both object and control tags; should be special registry/logic for them
                if from_name == to_name and t.lower() == 'chatmessage':
                    continue
                if t.lower() == 'textarea':  # avoid textarea to_name check (see DEV-1598)
                    continue
                if (
                    not check_control_in_config_by_regex(config_string, from_name)
                    or not check_toname_in_config_by_regex(config_string, to_name)
                    or t not in get_all_types(config_string)
                ):
                    diff_str.append(
                        f'{self.summary.created_annotations[ann_tuple]} '
                        f'with from_name={from_name}, to_name={to_name}, type={t}'
                    )
            if len(diff_str) > 0:
                diff_str = '\n'.join(diff_str)
                raise ValidationError(
                    f'Created annotations are incompatible with provided labeling schema, we found:\n{diff_str}'
                )

        # validate labels consistency
        labels_from_config, dynamic_label_from_config = get_all_labels(config_string)
        created_labels = merge_labels_counters(self.summary.created_labels, self.summary.created_labels_drafts)

        def display_count(count: int, type: str) -> Optional[str]:
            """Helper for displaying pluralized sources of validation errors,
            eg "1 draft" or "3 annotations"
            """
            if not count:
                return None
            return f'{count} {type}{"s" if count > 1 else ""}'

        parsed_config = parse_config(config_string)
        tag_types = {tag_info['type'] for _, tag_info in parsed_config.items()}

View on GitHub (pinned to 0b49e9b539)