HumanSignal/label-studio · error · ValidationError

Created annotations are incompatible with provided labeling

Error message

Created annotations are incompatible with provided labeling schema, we found:\n{diff_str}

What it means

Label Studio validates that annotations already created on tasks are still compatible with a newly submitted labeling config. When the new config changes a control tag (name, to_name, or type) so that existing annotation results no longer match the schema, this ValidationError is raised listing the incompatible results via diff_str. It prevents configs from silently invalidating existing annotations.

Source

Thrown at label_studio/projects/models.py:717

            for ann_tuple in different_annotations:
                from_name, to_name, t = ann_tuple.split('|')
                # TODO tags that operate as both object and control tags; should be special registry/logic for them
                if from_name == to_name and t.lower() == 'chatmessage':
                    continue
                if t.lower() == 'textarea':  # avoid textarea to_name check (see DEV-1598)
                    continue
                if (
                    not check_control_in_config_by_regex(config_string, from_name)
                    or not check_toname_in_config_by_regex(config_string, to_name)
                    or t not in get_all_types(config_string)
                ):
                    diff_str.append(
                        f'{self.summary.created_annotations[ann_tuple]} '
                        f'with from_name={from_name}, to_name={to_name}, type={t}'
                    )
            if len(diff_str) > 0:
                diff_str = '\n'.join(diff_str)
                raise ValidationError(
                    f'Created annotations are incompatible with provided labeling schema, we found:\n{diff_str}'
                )

        # validate labels consistency
        labels_from_config, dynamic_label_from_config = get_all_labels(config_string)
        created_labels = merge_labels_counters(self.summary.created_labels, self.summary.created_labels_drafts)

        def display_count(count: int, type: str) -> Optional[str]:
            """Helper for displaying pluralized sources of validation errors,
            eg "1 draft" or "3 annotations"
            """
            if not count:
                return None
            return f'{count} {type}{"s" if count > 1 else ""}'

        parsed_config = parse_config(config_string)
        tag_types = {tag_info['type'] for _, tag_info in parsed_config.items()}

View on GitHub (pinned to 0b49e9b539)

Solutions

1. Keep the original control tag names and types in the new config so existing annotation results stay valid
2. Delete or export-and-reset the incompatible annotations before applying the new config
3. If labels were merely renamed intentionally, migrate existing annotation results to the new names first (update result JSON from_name values)
4. Add the old tag back alongside the new one (keep both tags in config) so old results remain schema-valid

Example fix

// before
<View>
  <Text name='transcription' value='text'/>
  <Choices name='sentiment' toName='transcription'>...</Choices>
</View>
// after (keep 'sentiment' name/type; only change labels inside)
<View>
  <Text name='transcription' value='text'/>
  <Choices name='sentiment' toName='transcription'>
    <Choice value='Positive'/><Choice value='Negative'/><Choice value='Neutral'/>
  </Choices>
</View>

// Before saving config, diff control tags against existing annotations
const results = await api.get(`/api/projects/${projectId}/tasks`, {params:{page_size:1}});
if (results.count > 0) {
  // ensure all from_name/to_name/type combos used in annotations are unchanged
  const annotations = await api.get('/api/annotations', {params:{project: projectId}});
  const used = new Set(annotations.results.flatMap(a => a.result.map(r => `${r.from_name}|${r.to_name}|${r.type}`)));
  // parse new config and verify each tag in `used` still exists with same name/type
}

function keepsExistingControlTags(oldConfig, newConfig, usedTags) {
  return usedTags.every(t => newConfig.includes(`name="${t.fromName}"`));
}

try {
  await api.post(`/api/projects/${projectId}/validate-label-config`, {label_config: newConfig});
} catch (e) {
  if (e.response?.status === 400 && /incompatible with provided labeling schema/.test(e.response.data?.config ?? '')) {
    // surface diff to user; do not save until tags preserved or annotations reset
  } else throw e;
}

When it happens

Trigger: Calling POST /api/projects/<id>/validate-label-config or saving a project config (Project.validate_config) after annotations exist while renaming a from_name, changing to_name, or changing the control type (e.g. 'choices' to 'textarea') for results already stored.

Common situations: Editing an XML label config to rename a <Choices name='label'> tag after annotators have already submitted results; switching a control type mid-project; importing a project config from a template without checking existing annotations.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• There are {count} annotation(s) created with tag "{control_t
• Wrong old label name, it is not from labeling config: {old_l
• These labels still exist in annotations or drafts:\n{diff_st
• This task cannot be skipped.
• Error validating annotation: {validation_errors}

AI-assisted analysis of HumanSignal/label-studio@0b49e9b539 (2026-08-29). Data as JSON: /api/errors/a985c190664fa547. Report an issue: GitHub.