HumanSignal/label-studio · error · ValidationError

There are {count} annotation(s) created with tag "{control_t

Error message

There are {count} annotation(s) created with tag "{control_tag_from_data}", you can't remove it

What it means

When validating a new labeling config, Label Studio checks that control tags (from_name) referenced by existing annotations still exist in the config. If annotations were created with a tag that is absent from the new config (and not matched by dynamic label/regex rules), this ValidationError is raised, because removing the tag would orphan those annotation results.

Source

Thrown at label_studio/projects/models.py:769

            for sibling_tag, sibling_tag_info in parsed_config.items():
                if sibling_tag_info.get('type') != 'Labels':
                    continue
                if control_to_names.intersection(sibling_tag_info.get('to_name') or []):
                    labels_from_config_by_tag |= set(labels_from_config.get(sibling_tag, []))
            return labels_from_config_by_tag

        for control_tag_from_data, labels_from_data in created_labels.items():
            # Check if labels created in annotations, and their control tag has been removed
            if (
                labels_from_data
                and (
                    (control_tag_from_data not in labels_from_config)
                    and (control_tag_from_data not in dynamic_label_from_config)
                )
                and not check_control_in_config_by_regex(config_string, control_tag_from_data)
            ):
                raise ValidationError(
                    f'There are {sum(labels_from_data.values(), 0)} annotation(s) created with tag '
                    f'"{control_tag_from_data}", you can\'t remove it'
                )
            control_tag_from_config = get_original_fromname_by_regex(config_string, control_tag_from_data)
            labels_from_config_by_tag = set(labels_from_config[control_tag_from_config])
            labels_from_config_by_tag = add_separated_video_object_labels(
                control_tag_from_config, labels_from_config_by_tag
            )
            if 'Taxonomy' in tag_types:
                custom_tags = Label.objects.filter(links__project=self).values_list('value', flat=True)
                flat_custom_tags = set([item for sublist in custom_tags for item in sublist])
                labels_from_config_by_tag |= flat_custom_tags
            # check if labels from is subset if config labels
            if not set(labels_from_data).issubset(set(labels_from_config_by_tag)):
                different_labels = list(set(labels_from_data).difference(labels_from_config_by_tag))
                diff_str = ''
                for label in different_labels:
                    annotation_label_count = self.summary.created_labels.get(control_tag_from_data, {}).get(label, 0)

View on GitHub (pinned to 0b49e9b539)

Solutions

1. Re-add the control tag with that name to the labeling config
2. Delete the annotations that use the removed tag before saving the new config
3. Use dynamic labeling (regex) so removed concrete labels are tolerated while the tag itself stays
4. Export data first, then wipe project annotations before switching configs

Example fix

// before (tag removed)
<View>
  <Image name='image' value='$img'/>
  <Choices name='quality' toName='image'>...</Choices>
</View>
// after (restore tag that annotations reference)
<View>
  <Image name='image' value='$img'/>
  <Choices name='label' toName='image'>...</Choices>
  <Choices name='quality' toName='image'>...</Choices>
</View>

// Verify every control tag referenced by annotations exists in the new config
const annotations = await api.get('/api/annotations', {params:{project: projectId}});
const requiredTags = new Set(annotations.results.flatMap(a => a.result.map(r => r.from_name)));
for (const tag of requiredTags) {
  if (!newConfig.includes(`name="${tag}"`)) {
    throw new Error(`Config must keep control tag: ${tag}`);
  }
}

function configKeepsAllTags(config, tags) {
  return tags.every(tag => config.includes(`name="${tag}"`));
}

try {
  await projectApi.update(projectId, {label_config: newConfig});
} catch (e) {
  if (e.response?.status === 400 && /annotation\(s\) created with tag/.test(JSON.stringify(e.response.data))) {
    const tag = JSON.stringify(e.response.data).match(/tag \\"([^\\"]+)\\\\"/)?.[1];
    // re-add tag `${tag}` to config or delete related annotations, then retry
  } else throw e;
}

When it happens

Trigger: POST /api/projects/<id>/validate-label-config or project save where sum(labels_from_data.values()) > 0 (annotations exist for that control tag) and the tag name no longer appears in config_string nor in dynamic labels nor matches check_control_in_config_by_regex.

Common situations: Deleting a <Choices> or <Label> tag from the XML config after annotators used it; replacing a config wholesale with a new template; cleaning up 'unused' tags that actually still have annotation data.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Created annotations are incompatible with provided labeling
• Wrong old label name, it is not from labeling config: {old_l
• These labels still exist in annotations or drafts:\n{diff_st
• This task cannot be skipped.
• Error validating annotation: {validation_errors}

AI-assisted analysis of HumanSignal/label-studio@0b49e9b539 (2026-08-29). Data as JSON: /api/errors/f6cb834d6f973374. Report an issue: GitHub.