HumanSignal/label-studio · error · ValidationError

These labels still exist in annotations or drafts:\n{diff_st

Error message

These labels still exist in annotations or drafts:\n{diff_str}Please add labels to tag with name="{control_tag_from_data}".

What it means

After checking that the control tag still exists, Label Studio verifies that labels recorded in existing annotations or drafts still appear in the new config's label set. If specific labels (choices/values) used in annotations were removed from the tag and are neither dynamic nor matched by regex rules, this ValidationError lists the offending labels in diff_str.

Source

Thrown at label_studio/projects/models.py:805

                    annotation_label_count = self.summary.created_labels.get(control_tag_from_data, {}).get(label, 0)
                    draft_label_count = self.summary.created_labels_drafts.get(control_tag_from_data, {}).get(label, 0)
                    annotation_display_count = display_count(annotation_label_count, 'annotation')
                    draft_display_count = display_count(draft_label_count, 'draft')

                    display = [disp for disp in [annotation_display_count, draft_display_count] if disp]
                    if display:
                        diff_str += f'{label} ({", ".join(display)})\n'

                if (strict is True) and (
                    (control_tag_from_data not in dynamic_label_from_config)
                    and (
                        not check_control_in_config_by_regex(
                            config_string, control_tag_from_data, filter=dynamic_label_from_config.keys()
                        )
                    )
                ):
                    # raise error if labels not dynamic and not in regex rules
                    raise ValidationError(
                        f'These labels still exist in annotations or drafts:\n{diff_str}'
                        f'Please add labels to tag with name="{str(control_tag_from_data)}".'
                    )
                else:
                    logger.info(f'project_id={self.id} inconsistent labels in config and annotations: {diff_str}')

    def _label_config_has_changed(self):
        return self.label_config != self.__original_label_config

    @property
    def label_config_is_not_default(self):
        return self.label_config != Project._meta.get_field('label_config').default

    def should_none_model_version(self, model_version):
        """
        Returns True if the model version provided matches the object's model version,
        or no model version is set for the object but model version exists in ML backend.
        """

View on GitHub (pinned to 0b49e9b539)

Solutions

1. Re-add the missing labels to the control tag in the config
2. Convert the tag to dynamic labeling (dynamic labels / regex) so stored labels remain valid
3. Delete or rewrite the annotations/drafts that use the removed labels
4. Archive the old data and start a fresh project with the trimmed label set

Example fix

// before
<Choices name='sentiment' toName='text'>
  <Choice value='Positive'/><Choice value='Negative'/>
</Choices>
// after (re-add label still present in annotations)
<Choices name='sentiment' toName='text'>
  <Choice value='Positive'/><Choice value='Negative'/><Choice value='Neutral'/>
</Choices>

// Compare labels used in annotations against labels declared in the new config
const annotations = await api.get('/api/annotations', {params:{project: projectId}});
const usedLabels = new Set(annotations.results.flatMap(a =>
  a.result.flatMap(r => Object.values(r.value ?? {}).flat())));
const declaredLabels = new Set([...newConfig.matchAll(/value="([^"]+)"/g)].map(m => m[1]));
const missing = [...usedLabels].filter(l => !declaredLabels.has(l));
if (missing.length) throw new Error(`Add missing labels to config: ${missing.join(', ')}`);

function allUsedLabelsDeclared(used, declared) {
  return used.every(l => declared.has(l));
}

try {
  await projectApi.update(projectId, {label_config: newConfig});
} catch (e) {
  if (e.response?.status === 400 && /labels still exist in annotations or drafts/.test(JSON.stringify(e.response.data))) {
    // parse listed labels from the message and re-add them or migrate annotations
  } else throw e;
}

When it happens

Trigger: POST /api/projects/<id>/validate-label-config or project save where annotation/draft results contain labels (e.g. choice value 'Maybe') that are missing from the corresponding control tag in the new config and dynamic_label_from_config does not cover them.

Common situations: Trimming a list of <Choice> options from 5 to 3 after annotation has begun; renaming <Label> values in a segmentation template; replacing a taxonomy with a shorter one.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Wrong old label name, it is not from labeling config: {old_l
• Created annotations are incompatible with provided labeling
• There are {count} annotation(s) created with tag "{control_t
• Maximum task number is {settings.TASKS_MAX_NUMBER}, current
• Maximum total size of all files is {settings.TASKS_MAX_FILE_

AI-assisted analysis of HumanSignal/label-studio@0b49e9b539 (2026-08-29). Data as JSON: /api/errors/37dca607bd5f9f49. Report an issue: GitHub.