mlflow/mlflow · error · MlflowException

Template variables mismatch in prompt '{name}'. Missing: {mi

Error message

Template variables mismatch in prompt '{name}'. Missing: {missing}.

What it means

_validate_template_variables compares f-string style template variables (via string.Formatter) between original and improved prompts. If the improved prompt dropped a variable that the original required, this error names the prompt and the missing variables. It prevents silently breaking prompts whose variables are filled programmatically at inference time.

Source

Thrown at mlflow/genai/optimize/optimizers/metaprompt_optimizer.py:428

    def _validate_template_variables(
        self, original_prompts: dict[str, str], new_prompts: dict[str, str]
    ) -> bool:
        """Validate that all template variables are preserved in new prompts.

        Extra variables introduced by the LLM are automatically stripped from the
        generated prompt so that optimisation can succeed even when the model
        erroneously adds new ``{{variable}}`` patterns.
        """
        original_vars = self._extract_template_variables(original_prompts)
        new_vars = self._extract_template_variables(new_prompts)

        for name in original_prompts:
            missing = original_vars[name] - new_vars[name]
            extra = new_vars[name] - original_vars[name]

            if missing:
                raise MlflowException(
                    f"Template variables mismatch in prompt '{name}'. Missing: {missing}."
                )

            if extra:
                _logger.warning(f"Stripping extra template variables {extra} from prompt '{name}'.")
                text = new_prompts[name]
                for var in extra:
                    text = text.replace("{{" + var + "}}", "")
                new_prompts[name] = text

        return True

    def _build_zero_shot_meta_prompt(
        self,
        current_prompts: dict[str, str],
        template_variables: dict[str, set[str]],
    ) -> str:
        # Format the current prompts for each module

View on GitHub (pinned to 6a27f2decc)

Solutions

  1. Adjust guidelines to instruct the reflection model to preserve all template variables
  2. Re-add the missing variables manually to the improved prompt before use
  3. Catch MlflowException and fall back to the original prompt text
  4. Review the improved prompt in the error message and edit it to include the missing {vars}

Example fix

// before
prompts = optimizer.optimize(eval_fn, data, {'q': 'Answer {question} in {language}'})
// after
try:
    prompts = optimizer.optimize(eval_fn, data, {'q': 'Answer {question} in {language}'})
except MlflowException as e:
    if 'Template variables mismatch' in str(e):
        prompts = {'q': 'Answer {question} in {language}'}  # keep original
    else:
        raise
Defensive patterns

Strategy: try-catch

Validate before calling

import string
vars = {n: {v for _, v, _, _ in string.Formatter().parse(t) if v} for n, t in target_prompts.items()}

Type guard

def preserved_vars(orig: str, new: str) -> bool:
    import string
    ex = lambda s: {f for _, f, _, _ in string.Formatter().parse(s) if f}
    return ex(orig) <= ex(new)

Try / catch

try:
    prompts = optimizer.optimize(eval_fn, train_data, target_prompts)
except MlflowException as e:
    if 'Template variables mismatch' in str(e):
        prompts = dict(target_prompts)
    else:
        raise

Prevention

When it happens

Trigger: optimize() produced an improved prompt that no longer contains a variable present in the original, e.g. original 'Answer {question} about {context}' improved to 'Answer {question}'.

Common situations: The reflection model paraphrases and removes placeholders; developers then hit KeyError at runtime when formatting the optimized prompt with their data.

Related errors

AI-assisted analysis of mlflow/mlflow@6a27f2decc (2026-08-29). Data as JSON: /api/errors/8a506ce91f498b87. Report an issue: GitHub.