OpenBMB/MiniCPM-V · warning

Could not find response key `{response_template}` in the fol

Error message

Could not find response key `{response_template}` in the following instance: @===>{tokenizer.decode(res_input_ids)}<===@ Raw text is @===>{res_text}<===@Raw source is @===>{new_source}<===@This instance will be ignored in loss calculation. Note, if this happens often, consider increasing the `max_seq_length`.

What it means

omni_preprocess (the collator used to build supervised labels) tokenizes each sample and searches res_labels for the response template token ids (the assistant/response key, e.g. '<|Assistant|>'). If none of the tokenized sequence contains the response key, it emits this UserWarning and the instance contributes no loss (its response positions are not masked in). It is a warning, not an exception, triggered because tokenized input was truncated or formatted so the response marker vanished.

Source

Thrown at omnilmm/train/train_utils.py:111

        conversations_tokenized = _tokenize_fn([res_text], tokenizer)
        res_input_ids = conversations_tokenized["input_ids"][0]

        # since labels and input_ids are reference towards the same object
        res_labels = copy.deepcopy(conversations_tokenized["labels"][0])

        response_token_ids_idxs = []
        human_token_ids_idxs = []

        for assistant_idx in np.where(res_labels == response_token_ids[0])[0]:
            # find the indexes of the start of a response.
            if (response_token_ids == res_labels[assistant_idx: assistant_idx + len(
                        response_token_ids)].tolist()
                    ):
                response_token_ids_idxs.append(
                    assistant_idx + len(response_token_ids))

        if len(response_token_ids_idxs) == 0:
            warnings.warn(
                f"Could not find response key `{response_template}` in the "
                f'following instance: @===>{tokenizer.decode(res_input_ids)}<===@ '
                f'Raw text is @===>{res_text}<===@'
                f'Raw source is @===>{new_source}<===@'
                f"This instance will be ignored in loss calculation. "
                f"Note, if this happens often, consider increasing the `max_seq_length`."
            )
            res_labels[:] = ignore_index

        human_token_ids = instruction_token_ids
        for human_idx in np.where(res_labels == human_token_ids[0])[0]:
            # find the indexes of the start of a human answer.
            if human_token_ids == res_labels[human_idx: human_idx + len(human_token_ids)].tolist():
                human_token_ids_idxs.append(human_idx)

        if len(human_token_ids_idxs) == 0:
            warnings.warn(
                f"Could not find instruction key `{instruction_template}` in the "

View on GitHub (pinned to 7a11e2bec4)

Solutions

1. Increase max_seq_length in the collator/training args so the response key survives tokenization.
2. Verify each sample actually contains the response template string in its text; fix data generation to include it.
3. Print tokenizer.decode(res_input_ids) from the warning to see what survived; check whether the response key tokenizes to the expected id subsequence with your current tokenizer version.
4. Filter or repair offending samples in the dataset before training if they are malformed.

Example fix

// before
collator = DataCollatorForActionPrediction(tokenizer=tokenizer, max_seq_length=1024)
// after
collator = DataCollatorForActionPrediction(tokenizer=tokenizer, max_seq_length=4096)

encoded = tokenizer(text).input_ids
resp_ids = tokenizer(response_template, add_special_tokens=False).input_ids
assert any(encoded[i:i+len(resp_ids)] == resp_ids for i in range(len(encoded))), "response key missing or truncated"

import warnings
with warnings.catch_warnings(record=True) as w:
    warnings.simplefilter("always")
    batch = collator(features)
    dropped = [str(x.message) for x in w if 'Could not find response key' in str(x.message)]
if dropped:
    logging.warning("%d samples skipped in loss: raise max_seq_length or fix data", len(dropped))

When it happens

Trigger: Calling the training collator (via omni_preprocess, e.g. from wrap_question_for_omni_lmm pipelines) with a sample whose tokenized res_input_ids does not contain response_template's token ids — typically because max_seq_length truncated the sequence before the response key, or the sample text never contains the response marker.

Common situations: max_seq_length too small relative to long sources (long image descriptions/sources), so the assistant turn is cut off; prompt built without the assistant/response special token; tokenizer version change altering how the response key tokenizes (merged differently, so exact id subsequence no longer appears); empty response strings.

Related errors

• Could not find instruction key `{instruction_template}` in t

AI-assisted analysis of OpenBMB/MiniCPM-V@7a11e2bec4 (2026-08-30). Data as JSON: /api/errors/9bec842f0cdee519. Report an issue: GitHub.