open-mmlab/mmdetection · warning

Inputting a text that is too long will result in poor predic

Error message

Inputting a text that is too long will result in poor prediction performance. Please reduce the --chunked-size.

What it means

Grounding DINO's chunked prompt tokenizer warns that a chunk's tokenized caption exceeds the language model's max_tokens, degrading embedding/prediction quality; it recommends reducing the chunk size.

Source

Thrown at mmdet/models/detectors/grounding_dino.py:286

            list(range(1,
                       len(original_caption) + 1)), chunked_size)

        positive_map_label_to_token_chunked = []
        caption_string_chunked = []
        positive_map_chunked = []
        entities_chunked = []

        for i in range(len(ids_chunked)):
            if enhanced_text_prompts is not None:
                caption_string, tokens_positive = self.to_enhance_text_prompts(
                    original_caption_chunked[i], enhanced_text_prompts)
            else:
                caption_string, tokens_positive = self.to_plain_text_prompts(
                    original_caption_chunked[i])
            tokenized = self.language_model.tokenizer([caption_string],
                                                      return_tensors='pt')
            if tokenized.input_ids.shape[1] > self.language_model.max_tokens:
                warnings.warn('Inputting a text that is too long will result '
                              'in poor prediction performance. '
                              'Please reduce the --chunked-size.')
            positive_map_label_to_token, positive_map = self.get_positive_map(
                tokenized, tokens_positive)

            caption_string_chunked.append(caption_string)
            positive_map_label_to_token_chunked.append(
                positive_map_label_to_token)
            positive_map_chunked.append(positive_map)
            entities_chunked.append(original_caption_chunked[i])

        return positive_map_label_to_token_chunked, \
            caption_string_chunked, \
            positive_map_chunked, \
            entities_chunked

    def forward_transformer(
        self,

View on GitHub (pinned to cfd5d3a985)

Solutions

1. Lower the --chunked-size so each chunk fits max_tokens
2. Shorten/normalize category text

Example fix

# before
python demo/image_demo.py img.jpg grounding_dino config ckpt --texts 'many categories ...' --chunked-size 10
# after
python demo/image_demo.py img.jpg grounding_dino config ckpt --texts 'many categories ...' --chunked-size 3

tok = detector.language_model.tokenizer
while any(tok([c], return_tensors='pt').input_ids.shape[1] > detector.language_model.max_tokens for c in chunks):
    chunked_size = max(1, chunked_size // 2)
    chunks = rechunk(categories, chunked_size)

When it happens

Trigger: get_tokens_positive_and_prompts_chunked with a chunk whose tokenized input_ids length > language_model.max_tokens.

Common situations: Grounding DINO demos/inference with many categories and a large chunked-size value.

Related errors

• Inputting a text that is too long will result in poor predic
• If you want to reduce GPU memory usage,
• There is not post_norm in {name}
• Inputting a text that is too long will result in poor predic
• The unexpected output indicates an issue with named entity r

AI-assisted analysis of open-mmlab/mmdetection@cfd5d3a985 (2026-08-27). Data as JSON: /api/errors/bce8e262b855e040. Report an issue: GitHub.