{"record":{"id":"ee1c7a88ed1e7384","repo":"open-mmlab/mmdetection","slug":"inputting-a-text-that-is-too-long-will-result-in-p","errorCode":null,"errorMessage":"Inputting a text that is too long will result in poor prediction performance. Please reduce the text length.","messagePattern":"Inputting a text that is too long will result in poor prediction performance\\. Please reduce the text length\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"mmdet/models/detectors/glip.py","lineNumber":361,"sourceCode":"\n        chunked_size = self.test_cfg.get('chunked_size', -1)\n        if not self.training and chunked_size > 0:\n            assert isinstance(original_caption,\n                              (list, tuple)) or custom_entities is True\n            all_output = self.get_tokens_positive_and_prompts_chunked(\n                original_caption, enhanced_text_prompt)\n            positive_map_label_to_token, \\\n                caption_string, \\\n                positive_map, \\\n                entities = all_output\n        else:\n            tokenized, caption_string, tokens_positive, entities = \\\n                self.get_tokens_and_prompts(\n                    original_caption, custom_entities, enhanced_text_prompt)\n            positive_map_label_to_token, positive_map = self.get_positive_map(\n                tokenized, tokens_positive)\n            if tokenized.input_ids.shape[1] > self.language_model.max_tokens:\n                warnings.warn('Inputting a text that is too long will result '\n                              'in poor prediction performance. '\n                              'Please reduce the text length.')\n        return positive_map_label_to_token, caption_string, \\\n            positive_map, entities\n\n    def get_tokens_positive_and_prompts_chunked(\n            self,\n            original_caption: Union[list, tuple],\n            enhanced_text_prompts: Optional[ConfigType] = None):\n        chunked_size = self.test_cfg.get('chunked_size', -1)\n        original_caption = [clean_label_name(i) for i in original_caption]\n\n        original_caption_chunked = chunks(original_caption, chunked_size)\n        ids_chunked = chunks(\n            list(range(1,\n                       len(original_caption) + 1)), chunked_size)\n\n        positive_map_label_to_token_chunked = []","sourceCodeStart":343,"sourceCodeEnd":379,"githubUrl":"https://github.com/open-mmlab/mmdetection/blob/cfd5d3a985b0249de009b67d04f37263e11cdf3d/mmdet/models/detectors/glip.py#L343-L379","documentation":"GLIP warns when the tokenized caption exceeds the language model's max_tokens (e.g. BERT's 256). The text is not truncated by this check — downstream embedding/prediction quality degrades because the language encoder cannot represent the full prompt.","triggerScenarios":"Calling GLIP predict with a long original_caption whose tokenized input_ids length > language_model.max_tokens.","commonSituations":"Long descriptive prompts, concatenating many category names, or phrase-level prompts with many entities in zero-shot grounding.","solutions":["Shorten the caption / reduce number of categories per prompt","Split text into chunks and use get_tokens_positive_and_prompts_chunked with a smaller --chunked-size"],"exampleFix":"# before\nresults = detector(inputs, texts=['a very long caption with hundreds of words ...'])\n# after\nresults = detector(inputs, texts=['person . dog . car .'])","handlingStrategy":"validation","validationCode":"max_tokens = detector.language_model.max_tokens\nn = detector.language_model.tokenizer([caption], return_tensors='pt').input_ids.shape[1]\nassert n <= max_tokens, f'caption too long: {n} > {max_tokens}'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Pre-tokenize and check caption length before inference","Keep prompts as short category phrase lists"],"tags":["mmdet","glip","text-prompt","token-limit"],"backgroundTag":"input-exceeds-token-limit","analyzedSha":"cfd5d3a985b0249de009b67d04f37263e11cdf3d","analyzedAt":"2026-08-27T20:54:20.183Z","schemaVersion":2},"datasetVersion":"2026-08-28T00:17:15.603Z"}