{"record":{"id":"8da15397afa52686","repo":"docling-project/docling","slug":"neither-processor-batch-decode-nor-tokenizer-batch","errorCode":null,"errorMessage":"Neither processor.batch_decode nor tokenizer.batch_decode is available.","messagePattern":"Neither processor\\.batch_decode nor tokenizer\\.batch_decode is available\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"docling/models/inference_engines/vlm/transformers_engine.py","lineNumber":474,"sourceCode":"            gen_kwargs[\"do_sample\"] = False\n\n        if stopping_criteria_list:\n            gen_kwargs[\"stopping_criteria\"] = stopping_criteria_list\n\n        start_time = time.time()\n        with torch.inference_mode():\n            generated_ids = self.vlm_model.generate(**gen_kwargs)  # type: ignore[union-attr,operator]\n        generation_time = time.time() - start_time\n\n        # Decode\n        input_len = inputs[\"input_ids\"].shape[1]\n        trimmed_sequences = generated_ids[:, input_len:]\n\n        decode_fn = getattr(self.processor, \"batch_decode\", None)\n        if decode_fn is None and tokenizer is not None:\n            decode_fn = getattr(tokenizer, \"batch_decode\", None)\n        if decode_fn is None:\n            raise RuntimeError(\n                \"Neither processor.batch_decode nor tokenizer.batch_decode is available.\"\n            )\n\n        decoded_texts = decode_fn(trimmed_sequences, **decoder_config)\n\n        # Remove padding\n        pad_token = getattr(tokenizer, \"pad_token\", None)\n        if pad_token:\n            decoded_texts = [text.rstrip(pad_token) for text in decoded_texts]\n\n        if self.strip_stop_strings and first_input.stop_strings:\n            from docling.utils.vlm_utils import strip_stop_strings\n\n            decoded_texts = strip_stop_strings(decoded_texts, first_input.stop_strings)\n\n        # Create outputs\n        outputs = []\n        for i, text in enumerate(decoded_texts):","sourceCodeStart":456,"sourceCodeEnd":492,"githubUrl":"https://github.com/docling-project/docling/blob/61d76f1ff3f8428065465889f7b4577da7df704c/docling/models/inference_engines/vlm/transformers_engine.py#L456-L492","documentation":"After generation, TransformersVlmEngine decodes the output token ids via processor.batch_decode, falling back to tokenizer.batch_decode. If neither object exposes batch_decode, decoding cannot proceed and a RuntimeError is raised — this indicates an exotic or incompatible processor/tokenizer pair rather than a usage mistake.","triggerScenarios":"Loading a repo whose processor class lacks batch_decode (e.g. some new/legacy processor types) and whose tokenizer is also None or lacks the method, then running predict_batch to completion of generation.","commonSituations":"Very new transformers releases with changed processor APIs; custom or community VLM repos with non-standard preprocessing classes; models loaded with a processor-only config where no tokenizer object is attached.","solutions":["Switch to a known-good VLM repo (e.g. the Docling-supported SmolDocling, GraniteDocling, or Phi-4 with pinned transformers)","Update or pin transformers to a version compatible with the chosen model's processor","Inspect the processor: python -c \"from transformers import AutoProcessor; p = AutoProcessor.from_pretrained('<repo>'); print(hasattr(p, 'batch_decode'))\""],"exampleFix":"# before\nmodel_config = EngineModelConfig(repo_id='<exotic-community-vlm>')  # processor lacks batch_decode\n\n# after\nmodel_config = EngineModelConfig(repo_id='ds4sd/SmolDocling-256M-preview')  # supported processor","handlingStrategy":"validation","validationCode":"from transformers import AutoProcessor\n\np = AutoProcessor.from_pretrained(repo_id)\nhas_decode = hasattr(p, 'batch_decode')\n# also check a tokenizer if present\ntry:\n    tok = p.tokenizer\nexcept AttributeError:\n    tok = None\nassert has_decode or (tok is not None and hasattr(tok, 'batch_decode')), (\n    f'{repo_id} processor exposes no batch_decode; use a supported VLM repo'\n)","typeGuard":null,"tryCatchPattern":"try:\n    outputs = engine.predict_batch(inputs)\nexcept RuntimeError as e:\n    if 'batch_decode' in str(e):\n        raise SystemExit(f'Processor for {engine.model_config.repo_id} lacks batch_decode; switch to a supported VLM repo or compatible transformers version') from e\n    raise","preventionTips":["Stick to the model repos Docling lists as supported for the Transformers engine","Smoke-test one inference after loading any new/community model to catch decode incompatibilities early","Pin transformers to a version validated for your chosen model"],"tags":["vlm","transformers","decoding","incompatible-model","runtime-error"],"backgroundTag":null,"analyzedSha":"61d76f1ff3f8428065465889f7b4577da7df704c","analyzedAt":"2026-08-14T23:53:18.727Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}