docling-project/docling · error · RuntimeError

Neither processor.batch_decode nor tokenizer.batch_decode is

Error message

Neither processor.batch_decode nor tokenizer.batch_decode is available.

What it means

After generate(), the model decodes token IDs via processor.batch_decode, falling back to processor.tokenizer.batch_decode. If the loaded processor exposes neither (some vision processors wrap the tokenizer under different attribute names), RuntimeError is raised because decoded text cannot be produced.

Source

Thrown at docling/models/vlm_pipeline_models/hf_transformers_model.py:399

            gen_kwargs["do_sample"] = False

        if stopping_criteria is not None:
            gen_kwargs["stopping_criteria"] = stopping_criteria

        start_time = time.time()
        with torch.inference_mode():
            generated_ids = self.vlm_model.generate(**gen_kwargs)
        generation_time = time.time() - start_time

        input_len = inputs["input_ids"].shape[1]  # common right-aligned prompt length
        trimmed_sequences = generated_ids[:, input_len:]  # only newly generated tokens

        # -- Decode with the processor/tokenizer (skip specials, keep DocTags as text)
        decode_fn = getattr(self.processor, "batch_decode", None)
        if decode_fn is None and getattr(self.processor, "tokenizer", None) is not None:
            decode_fn = self.processor.tokenizer.batch_decode
        if decode_fn is None:
            raise RuntimeError(
                "Neither processor.batch_decode nor tokenizer.batch_decode is available."
            )

        decoded_texts: list[str] = decode_fn(
            trimmed_sequences,
            **decoder_config,
        )

        # -- Clip off pad tokens from decoded texts
        pad_token = self.processor.tokenizer.pad_token
        if pad_token:
            decoded_texts = [text.rstrip(pad_token) for text in decoded_texts]

        if (
            self.vlm_options.extra_generation_config.get("strip_stop_strings", False)
            and self.vlm_options.stop_strings
        ):
            from docling.utils.vlm_utils import strip_stop_strings

View on GitHub (pinned to 61d76f1ff3)

Solutions

1. Prefer a repo_id whose processor exposes batch_decode or a .tokenizer attribute (standard HF vision-language processors)
2. Before converting, attach the real tokenizer: model.processor.tokenizer = model.processor.<actual_tokenizer_attr>
3. If the processor is fundamentally incompatible, use the vLLM engine for that model, which handles decoding itself

Example fix

# before: processor has .tokenizer_wrapper but no .tokenizer
# RuntimeError: Neither processor.batch_decode nor tokenizer.batch_decode
# after
model.processor.tokenizer = model.processor.tokenizer_wrapper  # minimal shim
result = pipeline.convert(document)

proc = model.processor
has_decode = callable(getattr(proc, 'batch_decode', None)) or getattr(proc, 'tokenizer', None) is not None
if not has_decode:
    raise RuntimeError('processor cannot decode; attach its tokenizer before conversion')

try:
    result = pipeline.convert(document)
except RuntimeError as e:
    if 'batch_decode' in str(e):
        model.processor.tokenizer = model.processor.tokenizer_wrapper  # adapt to real attr name
        result = pipeline.convert(document)
    else:
        raise

When it happens

Trigger: Loading a newer/less-common processor whose tokenizer is stored under an attribute other than 'tokenizer' and which does not itself implement batch_decode, then running a conversion that reaches generation.

Common situations: Upgrading transformers so a processor class changes its attribute layout; using an experimental repo_id whose processor is minimally implemented; mismatches between processor and tokenizer versions in a custom env.

Related errors

• Model not loaded. Ensure EngineModelConfig was provided duri
• Neither processor.batch_decode nor tokenizer.batch_decode is
• Cannot decode field {field.name!r} of type {field.type.value
• No record layout matches record type {record_type!r}.
• Record length {length} is shorter than the {layout.prefix_si

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/1253102f5330f554. Report an issue: GitHub.