PaddlePaddle/PaddleOCR · error · ValueError

`decoder_start_token_id` expected to have length {batch_size

Error message

`decoder_start_token_id` expected to have length {batch_size} but got {len(decoder_start_token_id)}

What it means

In generation, when decoder_start_token_id is provided as a list, the code builds one start token per sequence and therefore requires len(list) == batch_size. A list of any other length raises ValueError.

Source

Thrown at ppocr/modeling/heads/rec_ppformulanet_head.py:992

    ):

        # 1. Check whether the user has defined `decoder_input_ids` manually. To facilitate in terms of input naming,
        # we also allow the user to pass it under `input_ids`, if the encoder does not use it as the main input.
        if model_kwargs is not None and "decoder_input_ids" in model_kwargs:
            decoder_input_ids = model_kwargs.pop("decoder_input_ids")
        elif "input_ids" in model_kwargs:
            decoder_input_ids = model_kwargs.pop("input_ids")
        else:
            decoder_input_ids = None

        # 2. Encoder-decoder models expect the `decoder_input_ids` to start with a special token. Let's ensure that.
        decoder_start_token_id = self._get_decoder_start_token_id(
            decoder_start_token_id, bos_token_id
        )

        if isinstance(decoder_start_token_id, list):
            if len(decoder_start_token_id) != batch_size:
                raise ValueError(
                    f"`decoder_start_token_id` expected to have length {batch_size} but got {len(decoder_start_token_id)}"
                )
            decoder_input_ids_start = paddle.to_tensor(
                decoder_start_token_id,
                dtype=paddle.int64,
            )
            decoder_input_ids_start = decoder_input_ids_start.view(-1, 1)
        else:
            use_parallel = self.config_decoder.use_parallel
            parallel_step = self.config_decoder.parallel_step

            if use_parallel:
                decoder_input_ids_start = (
                    paddle.ones(
                        (batch_size, parallel_step),
                        dtype=paddle.int64,
                    )
                    * decoder_start_token_id

View on GitHub (pinned to 2661c7c0ef)

Solutions

Make the list length equal batch_size, one start id per sample (or broadcast a single value)
If all samples share the same start token, pass the plain int decoder_start_token_id instead of a list
Compute the list from your batch at runtime: [start_for(sample) for sample in batch]

Example fix

# before
out = model.generate(input_ids=x, decoder_start_token_id=[0])  # batch is 4
# after
bs = x.shape[0]
out = model.generate(input_ids=x, decoder_start_token_id=[0]*bs)

Defensive patterns

Strategy: validation

Validate before calling

def normalize_start_ids(decoder_start_token_id, batch_size):
    if isinstance(decoder_start_token_id, (list, tuple)):
        if len(decoder_start_token_id) != batch_size:
            raise ValueError(f'start ids len {len(decoder_start_token_id)} != batch {batch_size}')
        return list(decoder_start_token_id)
    return decoder_start_token_id

Type guard

def start_ids_match_batch(v, batch) -> bool:
    return not isinstance(v, (list, tuple)) or len(v) == batch

Try / catch

try:
    out = model.generate(input_ids=x, decoder_start_token_id=start)
except ValueError as e:
    if 'decoder_start_token_id' in str(e) and isinstance(start, (list, tuple)):
        out = model.generate(input_ids=x, decoder_start_token_id=start[0])  # shared start token
    else:
        raise

Prevention

Pass a scalar start token when all samples share one
Compute per-sample lists from the actual batch at runtime
Add a batch-assert before generate in inference wrappers

When it happens

Trigger: Calling generate with decoder_start_token_id as a list whose length differs from the batch dimension of input_ids/inputs_embeds — e.g. 4 start ids for a batch of 2, or a single-element list where batch > 1.

Common situations: Prompt-specific start tokens (per-sample BOS) where the list wasn't padded/trimmed to batch size; passing a scalar wrapped in [token] while running batched inference; dynamic batching upstream while the start-id list stays fixed.

Related errors

AI-assisted analysis of PaddlePaddle/PaddleOCR@2661c7c0ef (2026-08-14). Data as JSON: /api/errors/4cd2f7d62091a09e. Report an issue: GitHub.