{"record":{"id":"afb32463bdb00a3e","repo":"PaddlePaddle/PaddleOCR","slug":"this-attention-mask-converter-is-causal-make-sure","errorCode":null,"errorMessage":"This attention mask converter is causal. Make sure to pass `key_value_length` to correctly create a causal mask.","messagePattern":"This attention mask converter is causal\\. Make sure to pass `key_value_length` to correctly create a causal mask\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"ppocr/modeling/heads/rec_ppformulanet_head.py","lineNumber":194,"sourceCode":"    ):\n        \"\"\"\n        Converts 2D attention mask to 4D attention mask by expanding mask to (bsz, head_dim=1, query_length,\n        key_value_length) shape and by adding a large negative bias to not-attended positions. If attention_mask is\n        causal, a causal mask will be added.\n        \"\"\"\n        input_shape = (attention_mask_2d.shape[0], query_length)\n\n        causal_4d_mask = None\n        if use_parallel:\n            step = parallel_step\n        else:\n            step = 1\n        if (\n            input_shape[-1] > step or self.sliding_window is not None\n        ) and self.is_causal:\n\n            if key_value_length is None:\n                raise ValueError(\n                    \"This attention mask converter is causal. Make sure to pass `key_value_length` to correctly create a causal mask.\"\n                )\n\n            past_key_values_length = key_value_length - query_length\n\n            if use_parallel:\n                causal_4d_mask = self._make_causal_mask_parallel(\n                    input_shape,\n                    dtype,\n                    past_key_values_length=past_key_values_length,\n                    sliding_window=self.sliding_window,\n                    parallel_step=parallel_step,\n                    is_export=is_export,\n                )\n            else:\n                causal_4d_mask = self._make_causal_mask(\n                    input_shape,\n                    dtype,","sourceCodeStart":176,"sourceCodeEnd":212,"githubUrl":"https://github.com/PaddlePaddle/PaddleOCR/blob/2661c7c0ef5c613e8f93c6e93b2e052399f0f854/ppocr/modeling/heads/rec_ppformulanet_head.py#L176-L212","documentation":"When the PP-FormulaNet mask converter is causal and the query length exceeds the parallel step (or a sliding window is configured), it must compute past_key_values_length = key_value_length - query_length. If key_value_length was not passed, that arithmetic is impossible, so it raises.","triggerScenarios":"Calling attn_mask_converter.to_4d(...) without key_value_length while query_length > parallel_step (usually 1, i.e. any multi-token decode step) or while sliding_window is set. Typically happens in custom forward code or a patched decode loop that omits the argument.","commonSituations":"Extending the decoder forward for a new generation mode and forgetting key_value_length; refactoring a step-decoding loop into a parallel one and dropping the kv-length plumbing; using cached inference where past_key_values length must be forwarded into mask creation.","solutions":["Pass key_value_length explicitly to to_4d: it should equal past_key_values_length + query_length (e.g. past_key_values[0][0].shape[2] + input_shape[-1])","If you truly have no cache, pass key_value_length=query_length so past_key_values_length becomes 0","Audit custom decode loops to keep the mask call in sync with the cache bookkeeping (_update_model_kwargs_for_generation)"],"exampleFix":"# before\nmask = attn_mask_converter.to_4d(attn_mask, input_shape[-1], dtype=dtype)\n# after\nkv_len = past_key_values[0][0].shape[2] + input_shape[-1] if past_key_values is not None else input_shape[-1]\nmask = attn_mask_converter.to_4d(attn_mask, input_shape[-1], key_value_length=kv_len, dtype=dtype)","handlingStrategy":"validation","validationCode":"def kv_length(past_key_values, q_len):\n    if past_key_values is None:\n        return q_len\n    return past_key_values[0][0].shape[2] + q_len\n# always call: converter.to_4d(mask, q_len, key_value_length=kv_length(past, q_len), dtype=dtype)","typeGuard":"null","tryCatchPattern":"try:\n    mask = converter.to_4d(mask, q_len, dtype=dtype)\nexcept ValueError as e:\n    if 'key_value_length' in str(e):\n        mask = converter.to_4d(mask, q_len, key_value_length=q_len, dtype=dtype)\n    else:\n        raise","preventionTips":["Treat key_value_length as a required argument in multi-token decoding","Centralize mask creation in one helper that also owns cache bookkeeping","Add a decode-2-steps unit test to every custom generation loop"],"tags":["attention-mask","decoding","ppformulanet","api-misuse"],"backgroundTag":null,"analyzedSha":"2661c7c0ef5c613e8f93c6e93b2e052399f0f854","analyzedAt":"2026-08-14T20:17:30.180Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}