{"record":{"id":"8d5e962ef22056b7","repo":"invoke-ai/InvokeAI","slug":"krea-2-regional-attention-mask-shape-tuple-region","errorCode":null,"errorMessage":"Krea-2 regional attention mask shape {tuple(regional_attention_mask.shape)} does not match the transformer sequence length {hidden_states.shape[1]}.","messagePattern":"Krea-2 regional attention mask shape (.+?) does not match the transformer sequence length (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"invokeai/backend/krea2/attention.py","lineNumber":55,"sourceCode":"\n\nclass Krea2MemoryEfficientAttnProcessor:\n    \"\"\"Drop-in replacement for ``Krea2AttnProcessor`` that avoids the ``enable_gqa`` math fallback.\"\"\"\n\n    def __init__(self, regional_prompting_state: Krea2RegionalPromptingState | None = None) -> None:\n        self.regional_prompting_state = regional_prompting_state\n\n    def __call__(\n        self,\n        attn,\n        hidden_states: torch.Tensor,\n        attention_mask: torch.Tensor | None = None,\n        image_rotary_emb: tuple[torch.Tensor, torch.Tensor] | None = None,\n    ) -> torch.Tensor:\n        if self.regional_prompting_state is not None and self.regional_prompting_state.attention_mask is not None:\n            regional_attention_mask = self.regional_prompting_state.attention_mask\n            if regional_attention_mask.shape != (hidden_states.shape[1], hidden_states.shape[1]):\n                raise ValueError(\n                    f\"Krea-2 regional attention mask shape {tuple(regional_attention_mask.shape)} does not match \"\n                    f\"the transformer sequence length {hidden_states.shape[1]}.\"\n                )\n            attention_mask = (\n                regional_attention_mask if attention_mask is None else attention_mask & regional_attention_mask\n            )\n\n        query = attn.to_q(hidden_states).unflatten(-1, (attn.num_heads, attn.head_dim))\n        key = attn.to_k(hidden_states).unflatten(-1, (attn.num_kv_heads, attn.head_dim))\n        value = attn.to_v(hidden_states).unflatten(-1, (attn.num_kv_heads, attn.head_dim))\n        gate = attn.to_gate(hidden_states)\n\n        query = attn.norm_q(query)\n        key = attn.norm_k(key)\n\n        if image_rotary_emb is not None:\n            query = apply_rotary_emb(query, image_rotary_emb, sequence_dim=1)\n            key = apply_rotary_emb(key, image_rotary_emb, sequence_dim=1)","sourceCodeStart":37,"sourceCodeEnd":73,"githubUrl":"https://github.com/invoke-ai/InvokeAI/blob/0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06/invokeai/backend/krea2/attention.py#L37-L73","documentation":"Krea-2 regional prompting builds a custom attention mask covering the full transformer sequence (text embeddings + image latents). Before applying it, each attention block validates that the mask is square and matches hidden_states.shape[1] (the sequence length). A mismatch means the mask was computed for a different sequence length — usually because latent resolution, number of text conditionings, or token counts changed after the mask was built.","triggerScenarios":"Running the Krea-2 transformer __call__ with regional_prompting_state.attention_mask whose shape differs from (seq_len, seq_len): changed image height/width after the extension was constructed, different text conditionings, or a mask reused across denoising stages with differing sequence lengths.","commonSituations":"Caching a Krea2RegionalPromptingExtension built for one resolution and reusing it at another; mixing regional masks between batches; changing prompt tokenization after mask construction.","solutions":["Rebuild the regional prompting extension via Krea2RegionalPromptingExtension.from_text_conditionings(...) with the same image_seq_len used by the current transformer pass.","Ensure image dimensions (latent size) are fixed for the lifetime of the regional attention mask.","Pass attention_mask=None or a fresh regional state if you intentionally changed resolution or prompts.","Log hidden_states.shape[1] and the mask shape to find where they diverge."],"exampleFix":"// before\next = build_regional_extension(text_conds, image_seq_len=4096)\n# ... later run at 2x resolution -> seq_len=16384, mask is 4096x4096\n// after\next = Krea2RegionalPromptingExtension.from_text_conditionings(text_conds, image_seq_len=transformer_seq_len)","handlingStrategy":"try-catch","validationCode":"if regional_state and regional_state.attention_mask is not None:\n    assert regional_state.attention_mask.shape == (seq_len, seq_len), \\\n        f'mask {regional_state.attention_mask.shape} != ({seq_len}, {seq_len})'","typeGuard":null,"tryCatchPattern":"try:\n    output = transformer(...)\nexcept ValueError as e:\n    if 'regional attention mask shape' in str(e):\n        regional_state = Krea2RegionalPromptingExtension.from_text_conditionings(\n            text_conditionings, image_seq_len=hidden_states_len)\n        output = transformer(...)  # rebuild and retry once\n    else:\n        raise","preventionTips":["Rebuild the regional extension whenever image resolution or prompts change","Keep one regional extension per (resolution, prompt-set) pair","Assert mask shape equals 2 * seq_len dims before transformer calls","Don't cache regional masks across different generation configs"],"tags":["krea2","regional-prompting","attention-mask","shape-mismatch"],"backgroundTag":"attention-mask-shape-mismatch","analyzedSha":"0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06","analyzedAt":"2026-08-29T04:46:49.967Z","schemaVersion":2},"datasetVersion":"2026-08-29T07:17:48.351Z"}