{"record":{"id":"7d723a6edce255f4","repo":"ruvnet/ruflo","slug":"flashattention-key-and-value-dimensions-must-matc","errorCode":null,"errorMessage":"FlashAttention: Key and value dimensions must match. Got K=${kDim}, V=${vDim}","messagePattern":"FlashAttention: Key and value dimensions must match\\. Got K=(.+?), V=(.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"v3/@claude-flow/neural/src/flash-attention.ts","lineNumber":791,"sourceCode":"\n    if (keys.length !== values.length) {\n      throw new Error(\n        `FlashAttention: Keys and values must have same count. Got ${keys.length} keys, ${values.length} values`,\n      );\n    }\n\n    const qDim = queries[0]?.length ?? 0;\n    const kDim = keys[0]?.length ?? 0;\n    const vDim = values[0]?.length ?? 0;\n\n    if (qDim !== kDim) {\n      throw new Error(\n        `FlashAttention: Query and key dimensions must match. Got Q=${qDim}, K=${kDim}`,\n      );\n    }\n\n    if (kDim !== vDim) {\n      throw new Error(\n        `FlashAttention: Key and value dimensions must match. Got K=${kDim}, V=${vDim}`,\n      );\n    }\n  }\n}\n\n// ============================================================================\n// Singleton Instance\n// ============================================================================\n\nlet flashAttentionInstance: FlashAttention | null = null;\n\n/**\n * Get singleton FlashAttention instance\n *\n * @param config - Optional configuration (only used on first call)\n * @returns FlashAttention instance\n */","sourceCodeStart":773,"sourceCodeEnd":809,"githubUrl":"https://github.com/ruvnet/ruflo/blob/fa13ee4ad60ac2090b1480656eb233521790d640/v3/@claude-flow/neural/src/flash-attention.ts#L773-L809","documentation":"After the Q/K check, validateInputs() also requires values[0].length === keys[0].length — in this implementation V must have the same per-vector dimension as K, and any difference throws with both numbers. This is stricter than attention APIs (e.g. PyTorch) that allow an independent value/output dimension d_v, which is the most common reason ported code trips it.","triggerScenarios":"Porting attention code from a framework where d_v differs from d_k; building values from a different feature source than keys; stale cached values after a dimension change on keys.","commonSituations":"Migrating transformer code from PyTorch/transformers.js; pipelines where values carry auxiliary features of another width.","solutions":["Make values the same dimension as keys (derive all three tensors from the same encoder output)","Project or pad values to the key dimension if the pipeline genuinely produces a different width","Re-encode K and V together from one tensor so dims cannot drift"],"exampleFix":"// before\nconst out = computeAttention(q, k, v /* v is 256-dim, k is 384-dim */);\n\n// after\nconst vProjected = project(v, k[0].length); // widen/narrow V to match K\nconst out = computeAttention(q, k, vProjected);","handlingStrategy":"validation","validationCode":"const kDim = keys[0]?.length ?? 0;\nconst vDim = values[0]?.length ?? 0;\nif (kDim !== vDim) {\n  throw new Error(`K/V dimension mismatch: ${kDim} vs ${vDim} - this implementation requires d_v === d_k`);\n}\nconst out = computeAttention(queries, keys, values);","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Do not port d_v != d_k transformer configs directly - this API requires equal dims","Build K and V from the same encoder output so dimensions cannot drift","Project or pad values to the key dimension when bridging different feature widths"],"tags":["neural","flash-attention","input-shape","embedding-dimension"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"fa13ee4ad60ac2090b1480656eb233521790d640","analyzedAt":"2026-08-18T21:34:22.708Z","schemaVersion":2},"datasetVersion":"2026-08-28T21:17:43.275Z"}