ruvnet/ruflo · error · Error

MoE attention not initialized

Error message

MoE attention not initialized

What it means

Thrown by computeMoEAttention() when moeAttention is null. Unlike flash attention, MoE is opt-in: initializeTraining() constructs MoEAttention.simple(dim, 8, 2) only when config.useMoE is truthy AND the optional @ruvector/attention package is importable. The error therefore most commonly means you forgot { useMoE: true } rather than a broken install.

Source

Thrown at v3/@claude-flow/cli/src/services/ruvector-training.ts:606

  values: Float32Array[]
): Float32Array {
  if (!flashAttention) {
    throw new Error('Flash attention not initialized');
  }

  return flashAttention.computeRaw(query, keys, values);
}

/**
 * Compute MoE routing
 */
export function computeMoEAttention(
  query: Float32Array,
  keys: Float32Array[],
  values: Float32Array[]
): Float32Array {
  if (!moeAttention) {
    throw new Error('MoE attention not initialized');
  }

  return moeAttention.computeRaw(query, keys, values);
}

/**
 * Compute hyperbolic attention (for hierarchical patterns)
 */
export function computeHyperbolicAttention(
  query: Float32Array,
  keys: Float32Array[],
  values: Float32Array[]
): Float32Array {
  if (!hyperbolicAttention) {
    throw new Error('Hyperbolic attention not initialized');
  }

  return hyperbolicAttention.computeRaw(query, keys, values);

View on GitHub (pinned to fa13ee4ad6)

Solutions

1. Initialize with { useMoE: true } and assert the returned features include 'MoE (8 experts, top-2)'.
2. If you did pass useMoE: true and still see the error, check init logs/warnings for '@ruvector/attention unavailable' and reinstall the dependency.
3. Gate MoE code paths on the init result's features list so callers degrade to computeFlashAttention instead of throwing.
4. Re-run initializeTraining after cleanup() before any MoE call.

Example fix

// before
await initializeTraining({});
const routed = computeMoEAttention(query, keys, values); // throws — MoE is opt-in

// after
const init = await initializeTraining({ useMoE: true });
if (!init.features.some(f => f.startsWith('MoE'))) {
  throw new Error('MoE unavailable: ' + init.features.join(', '));
}
const routed = computeMoEAttention(query, keys, values);

const init = await initializeTraining({ useMoE: true }); // MoE is opt-in
if (!init.features.some(f => f.startsWith('MoE'))) {
  throw new Error(`MoE unavailable — check @ruvector/attention install. Features: ${init.features.join(', ')}`);
}
const routed = computeMoEAttention(query, keys, values);

async function moeReady(cfg: TrainingConfig = {}): Promise<boolean> {
  const init = await initializeTraining(cfg);
  return init.features.some(f => f.startsWith('MoE'));
}

try {
  return computeMoEAttention(q, keys, values);
} catch (e) {
  if (e instanceof Error && e.message === 'MoE attention not initialized') {
    return computeFlashAttention(q, keys, values); // feature-degrade, don't crash
  }
  throw e;
}

When it happens

Trigger: Calling computeMoEAttention after init without the useMoE flag; passing { useMoE: true } in an environment where @ruvector/attention failed to import (init then silently skips all attention features, logging a warning); calling after cleanup().

Common situations: Copy-pasting MoE benchmark code into a pipeline whose init config was written before MoE existed; feature-flag configs where useMoE is conditioned on an env var that isn't set in the failing environment; slim production installs missing the optional attention package.

Related errors

• Flash attention not initialized
• Hyperbolic attention not initialized
• Hard negative miner not initialized
• Contrastive loss not initialized
• Optimizer not initialized

AI-assisted analysis of ruvnet/ruflo@fa13ee4ad6 (2026-08-18). Data as JSON: /api/errors/81942053d9722845. Report an issue: GitHub.