ruvnet/ruflo · critical · Error

tuneDistillation: train/held-out partitions are not disjoint

Error message

tuneDistillation: train/held-out partitions are not disjoint (rowid ${x} in both)

What it means

assertDisjoint checks that no rowid appears in both the train and held-out rowid lists. The split is produced by partitioning rows on rowid <= trainBoundaryRowid vs rowid > trainBoundaryRowid, which is structurally disjoint, so this error firing indicates an internal bug in the split/boundary logic or that a and b were constructed by different code paths.

Source

Thrown at v3/@claude-flow/cli/src/services/distill-tuning.ts:427

  const total = rows.length;
  if (total === 0) return { totalRows: 0, trainBoundaryRowid: 0, trainRowids: [], heldOutRowids: [] };

  const trainCount = Math.min(total, Math.max(1, Math.floor(total * trainFraction)));
  const trainBoundaryRowid = rows[trainCount - 1].rowid;
  const trainRowids: number[] = [];
  const heldOutRowids: number[] = [];
  for (const r of rows) {
    if (r.rowid <= trainBoundaryRowid) trainRowids.push(r.rowid);
    else heldOutRowids.push(r.rowid);
  }
  return { totalRows: total, trainBoundaryRowid, trainRowids, heldOutRowids };
}

function assertDisjoint(a: number[], b: number[]): void {
  const setA = new Set(a);
  for (const x of b) {
    if (setA.has(x)) {
      throw new Error(`tuneDistillation: train/held-out partitions are not disjoint (rowid ${x} in both)`);
    }
  }
}

// ── Query set + baseline index construction ─────────────────────────────

function buildQuerySet(
  db: any,
  opts: { loRowid: number; hiRowid: number; namespaces: string[] },
): QueryEntry[] {
  const { loRowid, hiRowid, namespaces } = opts;
  if (namespaces.length === 0) return [];
  const placeholders = namespaces.map(() => '?').join(',');
  const hiFinite = Number.isFinite(hiRowid);
  const sql = `SELECT rowid, id, namespace, content, embedding FROM memory_entries
    WHERE rowid > ? ${hiFinite ? 'AND rowid <= ?' : ''} AND embedding IS NOT NULL
      AND COALESCE(namespace,'default') IN (${placeholders})
    ORDER BY rowid`;

View on GitHub (pinned to 6b01dc5a68)

Solutions

1. Treat this as a defect — report it; the split function should be disjoint by construction.
2. If you call assertDisjoint directly, ensure the two arrays come from a non-overlapping partition (e.g. by a single boundary predicate).
3. Inspect the source table for duplicate rowids (rowid should be unique).
4. Add a regression test pinning buildTimeSplit's disjointness.

Example fix

// This error signals an internal invariant violation, not a usage mistake.
// Report it. As a caller you cannot 'fix' it; verify your data instead:
const rowids = await collectRowids(db);
if (new Set(rowids).size !== rowids.length) {
  throw new Error('source has duplicate rowids — investigate db integrity');
}

// Caller cannot prevent an internal invariant bug. Sanity-check source rowid uniqueness:
const rowids = collectRowids(db);
if (new Set(rowids).size !== rowids.length) {
  throw new Error('source db has duplicate rowids — integrity issue');
}

try {
  await tuneDistillation({ dbPath, ... });
} catch (e) {
  if ((e as Error).message.includes('not disjoint')) {
    // internal invariant violation — report upstream with db details; do not retry blindly
    reportBug(e);
  }
  throw e;
}

When it happens

Trigger: The split function returns trainRowids and heldOutRowids that share a rowid, which should be impossible given the <= boundary rule. Realistically this only fires if the boundary logic or the row collection is corrupted, or if assertDisjoint is reused with externally-built arrays.

Common situations: A code change to buildTimeSplit that breaks the <= / > partitioning; duplicate rowids in the source table confusing the partition; calling assertDisjoint with hand-built overlapping arrays in tests.

Related errors

• tuneDistillation: source DB checksum changed during tuning —
• tuneDistillation: source db not found at ${dbPath}
• tuneDistillation: better-sqlite3 unavailable — cannot run th
• tuneDistillation: empty grid — supply at least one value per
• tuneDistillation: every grid candidate was skipped — see can

AI-assisted analysis of ruvnet/ruflo@6b01dc5a68 (2026-08-12). Data as JSON: /api/errors/43d12296b30ff3ef. Report an issue: GitHub.