{"record":{"id":"43d12296b30ff3ef","repo":"ruvnet/ruflo","slug":"tunedistillation-train-held-out-partitions-are-no","errorCode":null,"errorMessage":"tuneDistillation: train/held-out partitions are not disjoint (rowid ${x} in both)","messagePattern":"tuneDistillation: train/held-out partitions are not disjoint \\(rowid (.+?) in both\\)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"critical","filePath":"v3/@claude-flow/cli/src/services/distill-tuning.ts","lineNumber":427,"sourceCode":"  const total = rows.length;\n  if (total === 0) return { totalRows: 0, trainBoundaryRowid: 0, trainRowids: [], heldOutRowids: [] };\n\n  const trainCount = Math.min(total, Math.max(1, Math.floor(total * trainFraction)));\n  const trainBoundaryRowid = rows[trainCount - 1].rowid;\n  const trainRowids: number[] = [];\n  const heldOutRowids: number[] = [];\n  for (const r of rows) {\n    if (r.rowid <= trainBoundaryRowid) trainRowids.push(r.rowid);\n    else heldOutRowids.push(r.rowid);\n  }\n  return { totalRows: total, trainBoundaryRowid, trainRowids, heldOutRowids };\n}\n\nfunction assertDisjoint(a: number[], b: number[]): void {\n  const setA = new Set(a);\n  for (const x of b) {\n    if (setA.has(x)) {\n      throw new Error(`tuneDistillation: train/held-out partitions are not disjoint (rowid ${x} in both)`);\n    }\n  }\n}\n\n// ── Query set + baseline index construction ─────────────────────────────\n\nfunction buildQuerySet(\n  db: any,\n  opts: { loRowid: number; hiRowid: number; namespaces: string[] },\n): QueryEntry[] {\n  const { loRowid, hiRowid, namespaces } = opts;\n  if (namespaces.length === 0) return [];\n  const placeholders = namespaces.map(() => '?').join(',');\n  const hiFinite = Number.isFinite(hiRowid);\n  const sql = `SELECT rowid, id, namespace, content, embedding FROM memory_entries\n    WHERE rowid > ? ${hiFinite ? 'AND rowid <= ?' : ''} AND embedding IS NOT NULL\n      AND COALESCE(namespace,'default') IN (${placeholders})\n    ORDER BY rowid`;","sourceCodeStart":409,"sourceCodeEnd":445,"githubUrl":"https://github.com/ruvnet/ruflo/blob/6b01dc5a687b26b3e218f796de45ec51f8fa9e8c/v3/@claude-flow/cli/src/services/distill-tuning.ts#L409-L445","documentation":"assertDisjoint checks that no rowid appears in both the train and held-out rowid lists. The split is produced by partitioning rows on rowid <= trainBoundaryRowid vs rowid > trainBoundaryRowid, which is structurally disjoint, so this error firing indicates an internal bug in the split/boundary logic or that a and b were constructed by different code paths.","triggerScenarios":"The split function returns trainRowids and heldOutRowids that share a rowid, which should be impossible given the <= boundary rule. Realistically this only fires if the boundary logic or the row collection is corrupted, or if assertDisjoint is reused with externally-built arrays.","commonSituations":"A code change to buildTimeSplit that breaks the <= / > partitioning; duplicate rowids in the source table confusing the partition; calling assertDisjoint with hand-built overlapping arrays in tests.","solutions":["Treat this as a defect — report it; the split function should be disjoint by construction.","If you call assertDisjoint directly, ensure the two arrays come from a non-overlapping partition (e.g. by a single boundary predicate).","Inspect the source table for duplicate rowids (rowid should be unique).","Add a regression test pinning buildTimeSplit's disjointness."],"exampleFix":"// This error signals an internal invariant violation, not a usage mistake.\n// Report it. As a caller you cannot 'fix' it; verify your data instead:\nconst rowids = await collectRowids(db);\nif (new Set(rowids).size !== rowids.length) {\n  throw new Error('source has duplicate rowids — investigate db integrity');\n}","handlingStrategy":"try-catch","validationCode":"// Caller cannot prevent an internal invariant bug. Sanity-check source rowid uniqueness:\nconst rowids = collectRowids(db);\nif (new Set(rowids).size !== rowids.length) {\n  throw new Error('source db has duplicate rowids — integrity issue');\n}","typeGuard":null,"tryCatchPattern":"try {\n  await tuneDistillation({ dbPath, ... });\n} catch (e) {\n  if ((e as Error).message.includes('not disjoint')) {\n    // internal invariant violation — report upstream with db details; do not retry blindly\n    reportBug(e);\n  }\n  throw e;\n}","preventionTips":["Treat disjointness failures as defects; report them with the db and grid used.","Verify source rowid uniqueness before tuning.","Add regression tests covering buildTimeSplit's <= / > partitioning.","Do not monkey-patch the split arrays out-of-band."],"tags":["distillation","integrity","testing","typescript"],"backgroundTag":null,"analyzedSha":"6b01dc5a687b26b3e218f796de45ec51f8fa9e8c","analyzedAt":"2026-08-12T13:20:50.148Z","schemaVersion":2},"datasetVersion":"2026-08-12T18:17:37.767Z"}