{"record":{"id":"68657c71a552645d","repo":"xai-org/x-algorithm","slug":"no-cached-cls-parquet-under-split-dir","errorCode":null,"errorMessage":"no cached cls parquet under {split_dir}","messagePattern":"no cached cls parquet under (.+?)","errorType":"exception","errorClass":"FileNotFoundError","httpStatus":null,"severity":"error","filePath":"bdsm/training/train_head.py","lineNumber":38,"sourceCode":"import pyarrow as pa\nimport pyarrow.parquet as pq\n\nimport heads\nimport labels\nimport loss as task_loss\nfrom metrics import average_precision\nfrom task_heads import SEED, head_logits, init_head_params\n\nlogging.basicConfig(level=logging.INFO, format=\"%(asctime)s %(message)s\", datefmt=\"%H:%M:%S\")\nlog = logging.getLogger(\"train_head\")\n\n\ndef load_cached(split_dir: str) -> dict:\n    xs, ys, uids, sources = [], [], [], []\n    n_nonfinite = 0\n    files = sorted(glob.glob(os.path.join(split_dir, \"*.parquet\")))\n    if not files:\n        raise FileNotFoundError(f\"no cached cls parquet under {split_dir}\")\n    for fpath in files:\n        t = pq.read_table(fpath)\n        strength = np.asarray(t.column(\"label_strength\").to_pylist())\n        cls_col = t.column(\"cls\").combine_chunks()\n        cls_all = (\n            cls_col.flatten()\n            .to_numpy(zero_copy_only=False)\n            .reshape(len(cls_col), -1)\n            .astype(np.float32)\n        )\n        finite = np.isfinite(cls_all).all(axis=1)\n        n_nonfinite += int((~finite).sum())\n        keep = (strength == \"strong\") & finite\n        cls = cls_all[keep]\n        lbls = [lb for lb, k in zip(t.column(\"labels\").to_pylist(), keep, strict=True) if k]\n        xs.append(cls)\n        ys.append(np.stack([labels.labels_to_vectors(lb)[0] for lb in lbls]))\n        uids.append(np.asarray(t.column(\"user_id\").to_pylist(), dtype=np.int64)[keep])","sourceCodeStart":20,"sourceCodeEnd":56,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/bdsm/training/train_head.py#L20-L56","documentation":"load_cached expects precomputed CLS-embedding parquet shards under split_dir; if glob('*.parquet') finds nothing it raises FileNotFoundError so callers do not silently train on zero rows.","triggerScenarios":"Calling load_cached('/data/train') when the extraction step has not run or wrote to a sibling dir; split_dir misspelled; parquet files named with a different extension or nested one level deeper; empty split directory in a fresh checkout.","commonSituations":"Running train_head before the cache-building job; wrong split path ('train' vs 'train_cls'); artifacts on a volume not mounted; shards still uploading.","solutions":["List split_dir and confirm *.parquet files exist at that exact path","Run the upstream feature-extraction job that writes the cls parquet shards","Fix the split_dir argument (commonly a missing train/val subdirectory level)","Wait for/retry if shards are still being uploaded by a producer"],"exampleFix":"# before\nload_cached('/data/cls')\n# after\nload_cached('/data/cls/train')  # dir containing part-*.parquet","handlingStrategy":"validation","validationCode":"import glob\nassert glob.glob(os.path.join(split_dir, '*.parquet')), f\"no parquet in {split_dir}\"","typeGuard":"def split_cached(split_dir: str) -> bool:\n    return bool(glob.glob(os.path.join(split_dir, '*.parquet')))","tryCatchPattern":"try:\n    data = load_cached(split_dir)\nexcept FileNotFoundError:\n    run_extraction(split_dir)\n    data = load_cached(split_dir)","preventionTips":["Make extraction a prerequisite step enforced by the training DAG","Check for shard files plus a completion marker before training","Log resolved absolute split paths at startup"],"tags":["python","parquet","dataset","file-not-found"],"backgroundTag":"empty-file-glob","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}