{"record":{"id":"422929a7bf93c044","repo":"abhigyanpatwari/GitNexus","slug":"skill-fingerprint-input-exceeds-the-bounded-eviden","errorCode":null,"errorMessage":"skill fingerprint input exceeds the bounded evidence limit","messagePattern":"skill fingerprint input exceeds the bounded evidence limit","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"eval/workflow_bench/evolution.py","lineNumber":473,"sourceCode":"            Path(\".claude\") / \"skills\" / skill_name,\n            label=\"skill fingerprint root\",\n        )\n\n    entries = sorted(\n        (path for skill_name in skill_names for path in (worktree / \".claude\" / \"skills\" / skill_name).rglob(\"*\")),\n        key=lambda path: path.relative_to(worktree).as_posix(),\n    )\n    files: list[Path] = []\n    total = 0\n    for path in entries:\n        metadata = path.lstat()\n        if stat.S_ISDIR(metadata.st_mode):\n            continue\n        if stat.S_ISLNK(metadata.st_mode) or not stat.S_ISREG(metadata.st_mode):\n            raise ValueError(f\"skill fingerprint input must be a regular non-symlink file: {path}\")\n        total += metadata.st_size\n        if total > MAX_SKILL_FINGERPRINT_BYTES:\n            raise ValueError(\"skill fingerprint input exceeds the bounded evidence limit\")\n        files.append(path)\n    return fingerprint_files(worktree, files)\n\n\ndef evaluate_candidate(\n    results: dict[str, dict[str, dict[str, Any]]],\n    *,\n    incumbent_arm: str,\n    candidate_arm: str,\n    model: str | None,\n    metric: str = \"cost_usd\",\n    min_runs: int = 3,\n    min_improvement_pct: float = 5.0,\n    max_task_regression_pct: float = 20.0,\n) -> dict[str, Any]:\n    \"\"\"Deterministically decide whether a prompt candidate is promotable.\n\n    Resolution is lexicographically primary: a cheaper candidate that fails","sourceCodeStart":455,"sourceCodeEnd":491,"githubUrl":"https://github.com/abhigyanpatwari/GitNexus/blob/d540b00184d71a896261ee02670da9a92d59d8f7/eval/workflow_bench/evolution.py#L455-L491","documentation":"In the same skill_fingerprint walk, the cumulative byte size of all regular files is capped at MAX_SKILL_FINGERPRINT_BYTES (4 MiB). Exceeding it raises ValueError. The bound keeps fingerprinting fast and the proposer evidence payload bounded, so a single oversized skill cannot dominate a generation.","triggerScenarios":"The arm's evaluated skills together exceed 4 MiB — e.g. large embedded JSON/JSONL examples, vendored grammars, or full file trees accidentally placed under `.claude/skills/`.","commonSituations":"A skill that bundles sample output, a long corpus, or a vendored library; skills directory used as a general asset store rather than prompt text.","solutions":["Measure first: `du -sh .claude/skills/<skill>` and `find .claude/skills/<skill> -type f -printf '%s %p\\n' | sort -nr | head`.","Move bulk data (fixtures, samples) out of the skills dir into a non-fingerprinted location.","Split an oversized skill, or trim embedded examples to the essentials."],"exampleFix":"# before: skill ships a 6 MiB example corpus inline\n.claude/skills/my-skill/examples/large_corpus.jsonl  # 6 MiB\n\n# after: reference it by path and keep only a trimmed sample under the limit\ncp examples/large_corpus.jsonl /tmp/assets/\n# in the skill: \"see /tmp/assets/large_corpus.jsonl\"\n# keep .claude/skills/my-skill/examples/sample.jsonl under ~100 KiB","handlingStrategy":"validation","validationCode":"import stat\nfrom pathlib import Path\nfrom eval.workflow_bench.evolution import MAX_SKILL_FINGERPRINT_BYTES, EVALUATED_ARM_SKILLS\n\ndef arm_skill_bytes_within(clone: Path, arm: str) -> bool:\n    names = EVALUATED_ARM_SKILLS.get(arm, [])\n    total = 0\n    for name in names:\n        for p in (clone / \".claude\" / \"skills\" / name).rglob(\"*\"):\n            m = p.lstat().st_mode\n            if not stat.S_ISREG(m) or stat.S_ISLNK(m):\n                continue\n            total += m  # placeholder; use p.lstat().st_size\n    # correct size sum:\n    total = sum(p.lstat().st_size for name in names for p in (clone/\".claude\"/\"skills\"/name).rglob(\"*\") if stat.S_ISREG(p.lstat().st_mode))\n    return total <= MAX_SKILL_FINGERPRINT_BYTES","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep skills as prompt text; move fixtures/data elsewhere.","Measure with `du -sh .claude/skills/<skill>` before publishing a candidate.","Treat the 4 MiB bound as a design signal: a skill over the limit is probably doing too much."],"tags":["workflow-bench","fingerprint","size-limit","validation"],"backgroundTag":null,"analyzedSha":"d540b00184d71a896261ee02670da9a92d59d8f7","analyzedAt":"2026-08-12T19:50:25.132Z","schemaVersion":2},"datasetVersion":"2026-08-12T23:17:12.415Z"}