{"record":{"id":"d5f8da3afc5b144e","repo":"huggingface/open-r1","slug":"cf-tests-folder-path-tests-folder-does-not-exi","errorCode":null,"errorMessage":"CF_TESTS_FOLDER path '{tests_folder}' does not exist! Please download the codeforces generated tests and set CF_TESTS_FOLDER to the folder path. See https://huggingface.co/datasets/open-r1/codeforces for more information.","messagePattern":"CF_TESTS_FOLDER path '(.+?)' does not exist! Please download the codeforces generated tests and set CF_TESTS_FOLDER to the folder path\\. See https://huggingface\\.co/datasets/open-r1/codeforces for more information\\.","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/open_r1/utils/competitive_programming/cf_scoring.py","lineNumber":71,"sourceCode":"        return False\n\n    return result\n\n\n@alru_cache(maxsize=32)  # TODO make this configurable\nasync def get_generated_contest_tests(contest_id: str) -> list[dict]:\n    import pandas as pd\n\n    import aiofiles\n    import aiofiles.os\n\n    tests_folder = os.environ.get(\"CF_TESTS_FOLDER\", None)\n    if not tests_folder:\n        raise ValueError(\n            \"CF_TESTS_FOLDER environment variable not set! Please download the codeforces generated tests and set CF_TESTS_FOLDER to the folder path. See https://huggingface.co/datasets/open-r1/codeforces for more information.\"\n        )\n    if not await aiofiles.os.path.exists(tests_folder):\n        raise ValueError(\n            f\"CF_TESTS_FOLDER path '{tests_folder}' does not exist! Please download the codeforces generated tests and set CF_TESTS_FOLDER to the folder path. See https://huggingface.co/datasets/open-r1/codeforces for more information.\"\n        )\n    parquet_path = os.path.join(tests_folder, f\"test_cases_{int(contest_id):04d}.parquet\")\n    if not await aiofiles.os.path.exists(parquet_path):\n        return {}\n\n    # Read parquet file asynchronously\n    async with aiofiles.open(parquet_path, \"rb\") as f:\n        content = await f.read()\n        df = pd.read_parquet(BytesIO(content))\n\n    # Group by problem_id and convert to dictionary of lists\n    grouped_tests = df.groupby(\"problem_id\").apply(lambda x: x[[\"input\", \"output\"]].to_dict(\"records\")).to_dict()\n\n    return grouped_tests\n\n\nasync def get_generated_tests(problem_id: str) -> list[dict]:","sourceCodeStart":53,"sourceCodeEnd":89,"githubUrl":"https://github.com/huggingface/open-r1/blob/1416fa0cf21595d2083b399a2a0bbddd7f6e9563/src/open_r1/utils/competitive_programming/cf_scoring.py#L53-L89","documentation":"This is the follow-up check to error 16: CF_TESTS_FOLDER is set, but the path it points to does not exist on the filesystem (checked asynchronously with aiofiles.os.path.exists). The same remediation guidance is embedded in the message, including the dataset link.","triggerScenarios":"get_generated_contest_tests runs with CF_TESTS_FOLDER set to a path that fails the exists() check — deleted/moved folder, relative path resolved against a different working directory, or the tests downloaded on another machine/volume not mounted on the node.","commonSituations":"Relative path in .env that doesn't resolve from the job's cwd; cluster node without the shared filesystem mounted; folder deleted to free disk space; typo in the path or trailing whitespace in the env value; missing download step on a new machine.","solutions":["Verify the path exists: ls \"$CF_TESTS_FOLDER\"","Use an absolute path that resolves identically on all nodes (shared/network filesystem for clusters)","Re-download the test parquet files from https://huggingface.co/datasets/open-r1/codeforces","Check the value for typos, whitespace, or wrong username/volume paths; confirm cwd assumptions for relative paths"],"exampleFix":"// before\n# .env: CF_TESTS_FOLDER=./tests  (dir absent on compute node)\nscore = await score_submission(problem, code)  # ValueError: path does not exist\n// after\n# .env: CF_TESTS_FOLDER=/mnt/shared/codeforces_tests\nscore = await score_submission(problem, code)","handlingStrategy":"validation","validationCode":"import os\nfrom pathlib import Path\n\ndef require_cf_tests_path():\n    raw = os.environ.get(\"CF_TESTS_FOLDER\", \"\")\n    path = Path(raw.strip()).expanduser().resolve()\n    if not path.is_dir():\n        raise SystemExit(\n            f\"CF_TESTS_FOLDER resolves to {path}, which does not exist. \"\n            \"Download tests from https://huggingface.co/datasets/open-r1/codeforces\"\n        )\n    if not any(path.glob(\"test_cases_*.parquet\")):\n        raise SystemExit(f\"No test_cases_*.parquet files found in {path}\")","typeGuard":"def cf_tests_folder_exists() -> bool:\n    import os\n    p = os.environ.get(\"CF_TESTS_FOLDER\")\n    return bool(p) and os.path.isdir(p)","tryCatchPattern":"try:\n    tests = await get_generated_tests(problem_id)\nexcept ValueError as e:\n    if \"does not exist\" in str(e):\n        logger.error(\"%s — check path, mounts, and that tests were downloaded on this node\", e)\n        raise SystemExit(1) from e\n    raise","preventionTips":["Store tests on a filesystem mounted identically on all compute nodes","ls the folder (and glob test_cases_*.parquet) in your launch script before scoring","Re-download after cleaning disk space; don't assume another machine's copy","Trim/expanduser the env value to avoid whitespace and ~ issues"],"tags":["python","filesystem","missing-path","configuration"],"backgroundTag":"missing-path","analyzedSha":"1416fa0cf21595d2083b399a2a0bbddd7f6e9563","analyzedAt":"2026-08-30T08:56:53.400Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}