{"record":{"id":"2adc03c389593f8c","repo":"microsoft/qlib","slug":"source-dir-and-target-dir-cannot-be-none","errorCode":null,"errorMessage":"source_dir and target_dir cannot be None","messagePattern":"source_dir and target_dir cannot be None","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"scripts/data_collector/base.py","lineNumber":275,"sourceCode":"        \"\"\"\n\n        Parameters\n        ----------\n        source_dir: str or Path\n            The directory where the raw data collected from the Internet is saved\n        target_dir: str or Path\n            Directory for normalize data\n        normalize_class: Type[YahooNormalize]\n            normalize class\n        max_workers: int\n            Concurrent number, default is 16\n        date_field_name: str\n            date field name, default is date\n        symbol_field_name: str\n            symbol field name, default is symbol\n        \"\"\"\n        if not (source_dir and target_dir):\n            raise ValueError(\"source_dir and target_dir cannot be None\")\n        self._source_dir = Path(source_dir).expanduser()\n        self._target_dir = Path(target_dir).expanduser()\n        self._target_dir.mkdir(parents=True, exist_ok=True)\n        self._date_field_name = date_field_name\n        self._symbol_field_name = symbol_field_name\n        self._end_date = kwargs.get(\"end_date\", None)\n        self._max_workers = max_workers\n        self.interval = kwargs.get(\"interval\", \"1d\")\n\n        self._normalize_obj = normalize_class(\n            date_field_name=date_field_name, symbol_field_name=symbol_field_name, **kwargs\n        )\n\n    def format_data(self, df: pd.DataFrame):\n        if self.interval == \"1d\":\n            try:\n                pd.to_datetime(df.iloc[-1][\"date\"], format=\"%Y-%m-%d\", errors=\"raise\")\n            except Exception:","sourceCodeStart":257,"sourceCodeEnd":293,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/scripts/data_collector/base.py#L257-L293","documentation":"Raised by BaseNormalize constructor in scripts/data_collector/base.py when source_dir or target_dir is falsy (None or empty string). The normalize pipeline needs both an input directory of collected files and an output directory and refuses to continue without them.","triggerScenarios":"Invoking a data_collector tool's normalize_data command without --source_dir or --target_dir, or passing an empty string for either (the check `if not (source_dir and target_dir)` treats '' the same as None).","commonSituations":"Copy-pasting the collector command from docs of a different collector whose argument names differ; shell variable that expands to empty (e.g. $SOURCE_DIR unset); running download_data phase only and assuming normalize infers default directories.","solutions":["Supply both flags: --source_dir <dir of downloaded raw files> --target_dir <dir for normalized files>.","Echo/inspect the shell variables before invoking the CLI to ensure neither is empty.","Check the collector's README example command; some collectors need extra args (e.g. --qlib_data_1d_dir) but source/target are always mandatory."],"exampleFix":"# before\npython collector.py normalize_data --source_dir $SRC --normalize_dir /data/nor\n\n# after\npython collector.py normalize_data --source_dir /data/source/1d --normalize_dir /data/nor","handlingStrategy":"validation","validationCode":"if not source_dir or not target_dir:\n    raise SystemExit(\"--source_dir and --target_dir (or --normalize_dir) are required\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Set and check shell variables before invoking collector CLIs.","Copy the exact example command from the collector's README.","Treat missing mandatory flags as config errors caught in CI, not at runtime."],"tags":["qlib","data-collector","normalize","argument-validation"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}