{"record":{"id":"e3827fe3b77e90c6","repo":"microsoft/qlib","slug":"pit-database-does-not-support-referring-to-future","errorCode":null,"errorMessage":"PIT database does not support referring to future period (e.g. expressions like `Ref('$$roewa_q', -1)` are not supported","messagePattern":"PIT database does not support referring to future period \\(e\\.g\\. expressions like `Ref\\('\\$\\$roewa_q', -1\\)` are not supported","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"qlib/data/pit.py","lineNumber":34,"sourceCode":"\nimport numpy as np\nimport pandas as pd\nfrom qlib.data.ops import ElemOperator\nfrom qlib.log import get_module_logger\nfrom .data import Cal\n\n\nclass P(ElemOperator):\n    def _load_internal(self, instrument, start_index, end_index, freq):\n        _calendar = Cal.calendar(freq=freq)\n        resample_data = np.empty(end_index - start_index + 1, dtype=\"float32\")\n\n        for cur_index in range(start_index, end_index + 1):\n            cur_time = _calendar[cur_index]\n            # To load expression accurately, more historical data are required\n            start_ws, end_ws = self.feature.get_extended_window_size()\n            if end_ws > 0:\n                raise ValueError(\n                    \"PIT database does not support referring to future period (e.g. expressions like `Ref('$$roewa_q', -1)` are not supported\"\n                )\n\n            # The calculated value will always the last element, so the end_offset is zero.\n            try:\n                s = self._load_feature(instrument, -start_ws, 0, cur_time)\n                resample_data[cur_index - start_index] = s.iloc[-1] if len(s) > 0 else np.nan\n            except FileNotFoundError:\n                get_module_logger(\"base\").warning(f\"WARN: period data not found for {str(self)}\")\n                return pd.Series(dtype=\"float32\", name=str(self))\n\n        resample_series = pd.Series(\n            resample_data, index=pd.RangeIndex(start_index, end_index + 1), dtype=\"float32\", name=str(self)\n        )\n        return resample_series\n\n    def _load_feature(self, instrument, start_index, end_index, cur_time):\n        return self.feature.load(instrument, start_index, end_index, cur_time)","sourceCodeStart":16,"sourceCodeEnd":52,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/qlib/data/pit.py#L16-L52","documentation":"Raised by the PIT (point-in-time) operator P (qlib/data/pit.py) when the wrapped feature's extended window has a positive end offset — i.e. the expression needs FUTURE data relative to each evaluation timestamp. Financial PIT data must only be read as-of the current date to avoid look-ahead bias, so forward references like Ref('$$roewa_q', -1) inside P(...) are rejected with this ValueError.","triggerScenarios":"P(Ref($$roewa_q, -1)) or any P-wrapped expression whose get_extended_window_size() returns end_ws > 0 — negative-count Ref (future reference in qlib's sign convention), or a nested operator that looks ahead, inside P().","commonSituations":"Building point-in-time fundamental features and naively reusing lag/lead patterns from price features; the sign convention trips people: Ref(x, -N) refers to the future in qlib, Ref(x, N) to the past.","solutions":["Remove the future-looking reference: use Ref($$roewa_q, N) with N >= 0 for past values inside P(...).","Restructure so P(...) wraps only the raw PIT field and apply any forward shift outside P (and reconsider whether forward shifting is legitimate at all for your backtest).","Inspect feature.get_extended_window_size() of the wrapped expression; it must return end_ws == 0."],"exampleFix":"# before (future reference inside PIT operator — look-ahead, rejected)\nexpr = \"P(Ref($$roewa_q, -1))\"\n\n# after (past reference only)\nexpr = \"P(Ref($$roewa_q, 1))\"","handlingStrategy":"validation","validationCode":"_, end_ws = feature.get_extended_window_size()\nif end_ws > 0:\n    raise ValueError(\"P(...) cannot wrap expressions that reference future data\")","typeGuard":"def is_pit_safe(feature) -> bool:\n    return feature.get_extended_window_size()[1] <= 0","tryCatchPattern":null,"preventionTips":["Never use negative N in Ref inside P(...); negative N means future in qlib.","Check get_extended_window_size() when composing custom PIT expressions."],"tags":["qlib","pit","lookahead-bias","expression"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}