{"record":{"id":"e0861581bcb92763","repo":"infiniflow/ragflow","slug":"bigquery-timestamp-column-self-timestamp-column","errorCode":null,"errorMessage":"BigQuery timestamp column '{self.timestamp_column}' was not found in the schema.","messagePattern":"BigQuery timestamp column '(.+?)' was not found in the schema\\.","errorType":"validation","errorClass":"ConnectorValidationError","httpStatus":null,"severity":"error","filePath":"common/data_source/bigquery_connector.py","lineNumber":251,"sourceCode":"            return table.schema\n        else:\n            if dry_run_job is None:\n                dry_run_job = client.query(\n                    self._wrap_query(self._build_base_query()),\n                    job_config=self._build_query_job_config(dry_run=True),\n                    location=self.location or None,\n                )\n            return dry_run_job.schema or []\n\n    def _get_cursor_column_field_type(self) -> str:\n        \"\"\"Resolve the BigQuery field type of the timestamp column.\"\"\"\n        client = self._get_client()\n        schema = self._resolve_schema(client)\n\n        for field in schema:\n            if field.name == self.timestamp_column:\n                return field.field_type\n        raise ConnectorValidationError(f\"BigQuery timestamp column '{self.timestamp_column}' was not found in the schema.\")\n\n    def _resolve_cursor_param_type(self) -> str:\n        if self._cursor_param_type is not None:\n            return self._cursor_param_type\n        field_type = (self._get_cursor_column_field_type() or \"\").upper()\n        param_type = _CURSOR_PARAM_TYPE_MAP.get(field_type)\n        if param_type is None:\n            raise ConnectorValidationError(f\"BigQuery timestamp column type '{field_type}' is not supported as a cursor.\")\n        self._cursor_param_type = param_type\n        return param_type\n\n    def _make_cursor_param(self, name: str, value: Any, param_type: str):\n        return bigquery.ScalarQueryParameter(name, param_type, value)\n\n    def _build_time_filtered_query(\n        self,\n        base_query: str,\n        start: Any = None,","sourceCodeStart":233,"sourceCodeEnd":269,"githubUrl":"https://github.com/infiniflow/ragflow/blob/554fb1133ac3861732235ad9c377eb5e0a770665/common/data_source/bigquery_connector.py#L233-L269","documentation":"Raised when the configured timestamp_column (used as the incremental-sync cursor) does not appear in the schema resolved from a dry-run of the base query. The connector needs that field to build time-filtered queries, so it fails fast.","triggerScenarios":"Setting timestamp_column to a name that is not returned by the custom query or the table (typo, casing mismatch, or the column exists in another table). _get_cursor_column_field_type iterates schema fields and finds no field.name == timestamp_column.","commonSituations":"Column renamed in the warehouse; custom SELECT that aliases or omits the timestamp column; quoting/casing differences between config and BigQuery schema.","solutions":["Correct timestamp_column to exactly match a column in the base query's schema (check case)","If using a custom query, make sure it SELECTs the timestamp column unaliased","Temporarily unset timestamp_column if incremental sync by time is not needed"],"exampleFix":"// before\nconfig = {\"timestamp_column\": \"update_time\", ...}  # column is actually updated_at\n\n// after\nconfig = {\"timestamp_column\": \"updated_at\", ...}","handlingStrategy":"validation","validationCode":"# Pre-check against information_schema before enabling sync\nsql = \"\"\"SELECT column_name FROM region-us.INFORMATION_SCHEMA.COLUMNS\nWHERE table_schema = @ds AND table_name = @tbl\"\"\"\ncols = {r[0] for r in client.query(sql, params=...).result()}\nif config[\"timestamp_column\"] not in cols:\n    raise ValueError(f\"timestamp_column {config['timestamp_column']!r} not in table\")","typeGuard":null,"tryCatchPattern":"try:\n    conn.validate_connector_settings()\nexcept ConnectorValidationError as e:\n    if \"timestamp column\" in str(e):\n        disable_or_fix_timestamp_column()  # correct name or drop incremental sync\n    else:\n        raise","preventionTips":["Run validate_connector_settings() whenever upstream schemas change","Pin column names from the resolved schema, not from memory","Add schema-drift alerts on the warehouse side"],"tags":["bigquery","schema","incremental-sync","validation"],"backgroundTag":null,"analyzedSha":"554fb1133ac3861732235ad9c377eb5e0a770665","analyzedAt":"2026-08-15T09:20:16.380Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}