infiniflow/ragflow · error · ConnectorValidationError

BigQuery timestamp column '{self.timestamp_column}' was not

Error message

BigQuery timestamp column '{self.timestamp_column}' was not found in the schema.

What it means

Raised when the configured timestamp_column (used as the incremental-sync cursor) does not appear in the schema resolved from a dry-run of the base query. The connector needs that field to build time-filtered queries, so it fails fast.

Source

Thrown at common/data_source/bigquery_connector.py:251

            return table.schema
        else:
            if dry_run_job is None:
                dry_run_job = client.query(
                    self._wrap_query(self._build_base_query()),
                    job_config=self._build_query_job_config(dry_run=True),
                    location=self.location or None,
                )
            return dry_run_job.schema or []

    def _get_cursor_column_field_type(self) -> str:
        """Resolve the BigQuery field type of the timestamp column."""
        client = self._get_client()
        schema = self._resolve_schema(client)

        for field in schema:
            if field.name == self.timestamp_column:
                return field.field_type
        raise ConnectorValidationError(f"BigQuery timestamp column '{self.timestamp_column}' was not found in the schema.")

    def _resolve_cursor_param_type(self) -> str:
        if self._cursor_param_type is not None:
            return self._cursor_param_type
        field_type = (self._get_cursor_column_field_type() or "").upper()
        param_type = _CURSOR_PARAM_TYPE_MAP.get(field_type)
        if param_type is None:
            raise ConnectorValidationError(f"BigQuery timestamp column type '{field_type}' is not supported as a cursor.")
        self._cursor_param_type = param_type
        return param_type

    def _make_cursor_param(self, name: str, value: Any, param_type: str):
        return bigquery.ScalarQueryParameter(name, param_type, value)

    def _build_time_filtered_query(
        self,
        base_query: str,
        start: Any = None,

View on GitHub (pinned to 554fb1133a)

Solutions

Correct timestamp_column to exactly match a column in the base query's schema (check case)
If using a custom query, make sure it SELECTs the timestamp column unaliased
Temporarily unset timestamp_column if incremental sync by time is not needed

Example fix

// before
config = {"timestamp_column": "update_time", ...}  # column is actually updated_at

// after
config = {"timestamp_column": "updated_at", ...}

Defensive patterns

Strategy: validation

Validate before calling

# Pre-check against information_schema before enabling sync
sql = """SELECT column_name FROM region-us.INFORMATION_SCHEMA.COLUMNS
WHERE table_schema = @ds AND table_name = @tbl"""
cols = {r[0] for r in client.query(sql, params=...).result()}
if config["timestamp_column"] not in cols:
    raise ValueError(f"timestamp_column {config['timestamp_column']!r} not in table")

Try / catch

try:
    conn.validate_connector_settings()
except ConnectorValidationError as e:
    if "timestamp column" in str(e):
        disable_or_fix_timestamp_column()  # correct name or drop incremental sync
    else:
        raise

Prevention

Run validate_connector_settings() whenever upstream schemas change
Pin column names from the resolved schema, not from memory
Add schema-drift alerts on the warehouse side

When it happens

Trigger: Setting timestamp_column to a name that is not returned by the custom query or the table (typo, casing mismatch, or the column exists in another table). _get_cursor_column_field_type iterates schema fields and finds no field.name == timestamp_column.

Common situations: Column renamed in the warehouse; custom SELECT that aliases or omits the timestamp column; quoting/casing differences between config and BigQuery schema.

Related errors

AI-assisted analysis of infiniflow/ragflow@554fb1133a (2026-08-15). Data as JSON: /api/errors/e0861581bcb92763. Report an issue: GitHub.