infiniflow/ragflow · error · ConnectorValidationError

BigQuery configured columns not found in schema: {', '.join(

Error message

BigQuery configured columns not found in schema: {', '.join(sorted(missing))}

What it means

After a dry-run resolves the schema, every configured column (content_columns plus optional metadata/id/timestamp columns) must exist in the schema; missing ones are reported as a sorted, comma-joined list.

Source

Thrown at common/data_source/bigquery_connector.py:631

                location=self.location or None,
            )
            estimated_bytes = getattr(dry_run_job, "total_bytes_processed", None)
            if estimated_bytes is not None:
                logging.info("BigQuery base query dry-run estimate: %s bytes processed.", estimated_bytes)

            schema = self._resolve_schema(client, dry_run_job)
            schema_columns = {field.name for field in schema}

            required = set(self.content_columns)
            optional = set(self.metadata_columns)
            if self.id_column:
                optional.add(self.id_column)
            if self.timestamp_column:
                optional.add(self.timestamp_column)

            missing = (required | optional) - schema_columns
            if missing:
                raise ConnectorValidationError(f"BigQuery configured columns not found in schema: {', '.join(sorted(missing))}")

            if self.timestamp_column:
                self._resolve_cursor_param_type()
        except (ConnectorValidationError, ConnectorMissingCredentialError):
            raise
        except Exception as exc:
            raise ConnectorValidationError(f"BigQuery validation failed: {exc}")

View on GitHub (pinned to 554fb1133a)

Solutions

1. Match the reported missing names against the table/query schema and fix the config (spelling, case, alias)
2. If a column is optional metadata you no longer need, remove it from metadata_columns
3. For custom queries, add the missing columns to the SELECT list

Example fix

// before
config = {"content_columns": ["bodies"], ...}  # schema has 'body'

// after
config = {"content_columns": ["body"], ...}

schema_cols = {f.name for f in client.get_table(f"{p}.{d}.{t}").schema}
missing = (set(config["content_columns"]) | set(config.get("metadata_columns", []))
           | ({config["id_column"]} if config.get("id_column") else set())
           | ({config["timestamp_column"]} if config.get("timestamp_column") else set())) - schema_cols
if missing:
    raise ValueError(f"columns not in schema: {sorted(missing)}")

try:
    conn.validate_connector_settings()
except ConnectorValidationError as e:
    if "not found in schema" in str(e):
        sync_config_with_actual_schema()  # rename/remove mapped columns
    else:
        raise

When it happens

Trigger: Any of content_columns, metadata_columns, id_column, or timestamp_column naming a field not produced by the base query or table. The set difference (required | optional) - schema_columns is non-empty.

Common situations: Renamed or dropped warehouse columns; custom query aliasing columns to new names; case mismatches; copying a config between tables with different schemas.

Related errors

• BigQuery requires either a custom query or both dataset_id a
• BigQuery timestamp column '{self.timestamp_column}' was not
• BigQuery project_id is required.
• At least one content column must be specified.
• main() must return a value. Use null for an empty result.

AI-assisted analysis of infiniflow/ragflow@554fb1133a (2026-08-15). Data as JSON: /api/errors/aa15c503a2e49fea. Report an issue: GitHub.