infiniflow/ragflow · error · ConnectorValidationError

BigQuery requires either a custom query or both dataset_id a

Error message

BigQuery requires either a custom query or both dataset_id and table_id.

What it means

Raised by BigQueryConnector._build_base_query when the connector has neither a custom SQL `query` nor the pair `dataset_id` + `table_id` set. The connector cannot form any SELECT statement, so it refuses before touching the BigQuery API.

Source

Thrown at common/data_source/bigquery_connector.py:205

                project=self.project_id or None,
                credentials=creds,
                location=self.location or None,
            )
        except Exception as exc:
            raise ConnectorValidationError(f"Failed to create BigQuery client: {exc}")

        return self._client

    # ------------------------------------------------------------------ #
    # Query construction
    # ------------------------------------------------------------------ #
    def _build_base_query(self) -> str:
        """Return the single base query (custom query takes precedence over table mode)."""
        if self.query:
            return self.query.rstrip(";")
        if self.dataset_id and self.table_id:
            return f"SELECT * FROM `{self.project_id}.{self.dataset_id}.{self.table_id}`"
        raise ConnectorValidationError("BigQuery requires either a custom query or both dataset_id and table_id.")

    @staticmethod
    def _wrap_query(base_query: str, select_clause: str = "*") -> str:
        return f"SELECT {select_clause} FROM ({base_query}) AS ragflow_src"

    def _build_query_job_config(
        self,
        query_parameters: Optional[List[Any]] = None,
        dry_run: bool = False,
    ):
        config = bigquery.QueryJobConfig()
        config.use_legacy_sql = False
        config.use_query_cache = self.use_query_cache
        if self.maximum_bytes_billed:
            config.maximum_bytes_billed = int(self.maximum_bytes_billed)
        if self.job_timeout_ms:
            config.job_timeout_ms = int(self.job_timeout_ms)
        if query_parameters:

View on GitHub (pinned to 554fb1133a)

Solutions

  1. Set a complete table target: both dataset_id and table_id (project_id comes from credentials/config), e.g. {'project_id': 'p', 'dataset_id': 'd', 'table_id': 't'}
  2. Or provide a custom query instead: config['query'] = 'SELECT ... FROM ...'
  3. Verify config keys are exactly query/dataset_id/table_id before constructing the connector

Example fix

// before
config = {"project_id": "my-proj", "dataset_id": "my_data"}  # table_id missing

// after
config = {"project_id": "my-proj", "dataset_id": "my_data", "table_id": "events"}
# or: config = {"project_id": "my-proj", "query": "SELECT * FROM `my-proj.my_data.events`"}
Defensive patterns

Strategy: validation

Validate before calling

def has_bigquery_target(cfg: dict) -> bool:
    return bool(cfg.get("query") or (cfg.get("dataset_id") and cfg.get("table_id")))

if not has_bigquery_target(config):
    raise ValueError("Set 'query' or both 'dataset_id' and 'table_id'")

Try / catch

try:
    conn.fetch(...)
except ConnectorValidationError as e:
    if "custom query" in str(e):
        fix_target_config()  # set query or dataset_id+table_id
    else:
        raise

Prevention

When it happens

Trigger: Calling any method that builds the base query (fetch, sync, validate_connector_settings, schema resolution) with a config where query is empty AND at least one of dataset_id/table_id is empty.

Common situations: Connector config dict created from a form where the user left the query field blank and filled only dataset_id, or only table_id; config keys typo'd (e.g. 'datasetId' vs 'dataset_id'); partially migrated configs after a schema change.

Related errors

AI-assisted analysis of infiniflow/ragflow@554fb1133a (2026-08-15). Data as JSON: /api/errors/91392c0debbc3974. Report an issue: GitHub.