apache/beam · error · Exception

Could not execute the query. Please check if the query is…

Error message

Could not execute the query. Please check if the query is properly formatted and the table exists. {e}

What it means

The outer except in _execute_query converts any exception (including the RuntimeError above) into Exception('Could not execute the query. Please check if the query is properly formatted and the table exists. ...'), hinting that the SQL itself or the target table is the likely cause.

Solutions

  1. Inspect the chained message for the exact DB error and fix the SQL.
  2. Confirm table_name/db_id point to an existing table in the connected database.
  3. Verify db_adapter matches the actual database engine.
  4. Catch the exception and log e.__cause__ for diagnostics.

Example fix

# before
CustomQueryConfig(query="SELECT * FROM userz WHERE id = {id}")
# after
CustomQueryConfig(query="SELECT * FROM users WHERE id = {id}")
Defensive patterns

Strategy: try-catch

When it happens

Trigger: Invalid SQL syntax, non-existent table or schema, wrong dialect for the configured db_adapter, or any wrapped database failure reaching the outer handler.

Common situations: Typo in the table name; using MySQL dialect against PostgreSQL; the table living in a different database/db_id; missing SELECT permission making the table appear absent.

Understand the failure class

Background: "query failed", "%w: SQL error" — wrapped database query errors in Go libraries explained — this error's family across 3 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9da7e01a92ad89e3. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/transforms/enrichment_handlers/cloudsql.py:407

        if params:
          result = connection.execute(text(query), params)
        else:
          result = connection.execute(text(query))
        # Materialize results while transaction is active.
        data: Union[list[dict[str, Any]], dict[str, Any]]
        if is_batch:
          data = [row._asdict() for row in result]
        else:
          result_row = result.first()
          data = result_row._asdict() if result_row else {}
        # Explicitly commit the transaction.
        transaction.commit()
        return data
      except Exception as e:
        transaction.rollback()
        raise RuntimeError(f"Database operation failed: {e}") from e
    except Exception as e:
      raise Exception(
          f'Could not execute the query. Please check if the query is properly '
          f'formatted and the table exists. {e}') from e
    finally:
      if connection:
        connection.close()

  def _build_batch_query(
      self, requests: list[beam.Row], batch_size: int) -> str:
    """Build batched query with unique parameter names for multiple requests.

    This method extracts parameter placeholders from the where_clause_template
    using regex and creates unique parameter names for each batch item. The
    parameter names in the template can be any valid identifiers (e.g., :id,
    :param_0, :user_name) and don't need to match field names exactly.

    For batch queries, placeholders are replaced with unique names like
    :batch_0_id, :batch_1_param_0, etc., based on the actual parameter names
    found in the template.

View on GitHub (pinned to 12126d8942)