apache/seatunnel · error · CatalogException

Failed to execute custom SQL

Error message

Failed to execute custom SQL

What it means

Thrown by BigQueryCatalog.executeSql when the bigquery.query call executing a custom SQL statement fails with any non-interrupt exception. The message is generic ('Failed to execute custom SQL') and the actual reason is in the wrapped cause.

Source

Thrown at seatunnel-connectors-v2/connector-bigquery/src/main/java/org/apache/seatunnel/connectors/bigquery/catalog/BigQueryCatalog.java:417

            return table != null && table.getNumRows().longValue() > 0;
        }
    }

    @Override
    public void executeSql(TablePath tablePath, String sql) {
        if (sql == null || sql.trim().isEmpty()) {
            log.warn("No custom SQL query provided for table {}, skipping execution.", tablePath);
            return;
        }
        log.info("Executing custom SQL in dataset {}: {}", getDatasetName(tablePath), sql);
        QueryJobConfiguration queryConfig = QueryJobConfiguration.newBuilder(sql).build();
        try {
            bigquery.query(queryConfig);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
            throw new CatalogException("Custom SQL execution interrupted", e);
        } catch (Exception e) {
            throw new CatalogException("Failed to execute custom SQL", e);
        }
    }

    private Field convertColumn(Column column) {
        StandardSQLTypeName bqType = mapToBigQueryType(column.getDataType().getSqlType());
        Field.Builder fieldBuilder = Field.newBuilder(column.getName(), bqType);

        if (column.isNullable()) {
            fieldBuilder.setMode(Field.Mode.NULLABLE);
        } else {
            fieldBuilder.setMode(Field.Mode.REQUIRED);
        }

        if (column.getDataType().getSqlType() == SqlType.ROW) {
            SeaTunnelRowType rowType = (SeaTunnelRowType) column.getDataType();
            List<Field> subFields = new ArrayList<>();
            for (int i = 0; i < rowType.getFieldNames().length; i++) {
                Column subColumn =

View on GitHub (pinned to cf67b549a7)

Solutions

1. Unwrap the cause in the stack trace for the precise BigQuery error (400 invalid query, 404 not found, 403 permission, 429 quota)
2. Run the SQL directly in the BigQuery console with the same credentials to reproduce and debug syntax/table references
3. Verify the target dataset/project in tablePath matches reality and permissions are granted
4. Split large statements and retry transient 429/5xx errors

Example fix

// before
catalog.executeSql(tablePath, "CREATE TABLE IF NOT EXISTS t (id INT)");
// after
try {
    catalog.executeSql(tablePath, "CREATE TABLE IF NOT EXISTS t (id INT64)");
} catch (CatalogException e) {
    LOG.error("SQL failed, check cause", e.getCause());
    throw e;
}

try {
    catalog.executeSql(tablePath, sql);
} catch (CatalogException e) {
    Throwable cause = e.getCause();
    if (cause instanceof BigQueryException && ((BigQueryException) cause).getCode() == 429) {
        // backoff and retry
    } else {
        LOG.error("Custom SQL failed: " + (cause == null ? e : cause.getMessage()), e);
        throw e;
    }
}

When it happens

Trigger: Calling executeSql(tablePath, sql) with syntactically invalid SQL, a query that references nonexistent tables/columns, insufficient permissions (bigquery.jobs.create / dataset access), invalid legacy-vs-standard SQL mode, or quota/network failures.

Common situations: Save-mode DDL statements (CREATE TABLE / MERGE) that don't match actual BigQuery schema; wrong default dataset configured; project-level query quota exceeded; using legacy SQL syntax while the client defaults to standard SQL.

Understand the failure class

Background: "query failed", "%w: SQL error" — wrapped database query errors in Go libraries explained — this error's family across 3 libraries.

Related errors

• Failed to truncate BigQuery table:
• Failed to create BigQuery dataset:
• Failed to drop BigQuery dataset:
• Custom SQL execution interrupted
• Preview action is not supported

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/3f282a8720db0359. Report an issue: GitHub.