{"record":{"id":"3f282a8720db0359","repo":"apache/seatunnel","slug":"failed-to-execute-custom-sql","errorCode":null,"errorMessage":"Failed to execute custom SQL","messagePattern":"Failed to execute custom SQL","errorType":"exception","errorClass":"CatalogException","httpStatus":null,"severity":"error","filePath":"seatunnel-connectors-v2/connector-bigquery/src/main/java/org/apache/seatunnel/connectors/bigquery/catalog/BigQueryCatalog.java","lineNumber":417,"sourceCode":"            return table != null && table.getNumRows().longValue() > 0;\n        }\n    }\n\n    @Override\n    public void executeSql(TablePath tablePath, String sql) {\n        if (sql == null || sql.trim().isEmpty()) {\n            log.warn(\"No custom SQL query provided for table {}, skipping execution.\", tablePath);\n            return;\n        }\n        log.info(\"Executing custom SQL in dataset {}: {}\", getDatasetName(tablePath), sql);\n        QueryJobConfiguration queryConfig = QueryJobConfiguration.newBuilder(sql).build();\n        try {\n            bigquery.query(queryConfig);\n        } catch (InterruptedException e) {\n            Thread.currentThread().interrupt();\n            throw new CatalogException(\"Custom SQL execution interrupted\", e);\n        } catch (Exception e) {\n            throw new CatalogException(\"Failed to execute custom SQL\", e);\n        }\n    }\n\n    private Field convertColumn(Column column) {\n        StandardSQLTypeName bqType = mapToBigQueryType(column.getDataType().getSqlType());\n        Field.Builder fieldBuilder = Field.newBuilder(column.getName(), bqType);\n\n        if (column.isNullable()) {\n            fieldBuilder.setMode(Field.Mode.NULLABLE);\n        } else {\n            fieldBuilder.setMode(Field.Mode.REQUIRED);\n        }\n\n        if (column.getDataType().getSqlType() == SqlType.ROW) {\n            SeaTunnelRowType rowType = (SeaTunnelRowType) column.getDataType();\n            List<Field> subFields = new ArrayList<>();\n            for (int i = 0; i < rowType.getFieldNames().length; i++) {\n                Column subColumn =","sourceCodeStart":399,"sourceCodeEnd":435,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-bigquery/src/main/java/org/apache/seatunnel/connectors/bigquery/catalog/BigQueryCatalog.java#L399-L435","documentation":"Thrown by BigQueryCatalog.executeSql when the bigquery.query call executing a custom SQL statement fails with any non-interrupt exception. The message is generic ('Failed to execute custom SQL') and the actual reason is in the wrapped cause.","triggerScenarios":"Calling executeSql(tablePath, sql) with syntactically invalid SQL, a query that references nonexistent tables/columns, insufficient permissions (bigquery.jobs.create / dataset access), invalid legacy-vs-standard SQL mode, or quota/network failures.","commonSituations":"Save-mode DDL statements (CREATE TABLE / MERGE) that don't match actual BigQuery schema; wrong default dataset configured; project-level query quota exceeded; using legacy SQL syntax while the client defaults to standard SQL.","solutions":["Unwrap the cause in the stack trace for the precise BigQuery error (400 invalid query, 404 not found, 403 permission, 429 quota)","Run the SQL directly in the BigQuery console with the same credentials to reproduce and debug syntax/table references","Verify the target dataset/project in tablePath matches reality and permissions are granted","Split large statements and retry transient 429/5xx errors"],"exampleFix":"// before\ncatalog.executeSql(tablePath, \"CREATE TABLE IF NOT EXISTS t (id INT)\");\n// after\ntry {\n    catalog.executeSql(tablePath, \"CREATE TABLE IF NOT EXISTS t (id INT64)\");\n} catch (CatalogException e) {\n    LOG.error(\"SQL failed, check cause\", e.getCause());\n    throw e;\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    catalog.executeSql(tablePath, sql);\n} catch (CatalogException e) {\n    Throwable cause = e.getCause();\n    if (cause instanceof BigQueryException && ((BigQueryException) cause).getCode() == 429) {\n        // backoff and retry\n    } else {\n        LOG.error(\"Custom SQL failed: \" + (cause == null ? e : cause.getMessage()), e);\n        throw e;\n    }\n}","preventionTips":["Test custom SQL in the BigQuery console with the same service account first","Use standard SQL syntax matching the client's SQL dialect default","Grant bigquery.jobs.create and dataset-level access for the caller","Add backoff/retry for quota (429) and transient 5xx errors"],"tags":["bigquery","custom-sql","sql-query","catalog"],"backgroundTag":"sql-query-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}