apache/beam · error · IllegalArgumentException

Table reference [ ] must include at least a dataset and a…

Error message

Table reference [%s] must include at least a dataset and a table.

What it means

BigQueryUtils.toTableSpec builds a table spec string (project.dataset.table or dataset.table) from a TableReference. It requires at least datasetId and tableId; if either is null it throws IllegalArgumentException naming the incomplete reference.

Solutions

1. Set both datasetId and tableId on the TableReference before calling toTableSpec
2. Validate the source string with BigQueryHelpers.parseTableSpec or TableReference.fromDatasetId instead of manual string building
3. Log/inspect tableReference in the message to find which field is missing
4. Add a pre-call guard asserting the fields are non-null

Example fix

// before
TableReference ref = TableReference.newBuilder().setProjectId("p").build();
String spec = BigQueryUtils.toTableSpec(ref);
// after
TableReference ref = TableReference.newBuilder()
    .setProjectId("p").setDatasetId("mydataset").setTableId("mytable").build();
String spec = BigQueryUtils.toTableSpec(ref);

if (ref.getDatasetId() == null || ref.getTableId() == null) {
  throw new IllegalArgumentException("TableReference needs datasetId and tableId: " + ref);
}
String spec = BigQueryUtils.toTableSpec(ref);

boolean isCompleteRef(TableReference r) { return r.getDatasetId() != null && r.getTableId() != null; }

try { spec = BigQueryUtils.toTableSpec(ref); } catch (IllegalArgumentException e) { LOG.error("Incomplete table reference: %s", e.getMessage()); throw e; }

When it happens

Trigger: Calling BigQueryUtils.toTableSpec (or Write/Read paths that call it) with a TableReference constructed without setting datasetId or tableId — e.g. a builder that only set the project, or a reference parsed from a malformed table spec.

Common situations: Parsing user-supplied table IDs like "mytable" or "myproject." that lack dataset parts; building TableReference programmatically and forgetting setDatasetId/setTableId; config values with typos (dataset vs datasetId); dynamic destinations where destination lookup returned an empty table.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• Expected a table reference (PROJECT:DATASET.TABLE or…
• Table specification [ ] is not in one of the expected…
• A BigQuery table or a query must be specified
• A function must be provided to convert the input type into…
• A schema is required in order to prepare rows for writing…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/27f0c44af2852af9. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryUtils.java:1289

    if (m.matches()) {
      // The named groups are not optional, so they are non-null whenever the pattern matches.
      return new TableReference()
          .setProjectId(Preconditions.checkStateNotNull(m.group("PROJECT")))
          .setDatasetId(Preconditions.checkStateNotNull(m.group("DATASET")))
          .setTableId(Preconditions.checkStateNotNull(m.group("TABLE")));
    }
    return null;
  }

  /**
   * Returns a String representation of the table destination in the form:
   * `myproject.mydataset.mytable`.
   *
   * @param tableReference - a BigQueryTableIdentifier that may or may not include the project.
   */
  public static @Nullable String toTableSpec(TableReference tableReference) {
    if (tableReference.getDatasetId() == null || tableReference.getTableId() == null) {
      throw new IllegalArgumentException(
          String.format(
              "Table reference [%s] must include at least a dataset and a table.", tableReference));
    }
    String tableSpec =
        String.format("%s.%s", tableReference.getDatasetId(), tableReference.getTableId());
    if (!Strings.isNullOrEmpty(tableReference.getProjectId())) {
      tableSpec = String.format("%s.%s", tableReference.getProjectId(), tableSpec);
    }
    return tableSpec;
  }

  static TableSchema trimSchema(TableSchema schema, @Nullable List<String> selectedFields) {
    if (selectedFields == null || selectedFields.isEmpty()) {
      return schema;
    }

    List<TableFieldSchema> trimmedFields =
        schema.getFields().stream()

View on GitHub (pinned to 12126d8942)