apache/beam · error · IOException

${format}

Error message

${format}

What it means

FakeDatasetService.throwNotFound is a @FormatMethod helper that throws an IOException wrapping an HTTP 404 HttpResponseException with a formatted message. The fake calls it for any simulated BigQuery resource lookup failure (missing table, dataset, job, etc.), so callers see a 404-shaped IOException whose message is the format string filled with the offending resource id.

Solutions

1. Create the dataset/table in the fake before the operation: call the fake's createDataset/createTable (or insert rows to auto-create) with the exact identifiers used by the pipeline.
2. Verify the table spec string (project:dataset.table) matches what the test registered.
3. Catch IOException and inspect the embedded HttpResponseException status code (404) to distinguish not-found from other fake failures.
4. Re-run with the fake's state dump (e.g., list tables) to confirm the resource exists.

Example fix

// before
pipeline.apply(BigQueryIO.read().from("proj:data.missing_table"));
// after
fakeDatasetService.createDataset("proj", "data", "US");
fakeDatasetService.createTable("proj", "data", "missing_table", schema);
pipeline.apply(BigQueryIO.read().from("proj:data.missing_table"));

if (fake.getTable(tableSpec) == null) {
  throw new IllegalStateException("Table not registered in fake: " + tableSpec);
}

try {
  runPipelineAgainst(fake);
} catch (IOException e) {
  Throwable cause = e.getCause();
  if (cause instanceof HttpResponseException && ((HttpResponseException) cause).getStatusCode() == 404) {
    throw new AssertionError("Fake BigQuery resource missing (did you create dataset/table?): " + e.getMessage(), e);
  }
  throw e;
}

When it happens

Trigger: Any fake-backed BigQuery operation referencing a resource that was not registered in the fake: reading/writing a table or dataset never inserted via the fake's create/put methods, loading a job id that does not exist, or querying a nonexistent table.

Common situations: Tests where fixture setup (creating dataset/table in the fake) was skipped or ran after the code under test; wrong project/dataset/table identifiers in test configuration; typos in table specs passed to BigQueryIO.read/write.

Understand the failure class

Background: "API error: {status}" and "HTTP 401/403/404/429/5xx" errors: non-2xx HTTP responses explained — this error's family across 27 libraries.

Related errors

• BigQuery %1$s not found for table "%2$s" . Please create…
• Duplicate job id
• No such stream
• A function must be provided to convert the input type into…
• BigQuery data contained value

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/fd3e5262a52b9350. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/testing/FakeDatasetService.java:1006

        Maps.newHashMap();
    synchronized (FakeDatasetService.class) {
      for (Map.Entry<String, List<String>> entry : this.insertErrors.entrySet()) {
        TableRow tableRow = BigQueryHelpers.fromJsonString(entry.getKey(), TableRow.class);
        List<TableDataInsertAllResponse.InsertErrors> allErrors = Lists.newArrayList();
        for (String errorsString : entry.getValue()) {
          allErrors.add(
              BigQueryHelpers.fromJsonString(
                  errorsString, TableDataInsertAllResponse.InsertErrors.class));
        }
        parsedInsertErrors.put(tableRow, allErrors);
      }
    }
    return parsedInsertErrors;
  }

  @FormatMethod
  void throwNotFound(@FormatString String format, Object... args) throws IOException {
    throw new IOException(
        String.format(format, args),
        new HttpResponseException.Builder(404, String.format(format, args), new HttpHeaders())
            .build());
  }
}

View on GitHub (pinned to 12126d8942)