apache/beam · error · RuntimeException

Upserts only allowed when using primary keys

Error message

Upserts only allowed when using primary keys

What it means

TableContainer.upsertRow only supports UPDATE/MERGE-style DML for tables that define a primary key; without a key there is nothing to match on. The fake throws this RuntimeException to refuse the upsert, matching BigQuery DML requirements.

Solutions

1. Configure the fake table's schema with a primary key matching your MERGE/UPDATE join keys
2. Switch the test to the real BigQuery service if PK semantics are not needed
3. Verify how TableContainer was constructed so the key field names match row field names

Example fix

// before
new TableContainer(schema, null /* no PK */);
// after
new TableContainer(schema, "user_id"); // enable upsert support

if (primaryKeyFieldName == null) {
  throw new IllegalStateException("table must define a primary key for upserts");
}

try {
  container.upsertRow(row, seq);
} catch (RuntimeException e) {
  if (e.getMessage().contains("Upserts only allowed")) {
    // reconfigure table with a primary key
  } else throw e;
}

When it happens

Trigger: Calling upsertRow (e.g. via insertAll with DML-style upsert rows) against a fake table whose schema has no primary key.

Common situations: Testing a MERGE/UPDATE pipeline against a fake table created without PK columns configured in TableContainer.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Could not create destination for extract job
• Failed to convert the row to JSON
• Failed to fetch BigQuery data.
• Primary key validation error! Multiple inserts with the…
• A function must be provided to convert the input type into…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b4d45c11cfb14844. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/testing/TableContainer.java:138

      if (id != null) {
        ids.add(id);
      }
    }

    long tableSize = table.getNumBytes() == null ? 0L : table.getNumBytes();
    try {
      long rowSize = TableRowJsonCoder.of().getEncodedElementByteSize(row);
      table.setNumBytes(tableSize + rowSize);
      return rowSize;
    } catch (Exception ex) {
      throw new RuntimeException("Failed to convert the row to JSON", ex);
    }
  }

  void upsertRow(TableRow row, long sequenceNumber) {
    List<Object> primaryKey = getPrimaryKey(row);
    if (primaryKey == null) {
      throw new RuntimeException("Upserts only allowed when using primary keys");
    }
    long lastSequenceNumberForKey = lastSequenceNumber.getOrDefault(primaryKey, Long.MIN_VALUE);
    if (sequenceNumber <= lastSequenceNumberForKey) {
      // Out-of-order upsert - ignore it as we've already seen a more-recent update.
      return;
    }

    TableRow oldValue = keyedRows.put(primaryKey, row);
    try {
      long tableSize = table.getNumBytes() == null ? 0L : table.getNumBytes();
      if (oldValue != null) {
        tableSize -= TableRowJsonCoder.of().getEncodedElementByteSize(oldValue);
      }
      tableSize += TableRowJsonCoder.of().getEncodedElementByteSize(row);
      table.setNumBytes(tableSize);
    } catch (Exception e) {
      throw new RuntimeException("Failed to convert the row to JSON", e);
    }

View on GitHub (pinned to 12126d8942)