{"record":{"id":"b4d45c11cfb14844","repo":"apache/beam","slug":"upserts-only-allowed-when-using-primary-keys","errorCode":null,"errorMessage":"Upserts only allowed when using primary keys","messagePattern":"Upserts only allowed when using primary keys","errorType":"validation","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/testing/TableContainer.java","lineNumber":138,"sourceCode":"      if (id != null) {\n        ids.add(id);\n      }\n    }\n\n    long tableSize = table.getNumBytes() == null ? 0L : table.getNumBytes();\n    try {\n      long rowSize = TableRowJsonCoder.of().getEncodedElementByteSize(row);\n      table.setNumBytes(tableSize + rowSize);\n      return rowSize;\n    } catch (Exception ex) {\n      throw new RuntimeException(\"Failed to convert the row to JSON\", ex);\n    }\n  }\n\n  void upsertRow(TableRow row, long sequenceNumber) {\n    List<Object> primaryKey = getPrimaryKey(row);\n    if (primaryKey == null) {\n      throw new RuntimeException(\"Upserts only allowed when using primary keys\");\n    }\n    long lastSequenceNumberForKey = lastSequenceNumber.getOrDefault(primaryKey, Long.MIN_VALUE);\n    if (sequenceNumber <= lastSequenceNumberForKey) {\n      // Out-of-order upsert - ignore it as we've already seen a more-recent update.\n      return;\n    }\n\n    TableRow oldValue = keyedRows.put(primaryKey, row);\n    try {\n      long tableSize = table.getNumBytes() == null ? 0L : table.getNumBytes();\n      if (oldValue != null) {\n        tableSize -= TableRowJsonCoder.of().getEncodedElementByteSize(oldValue);\n      }\n      tableSize += TableRowJsonCoder.of().getEncodedElementByteSize(row);\n      table.setNumBytes(tableSize);\n    } catch (Exception e) {\n      throw new RuntimeException(\"Failed to convert the row to JSON\", e);\n    }","sourceCodeStart":120,"sourceCodeEnd":156,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/testing/TableContainer.java#L120-L156","documentation":"TableContainer.upsertRow only supports UPDATE/MERGE-style DML for tables that define a primary key; without a key there is nothing to match on. The fake throws this RuntimeException to refuse the upsert, matching BigQuery DML requirements.","triggerScenarios":"Calling upsertRow (e.g. via insertAll with DML-style upsert rows) against a fake table whose schema has no primary key.","commonSituations":"Testing a MERGE/UPDATE pipeline against a fake table created without PK columns configured in TableContainer.","solutions":["Configure the fake table's schema with a primary key matching your MERGE/UPDATE join keys","Switch the test to the real BigQuery service if PK semantics are not needed","Verify how TableContainer was constructed so the key field names match row field names"],"exampleFix":"// before\nnew TableContainer(schema, null /* no PK */);\n// after\nnew TableContainer(schema, \"user_id\"); // enable upsert support","handlingStrategy":"validation","validationCode":"if (primaryKeyFieldName == null) {\n  throw new IllegalStateException(\"table must define a primary key for upserts\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  container.upsertRow(row, seq);\n} catch (RuntimeException e) {\n  if (e.getMessage().contains(\"Upserts only allowed\")) {\n    // reconfigure table with a primary key\n  } else throw e;\n}","preventionTips":["Always construct TableContainer fixtures with the PK used by the pipeline's MERGE/UPDATE","Mirror real BigQuery table PK configuration in fakes"],"tags":["java","bigquery","testing","dml","primary-key"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}