apache/beam · error · IllegalArgumentException

Unknown ValueKind

Error message

Unknown ValueKind: {}

What it means

CdcSortKey.kindRank maps a CDC ValueKind (DELETE, UPDATE_BEFORE, UPDATE_AFTER, INSERT) to a one-byte rank used in the deterministic sort key. A ValueKind outside these four throws IllegalArgumentException, guaranteeing the sort encoding stays byte-comparable and deterministic for all known kinds.

Solutions

1. Only use the four ValueKinds supported by CdcSortKey
2. Add a case for the new ValueKind with a defined rank before encoding sort keys
3. Validate kinds at record-construction time to fail early
4. Ensure producer and consumer pipeline versions agree on the kind enum

Example fix

// before
CdcUpdate update = new CdcUpdate(customKind, pk, seq, row);
byte[] key = CdcSortKey.encode(update); // throws
// after
if (kind != ValueKind.DELETE && kind != ValueKind.UPDATE_BEFORE
    && kind != ValueKind.UPDATE_AFTER && kind != ValueKind.INSERT) {
  throw new IllegalArgumentException("Kind not supported for sort key: " + kind);
}
byte[] key = CdcSortKey.encode(update);

if (kind != ValueKind.DELETE && kind != ValueKind.UPDATE_BEFORE
    && kind != ValueKind.UPDATE_AFTER && kind != ValueKind.INSERT) {
  throw new IllegalArgumentException("ValueKind not encodable in sort key: " + kind);
}

static boolean isEncodableKind(ValueKind kind) {
  return kind == ValueKind.DELETE
      || kind == ValueKind.UPDATE_BEFORE
      || kind == ValueKind.UPDATE_AFTER
      || kind == ValueKind.INSERT;
}

try {
  byte[] key = CdcSortKey.encode(update);
} catch (IllegalArgumentException e) {
  LOG.error("Unencodable ValueKind for sort key: {}", e.getMessage());
  throw e;
}

When it happens

Trigger: Calling kindRank (directly or via encode) with a ValueKind not in {DELETE, UPDATE_BEFORE, UPDATE_AFTER, INSERT} — e.g. a custom kind or a new kind added by a newer CDC record model.

Common situations: Upgrading the record model and adding a ValueKind without updating kindRank; constructing CdcUpdate kinds manually with an out-of-enum value; deserializing records produced by a different pipeline version with extended kinds.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• Unsupported CDC ValueKind
• Equality field is not a top-level column of schema
• Expected at least one overlapping task in bidirectional list
• Invalid starting strategy. Valid values are
• Position delete index cardinality exceeds Integer.MAX_VALUE

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/349cd77e4c5346f3. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/cdc/sink/CdcSortKey.java:47

 * <p>The key is {@code [pkLen:4][pkBytes][seq ^ Long.MIN_VALUE:8][kindRank:1]}, big-endian.
 */
final class CdcSortKey {

  private CdcSortKey() {}

  /** Ranks change kinds so before-images sort before after-images at an equal {@code seq}. */
  public static byte kindRank(ValueKind kind) {
    switch (kind) {
      case UPDATE_BEFORE:
        return 0;
      case DELETE:
        return 1;
      case UPDATE_AFTER:
        return 2;
      case INSERT:
        return 3;
      default:
        throw new IllegalArgumentException("Unknown ValueKind: " + kind);
    }
  }

  /**
   * Encodes the deterministic, byte-comparable sort key {@code [pkLen:4][pkBytes][seq ^
   * Long.MIN_VALUE:8][kindRank:1]} for one CDC record.
   *
   * <p>SortValues compares unsigned lexicographic byte order. The length prefix is needed to
   * accurately compare two primary keys of varying byte-lengths. Flipping the sequence number's
   * sign bit makes unsigned byte order match signed numeric order. kindRank breaks equal-seq ties.
   */
  public static byte[] encode(byte[] pkBytes, long seq, ValueKind kind) {
    return ByteBuffer.allocate(4 + pkBytes.length + 9)
        .putInt(pkBytes.length)
        .put(pkBytes)
        .putLong(seq ^ Long.MIN_VALUE)
        .put(kindRank(kind))
        .array();

View on GitHub (pinned to 12126d8942)