apache/beam · error · IllegalStateException

Failed to encode values for multimap user state id

Error message

Failed to encode values for multimap user state id %s.

What it means

MultimapUserState.encodeValues serializes each pending value of a multimap user state entry with the value Coder when flushing writes to the Fn API state service. An IOException during encoding is wrapped in an IllegalStateException naming the multimap user state id. The values written are not encodable by the declared value coder.

Solutions

1. Ensure all values written match the declared value Coder type and are non-null if the coder forbids nulls.
2. Fix custom Coders that throw for the values being written.
3. Filter or sanitize values before adding them to user state.
4. Verify the state id's coder spec in the pipeline graph matches what user code writes.

Example fix

// before
userState.put(key, maybeNullValue);
// after
if (maybeNullValue != null) {
  userState.put(key, maybeNullValue);
}

for (V v : values) {
  if (v == null || !valueCoder.getType().isInstance(v)) {
    throw new IllegalArgumentException("Value not encodable for user state: " + v);
  }
}

boolean encodableValue(V v, Coder<V> coder) {
  return v != null && coder.getType().isInstance(v);
}

try {
  userState.flush();
} catch (IllegalStateException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Failed to encode values")) {
    LOG.error("User state values unencodable: {}", e.getMessage(), e);
  } else { throw e; }
}

When it happens

Trigger: Calling write/add on multimap user state and later flush/startStateApiWrites where any value V fails valueCoder.encode — null values, coder/type mismatch, or a custom coder throwing on that instance.

Common situations: User state populated with values of a different runtime type than the declared coder; null values added to the multimap; coder changes after pipeline refactor; Kryo/Java-serialized objects that became non-serializable after code changes.

Understand the failure class

Background: json.Marshal / "failed to marshal" errors in Go: why "unsupported type" happens and how to fix it — this error's family across 22 libraries.

Related errors

• err
• failed encoding key for
• Failed to encode key
• The input value cannot be encoded
• The input value cannot be encoded

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/787f7326eb38e2ff. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/harness/src/main/java/org/apache/beam/fn/harness/state/MultimapUserState.java:505

        KV<K, CachingStateIterable<V>> value = persistedValues.get(entry.getKey());
        // We don't do anything for keys that haven't been loaded since we have no knowledge whether
        // the key is empty or not.
        if (value != null) {
          value.getValue().append(entry.getValue().getValue());
        }
      }
    }
  }

  private ByteString encodeValues(Iterable<V> values) {
    try {
      ByteStringOutputStream output = new ByteStringOutputStream();
      for (V value : values) {
        valueCoder.encode(value, output);
      }
      return output.toByteString();
    } catch (IOException e) {
      throw new IllegalStateException(
          String.format(
              "Failed to encode values for multimap user state id %s.",
              keysStateRequest.getStateKey().getMultimapKeysUserState().getUserStateId()),
          e);
    }
  }

  private StateRequest createUserStateRequest(K key) {
    try {
      ByteStringOutputStream output = new ByteStringOutputStream();
      mapKeyCoder.encode(key, output);
      StateRequest.Builder request = userStateRequest.toBuilder();
      request.getStateKeyBuilder().getMultimapUserStateBuilder().setMapKey(output.toByteString());
      return request.build();
    } catch (IOException e) {
      throw new IllegalStateException(
          String.format(
              "Failed to encode key for multimap user state id %s.",

View on GitHub (pinned to 12126d8942)