apache/beam · error · IllegalStateException

Failed to encode key for multimap user state id

Error message

Failed to encode key for multimap user state id %s.

What it means

Beam Fn harness throws this IllegalStateException from createMultimapKeysUserState/createUserStateRequest when it fails to protobuf-encode the user's map key with the registered key coder while building a StateRequest for a multimap user state. Encoding an in-memory key should never fail, so an IOException here signals a broken/mismatched coder rather than a data problem. The original IOException is attached as the cause.

Solutions

1. Inspect the wrapped IOException cause to find which coder and value failed to encode
2. Verify the multimap state's key coder matches the actual key type written to the state
3. Fix the custom coder's encode() so it never throws IOException for valid key values
4. Check Beam SDK/harness version alignment between runner and SDK coders

Example fix

// before: custom coder throws on null elements
public void encode(String value, OutputStream out) {
  if (value == null) throw new IOException("null key");
  ...
}
// after: handle all valid values deterministically
public void encode(String value, OutputStream out) throws IOException {
  StringUtf8Coder.of().encode(value == null ? "" : value, out);
}

// Ensure the key coder accepts the value before writing state
try (ByteArrayOutputStream unused = new ByteArrayOutputStream()) {
  mapKeyCoder.verifyDeterministic(); // and ensure key is non-null
} catch (CoderException | NonDeterministicException e) {
  throw new IllegalArgumentException("key not encodable", e);
}

When it happens

Trigger: The multimap state key coder's encode() throws IOException for the given key value while constructing the StateRequest that sets the map key on a multimap user state.

Common situations: A custom Coder whose encode() performs I/O or throws on certain values; pipeline deserialization producing a misconfigured coder; a coder registered for the state key that does not actually support the key type being written.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Cannot create from non-Java
• Malformed class : state declaration field is not accessible.
• response.get().getError()
• Unknown
• A function must be provided to convert the input type into…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/2fa24133b828f116. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/harness/src/main/java/org/apache/beam/fn/harness/state/MultimapUserState.java:521

      return output.toByteString();
    } catch (IOException e) {
      throw new IllegalStateException(
          String.format(
              "Failed to encode values for multimap user state id %s.",
              keysStateRequest.getStateKey().getMultimapKeysUserState().getUserStateId()),
          e);
    }
  }

  private StateRequest createUserStateRequest(K key) {
    try {
      ByteStringOutputStream output = new ByteStringOutputStream();
      mapKeyCoder.encode(key, output);
      StateRequest.Builder request = userStateRequest.toBuilder();
      request.getStateKeyBuilder().getMultimapUserStateBuilder().setMapKey(output.toByteString());
      return request.build();
    } catch (IOException e) {
      throw new IllegalStateException(
          String.format(
              "Failed to encode key for multimap user state id %s.",
              keysStateRequest.getStateKey().getMultimapKeysUserState().getUserStateId()),
          e);
    }
  }

  private CachingStateIterable<V> getPersistedValues(Object structuralKey, K key) {
    return persistedValues
        .computeIfAbsent(
            structuralKey,
            unused -> {
              StateRequest request = createUserStateRequest(key);
              return KV.of(
                  key,
                  StateFetchingIterators.readAllAndDecodeStartingFrom(
                      Caches.subCache(
                          cache,

View on GitHub (pinned to 12126d8942)