apache/beam · error · ValueError

Invalid model update: {key} appears in multiple update lists

Error message

Invalid model update: {key} appears in multiple update lists. A single model update must provide exactly one updated path per key.

What it means

Raised by update_model_paths when the same key appears in more than one update entry. A single model update must provide exactly one new path per key; duplicate keys make the intended new path ambiguous.

Source

Thrown at sdks/python/apache_beam/ml/inference/base.py:978

    # cohort_path_mapping will be structured as follows:
    # {
    # original_cohort_id: {
    #    'update/path/1': ['key1FromOriginalCohort', key2FromOriginalCohort'],
    #    'update/path/2': ['key3FromOriginalCohort', key4FromOriginalCohort'],
    #    }
    # }
    cohort_path_mapping: dict[KeyT, dict[str, list[KeyT]]] = {}
    key_modelid_mapping: dict[KeyT, str] = {}
    seen_keys = set()
    for mp in model_paths:
      keys = mp.keys
      update_path = mp.update_path
      model_id = mp.model_id
      if len(update_path) == 0:
        raise ValueError(f'Invalid model update, path for {keys} is empty')
      for key in keys:
        if key in seen_keys:
          raise ValueError(
              f'Invalid model update: {key} appears in multiple '
              'update lists. A single model update must provide exactly one '
              'updated path per key.')
        seen_keys.add(key)
        if key not in self._key_to_id_map:
          raise ValueError(
              f'Invalid model update: {key} appears in '
              'update, but not in the original configuration.')
        key_modelid_mapping[key] = model_id
        cohort_id = self._key_to_id_map[key]
        if cohort_id not in cohort_path_mapping:
          cohort_path_mapping[cohort_id] = defaultdict(list)
        cohort_path_mapping[cohort_id][update_path].append(key)
    for key in self._key_to_id_map:
      if key not in seen_keys:
        raise ValueError(
            f'Invalid model update: {key} appears in the '
            'original configuration, but not the update.')

View on GitHub (pinned to 12126d8942)

Solutions

1. Deduplicate keys across all update objects so each key appears exactly once.
2. Merge overlapping update entries into one entry with the single intended new path.
3. Validate before calling: collect all keys in a set and assert no duplicates.

Example fix

// before
handler.update_model_paths([
    UpdateModelPath(keys=['k1'], update_path='gs://b/m2'),
    UpdateModelPath(keys=['k1', 'k2'], update_path='gs://b/m3')])
// after
handler.update_model_paths([
    UpdateModelPath(keys=['k1'], update_path='gs://b/m2'),
    UpdateModelPath(keys=['k2'], update_path='gs://b/m3')])

all_keys = [k for u in updates for k in u.keys]
if len(all_keys) != len(set(all_keys)):
    dupes = {k for k in all_keys if all_keys.count(k) > 1}
    raise ValueError(f'duplicate keys in update: {dupes}')

When it happens

Trigger: Passing two or more update objects to update_model_paths whose keys lists overlap (e.g. key 'k1' in two entries), or one entry whose keys list repeats a key already seen.

Common situations: Merging update specs from multiple sources without deduplicating keys; batching per-model updates where a model belongs to several cohorts; accidentally submitting the same update object twice in the list.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

• key {key} maps to multiple model handlers. All keys must map
• Cannot override RemoteModelHandler.load_model, implement cre
• Cannot override RemoteModelHandler.run_inference, implement
• Rate Limit Exceeded, Could not process this batch.
• Cannot make make an unkeyed model handler with pre or postpr

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/3765ce2ca0a4d3d1. Report an issue: GitHub.