apache/beam · error · KeyError

k_val

Error message

k_val

What it means

Inside unwrap_xs (used by DataFrame.xs in the Beam DataFrame API), this reference identifies the k_val field: the key being cross-sectioned. The surrounding code builds a dummy index from k_val and reindexes to emulate xs lazily; if k_val's type/value cannot be handled (e.g. reindexing fails), the wrapped error mentioning 'k_val' surfaces. It is a lazy-API implementation detail rather than a user-facing validation.

Source

Thrown at sdks/python/apache_beam/dataframe/frames.py:1158

      dummy_index = (
          pd.MultiIndex.from_tuples([k_val], names=proxy_frame.index.names) if
          isinstance(k_val, tuple) else pd.Index([k_val],
                                                 name=proxy_frame.index.name))

      if isinstance(proxy_frame, pd.DataFrame):
        dummy_obj = proxy_frame.reindex(dummy_index)
        xs_proxy = dummy_obj.xs(k_val, **kwargs)
        if isinstance(xs_proxy, (pd.DataFrame, pd.Series)):
          xs_proxy = xs_proxy.iloc[:0]
      else:
        try:
          xs_proxy = proxy_frame.dtype.type()
        except TypeError:
          xs_proxy = proxy_frame.reindex(dummy_index).iloc[0]

      def unwrap_xs(ser):
        if ser.empty:
          raise KeyError(k_val)
        return ser.iloc[0]

      with expressions.allow_non_parallel_operations(True):
        return frame_base.DeferredFrame.wrap(
            expressions.ComputedExpression(
                'xs',
                unwrap_xs, [intermediate],
                proxy=xs_proxy,
                requires_partition_by=partitionings.Singleton(),
                preserves_partition_by=partitionings.Singleton()))

  @property
  def dtype(self):
    return self._expr.proxy().dtype

  isin = frame_base._elementwise_method('isin', base=pd.DataFrame)
  combine_first = frame_base._elementwise_method(
      'combine_first', base=pd.DataFrame)

View on GitHub (pinned to 12126d8942)

Solutions

1. Check the key exists at the given level before xs (e.g. key in df.index.get_level_values(level))
2. Wrap in try/except KeyError and handle the missing-key case explicitly
3. Verify index dtypes match the key type (int vs str labels)

Example fix

// before
row = df.xs('2024', level='year')
// after
if '2024' in df.index.get_level_values('year'):
    row = df.xs('2024', level='year')
else:
    row = None

level_vals = df.index.get_level_values(level) if level is not None else df.index
if not all(k in level_vals for k in (key if isinstance(key, tuple) else (key,))):
    raise KeyError(f"xs key {key!r} not present in index")

def key_in_index(df, key, level=None):
    vals = df.index.get_level_values(level) if level is not None else df.index
    return key in vals

try:
    out = df.xs(key, level=level)
except KeyError as e:
    logging.warning("xs key %s missing, returning empty", key)
    out = df.iloc[0:0]

When it happens

Trigger: df.xs(missing_key, level='lvl') where no row has that level value; xs after a filter that removed all matching rows; xs with a tuple key whose level values don't co-occur in any row.

Common situations: Typos in level values ('2023' vs 2023 type mismatches); data-dependent pipelines where a key present yesterday is absent today; xs on an empty intermediate frame.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• label
• Cannot specify both 'labels' and 'index'/'columns'
• axis must be one of (0, 1, 'index', 'columns'), got '%s'
• groupby(as_index=False)
• You have to supply one of 'by' and 'level'

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/30e961df6b1aaed0. Report an issue: GitHub.