apache/beam · error · ValueError

Encountered an unnamed index. Cannot convert to a schema-awa

Error message

Encountered an unnamed index. Cannot convert to a schema-aware PCollection with include_indexes=True. Please name all indexes or consider not including indexes.

What it means

When converting a DataFrame proxy to a schema-aware PCollection with include_indexes=True, every index level must have a name so it can map to a schema field. An index level whose name is None makes the mapping ambiguous, so ValueError is raised.

Source

Thrown at sdks/python/apache_beam/dataframe/schemas.py:143

          "include_indexes=True for a Series input. Note that this "
          "parameter is _not_ respected for DeferredSeries "
          "conversion.")
    return dtype_to_fieldtype(proxy.dtype)
  else:
    raise TypeError(f"Proxy '{proxy}' has unsupported type '{type(proxy)}'")


def element_typehint_from_dataframe_proxy(
    proxy: pd.DataFrame, include_indexes: bool = False) -> RowTypeConstraint:

  output_columns = []
  if include_indexes:
    remaining_index_names = list(proxy.index.names)
    i = 0
    while len(remaining_index_names):
      index_name = remaining_index_names.pop(0)
      if index_name is None:
        raise ValueError(
            "Encountered an unnamed index. Cannot convert to a "
            "schema-aware PCollection with include_indexes=True. "
            "Please name all indexes or consider not including "
            "indexes.")
      elif index_name in remaining_index_names:
        raise ValueError(
            "Encountered multiple indexes with the name '%s'. "
            "Cannot convert to a schema-aware PCollection with "
            "include_indexes=True. Please ensure all indexes have "
            "unique names or consider not including indexes." % index_name)
      elif index_name in proxy.columns:
        raise ValueError(
            "Encountered an index that has the same name as one "
            "of the columns, '%s'. Cannot convert to a "
            "schema-aware PCollection with include_indexes=True. "
            "Please ensure all indexes have unique names or "
            "consider not including indexes." % index_name)
      else:

View on GitHub (pinned to 12126d8942)

Solutions

Name the index before conversion: df.index.name = 'id' (or rename each MultiIndex level via df.index.names = [...])
Call reset_index() and rename the resulting columns, then convert with include_indexes=False
Set include_indexes=False if the index values are not needed in the schema

Example fix

// before
df = pd.DataFrame({'a': [1,2]}); beam.dataframe.convert.to_pcollection(df, include_indexes=True)
// after
df.index.name = 'row_id'
beam.dataframe.convert.to_pcollection(df, include_indexes=True)

Defensive patterns

Strategy: validation

Validate before calling

if include_indexes and any(n is None for n in proxy.index.names):
    proxy.index.names = [n or f'index_{i}' for i, n in enumerate(proxy.index.names)]

Type guard

def has_named_index(df) -> bool:
    return all(n is not None for n in df.index.names)

Try / catch

try:
    pc = beam.dataframe.convert.to_pcollection(df, include_indexes=True)
except ValueError as e:
    if 'unnamed index' in str(e):
        df.index.name = 'row_id'
        pc = beam.dataframe.convert.to_pcollection(df, include_indexes=True)

Prevention

Name all index levels before schema-aware conversion
Prefer reset_index() + named columns over include_indexes=True
Validate index.names during pipeline construction, not at runtime

When it happens

Trigger: Calling infer_output_type / element_type_from_dataframe (or a schema-aware output of a DataFrame transform) with include_indexes=True on a DataFrame whose index (or any MultiIndex level) is unnamed.

Common situations: DataFrames created from raw lists/arrays with default RangeIndex; after groupby/reset operations that leave None-named index levels; porting pandas code that never names its index.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f0b315ca5f79b412. Report an issue: GitHub.