pandas-dev/pandas · error · ValueError

Duplicate index entries are not allowed in to_coo transforma

Error message

Duplicate index entries are not allowed in to_coo transformation.

What it means

Thrown by sparse_series_to_coo in pandas/core/arrays/sparse/scipy_sparse.py:159 when the Series index has duplicate entries. The COO conversion maps each index tuple to a unique (row, col) coordinate; duplicates would alias cells and produce an ambiguous matrix, so pandas rejects them up front.

Source

Thrown at pandas/core/arrays/sparse/scipy_sparse.py:159

def sparse_series_to_coo(
    ss: Series,
    row_levels: Iterable[int] = (0,),
    column_levels: Iterable[int] = (1,),
    sort_labels: bool = False,
) -> tuple[scipy.sparse.coo_matrix, list[IndexLabel], list[IndexLabel]]:
    """
    Convert a sparse Series to a scipy.sparse.coo_matrix using index
    levels row_levels, column_levels as the row and column
    labels respectively. Returns the sparse_matrix, row and column labels.
    """
    import scipy.sparse

    if ss.index.nlevels < 2:
        raise ValueError("to_coo requires MultiIndex with nlevels >= 2.")
    if not ss.index.is_unique:
        raise ValueError(
            "Duplicate index entries are not allowed in to_coo transformation."
        )

    # to keep things simple, only rely on integer indexing (not labels)
    row_levels = [ss.index._get_level_number(x) for x in row_levels]
    column_levels = [ss.index._get_level_number(x) for x in column_levels]

    v, i, j, rows, columns = _to_ijv(
        ss, row_levels=row_levels, column_levels=column_levels, sort_labels=sort_labels
    )
    sparse_matrix = scipy.sparse.coo_matrix(
        (v, (i, j)), shape=(len(rows), len(columns))
    )
    return sparse_matrix, rows, columns


def coo_to_sparse_series(
    A: scipy.sparse.coo_matrix, dense_index: bool = False

View on GitHub (pinned to 3b7651241d)

Solutions

1. Reset and rebuild a unique index: ss = ss.reset_index(drop=True); ss.index = pd.MultiIndex.from_arrays(...).
2. Aggregate duplicates first: ss = ss.groupby(level=list(range(ss.index.nlevels))).sum().
3. Check uniqueness beforehand: assert ss.index.is_unique.

Example fix

// before
ss = pd.Series([1,2], index=pd.MultiIndex.from_tuples([('a','x'),('a','x')])).astype('Sparse[int]')
ss.sparse.to_coo()  # raises

// after
ss = ss.groupby(level=[0,1]).sum()
ss.sparse.to_coo()

def unique_index_to_coo(ss, **kw):
    if not ss.index.is_unique:
        ss = ss.groupby(level=list(range(ss.index.nlevels))).sum()
    return ss.sparse.to_coo(**kw)

null

null

When it happens

Trigger: Calling ss.sparse.to_coo() on a sparse Series whose MultiIndex contains repeated (row_label, col_label) tuples. Common after groupby/concat operations that retain non-unique indices.

Common situations: Index not reset after a groupby aggregation. Concatenating sparse Series without verify_integrity. Real-world data with duplicated composite keys fed straight into to_coo.

Related errors

• Is not a partition because intersection is not null.
• Is not a partition because union is not the whole.
• to_coo requires MultiIndex with nlevels >= 2.
• Expected coo_matrix. Got {type(A).__name__} instead.
• Column length mismatch: {len(columns)} vs. {K}

AI-assisted analysis of pandas-dev/pandas@3b7651241d (2026-08-11). Data as JSON: /api/errors/5fce744464fe8fe7. Report an issue: GitHub.