pola-rs/polars · error · ComputeError

can't pass a Series with missing data to a generalized ufunc

Error message

can't pass a Series with missing data to a generalized ufunc, as it might give unexpected results. See https://docs.pola.rs/user-guide/expressions/missing-data/ for suggestions on how to remove or fill in missing data.

What it means

Raised as polars ComputeError when a generalized ufunc (gufunc - a ufunc with a signature like '(n)->()' or '(m,n),(n,p)->(m,p)') is applied to a Series containing nulls. Gufuncs consume the whole backing buffer at once, so masked-out nulls would silently corrupt the math; Polars refuses instead of guessing how to handle missing values.

Source

Thrown at py-polars/src/polars/series/series.py:1682

                    dtype_char_minimum = dtype_ufunc
                    break

            # Override minimum dtype if requested.
            dtype_char = (
                np.dtype(kwargs.pop("dtype")).char
                if "dtype" in kwargs
                else dtype_char_minimum
            )

            # Only generalized ufuncs have a signature set:
            is_generalized_ufunc = bool(ufunc.signature)

            if is_generalized_ufunc:
                # Generalized ufuncs will operate on the whole array, so
                # missing data can corrupt the results.
                if self.has_nulls():
                    msg = "can't pass a Series with missing data to a generalized ufunc, as it might give unexpected results. See https://docs.pola.rs/user-guide/expressions/missing-data/ for suggestions on how to remove or fill in missing data."
                    raise ComputeError(msg)
                # If the input and output are the same size, e.g. "(n)->(n)" we
                # can allocate ourselves and save a copy. If they're different,
                # we let the ufunc do the allocation, since only it knows the
                # output size.
                assert ufunc.signature is not None  # pacify MyPy
                ufunc_input, ufunc_output = ufunc.signature.split("->")
                if ufunc_output == "()":
                    # If the result a scalar, just let the function do its
                    # thing, no need for any song and dance involving
                    # allocation:
                    return ufunc(*args, dtype=dtype_char, **kwargs)
                else:
                    allocate_output = ufunc_input == ufunc_output
            else:
                allocate_output = True

            f = get_ffi_func("apply_ufunc_<>", numpy_char_code_to_dtype(dtype_char), s)

View on GitHub (pinned to df599052da)

Solutions

Drop nulls first: `s.drop_nulls()` before the gufunc (if the semantics allow omitting samples).
Fill nulls explicitly: `s.fill_null(0)` / `s.fill_null(strategy='mean')` / `s.interpolate()`.
Handle null-aware logic outside: split into `s.filter(s.is_not_null())` plus a mask, then stitch results back.
Catch polars.exceptions.ComputeError in generic numeric wrappers and surface a clearer message about missing data.

Example fix

// before
s = pl.Series([3.0, None, 4.0])
np.linalg.norm(s)  # ComputeError

// after
np.linalg.norm(s.drop_nulls())
# or
np.linalg.norm(s.fill_null(0.0))

Defensive patterns

Strategy: validation

Validate before calling

def clean_for_gufunc(s: pl.Series, fill=None) -> pl.Series:
    if s.has_nulls():
        return s.fill_null(fill) if fill is not None else s.drop_nulls()
    return s

out = np.linalg.norm(clean_for_gufunc(s, fill=0.0))

Type guard

def is_gufunc_safe(s: pl.Series) -> bool:
    return not s.has_nulls()

Try / catch

try:
    out = np.linalg.norm(s)
except pl.exceptions.ComputeError:
    out = np.linalg.norm(s.drop_nulls())

Prevention

Check s.has_nulls() before any gufunc (signature != None) call: norm, dot, matmul.
Decide drop vs fill policy explicitly; never rely on gufuncs to ignore nulls.
Elementwise ufuncs tolerate nulls - the strict rule is gufunc-specific.

When it happens

Trigger: `np.linalg.norm(s)` or `np.dot(a_s, b_s)` where either Series has nulls; `np.matmul(series_2d_view, other)` with missing data; any gufunc (ufunc.signature non-empty) dispatching through __array_ufunc__ while self.has_nulls() is true. Elementwise ufuncs are fine - nulls are re-masked afterwards.

Common situations: Linalg/feature math on real-world columns with missing values; joins or parses producing nulls that the developer forgot to handle; switching from elementwise ops (which tolerate nulls) to a gufunc like norm/dot and hitting the stricter rule.

Related errors

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/9affb2181372517e. Report an issue: GitHub.