xai-org/x-algorithm · error · ValueError

Only 1D arrays are supported for unique.

Error message

Only 1D arrays are supported for unique.

What it means

The custom unique kernel only supports 1D arrays, matching the semantics of a single flat unique operation. Multi-dimensional inputs are rejected with a ValueError before the GPU path (or the jnp.unique fallback) is taken, because the kernel cannot iterate higher-rank layouts.

Source

Thrown at phoenix/xrex/cuda/unique/__init__.py:20

# Copyright 2026 X.AI Corp.
import jax
import jax.numpy as jnp

try:
    from xrex.cuda.unique.src import unique_api
except ImportError:
    unique_api = None
else:
    jax.ffi.register_ffi_target("xrex_unique", fn=unique_api.unique(), platform="CUDA")


def unique(
    x: jax.Array, return_inverse: bool, size: int, fill_value: int
) -> tuple[jax.Array, jax.Array]:
    if x.dtype != jnp.int32:
        raise ValueError("Only int32 is supported for unique.")
    if x.ndim != 1:
        raise ValueError("Only 1D arrays are supported for unique.")

    if unique_api is None or jax.default_backend() != "gpu":
        unique_vals, unique_inverse = jnp.unique(
            x, return_inverse=return_inverse, size=size, fill_value=fill_value
        )
        return unique_vals.astype(x.dtype), unique_inverse.astype(x.dtype)

    call = jax.ffi.ffi_call(
        "xrex_unique",
        [
            jax.ShapeDtypeStruct(shape=[size], dtype=x.dtype),
            jax.ShapeDtypeStruct(shape=x.shape, dtype=x.dtype),
            jax.ShapeDtypeStruct(shape=x.shape, dtype=x.dtype),
            jax.ShapeDtypeStruct(shape=x.shape, dtype=x.dtype),
            jax.ShapeDtypeStruct(shape=x.shape, dtype=x.dtype),
        ],
    )
    unique_vals, unique_inverse, _, _, _ = call(

View on GitHub (pinned to 24c60942c5)

Solutions

1. Flatten before calling: unique(x.reshape(-1), ...) and reshape the inverse output back afterward
2. If per-row uniqueness is needed, vmap or loop over rows instead of passing 2D
3. Check x.ndim and x.shape immediately after data loading to catch this early

Example fix

// before
vals, inv = unique(ids_2d, return_inverse=True, size=size, fill_value=-1)
// after
vals, inv = unique(ids_2d.reshape(-1), return_inverse=True, size=size, fill_value=-1)
inv = inv.reshape(ids_2d.shape)

if x.ndim != 1:
    orig_shape = x.shape
    x = x.reshape(-1)
vals, inv = unique(x, ...)
# inv = inv.reshape(orig_shape) if needed

def is_flat_int32(x: jax.Array) -> bool:
    return x.ndim == 1 and x.dtype == jnp.int32

When it happens

Trigger: Calling unique(x, ...) where x.ndim != 1, e.g. a (batch, seq) tensor of ids from prefetcher_loop or sample_engaged_posts.

Common situations: Switching from per-row unique to batched tensors without reshaping; forgetting that a (1, N) shaped slice still has ndim == 2; passing token-id matrices instead of flattened id lists.

Related errors

• Only int32 is supported for unique.
• ASTNode %s expected %d arguments, %d passed.
• k ({k}) must be <= n ({n})
• Block sparse tensors{context} must have shapes (B, H, M) and
• Not a canonical UMMA_K Layout: Expected MN-size multiple of

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/e9177c8767f962a2. Report an issue: GitHub.