{"record":{"id":"76da6d69da93482d","repo":"jax-ml/jax","slug":"indices-dtype-does-not-have-enough-range-to-gen","errorCode":null,"errorMessage":"{indices_dtype=} does not have enough range to generate sparse indices of size {sparse_size}.","messagePattern":"(.+?) does not have enough range to generate sparse indices of size (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"jax/experimental/sparse/random.py","lineNumber":87,"sourceCode":"  if n_batch < 0 or n_dense < 0 or n_batch + n_dense > len(shape):\n    raise ValueError(f\"Invalid {n_batch=}, {n_dense=} for {shape=}\")\n  n_sparse = len(shape) - n_batch - n_dense\n  batch_shape, sparse_shape, dense_shape = map(tuple, split_list(shape, [n_batch, n_sparse]))\n  batch_size = math.prod(batch_shape)\n  sparse_size = math.prod(sparse_shape)\n  if not 0 <= nse < sparse_size:\n    raise ValueError(f\"got {nse=}, expected to be between 0 and {sparse_size}\")\n  if 0 < nse < 1:\n    nse = int(math.ceil(nse * sparse_size))\n  assert not isinstance(nse, float)\n  nse = operator.index(nse)\n\n  data_shape = batch_shape + (nse,) + dense_shape\n  indices_shape = batch_shape + (nse, n_sparse)\n  if indices_dtype is None:\n    indices_dtype = dtypes.default_int_dtype()\n  if sparse_size > jnp.iinfo(indices_dtype).max:\n    raise ValueError(f\"{indices_dtype=} does not have enough range to generate \"\n                     f\"sparse indices of size {sparse_size}.\")\n  @vmap\n  def _indices(key):\n    if not sparse_shape:\n      return jnp.zeros((nse, n_sparse), dtype=indices_dtype)\n    flat_ind = random.choice(key, sparse_size, shape=(nse,),\n                             replace=not unique_indices).astype(indices_dtype)\n    return jnp.column_stack(jnp.unravel_index(flat_ind, sparse_shape))\n\n  keys = random.split(key, batch_size + 1)\n  data_key, index_keys = keys[0], keys[1:]\n  data = generator(data_key, shape=data_shape, dtype=dtype, **kwds)\n  indices = _indices(index_keys).reshape(indices_shape)\n  mat = sparse.BCOO((data, indices), shape=shape)\n  return mat.sort_indices() if sorted_indices else mat\n","sourceCodeStart":69,"sourceCodeEnd":103,"githubUrl":"https://github.com/jax-ml/jax/blob/1e1c6a8fc06dfcd1247076ec5cae4640cea5d7bb/jax/experimental/sparse/random.py#L69-L103","documentation":"random_bcoo generates flat sparse indices with random.choice over the sparse extent; if the chosen indices_dtype (default int32) cannot represent sparse_size-1, index generation would overflow, so it raises preemptively.","triggerScenarios":"Generating a BCOO whose sparse dimensions multiply to more than jnp.iinfo(indices_dtype).max — e.g. a 100000 x 100000 sparse layout with default int32 indices.","commonSituations":"Very large sparse matrices with default int32; explicitly requesting int8/int16 indices; enabling x64 only for data but not indices.","solutions":["Pass indices_dtype=jnp.int64 (and enable 64-bit ints via jax_enable_x64 if needed)","Reduce the sparse dimensions' product below the dtype's max","Leave indices_dtype=None to use the default int dtype, which must still be large enough"],"exampleFix":"// before\nM = random_bcoo(key, shape=(10**6, 10**6), nse=100)\n// after\njax.config.update('jax_enable_x64', True)\nM = random_bcoo(key, shape=(10**6, 10**6), nse=100, indices_dtype=jnp.int64)","handlingStrategy":"validation","validationCode":"import math, jax.numpy as jnp\nsparse_size = math.prod(shape)\nif sparse_size > jnp.iinfo(indices_dtype or jnp.int32).max:\n    jax.config.update('jax_enable_x64', True)\n    indices_dtype = jnp.int64","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Enable 64-bit mode and pass indices_dtype=jnp.int64 for very large sparse layouts","Estimate product of sparse dims against the dtype max before generation"],"tags":["jax","sparse","bcoo","random","index-overflow"],"backgroundTag":"integer-index-overflow","analyzedSha":"1e1c6a8fc06dfcd1247076ec5cae4640cea5d7bb","analyzedAt":"2026-08-27T09:53:25.647Z","schemaVersion":2},"datasetVersion":"2026-08-27T13:17:12.746Z"}