apache/beam · error · IndexOutOfBoundsException

Partition function returned out of bounds index:

Error message

Partition function returned out of bounds index:

What it means

During DoFn.processElement, Beam invokes the user-provided partition function to decide which output tag an element goes to. If the returned index is negative or >= numPartitions, no matching output exists, so Beam throws IndexOutOfBoundsException naming the invalid index and valid range.

Solutions

Fix the partition function to always return an index in [0, numPartitions)
Clamp the index inside the function, e.g. Math.floorMod(key, numPartitions)
If elements should be filtered out, filter them before Partition instead of returning a sentinel index

Example fix

// before
return element.length() == 0 ? -1 : element.length() % numPartitions;
// after
if (element.isEmpty()) return 0; // or filter empties beforehand
return Math.floorMod(element.length(), numPartitions);

Defensive patterns

Strategy: validation

Validate before calling

int idx = partitionFn.apply(element);
if (idx < 0 || idx >= numPartitions) throw new IllegalArgumentException("partition fn returned " + idx);

Try / catch

try { input.apply(Partition.of(n, fn)); } catch (IndexOutOfBoundsException e) { LOG.error("partition function returned bad index", e); throw new IllegalArgumentException("Fix partition fn", e); }

Prevention

Use Math.floorMod for modulo-based partitioning to avoid negatives
Filter out undesired elements before Partition rather than returning sentinel indices
Keep the partition function's output range coupled to numPartitions in one place

When it happens

Trigger: A partition function (e.g. fn -> element.length() % 0, integer division modulo with 0, or a function returning indices based on a dataset value exceeding the declared partition count, or returning -1 as a sentinel) produces an index outside [0, numPartitions).

Common situations: Partition function logic changed after partition count was fixed; using % with zero; sentinel values like -1 for 'drop' elements; mismatch between number of outputs declared and indices the function can return.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/debc01b7ea9c9bcf. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Partition.java:238

        buildOutputTags = buildOutputTags.and(new TupleTag<X>());
      }
      outputTags = buildOutputTags;
    }

    public TupleTagList getOutputTags() {
      return outputTags;
    }

    @ProcessElement
    public void processElement(ProcessContext c) throws Exception {
      X input = c.element();
      int partition = ctxFn.getClosure().apply(input, Contextful.Fn.Context.wrapProcessContext(c));
      if (0 <= partition && partition < numPartitions) {
        @SuppressWarnings("unchecked")
        TupleTag<X> typedTag = (TupleTag<X>) outputTags.get(partition);
        c.output(typedTag, input);
      } else {
        throw new IndexOutOfBoundsException(
            "Partition function returned out of bounds index: "
                + partition
                + " not in [0.."
                + numPartitions
                + ")");
      }
    }

    @Override
    public void populateDisplayData(DisplayData.Builder builder) {
      super.populateDisplayData(builder);
      builder
          .add(DisplayData.item("numPartitions", numPartitions).withLabel("Partition Count"))
          .add(
              DisplayData.item("partitionFn", originalFnClassForDisplayData.getClass())
                  .withLabel("Partition Function"));
    }

View on GitHub (pinned to 12126d8942)