What causes "Illegal shard number: {shardNum}" in apache/beam?

Ensure the shard number is computed as a valid index: 0 <= shardNum < number of open shard writers.

apache/beam · error · IllegalArgumentException

Illegal shard number

Error message

Illegal shard number: {shardNum}

What it means

ShardingWritableByteChannel.write dispatches writes to one of a fixed set of shard writers based on the shard number; if shardNum falls outside the configured shard range it throws this IllegalArgumentException. It indicates an internal dispatch failure — the caller attempted to write to a shard that does not exist.

Solutions

Ensure the shard number is computed as a valid index: 0 <= shardNum < number of open shard writers.
Verify the channel was constructed/configured with the same numShards used when selecting the shard.
If subclassing or calling writeToShard directly, clamp or validate shardNum before dispatching.
Report as a bug if it arises from standard Beam file sinks with a consistent shard configuration.

Example fix

// before
channel.writeToShard(shardNum, buffer); // shardNum may exceed writers
// after
if (shardNum < 0 || shardNum >= writers.size()) {
  throw new IllegalArgumentException("shardNum=" + shardNum + " outside 0.." + (writers.size() - 1));
}
channel.writeToShard(shardNum, buffer);

Defensive patterns

Strategy: validation

Validate before calling

if (shardNum < 0 || shardNum >= numShards) {
  throw new IllegalArgumentException("shardNum " + shardNum + " must be in [0, " + numShards + ")");
}

Try / catch

try {
  channel.writeToShard(shardNum, buffer);
} catch (IllegalArgumentException e) {
  // correct shard selection or re-init channel
}

Prevention

Compute shard indexes with Math.floorMod(value, numShards) to avoid negatives.
Keep shard count configuration consistent between channel creation and write paths.
Never call writeToShard directly unless you control the shard index space.

When it happens

Trigger: Calling write(ByteBuffer) on a channel whose computed target shard number (from the shard-modulo logic or explicitly set shardNum) is negative or >= number of writers; only reachable via a misconfigured sharding setup or internal bug in writeToShard.

Common situations: File-based sinks writing sharded outputs where shard count configuration is inconsistent (e.g. numShards changed between channel construction and write), or custom code subclassing/invoking writeToShard with an out-of-range shard index.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/09bd037310466e9b. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/util/ShardingWritableByteChannel.java:80

   *
   * @return The total number of bytes written. If the shard number is {@link #ALL_SHARDS}, then the
   *     total is the sum of each individual shard write.
   */
  public int writeToShard(int shardNum, ByteBuffer src) throws IOException {
    if (shardNum >= 0) {
      return writers.get(shardNum).write(src);
    }

    switch (shardNum) {
      case ALL_SHARDS:
        int size = 0;
        for (WritableByteChannel writer : writers) {
          size += writer.write(src);
        }
        return size;

      default:
        throw new IllegalArgumentException("Illegal shard number: " + shardNum);
    }
  }

  /**
   * Writes a buffer to all shards.
   *
   * <p>Same as calling {@code writeToShard(ALL_SHARDS, buf)}.
   */
  @Override
  public int write(ByteBuffer src) throws IOException {
    return writeToShard(ALL_SHARDS, src);
  }

  @Override
  public boolean isOpen() {
    for (WritableByteChannel writer : writers) {
      if (!writer.isOpen()) {
        return false;

View on GitHub (pinned to 12126d8942)