nathanmarz/storm · error · RuntimeException

Task ID should not exceed

Error message

Task ID should not exceed ${Short.MAX_VALUE}

What it means

The wire format writes the task id as a 2-byte short. writeTaskMessage (called from MessageBatch.buffer) validates this and throws when a task id exceeds Short.MAX_VALUE (32767), since it cannot be encoded as writeShort(task_id).

Solutions

1. Reduce topology parallelism (fewer executors/tasks) so task ids stay below 32768.
2. Upgrade Storm to a version whose transport uses int task ids instead of shorts.
3. Validate task ids in custom message-producing code against Short.MAX_VALUE before sending.
4. Patch the transport to widen the task id field to 4 bytes (both sender and receiver must agree).

Example fix

// before
int taskId = 40000;
batch.add(new TaskMessage(taskId, payload));
// after
assert taskId <= Short.MAX_VALUE;
batch.add(new TaskMessage(taskId, payload)); // or restructure topology to keep ids < 32768

if (taskId > Short.MAX_VALUE || taskId < 0) {
    throw new IllegalArgumentException("Task ID must fit in a short: " + taskId);
}

try {
    byte[] buf = batch.buffer();
} catch (RuntimeException e) {
    if (e.getMessage().contains("Task ID should not exceed")) {
        // restructure topology parallelism or upgrade transport
    } else { throw e; }
}

When it happens

Trigger: A topology whose task IDs exceed 32767 — e.g. very large topologies with many executors — when batching TaskMessages for sending over netty and calling buffer().

Common situations: Massively parallel topologies where storm-assigned task ids exceed the short range; custom code fabricating task ids without checking the short bound; ports of the transport to newer Storm versions with larger task counts.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• Client is being closed, and does not take requests any more
• Client connection should not receive any messages
• null object forbidded in message batch
• Unsuppoted object type
• Unsupported encoding of object of class

AI-assisted analysis of nathanmarz/storm@cdb116e942 (2026-09-12). Data as JSON: /api/errors/2c4758069957949f. Report an issue: GitHub.

Appendix: source

Thrown at storm-netty/src/jvm/backtype/storm/messaging/netty/MessageBatch.java:163

        return bout.buffer();
    }

    /**
     * write a TaskMessage into a stream
     *
     * Each TaskMessage is encoded as:
     *  task ... short(2)
     *  len ... int(4)
     *  payload ... byte[]     *  
     */
    private void writeTaskMessage(ChannelBufferOutputStream bout, TaskMessage message) throws Exception {
        int payload_len = 0;
        if (message.message() != null)
            payload_len =  message.message().length;

        int task_id = message.task();
        if (task_id > Short.MAX_VALUE)
            throw new RuntimeException("Task ID should not exceed "+Short.MAX_VALUE);
        
        bout.writeShort((short)task_id);
        bout.writeInt(payload_len);
        if (payload_len >0)
            bout.write(message.message());
    }
}

View on GitHub (pinned to cdb116e942)