nathanmarz/storm · error · IllegalArgumentException

Multireducer groupFields and inputFields must be the same…

Error message

Multireducer groupFields and inputFields must be the same size

What it means

GroupedMultiReducerExecutor wraps a GroupedMultiReducer and maintains parallel lists of group fields and input fields, one entry per reduce stream. The constructor requires inputFields.size() == groupFields.size() and throws IllegalArgumentException otherwise, since each stream's group spec needs a matching input spec.

Solutions

1. Supply one input Fields per stream so the lists are parallel: multiReduce(groupFields, inputFields, streams, reducer)
2. If all streams share the same grouping/inputs, use the single-Fields multiReduce overload instead of lists
3. Check list sizes before constructing and log both lists when debugging

Example fix

// before
multiReduce(Arrays.asList(gf1), stream1.and(stream2), Arrays.asList(in1, in2), reducer); // sizes mismatch
// after
multiReduce(Arrays.asList(gf1, gf1), Arrays.asList(in1, in2), stream1.and(stream2), reducer);

if (groupFields.size() != inputFields.size()) {
    throw new IllegalArgumentException("groupFields and inputFields lists must be parallel: " + groupFields.size() + " vs " + inputFields.size());
}

boolean listsAreParallel(List<Fields> a, List<Fields> b) {
    return a != null && b != null && a.size() == b.size();
}

try {
    stream.multiReduce(groupFields, inputFields, streams, reducer);
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("must be the same size")) {
        throw new IllegalStateException("Provide one group Fields and one input Fields per stream", e);
    } throw e;
}

When it happens

Trigger: Calling multiReduce(groupFields, streams, reducer) or constructing GroupedMultiReducerExecutor directly where the List<Fields> for grouping and for inputs have different lengths (e.g. one group Fields per stream but a single combined input Fields).

Common situations: Passing a single Fields for all streams' inputs while listing per-stream group fields; programmatically building the lists and dropping an entry; API confusion between the two multiReduce overloads.

Related errors

• Require input fields for each aggregator
• numBuckets must be >= 2
• Trying to select non-existent field
• Regular rich spouts not supported yet... try wrapping in a…
• Cannot join DRPC stream with streams originating from other…

AI-assisted analysis of nathanmarz/storm@cdb116e942 (2026-09-12). Data as JSON: /api/errors/3975dbd9938bd5c8. Report an issue: GitHub.

Appendix: source

Thrown at storm-core/src/jvm/storm/trident/operation/impl/GroupedMultiReducerExecutor.java:42

import java.util.Map;
import storm.trident.operation.GroupedMultiReducer;
import storm.trident.operation.MultiReducer;
import storm.trident.operation.TridentCollector;
import storm.trident.operation.TridentMultiReducerContext;
import storm.trident.tuple.TridentTuple;
import storm.trident.tuple.TridentTupleView.ProjectionFactory;


public class GroupedMultiReducerExecutor implements MultiReducer<Map<TridentTuple, Object>> {
    GroupedMultiReducer _reducer;
    List<Fields> _groupFields;
    List<Fields> _inputFields;
    List<ProjectionFactory> _groupFactories = new ArrayList<ProjectionFactory>();
    List<ProjectionFactory> _inputFactories = new ArrayList<ProjectionFactory>();
    
    public GroupedMultiReducerExecutor(GroupedMultiReducer reducer, List<Fields> groupFields, List<Fields> inputFields) {
        if(inputFields.size()!=groupFields.size()) {
            throw new IllegalArgumentException("Multireducer groupFields and inputFields must be the same size");
        }
        _groupFields = groupFields;
        _inputFields = inputFields;
        _reducer = reducer;
    }
    
    @Override
    public void prepare(Map conf, TridentMultiReducerContext context) {
        for(int i=0; i<_groupFields.size(); i++) {
            _groupFactories.add(context.makeProjectionFactory(i, _groupFields.get(i)));
            _inputFactories.add(context.makeProjectionFactory(i, _inputFields.get(i)));
        }
        _reducer.prepare(conf, new TridentMultiReducerContext((List) _inputFactories));
    }

    @Override
    public Map<TridentTuple, Object> init(TridentCollector collector) {
        return new HashMap();

View on GitHub (pinned to cdb116e942)