nathanmarz/storm · error · IllegalArgumentException

duplicate field

Error message

duplicate field '%s'

What it means

The Fields class models an ordered, unique list of output field names. Its constructor validates uniqueness up front, throwing IllegalArgumentException if the input list contains any duplicate name, since duplicate field names would make index lookup ambiguous.

Solutions

1. Deduplicate the input list before constructing Fields, e.g. new ArrayList<>(new LinkedHashSet<>(fields))
2. Fix the source of duplicate names (SQL aliases, header parsing) so each field is declared exactly once
3. If order must be preserved and duplicates intentional, rename the duplicate fields instead of reusing the name

Example fix

// before
Fields fields = new Fields(Arrays.asList("word", "word", "count")); // throws
// after
Fields fields = new Fields(new ArrayList<>(new LinkedHashSet<>(Arrays.asList("word", "word", "count"))));
// or fix names:
Fields fields = new Fields("word", "word2", "count");

Set<String> seen = new HashSet<>();
for (String f : fields) {
    if (!seen.add(f)) {
        throw new IllegalArgumentException("duplicate output field: " + f);
    }
}
Fields fields = new Fields(new ArrayList<>(seen));

boolean hasUniqueFields(java.util.List<String> fields) {
    return new HashSet<>(fields).size() == fields.size();
}

try {
    Fields f = new Fields(fieldList);
} catch (IllegalArgumentException e) {
    LOG.error("Duplicate field names in tuple declaration", e);
    fieldList = new ArrayList<>(new LinkedHashSet<>(fieldList));
    Fields f = new Fields(fieldList);
}

When it happens

Trigger: Constructing new Fields(...) with a List<String> (or varargs/Values) that contains the same field name twice, e.g. new Fields(Arrays.asList("id","id","val")).

Common situations: Dynamically building field lists from database result set metadata or CSV headers that contain duplicate column names; string concatenation/splitting bugs producing repeated names; typos like emit("a","b") declared as declareOutputFields(new Fields("a","b","b")).

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Additive operations cannot add fields with same name as…
• Fields order must be same length as pointers map
• The same metric name
• Output fields for chained aggregators must be distinct
• Combiner aggs only take a single field as input. Got this…

AI-assisted analysis of nathanmarz/storm@cdb116e942 (2026-09-12). Data as JSON: /api/errors/c9da28ef67dd16e4. Report an issue: GitHub.

Appendix: source

Thrown at storm-core/src/jvm/backtype/storm/tuple/Fields.java:40

import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.io.Serializable;

public class Fields implements Iterable<String>, Serializable {
    private List<String> _fields;
    private Map<String, Integer> _index = new HashMap<String, Integer>();
    
    public Fields(String... fields) {
        this(Arrays.asList(fields));
    }
    
    public Fields(List<String> fields) {
        _fields = new ArrayList<String>(fields.size());
        for (String field : fields) {
            if (_fields.contains(field))
                throw new IllegalArgumentException(
                    String.format("duplicate field '%s'", field)
                );
            _fields.add(field);
        }
        index();
    }
    
    public List<Object> select(Fields selector, List<Object> tuple) {
        List<Object> ret = new ArrayList<Object>(selector.size());
        for(String s: selector) {
            ret.add(tuple.get(_index.get(s)));
        }
        return ret;
    }

    public List<String> toList() {
        return new ArrayList<String>(_fields);
    }

View on GitHub (pinned to cdb116e942)