nathanmarz/storm · error · IllegalArgumentException

Trying to select non-existent field

Error message

Trying to select non-existent field: '${field}' from stream containing fields fields: <${allFields}>

What it means

Trident validates at topology-build time that every field named in a projection actually exists on the stream's output schema. When a field in the projection list is not present in getOutputFields(), Stream.projectionValidation throws this IllegalArgumentException. It is an early fail-fast check so bad field names surface locally instead of as runtime tuple errors on workers.

Solutions

Fix the field name in the projection/operation call to match the upstream stream's output fields
Check what fields the stream actually declares (e.g. each(new Function, new Fields("a","b")) output) and print/inspect getOutputFields
Update the upstream operation's output Fields declaration so the field exists
Reorder the pipeline so the projection happens after the operation that produces the field

Example fix

// before
stream.each(new Fields("userId"), filter);
// after (field is actually named "user_id")
stream.each(new Fields("user_id"), filter);

Defensive patterns

Strategy: validation

Validate before calling

Fields out = stream.getOutputFields();
for (String f : projectionFields) {
    if (!out.contains(f)) throw new IllegalStateException("Field '" + f + "' not in stream fields " + out);
}

Type guard

boolean isProjectedValidly(Fields out, Fields proj) {
    return Arrays.stream(proj.toList()).allMatch(out::contains);
}

Try / catch

try {
    stream.project(fields);
} catch (IllegalArgumentException e) {
    if (e.getMessage().startsWith("Trying to select non-existent field")) {
        log.error("Field mismatch; stream fields=" + stream.getOutputFields(), e);
    } else throw e;
}

Prevention

Inspect getOutputFields() before projecting
Keep output-field declarations next to each operation and update consumers when renaming
Write a unit test that builds the topology (topology.build()) to fail fast on field typos

When it happens

Trigger: Calling stream.project(Fields), groupBy, partitionBy, each, partitionAggregate, or stateQuery with a Fields object naming a field not produced by the upstream stream's declared output fields.

Common situations: Typos in field names; an upstream function/aggregate whose declared output fields changed; copy-pasted pipeline code where a renamed field was not propagated; chaining operations assuming extra fields survive operations that only pass declared fields through.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of nathanmarz/storm@cdb116e942 (2026-09-12). Data as JSON: /api/errors/9d9a423152255620. Report an issue: GitHub.

Appendix: source

Thrown at storm-core/src/jvm/storm/trident/Stream.java:370

            return s.global();
        }

        @Override
        public BatchToPartition singleEmitPartitioner() {
            return new GlobalBatchToPartition();
        }
        
    }

    private void projectionValidation(Fields projFields) {
        if (projFields == null) {
            return;
        }

        Fields allFields = this.getOutputFields();
        for (String field : projFields) {
            if (!allFields.contains(field)) {
                throw new IllegalArgumentException("Trying to select non-existent field: '" + field + "' from stream containing fields fields: <" + allFields + ">");
            }
        }
    }
}

View on GitHub (pinned to cdb116e942)