apache/flink · error · IllegalArgumentException

The filter factor cannot be smaller than zero.

Error message

The filter factor cannot be smaller than zero.

What it means

Thrown by CompilerHints.setFilterFactor() when the supplied value is negative. The filter factor is an optimizer hint estimating the fraction of input records a filter/select operator retains (e.g. 0.5 = half survive, 0.0 = all dropped, 1.0 = none). The default is -1 (unset); any explicit value must be >= 0 so the optimizer's cardinality estimate stays non-negative.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/common/operators/CompilerHints.java:93

    public float getAvgOutputRecordSize() {
        return this.avgOutputRecordSize;
    }

    public void setAvgOutputRecordSize(float avgOutputRecordSize) {
        if (avgOutputRecordSize <= 0) {
            throw new IllegalArgumentException("The size of produced records must be positive.");
        }

        this.avgOutputRecordSize = avgOutputRecordSize;
    }

    public float getFilterFactor() {
        return filterFactor;
    }

    public void setFilterFactor(float filterFactor) {
        if (filterFactor < 0) {
            throw new IllegalArgumentException("The filter factor cannot be smaller than zero.");
        }

        this.filterFactor = filterFactor;
    }

    // --------------------------------------------------------------------------------------------
    //  Uniqueness
    // --------------------------------------------------------------------------------------------

    /**
     * Gets the FieldSets that are unique
     *
     * @return List of FieldSet that are unique
     */
    public Set<FieldSet> getUniqueFields() {
        return this.uniqueFields;
    }

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Ensure the value is in [0.0, 1.0]; a factor of 0 is valid and means all records are filtered out.
2. Guard the assignment: only call setFilterFactor when your computed value is >= 0, and leave it unset (default -1) when unknown.
3. If using a ratio like retained/total, special-case total==0 to skip the hint instead of computing a negative ratio.

Example fix

// before
hints.setFilterFactor(passed - total);
// after
if (total > 0 && passed >= 0) {
    hints.setFilterFactor((float) passed / total);
}

if (factor >= 0f) {
    hints.setFilterFactor(factor);
} else {
    // leave unset (default -1) when unknown
}

When it happens

Trigger: Calling Operator.getCompilerHints().setFilterFactor(x) with x < 0, or passing a computed ratio (dropped/total) that evaluates negative due to an off-by-one or a total count of zero producing a negative intermediate.

Common situations: Setting per-operator selectivity hints in the old DataSet API; computing a filter factor from job statistics where the denominator is 0 or negative; copying hint values from a config that allowed negative sentinels.

Related errors

• The output size cannot be smaller than zero.
• The output cardinality cannot be smaller than zero.
• The size of produced records must be positive.
• The number of iterations must be at least one.
• Operator for initial partial solution is not set.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/26e3685819770b7c. Report an issue: GitHub.