apache/flink · error · RuntimeException

Error: Invalid number of samples: ${numSamples}

Error message

Error: Invalid number of samples: ${numSamples}

What it means

During getStatistics(), DelimitedInputFormat computes the number of samples from the total input size and clamps it between configured min/max bounds. A negative resulting numSamples indicates an internal arithmetic/config inconsistency (e.g. DEFAULT_MIN_NUM_SAMPLES or DEFAULT_MAX_NUM_SAMPLES misconfigured to negative). The format throws a RuntimeException (not IllegalArgumentException) because this is not a direct user argument but a derived, unexpected value.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/common/io/DelimitedInputFormat.java:399

            // compute how many samples to take, depending on the defined upper and lower bound
            final int numSamples;
            if (this.numLineSamples != NUM_SAMPLES_UNDEFINED) {
                numSamples = this.numLineSamples;
            } else {
                // make the samples small for very small files
                final int calcSamples = (int) (stats.getTotalInputSize() / 1024);
                numSamples =
                        Math.min(
                                DEFAULT_MAX_NUM_SAMPLES,
                                Math.max(DEFAULT_MIN_NUM_SAMPLES, calcSamples));
            }

            // check if sampling is disabled.
            if (numSamples == 0) {
                return stats;
            }
            if (numSamples < 0) {
                throw new RuntimeException("Error: Invalid number of samples: " + numSamples);
            }

            // make sure that the sampling times out after a while if the file system does not
            // answer in time
            this.openTimeout = 10000;
            // set a small read buffer size
            this.bufferSize = 4 * 1024;
            // prevent overly large records, for example if we have an incorrectly configured
            // delimiter
            this.lineLengthLimit = MAX_SAMPLE_LEN;

            long offset = 0;
            long totalNumBytes = 0;
            long stepSize = stats.getTotalInputSize() / numSamples;

            int fileNum = 0;
            int samplesTaken = 0;

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Check flink-conf.yaml / OptimizerOptions: ensure optimizer.delimited-format.min-line-samples and max-line-samples are >= 0.
2. Remove overrides so the documented defaults are used.
3. If you do not need sampling, set the format's numLineSamples to 0 via setNumLineSamples(0) so getStatistics short-circuits before the negative check.
4. Inspect the log: loadConfigParameters logs 'Invalid default ... number of line samples' when defaults are bad.

Example fix

// before: bad global config
// optimizer.delimited-format.min-line-samples: -5
stats = format.getStatistics(); // throws RuntimeException

// after: fix config (or disable sampling)
// optimizer.delimited-format.min-line-samples: 2
// or in code:
format.setNumLineSamples(0);
stats = format.getStatistics();

// Before getStatistics, ensure global optimizer sample bounds are non-negative.
int minS = parameters.get(OptimizerOptions.DELIMITED_FORMAT_MIN_LINE_SAMPLES);
int maxS = parameters.get(OptimizerOptions.DELIMITED_FORMAT_MAX_LINE_SAMPLES);
if (minS < 0 || maxS < 0) {
    // fix config or disable sampling
    format.setNumLineSamples(0);
}

try {
    BaseStatistics stats = format.getStatistics();
} catch (RuntimeException e) {
    if (e.getMessage() != null && e.getMessage().contains("Invalid number of samples")) {
        // global optimizer sample config is bad; fall back to no sampling
        format.setNumLineSamples(0);
        stats = format.getStatistics();
    } else {
        throw e;
    }
}

When it happens

Trigger: Calling format.getStatistics() when the global optimizer config sets DELIMITED_FORMAT_MIN_LINE_SAMPLES or DELIMITED_FORMAT_MAX_LINE_SAMPLES to values that yield a negative derived numSamples; or an environment where the loaded defaults produced a negative result.

Common situations: Custom flink-conf.yaml with a typo'd/negative value for optimizer.delimited-format.min-line-samples or max-line-samples; running with a corrupted or hand-edited configuration; a build where loadConfigParameters logged an invalid default but execution continued.

Related errors

• Number of line samples must not be negative.
• Delimiter must not be null
• Line length limit must be at least 1.
• Buffer size must be at least 2.
• Buffer size must be greater than length of delimiter.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/7903f99357d3fcca. Report an issue: GitHub.