What causes "'chunk-size' is not a valid number: `${chunkSizeText}`" in apache/shardingsphere?

Set chunk-size to a plain positive integer literal, e.g. chunk-size: 1000 (unquoted in YAML so it stays numeric).

apache/shardingsphere · error · PipelineInvalidParameterException

'chunk-size' is not a valid number: `${chunkSizeText}`

Error message

'chunk-size' is not a valid number: `${chunkSizeText}`

What it means

DataMatchTableDataConsistencyChecker parses the 'chunk-size' property of the DATA_MATCH consistency-check algorithm configuration. If the value is present but not parseable by Integer.parseInt, a PipelineInvalidParameterException with the offending text is thrown at checker construction time. This is fail-fast validation of algorithm props, not a runtime data error.

Source

Thrown at kernel/data-pipeline/core/src/main/java/org/apache/shardingsphere/data/pipeline/core/consistencycheck/table/DataMatchTableDataConsistencyChecker.java:70

    
    private StreamingRangeType streamingRangeType;
    
    @Override
    public void init(final Properties props) {
        chunkSize = getChunkSize(props);
        streamingRangeType = getStreamingRangeType(props);
    }
    
    private int getChunkSize(final Properties props) {
        String chunkSizeText = props.getProperty(CHUNK_SIZE_KEY);
        if (Strings.isNullOrEmpty(chunkSizeText)) {
            return DEFAULT_CHUNK_SIZE;
        }
        int result;
        try {
            result = Integer.parseInt(chunkSizeText);
        } catch (final NumberFormatException ignore) {
            throw new PipelineInvalidParameterException("'chunk-size' is not a valid number: `" + chunkSizeText + "`");
        }
        if (result <= 0) {
            throw new PipelineInvalidParameterException("Invalid 'chunk-size' value: `" + result + "`, it should be a positive integer.");
        }
        return result;
    }
    
    private StreamingRangeType getStreamingRangeType(final Properties props) {
        String streamingRangeTypeText = props.getProperty(STREAMING_RANGE_TYPE_KEY);
        if (Strings.isNullOrEmpty(streamingRangeTypeText)) {
            return DEFAULT_STREAMING_RANGE_TYPE;
        }
        try {
            return StreamingRangeType.valueOf(streamingRangeTypeText.toUpperCase());
        } catch (final IllegalArgumentException ex) {
            throw new PipelineInvalidParameterException("Invalid 'streaming-range-type' value: `" + streamingRangeTypeText
                    + "`, expected values are " + Arrays.toString(StreamingRangeType.values()));
        }

View on GitHub (pinned to e952770a21)

Solutions

Set chunk-size to a plain positive integer literal, e.g. chunk-size: 1000 (unquoted in YAML so it stays numeric).
Remove the chunk-size key entirely to fall back to DEFAULT_CHUNK_SIZE if the default is acceptable.
Validate props with a preflight script or unit test that Integer.parseInt's each numeric pipeline prop before deploying.

Example fix

# before (YAML)
props:
  chunk-size: "1,000"

# after
props:
  chunk-size: 1000

Defensive patterns

Strategy: validation

Validate before calling

String chunkSizeText = props.getProperty("chunk-size");
if (null != chunkSizeText && !chunkSizeText.matches("\\d+")) {
    throw new IllegalArgumentException("chunk-size must be numeric, got: " + chunkSizeText);
}

Try / catch

try {
    checker = new DataMatchTableDataConsistencyChecker(props);
} catch (final PipelineInvalidParameterException ex) {
    // surface config error to the operator with ex.getMessage()
}

Prevention

Keep chunk-size unquoted and purely numeric in YAML/DISTSQL props.
Validate all pipeline algorithm props in a pre-deploy lint step.
Never carry commas, units, or expressions into numeric pipeline props.

When it happens

Trigger: Configuring the data consistency check algorithm with props like chunk-size: "1000" quoted oddly, chunk-size: 1e4, chunk-size containing whitespace, or a YAML value parsed as a non-integer string; then starting a migration/check job that instantiates the checker.

Common situations: Hand-edited distsql or YAML with a typo in chunk-size; environment-variable substitution injecting an empty-but-present or malformed value; upgrading from a version that silently ignored unknown props to one that validates.

Related errors

AI-assisted analysis of apache/shardingsphere@e952770a21 (2026-08-14). Data as JSON: /api/errors/badebeeb560b1248. Report an issue: GitHub.