apache/flink · error · NumberFormatException

Invalid character.

Error message

Invalid character.

What it means

ShortParser.parseField throws NumberFormatException("Invalid character.") when a byte inside the numeric field is outside the ASCII digit range '0'..'57'. The parser is a strict byte-level parser: no signs mid-field, no spaces, no decimal points, no thousands separators, no scientific notation.

Source

Thrown at flink-core/src/main/java/org/apache/flink/types/parser/ShortParser.java:145

        if (bytes[startPos] == delimiter) {
            throw new NumberFormatException("Empty field.");
        }

        if (bytes[startPos] == '-') {
            neg = true;
            startPos++;
            length--;
            if (length == 0 || bytes[startPos] == delimiter) {
                throw new NumberFormatException("Orphaned minus sign.");
            }
        }

        for (; length > 0; startPos++, length--) {
            if (bytes[startPos] == delimiter) {
                return (short) (neg ? -val : val);
            }
            if (bytes[startPos] < 48 || bytes[startPos] > 57) {
                throw new NumberFormatException("Invalid character.");
            }
            val *= 10;
            val += bytes[startPos] - 48;

            if (val > OVERFLOW_BOUND && (!neg || val > UNDERFLOW_BOUND)) {
                throw new NumberFormatException("Value overflow/underflow");
            }
        }

        return (short) (neg ? -val : val);
    }
}

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Inspect the exact bytes of the failing field (hex dump the line) and clean the data: strip whitespace, remove separators, drop '+' signs, remove BOM
2. Verify the CSV delimiter and quote configuration so non-numeric columns are not merged into the short column
3. Pre-filter or map fields to sanitized strings before parsing to short
4. Switch the column type to STRING and cast/validate in application code if the data is inherently dirty

Example fix

// before
RowCsvInputFormat fmt = new RowCsvInputFormat(path, Types.SHORT); // field: " 12"

// after
// sanitize first: read as STRING, trim, then parse
String s = row.getFieldAs(0).trim();
short v = Short.parseShort(s.startsWith("+") ? s.substring(1) : s);

static boolean isCleanShortToken(String s) {
    return s != null && s.matches("-?[0-9]+") && s.replace("-", "").length() <= 5;
}

catch (NumberFormatException e) { /* quarantine row with offending token */ }

When it happens

Trigger: A short-typed CSV field containing letters ('12a'), a plus sign ('+5' — only '-' is handled), whitespace (' 12' or '12 '), a decimal point ('3.5'), or a locale-style separator ('1,000' when ',' is not the delimiter).

Common situations: Excel-exported CSVs with thousands separators or trailing spaces; locales using different encodings; UTF-8 BOM bytes at the start of the first field; mismatched delimiter making text from the next column leak into a numeric column; plus-signed numbers produced by some exporters.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Empty field.
• Orphaned minus sign.
• Value overflow/underflow
• value overflow
• Row too short: {}

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/6282bb6f41d4408b. Report an issue: GitHub.