alibaba/canal · error · IllegalArgumentException

Invalid charset id: {}

Error message

Invalid charset id: {}

What it means

Thrown by CharsetConversion.getEntry(id) when a MySQL character-set id passed during binlog row decoding is negative or >= 2048 (the fixed size of the static 'entries' lookup array). The library maps numeric charset ids (carried in TABLE_MAP and row events) to Java Charset objects at load time via putEntry; ids in range but unregistered return null rather than throwing, so this fires only for genuinely out-of-range values.

Source

Thrown at dbsync/src/main/java/com/taobao/tddl/dbsync/binlog/CharsetConversion.java:43

        Entry(final int id, String mysqlCharset, // NL
              String mysqlCollation, String javaCharset){
            this.charsetId = id;
            this.mysqlCharset = mysqlCharset;
            this.mysqlCollation = mysqlCollation;
            this.javaCharset = javaCharset;
            this.charset = Charset.isSupported(javaCharset) ? Charset.forName(javaCharset) : null;
        }
    }

    // Character set data used in lookups. The array will be sparse.
    static final Entry[] entries = new Entry[2048];

    static Entry getEntry(final int id) {
        if (id >= 0 && id < entries.length) {
            return entries[id];
        } else {
            throw new IllegalArgumentException("Invalid charset id: " + id);
        }
    }

    // Loads character set information.
    static void putEntry(final int charsetId, String mysqlCharset, String mysqlCollation, String javaCharset) {
        entries[charsetId] = new Entry(charsetId, mysqlCharset, // NL
            mysqlCollation,
            javaCharset);
    }

    // Loads character set information.
    @Deprecated
    static void putEntry(final int charsetId, String mysqlCharset, String mysqlCollation) {
        entries[charsetId] = new Entry(charsetId, mysqlCharset, // NL
            mysqlCollation, /* Unknown java charset */
            null);
    }

View on GitHub (pinned to 87be50e876)

Solutions

1. Confirm the id value in the message; if negative, suspect a signed-read bug in the upstream LogBuffer call feeding getEntry and verify the binlog is not truncated or corrupt.
2. Re-stream the binlog from a known-good position (earlier GTID/binlog file) to rule out transient corruption at the master.
3. If the id is legitimately valid but unregistered in this build, extend the putEntry table in CharsetConversion to register it and rebuild dbsync.
4. Validate the binlog with mysqlbinlog on the source server; if it errors there too, the binlog file itself is damaged and must be skipped/re-seeded.

Example fix

// before
Entry e = CharsetConversion.getEntry(charsetId); // throws for out-of-range id

// after - guard before lookup
if (charsetId < 0 || charsetId >= CharsetConversion.entries.length) {
    logger.warn("Skipping column with out-of-range charset id: " + charsetId);
    return null;
}
Entry e = CharsetConversion.getEntry(charsetId);

// Validate charset id against the lookup array bounds before getEntry
public static Entry safeGetEntry(int id) {
    if (id < 0 || id >= CharsetConversion.entries.length) {
        return null; // caller decides how to treat unknown charset
    }
    return CharsetConversion.getEntry(id);
}

try { Entry e = CharsetConversion.getEntry(id); }
catch (IllegalArgumentException ex) {
    // log + degrade: treat column as binary/unknown charset
    logger.warn("Out-of-range charset id " + id + ", treating as binary", ex);
}

When it happens

Trigger: A binlog event supplies a column charset/collation numeric id that, when handed to CharsetConversion.getEntry(id), is < 0 or >= 2048. This happens while decoding column metadata in a TABLE_MAP_EVENT or writing row column values whose MysqlType carries a charset number.

Common situations: Corrupt or partially-written binlog where the 2-byte charset id was read as signed (e.g. a high bit set turning a valid id negative), a hand-crafted/malformed binlog stream from a non-standard source, or a future MySQL version emitting a collation id beyond the library's known set combined with a sign-extension bug in the read path.

Related errors

• illegal json data
• parse row data failed.
• No such method: '{}' @ {}
• Cannot invoke method: '{}' @ {}
• Invoke method failed: '{}' @ {}

AI-assisted analysis of alibaba/canal@87be50e876 (2026-08-14). Data as JSON: /api/errors/8012e1d827a6ea0b. Report an issue: GitHub.