alibaba/canal · error · RuntimeException

Expected length of at least {} bytes, but had {}

Error message

Expected length of at least {} bytes, but had {}

What it means

PhTypeUtil.checkForSufficientLength() guards every PHOENIX-mode decode: it verifies the byte array is long enough to hold the required type at the given offset. When reading a value with the wrong PHOENIX PhType (or truncated/corrupt bytes), the array is shorter than the type demands and it throws a RuntimeException naming expected vs actual byte counts.

Source

Thrown at client-adapter/hbase/src/main/java/com/alibaba/otter/canal/client/adapter/hbase/support/PhTypeUtil.java:604

            result[--index] = (byte) (digit * multiplyBy + digitOffset);
            multiplyBy = 1;
            divideBy = ONE_HUNDRED;
        }
        long l = bi.longValue();
        do {
            long divBy = 100 / multiplyBy;
            long digit = l % divBy;
            l /= divBy;
            result[--index] = (byte) (digit * multiplyBy + digitOffset);
            multiplyBy = 1;
        } while (l != 0);

        return length;
    }

    private static void checkForSufficientLength(byte[] b, int offset, int requiredLength) {
        if (b.length < offset + requiredLength) {
            throw new RuntimeException(
                "Expected length of at least " + requiredLength + " bytes, but had " + (b.length - offset));
        }
    }

}

View on GitHub (pinned to 87be50e876)

Solutions

1. Align the column's configured PHOENIX type with the type originally used to write the bytes (re-ETL if the type changed).
2. Verify the mapping 'columns' type annotation (the part after '$') matches the actual stored PHOENIX type.
3. Audit the HBase cell bytes for truncation/corruption.
4. If migrating types, do a full re-import (ETL) so all bytes use the new encoding.

Example fix

# before (type mismatch after data written)
columns:
  amount: CF:amount$INTEGER

# after (match the stored PHOENIX type)
columns:
  amount: CF:amount$DECIMAL

// Before decoding in PHOENIX mode, sanity-check byte length
PhType ph = PhType.getType(columnItem.getType());
int needed = expectedLengthFor(ph); // your helper mapping PhType->Bytes.SIZEOF_*
if (bytes.length < needed) {
    throw new IllegalStateException("byte length " + bytes.length + " < required " + needed + " for " + ph);
}

try {
    Object v = PhTypeUtil.toObject(bytes, phType);
} catch (RuntimeException e) {
    if (e.getMessage() != null && e.getMessage().contains("Expected length")) {
        logger.error("Stored bytes shorter than PHOENIX type requires; likely a type change or corruption - re-ETL");
    }
    throw e;
}

When it happens

Trigger: Reading HBase cells in PHOENIX mode where the stored bytes were written with a different type than the configured PhType, or the byte array is truncated/corrupt; e.g. decoding 3 bytes as an INT (needs 4) or 7 bytes as a LONG (needs 8).

Common situations: Changing a column's configured PHOENIX type after data was already written with the old type; data corruption in HBase; a column type misconfiguration in the mapping; mode mismatch between write and read.

Related errors

• mismatch class type
• not allow to change outAdapterKey
• hbaseMapping.database
• hbaseMapping.table
• hbaseMapping.hbaseTable

AI-assisted analysis of alibaba/canal@87be50e876 (2026-08-14). Data as JSON: /api/errors/5f6705acc31b68ab. Report an issue: GitHub.