apache/seatunnel · error · SQLException

Invalid YashanDB 2-byte vector buffer length:

Error message

Invalid YashanDB 2-byte vector buffer length: 

What it means

toPrimitiveShortArray() expects the FLOAT16/BFLOAT16 vector ByteBuffer to contain a whole number of 2-byte (short) elements, one per vector element. A buffer whose remaining bytes are not a multiple of Short.BYTES (2) is malformed, so an SQLException is thrown with the actual byte count.

Solutions

1. Ensure the producer encodes fp16 vectors via VectorUtils.toByteBuffer(Short[]) so each element is exactly 2 bytes
2. Check the ByteBuffer has even remaining() and full elements (no truncation) before writing
3. Verify the vector column type matches the data: FLOAT32 vectors in a FLOAT16 column will mismatch element sizes
4. Regenerate the embedding data if buffers are corrupted upstream

Example fix

// before
byteBuffer.position(0); // slice accidentally excludes last byte -> odd remaining()
// after
if (byteBuffer.remaining() % 2 != 0) {
    throw new IllegalArgumentException("fp16 buffer must be even-length, got " + byteBuffer.remaining());
}

if (buf.remaining() % 2 != 0) throw new IllegalStateException("fp16 vector buffer must be even length: " + buf.remaining());

static boolean isWellFormedFp16Buffer(ByteBuffer b) {
    ByteBuffer d = b.duplicate(); d.rewind();
    return d.remaining() % Short.BYTES == 0;
}

try {
    stmt.setObject(idx, Vector.ofFloat16Values(shortArray(value)));
} catch (SQLException e) {
    if (e.getMessage().contains("Invalid YashanDB 2-byte vector buffer length")) {
        throw new DataValidationException("corrupt fp16 vector from upstream", e);
    } else throw e;
}

When it happens

Trigger: Writing a FLOAT16_VECTOR or BFLOAT16_VECTOR value where the underlying ByteBuffer has odd byte length — e.g. truncated buffer, wrong endianness/serialization, or a 1-byte-per-element encoding passed instead of 2-byte shorts.

Common situations: Upstream producer serialized fp16 vectors as 8-bit halves or dropped a byte (off-by-one/truncation); ByteBuffer slice with wrong position/limit; mixing FLOAT32 (4-byte) data into a FLOAT16 vector column config.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• The column with type ARRAY will be converted to VECTOR in…
• The column with type will be converted to VECTOR in YashanDB
• Unsupported YashanDB vector value type
• Column not found in table
• COMMON-17

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/18e88a1fdee98787. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/yashandb/YashanDbJdbcRowConverter.java:291

    }

    private static float[] toBFloat16Array(Object value) throws SQLException {
        short[] shortArray = toPrimitiveShortArray(value);
        float[] floatArray = new float[shortArray.length];
        for (int i = 0; i < shortArray.length; i++) {
            floatArray[i] = bfloat16ToFloat(shortArray[i]);
        }
        return floatArray;
    }

    private static short[] toPrimitiveShortArray(Object value) throws SQLException {
        if (value instanceof ByteBuffer) {
            ByteBuffer byteBuffer = ((ByteBuffer) value).duplicate();
            byteBuffer.rewind();
            // SeaTunnel FLOAT16_VECTOR and BFLOAT16_VECTOR ByteBuffers are encoded as one
            // big-endian 2-byte short per vector element via VectorUtils.toByteBuffer(Short[]).
            if (byteBuffer.remaining() % Short.BYTES != 0) {
                throw new SQLException(
                        "Invalid YashanDB 2-byte vector buffer length: " + byteBuffer.remaining());
            }
            short[] shortArray = new short[byteBuffer.remaining() / Short.BYTES];
            for (int i = 0; i < shortArray.length; i++) {
                shortArray[i] = byteBuffer.getShort();
            }
            return shortArray;
        }

        Object[] objectArray = (Object[]) value;
        short[] shortArray = new short[objectArray.length];
        for (int i = 0; i < objectArray.length; i++) {
            shortArray[i] = ((Number) objectArray[i]).shortValue();
        }
        return shortArray;
    }

    private static float float16ToFloat(short value) {

View on GitHub (pinned to cf67b549a7)