pinpoint-apm/pinpoint · error · ArrayIndexOutOfBoundsException

invalid varLong. start offset:${offset} readOffset:${offset}

Error message

invalid varLong. start offset:${offset} readOffset:${offset}

What it means

BytesUtils.readVar64SlowPath decodes a LEB128-style variable-length integer (varint/varLong) from a byte array. After 10 continuation bytes (shift reaching 64) without encountering a byte whose high bit is 0, the encoded value is malformed, and the method throws ArrayIndexOutOfBoundsException with 'invalid varLong'. This means the byte stream is corrupted, truncated, or the caller passed a wrong offset. It is thrown by bytesToVar32/bytesToVar64 when reading agents' binary payloads (e.g.agent metadata, network-encoded values).

Source

Thrown at commons/src/main/java/com/navercorp/pinpoint/common/util/BytesUtils.java:250

            }
            return x;
        }
        return readVar64SlowPath(buffer, offset);
    }

    /** Variant of readRawVarint64 for when uncomfortably close to the limit. */
    /* Visible for testing */
    static long readVar64SlowPath(final byte[] buffer, int offset) {

        long result = 0;
        for (int shift = 0; shift < 64; shift += 7) {
            final byte b = buffer[offset++];
            result |= (long) (b & 0x7F) << shift;
            if ((b & 0x80) == 0) {
                return result;
            }
        }
        throw new ArrayIndexOutOfBoundsException("invalid varLong. start offset:" +  offset + " readOffset:" + offset);
    }

    public static short bytesToShort(final byte byte1, final byte byte2) {
        return (short) (((byte1 & 0xff) << 8) | ((byte2 & 0xff)));
    }


    public static int writeLong(final long value, final byte[] buf, int offset) {
        if (buf == null) {
            throw new NullPointerException("buf");
        }
        checkBounds(buf, offset, LONG_BYTE_LENGTH);

        buf[offset++] = (byte) (value >> 56);
        buf[offset++] = (byte) (value >> 48);
        buf[offset++] = (byte) (value >> 40);
        buf[offset++] = (byte) (value >> 32);
        buf[offset++] = (byte) (value >> 24);

View on GitHub (pinned to 744c3d3075)

Solutions

  1. Verify the offset passed to bytesToVar32/bytesToVar64 points at the first byte of a varint, not mid-value; recompute offsets from prior read lengths.
  2. Check that the byte array is complete and not truncated before decoding; confirm the writer and reader use the same Pinpoint version and matching bytesToVar/writeVar (var32 vs var64) pair.
  3. Validate the input bytes: at most 10 bytes for a 64-bit varint and the 10th byte must have high bit 0; reject payloads violating this before calling the API.
  4. Catch ArrayIndexOutOfBoundsException around the decode call and treat the payload as corrupt: log offset and skip/discard the record instead of crashing.
  5. If data comes from the wire, add a checksum/length prefix when writing so truncated or shifted frames can be detected before parsing.

Example fix

// before: decode without validation, mid-array offset from manual bookkeeping
long value = BytesUtils.bytesToVar64(buffer, offset);

// after: bounds + continuation-bit sanity check before decoding
long value;
if (offset < 0 || offset >= buffer.length) {
    throw new IllegalArgumentException("bad offset: " + offset);
}
int maxBytes = Math.min(10, buffer.length - offset);
boolean terminated = false;
for (int i = 0; i < maxBytes; i++) {
    if ((buffer[offset + i] & 0x80) == 0) { terminated = true; break; }
}
if (!terminated) {
    throw new IllegalArgumentException("malformed varLong at offset " + offset);
}
value = BytesUtils.bytesToVar64(buffer, offset);
Defensive patterns

Strategy: validation

Validate before calling

// Pre-validate before calling BytesUtils.bytesToVar64/bytesToVar32
public static boolean isValidVarLongAt(byte[] buffer, int offset) {
    if (buffer == null || offset < 0 || offset >= buffer.length) return false;
    int max = Math.min(10, buffer.length - offset);
    for (int i = 0; i < max; i++) {
        if ((buffer[offset + i] & 0x80) == 0) return true; // terminator found
    }
    return false; // 10 continuation bytes or buffer ends mid-varint
}

Type guard

// Java has no runtime type narrowing; guard structurally with an Optional-style check
public static Integer safeVar32Offset(byte[] buffer, int offset) {
    return isValidVarLongAt(buffer, offset) ? Integer.valueOf(offset) : null;
}

Try / catch

try {
    long value = BytesUtils.bytesToVar64(buffer, offset);
    // use value
} catch (ArrayIndexOutOfBoundsException e) {
    if (e.getMessage() != null && e.getMessage().contains("invalid varLong")) {
        log.warn("Corrupt varLong in payload at offset {}, discarding record", offset, e);
        return null; // or skip record / mark payload corrupt
    }
    throw e;
}

Prevention

When it happens

Trigger: Calling BytesUtils.bytesToVar32(buffer, offset) or BytesUtils.bytesToVar64(buffer, offset) with (a) 10 or more consecutive bytes all having the continuation bit (0x80) set starting at offset, (b) a buffer truncated so buffer[offset++] itself goes out of bounds while scanning, or (c) a misaligned offset that lands mid-varint so the terminator byte is never seen within 10 bytes.

Common situations: Deserializing a Pinpoint agent's binary column or network payload that was written by a different/incompatible version (varint encoder changed), reading from an offset not returned by a previous varint read (manual offset bookkeeping off by one), passing a byte array slice that cuts off mid-varint, or feeding corrupted/truncated data from storage or the wire into bytesToVar64/bytesToVar32.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

AI-assisted analysis of pinpoint-apm/pinpoint@744c3d3075 (2026-09-07). Data as JSON: /api/errors/7d5651b33b4d3058. Report an issue: GitHub.