t8y2/dbx · error

length %d exceeds limit

Error message

length %d exceeds limit

What it means

This error means a field length in the delegation token exceeded the 64 MiB safety limit enforced by readHadoopByteArray. The driver caps parsed token fields to prevent absurd allocations from corrupt or hostile input. A length over 64*1024*1024 bytes means the token is malformed or not a delegation token.

Source

Thrown at agents/drivers/hive-go/config.go:810

	if _, err := readHadoopByteArray(reader); err != nil {
		return nil, nil, fmt.Errorf("service: %w", err)
	}
	if reader.Len() != 0 {
		return nil, nil, errors.New("token contains trailing data")
	}
	return identifier, password, nil
}

func readHadoopByteArray(reader io.ByteReader) ([]byte, error) {
	length, err := readHadoopVInt(reader)
	if err != nil {
		return nil, err
	}
	if length < 0 {
		return nil, fmt.Errorf("negative length %d", length)
	}
	if length > 64*1024*1024 {
		return nil, fmt.Errorf("length %d exceeds limit", length)
	}
	value := make([]byte, int(length))
	byteReader, ok := reader.(io.Reader)
	if !ok {
		return nil, errors.New("reader cannot read token payload")
	}
	if _, err := io.ReadFull(byteReader, value); err != nil {
		return nil, err
	}
	return value, nil
}

func readHadoopVInt(reader io.ByteReader) (int64, error) {
	firstByte, err := reader.ReadByte()
	if err != nil {
		return 0, err
	}
	first := int8(firstByte)

View on GitHub (pinned to c0390bff16)

Solutions

1. Verify only the actual delegation token string is passed to delegationToken/token/password
2. Re-fetch the token from the Hive server; it should be a few hundred bytes
3. Check that the token's base64 encoding was not double-encoded, shifting length bytes

func tokenSizeReasonable(token string) bool {
	raw, err := base64.StdEncoding.DecodeString(token)
	return err == nil && len(raw) < 64*1024*1024 && len(raw) > 0
}

if err != nil && strings.Contains(err.Error(), "exceeds limit") {
	return fmt.Errorf("not a delegation token (too large): %w", err)
}

When it happens

Trigger: A delegation token whose VInt length prefix decodes to more than 67108864 bytes, usually from decoding an unrelated large binary blob as a token.

Common situations: Passing a large certificate, keystore, or other base64 blob in the delegationToken field; corrupted multi-byte VInt inflating the decoded length.

Related errors

• token identifier and password must be non-empty
• token contains trailing data
• identifier: %w
• password: %w
• kind: %w

AI-assisted analysis of t8y2/dbx@c0390bff16 (2026-09-05). Data as JSON: /api/errors/c478b198f56e2162. Report an issue: GitHub.