CHAPTER 1 INTRODUCTION -...
Transcript of CHAPTER 1 INTRODUCTION -...
1
CHAPTER 1
INTRODUCTION
1.1 GENERAL
In the last two decades, there has been a substantial improvement in
the computing and network performance. The recent advances in technologies
have enabled the availability of Personal Computers (PCs), workstations,
Symmetric Multiple Processors (SMPs) and nomadic devices at a cheaper
cost, making it possible to achieve high performance and high throughput
computing. The low cost high performance computing systems motivated
researchers to solve the resource and data intensive problems in a number of
application domains. The availability of powerful computing resources and
high speed networks allowed scientists to broaden their simulations and
experiments to accommodate more parameters than ever before.
The scientific applications often need a huge amount of
computational and storage power. Hence, seamless access to such resources
that are distributed geographically are needed. The power of computing and
networking made it possible to share resources such as data from instruments,
results of experiments with collaborators, and high performance computers
around the globe almost instantaneously. Several attempts have been made to
provide support for flexible and controlled sharing of various types of
resources that are required to solve computationally intensive applications.
For instance, the cluster computing paradigm fails to address the management
2
of heterogeneous resources, whereas distributed computing paradigm does not
support the management of resources belonging to diverse organizations.
To address this issue, an extended distributed computing
technology, termed as 'Grid', was coined by Ian Foster et al (2001 and 2002)
that support the aggregation of distributed computational resources that spans
beyond the organizational boundaries, and their coordinated use to meet the
requirements of advanced science and engineering. Grid can be distinguished
from conventional distributed computing by its focus on large scale resource
sharing, high performance, and computation / data intensive applications.
Grid supports researchers and scientists from diverse organizations to share
information, instruments, data, computation and storage resources
dynamically in a flexible and secure manner, thereby forming a ‘Virtual
Organization’(VO) to solve challenging applications.
To realize such an infrastructure, layered reference architecture
with a set of protocols was devised and is shown in the Figure 1.1. The
bottom layer is the resource (fabric) and top layer comprises of demanding e-
science applications.
Figure 1.1 Grid Protocol Architecture
3
The middle three layers are also called as Middleware provides an
abstraction over the resources and relieves the user from the tenets of
management issues, like locating the suitable resource, job migration, security
and monitoring. Taking the analogy of Internet, which is a network of
networks, the grid is just an aggregation of clusters. The rationale of choosing
a cluster as the basic entity of the grid is that it can encapsulate a substantial
number of nodes and provide a single system image. Also the grid is
constructed by the contribution from various organizational resources. It is
easy to maintain the contributing organizations cluster than manage the
machines one by one.
The Resource and Connectivity layers contribute a set of services,
which performs the individual cluster management. The Collective layer on
top performs grid wide management, which is a super scheduler.
In addition to the middleware, to facilitate data centric applications
on the grid, domain specific functionalities can be complemented with the
middleware and classified as core services and higher level services as
depicted in Figure 1.2. The core services such as data discovery, job
submission and replication provide transparent access to the distributed data
and computation.
Figure 1.2 High Level Services for Data Centric Applications
4
Higher level service includes resource brokering which selects
resources for a user based on his requirement and replica management service
which decides about the creation and distribution of multiple copies of the
same data item. These services can be constructed from the core middleware
services to complement the collective layer.
Grid is already being successfully used in many scientific
applications such as high-energy physics, bio-medicine, aerospace and earth
sciences and is continuing to evolve and expand. However, it is also the basis
and enabling technology for pervasive and utility computing due to its ability
of being open, highly heterogeneous and scalable. Yet, it has become essential
to support seamless computing and communication for mobile users, which
would offer flexibility and increased information availability.
At the same time, a modern computing technology arises with the
consumer electronic devices which is termed as Mobile Computing. Mobile
computing is an expansion of the traditional distributed computing, which
includes devices like Laptops with Wireless Local Area Networking (WLAN)
technologies, Mobile Phones and Personal Digital Assistants. This computing
model focuses on the requirement of providing access to information,
communication and services everywhere, anytime.
The rapid technology advancements made mobile devices more
powerful (in terms of computing) and ubiquitous (in terms of connectivity).
The evolution of 3G and 4G wireless technologies enhances the spectral
efficiency, to support high data rates and mobile broad band everywhere.
Nowadays, the wireless devices such as laptops are enriched with a
substantial amount of computing power equal to their static counterparts and
the ability to change the locations with the same IP address is also achieved.
These advances made the wireless devices to offer high performance
capabilities in a flexible manner seamlessly.
5
Figure 1.3 depicts the common mobile application architecture
proposed by Microsoft patterns and Practices team (2009) to enable the
applications on mobile computing environment.
Figure 1.3 Mobile Application Architecture
In this architecture, the mobile devices are connected with the base stations using wireless communication medium and the connectivity extended with the mobile support infrastructure. The two main components of the architecture are
1. Application support layer
2. Infrastructure support layer.
This architecture enables the realization of data centric applications
on the mobile domain by having a separate layer called data layer. The data
specific provisions like replica management, replica selection and data access
utilities can be integrated with this data layer. Rapid advances in the
6
increasing processing power of wireless devices, along with the wireless
technologies such as Wi-Fi, Wi -Max has made them increasingly capable of
participating in grid networks.
The combinations of these two computing models have the
potential to realize significant developments in the adoption of high
performance grid through mobile devices. As a result of several case studies
from various e-Science projects as discussed by Hey T and Trefethen A E
(2002), the mobile device needs the grid for computation and integration,
while the grid needs the mobile devices to interface with the physical world.
With the incorporation of third generation mobile networks and high
bandwidth wireless technologies, grid computing has moved from the
traditional parallel and distributed model to the mobility based model (Sanjay
Ahuja and Jack Myers 2006).
This paradigm shift has given rise to a new mechanism called
Mobile Grid (M-Grid). The Mobile Grid enables both the fixed and mobile
users to have access to both the fixed and mobile grid resources transparently
using the underlying technologies. The integration of the fixed and mobile
devices into the grid environment is presented in the Figure 1.4.
Figure 1.4 Mobile Grid Computing Model
Interface to both fixed and
mobile grid resources
7
In the above mobile grid model, the two possible roles of mobile
devices are,
Mobile device as a Grid Interface
o Mobile devices transfer their tasks to the grid rather than
performing it themselves and monitor them being
executed on the grid. In this role, mobile devices act as
Grid Service consumers as shown in the Figure 1.5.
Figure 1.5 Mobile Devices as Grid Interface
Mobile device as a compute node in grid infrastructure
o This integration makes mobile devices aggregated into
the grid environment as resource providers for effective
computational distribution. In this, by forming Mobile
Dynamic Virtual Organization (MDVO) (Martin
Waldburger and Burkhard Stiller 2005) the wireless
devices participate in the grid as resources. The Figure
1.6 shows the mobile devices as resource provider in
grid.
8
Figure 1.6 Mobile Devices as Grid Resources
While integrating mobile devices with the grid environment which
has a dynamic nature, the conventional grid middleware needs extension to
support heterogeneous device characteristics, disconnected operations,
resource optimization in resource constrained environment and energy
optimization. Also, the main objective of realizing mobile grid is running data
centric applications. In such applications, large amount of data will be stored,
accessed and processed. To enhance these operations, an effective high level
data management service like data replication can be utilized.
1.2 DATA REPLICATION IN GRID DOMAIN
Data replication is a well known strategy to improve the
performance and reliability of any distributed computing platforms. When the
users of a system are distributed over a resource sharing network such as grid,
keeping data at a single location can affect data access in three ways
Latency: The data access time varies with the distance and link
bandwidth of the user from the data storage, and it is subject to network
problems related to the network environment.
Grid Node
9
Availability: Having a single data storage site is risky for critical
applications. When storage is temporarily unavailable (for faults or
maintenance reasons), or the storage site is not reachable due to network
problems, users do not have access to data.
Congestion: The single data storage site must sustain a potentially
high number of user’s requests. The hardware used for data storage can be
very expensive or fail to satisfy the user’s requests.
For these reasons, to facilitate the data centric applications on the
grid domain, data replication services seek to enhance the network traffic by
copying heavily accessed data to appropriate locations and managing them.
The replication mechanism attempts to determine when, where and what data
are to be replicated across the grid nodes that may be fixed or mobile. The
Figure 1.7 illustrates a replication environment.
Figure 1.7 A Simple Replication Scenario
In the Figure 1.7, Site 1, Site 2, …, Site n are distributed site
locations and connected through a middleware infrastructure. Data stored in a
file, File X, is stored in site 2and is replicated at all other sites. The benefits
10
due to replication of files can be accessed at a cheaper cost i.e. closer to the
user and the file can be still accessed even if 3 out of the 4 sites are down.
Data replication increases the availability and reliability of the data. It also
reduces data access delay at the cost of data storage.
However, in a replicated environment different challenges must be addressed, and the following are some of the key points,
Data Consistency
o Maintaining uniformity among the distributed data
Maintenance Overhead
o Managing more number of copies makes it difficult to administrate in terms of storage and update propagation.
Lower Write performance
o Applications require more updates in a replicated environment; multiple copies may have to be updated.
Due to the above mentioned challenges, it is necessary to manage
the resources in a resource restricted (intermittent connectivity, battery power)
mobile grid environment in an optimized manner and the service provisions
should be reliable.
1.3 ADOPTION OF MOBILE GRID
The adoption of this new computing model research is already
accomplished in the areas of disaster handling, e-health and crisis
management. Some of the currently existing mobile grid projects are given
below,
AKOGRIMO - Developed mobile collaborative business grids for
e-health, e-learning and disaster management. (www.akogrimo.org)
11
WiGiT - A middle ware extension layer called Edgeware to
facilitate collaboration among grid, cloud and mobile networks.
(www.wigit.ischool.syr.edu)
Garuda Grid - Provides Garuda Access Portal 2.1 with mobile
services, supporting access and monitor the grid resources through mobile
phones. (http://portal.garudaindia.in/gap2/gap2)
The motivation and contributions of the present research, to extend
the middleware capabilities to a mobile grid and enhance data centric
applications on it are discussed in the next section.
1.4 MOTIVATION AND CONTRIBUTIONS
The integration of the grid and mobile computing models provide a
remarkable development ie, the adoption of Wireless Cooperative Clusters
(WCC) into the high performance grid. Grid can include these clusters as task
execution environment by forming MDVO.
The core requirements to realize such an environment can be
grouped into two levels of abstractions as mentioned below,
Infrastructure Management
o Managing intermittent connectivity
o Autonomic configuration
o Energy optimization
o Resource optimization
Data Management
o Availability ( Replication)
o Reliability ( Fault tolerant)
12
o Reliable task achievement
o Maintainability
1.4.1 Infrastructure Management
In a mobile environment, the available resources comprise of static
fixed nodes and dynamic mobile nodes. To accommodate the dynamic nature
of the mobile grid environment such as link quality and mobility of nodes, the
conventional middleware components does not seem to be flexible and
scalable. The formation of a mobile grid requires additional functionalities to
be added at the infrastructure level in a flexible and open way which are listed
below,
Resource management and optimization
Mobility management
Asymmetry connectivity
Changing loads on the participating node
Energy optimization and
Formation of MDVO
The volatile and dynamic mobile grid environment requires the use
of sophisticated mechanisms for resource discovery and selection. The
selection of resources that meet the time and cost constraints should be
imposed by the adapting mechanism. Additional match making parameters
like resource accessibility, system workload and network performance have to
be considered by the approach.
Constraints that would complicate the mobile grid job scheduling
are the changing devices connection status and their physical locations.
Hence, an optimization criterion for job scheduling mechanism should
13
consider not only the cost and performance of each resource in terms of task
execution, but also the current availability of the reliable resource in a
MDVO. One such network characteristic to achieve reliability in the mobile
environment is the Signal – to – Noise Ratio (SNR). SNR directly impacts the
performance of a wireless LAN connection. A higher SNR value (in dB)
means that the signal strength is stronger in relation to the noise levels, which
allows higher data rates and fewer re-transmissions.
The bandwidth, storage, computational and energy resources spent
for task execution, is wasted if the assigned task is aborted upon
disconnection. It is stressed that the approaches used for reliable task
execution, should address the problem of intermittent connectivity. Most of
the approaches in this direction are not mutually exclusive. Building
connectivity profiles for each node may assist with an intelligent scheduler
and such intelligence can be achieved using agent based technology.
To make grid functionalities feasible among the group of mobile
devices, energy efficiency is one of the major concern for their
implementation on each node. To find a technique to reduce the power
consumption for communication and storage is an issue. It is mandatory to
provide a mechanism, to periodically exchange the power level of nodes, so
that the node with the highest power can be considered to hold the services, to
improve the availability.
1.4.2 Data Management
The idea of integrating mobile devices with the back end parallel
computing cluster will be essential for different classes of parallel
applications such as image rendering on battlefield, surveillance,
reconnaissance and other classes of military applications. In these scenarios,
attack task forces can form a mobile grid network to accomplish the mission
14
(also called mobile tactical networks Yang Zhang et al (2011)). As the
mission progresses, a large amount of information must be shared and
processed among the soldiers and squads by their wireless devices. In such
cases the data replication services are a vital requirement to facilitate the
execution of data centric applications on the mobile grid.
The key challenge in mobile grid with respect to data service is the
effective replication of data over mobile nodes. To improve and maintain the
overall throughput of grid jobs that access files, the environment has to
provide an optimal distribution and replication of data files. With a number
of replicas available, a client site can access the file with minimal distance.
However, maintaining multiple copies in a resource constrained mobile grid
system is expensive. Therefore, the number of replica should be bounded to
the optimum number of copies.
Most of the replica management procedure supports only
immutable file operations in the conventional grid implementation. It is
mandatory to assist mutable file transactions to realize the grid in the
commercial domain. The new computing model like Mobile Grid should
consider both the read and update transactions to effectively manage the
replication in this domain.
Due to the dynamic characteristics of mobile grid such as mobility
of the nodes, the node which holds replica currently may not be the best site
to facilitate the access request. Therefore, relocation is to be considered if the
performance is to be maintained. While relocating a replica, user access
patterns may be considered to minimize the access cost. Based on the
accumulated Read / Write statistics of a node, it can be selected to hold the
replica.
15
In order to handle the intermittent connectivity among the mobile
devices, it is necessary to consider the signal quality in terms of SNR. During
the selection of the nodes to hold the replica, the node which has the higher
SNR can be considered. Hence, one can prevent data replication at nodes
which has the tendency towards connection failure. Hence, the reliability and
stability of the network can be maximized.
Apart from considering mechanisms to find the location of replica,
relocation of replica, consistency maintenance and infrastructure for mobile
grid, use of good replica placement strategies is necessary to optimize the
number of replicas and to improve the performance of replica aware services.
Additional challenges to be considered with regard to resource
management are optimized resource utilization, energy optimization,
autonomic configuration and mobility management. It requires a well defined
fault tolerant provision to improve the performance in terms of scalability and
reliability.
In conclusion the management of data replication has to be
achieved in an effective manner in the mobile grid environment requiring
additional efforts. It has to cover all the aspects of wired and wireless grid
characteristics. Considering only the static characteristics will not be a
solution for replica management. In the mobile grid a lot of dynamic
characteristics such as topology management, transmission capability,
handling of flash request with minimum resources and signal strength (SNR)
of the environment must be considered. In addition, agent based intelligence
can be used to support the replication process.
The problems considered in the proposed research are depicted in
the Figure 1.8.
16
Replication Support
Infrastructure Support
Figure 1.8 Architecture of the Proposed Work
The present research work presented in this thesis has addressed a
multi agent based framework to enhance the middleware to support mobility
and a replica management mechanism to support data services on the mobile
grid. The proposed approach is enhanced with consistency management
among the replications and fault tolerant provisions to manage the failure.
The work considers the dynamic characteristics such as Signal quality,
Storage and disconnection.
In brief, the contributions of the research are summarized as
follows,
User Applications
Grid Middleware
Mobility support agent based framework
High level Data Services Optimized Replication Manager
Grid Portal
Consistency Maintenance Support
Fault Tolerant Support
Static Resources Dynamic Resources
17
1. Proposed an optimized multi-site replica placement mechanism based on Signal-to-Noise ratio, which is intended to find the appropriate and reliable candidate nodes to place the replica. The proposed algorithm finds, deletes, and relocates the candidate node, based on the read and update frequency, and communication cost along with the threshold Signal –to– Noise ratio. Also, this mechanism will select only minimum number of locations to place the replica which can satisfy large spike of unpredictable requests. Hence, maintenance becomes easy with good availability.
2. Existing middleware mechanisms supports the integration of mobile devices as resource consumers. However, it is necessary to build a mobility aware framework to allow the wireless devices to participate in the grid as resource provider by pooling the resources. To realize this, a multi-agent based replication framework is proposed which provides a comprehensive infrastructure for improving data availability with a small number of replicas. The functional components that are required to support mobility such as system state monitor, context analyzer, strategy manager, localization manager, replica manager and consistency manager are proposed. The architecture is enhanced with multi agents, to intelligently manage the replica and ensure consistency in the resource constrained environment.
3. In a replicated environment, it is mandatory to maintain the consistency among the multiple copies of the data distributed in multiple locations. In order to achieve that, a consistency management approach which uses the top down update propagation method is proposed. In this context, the
18
base station manages the update conflict by maintaining the mutual exclusion to access the data resources.
4. The ubiquitous distributed computing environment should maintain the stability of the environment and it should be continuous to operate despite the failures. A fault tolerance algorithm, which exhibits a well defined failure behavior, is proposed. This approach triggers in case of node failure in the environment.
5. The simulation environment is adopted to evaluate the system performance. Different scenarios have been designed and analyzed based on the signal-to-noise ratio, communication cost, data access cost, mobility information, load information and battery information.
1.5 ORGANIZATION OF THE THESIS
The chapters are organized as follows. The Chapter 2 presents the
existing research contributions, in terms of mobile grid middleware support
for various applications. Further research issues towards data replication with
mobility support and adaptive data management on mobile grid environment
are studied. The different middleware are compared for their service support
such as resource allocation support, dynamic environment support and mobile
device execution support. In addition various replication mechanisms for
mobile grid environment are analyzed with their advantages and limitations to
understand the strategies.
In Chapter 3, the extension at higher level data service layer is
explained. An optimized multi site replica placement algorithm is elaborated.
The enhanced method works with three tests namely, expansion test,
contraction test and switch test. The proposed placement mechanism is based
19
on the Signal – to – Noise ratio for mobile grids, which eliminates some of the
drawbacks of the existing systems. Multiple nodes for replica placement are
selected in a single iteration, catering the needs of sudden burst of requests.
The internal components for infrastructure level extension are
elaborated in Chapter 4. The implementation of the proposed replication
framework using agent technology to support the mobile grid is presented.
Two logical components are provided at the base station level and the mobile
host level. Design considerations of the proposed functional components that
are required to support mobility are discussed. To support replica
management effectively with this framework, three types of agents namely,
Base Station agent (BS agent), Node agent (N agent) and Update agent
(U agent) are realized. The experimental results demonstrate the benefits of
the usage of agents in the proposed framework.
In a dynamic replicated environment, maintaining consistency and
failure handling are important aspects. The Chapter 5 discusses the update
propagation mechanism to maintain consistency among the replicas. Different
agent states have been discussed and the interactions between these states are
elaborated. The proposed approach uses ROWA (Read One Write All) model.
The deferred update procedure is used to ensure replica consistency after each
write operation and resolving update conflicts. Along with the consistency
maintenance a fault tolerant algorithm is proposed and explained in Chapter 6.
To maintain the stability of the system in case of node failures, the algorithm
considers the mobility, battery and load information for selecting the node to
place the replica. Chapter 7 concludes by reviewing the present research and
discussing the future directions of the mobile grid computing.