A ﬂexible multi-dimensional QoS performance measure …hj/journals/85.pdf · 2006. 10. 6. ·...

Cluster Comput (2006) 9:281–296DOI 10.1007/s10586-006-9741-8

A flexible multi-dimensional QoS performance measureframework for distributed heterogeneous systemsJong-Kook Kim · Debra A. Hensgen · Taylor Kidd · Howard Jay Siegel ·David St. John · Cynthia Irvine · Tim Levin · N. Wayne Porter · Viktor K. Prasanna ·Richard F. Freund

Received: 1 September 2003 / Revised: 1 March 2004 / Accepted: 1 May 2004C© Springer Science + Business Media, LLC 2006

Abstract When users’ tasks in a distributed heterogeneouscomputing environment (e.g., cluster of heterogeneous com-puters) are allocated resources, the total demand placed onsome system resources by the tasks, for a given interval oftime, may exceed the availability of those resources. In such acase, some tasks may receive degraded service or be droppedfrom the system. One part of a measure to quantify the suc-cess of a resource management system (RMS) in such a dis-tributed environment is the collective value of the tasks com-pleted during an interval of time, as perceived by the user,

This research was supported by the DARPA/ITO Quorum Program, bythe DARPA/ISO BADD Program and the Office of Naval Researchunder ONR grant number N00014-97-1-0804, by the DARPA/ITOAICE program under contract numbers DABT63-99-C-0010 andDABT63-99-C-0012, and by the Colorado State University George T.Abell Endowment. Intel and Microsoft donated some of theequipment used in this research.

J.-K. Kim (�)Electrical and Computer Engineering School, Purdue University,W. Lafayette, IN 47907-1285, USAe-mail: [email protected]

D. A. Hensgen . T. Kidd . D. St. John . C. Irvine . N. W. PorterDepartment of Computer Science, Naval Postgraduate School,Monterey, CA 93943, USAe-mail: [email protected]

T. Kidde-mail: [email protected]

D. St. Johne-mail: [email protected]

C. Irvinee-mail: [email protected]

N. W. Portere-mail: [email protected]

H. J. SiegelDepartment of Electrical and Computer Engineering andDepartment of Computer Science, Colorado State University,Ft. Collins, CO 80523-1373, USAe-mail: [email protected]

T. LevinAnteon Corporation, 2600 Garden Rd., Ste. 220A, Monterey, CA93940, USAe-mail: [email protected]; [email protected]

V. K. PrasannaDepartment of Electrical Engineering-Systems, University ofSouthern California, Los Angeles, CA 90089, USAe-mail: [email protected]

R. F. FreundGridIQ, 13833 Adrian St. Poway, CA 92064-3453, USAe-mail: [email protected]

application, or policy maker. The Flexible Integrated SystemCapability (FISC) measure presented here is a measure forquantifying this collective value. The FISC measure is a flex-ible multi-dimensional measure such that any task attributecan be inserted and may include priorities, versions of a taskor data, deadlines, situational mode, security, application-and domain-specific QoS, and task dependencies. For an en-vironment where it is important to investigate how well datacommunication requests are satisfied, the data communica-tion request satisfied can be the basis of the FISC measureinstead of tasks completed. The motivation behind the FISCmeasure is to determine the performance of resource man-agement schemes if tasks have multiple attributes that needsto be satisfied. The goal of this measure is to compare theresults of different resource management heuristics that are

Springer

282 Cluster Comput (2006) 9:281–296

trying to achieve the same performance objective but withdifferent approaches.

Keywords Distributed computing . Heterogeneouscomputing . Performance metrics . Resource managementsystem . Scheduling

1 Introduction

In many distributed heterogeneous environments, the tasksthat are executing have different quality of service (QoS) re-quirements. These different QoS requirements impose dif-ferent machine and resource requirements. Furthermore,these tasks may require input data from a variety of dis-tributed sources. Mixed-machine heterogeneous computing(HC) environments (e.g., a cluster of heterogeneous comput-ers) provide a distributed suite of different types of machines,connected by diverse types of networks, to meet the variedcomputational and input requirements of widely varying taskmixtures (e.g., [6, 13, 29]). The goals of a resource manage-ment system (RMS) in an HC environment are to assign com-munication, computation, and other resources in an attemptto satisfy users’ requests, which may require different typesand levels of QoS. When users’ tasks in a distributed het-erogeneous computing environment are allocated resources,the total demand placed on some system resources by thetasks, for a given interval of time, may exceed the availabil-ity of those resources. In this case, some tasks may receivedegraded service, or be dropped from the system. In the evalu-ation of the performance of an RMS, it is essential to include:(1) how well it performed its goals or functions and (2) howwell it performed compared to other RMSs.

The goal of this research is to quantify the collective valueof the tasks completed during an interval of time, as perceivedby the user, application, or policy maker. Intuitively, if oneRMS performs better than another in a given environment,the better RMS would have a higher collective value. Thismeasure can be a part of a metric to assess the success of anRMS in a certain environment (other parts may include exe-cution time, ability to work with different operating systems,and user interfaces). This research describes attributes thatcan be included in such a performance measure, providesa way to combine them, and discusses other issues such asmultiple versions of tasks, relative importance of the differ-ent attributes, a generalization of the measure, and prioritylevels with classes.

The proposed approach is called the Flexible IntegratedSystem Capability (FISC) measure. The FISC measure isa flexible multi-dimensional measure such that any taskattribute can be inserted and may include priorities, ver-sions of a task or data, deadlines, situational mode, security,application- and domain-specific QoS, and task dependen-

cies. The FISC measure is a framework for quantifying thecollective value of a set of tasks completed during a giveninterval of time. It provides one way of combining the factorslisted above and any other attributes can be included in themeasure. For an environment where it is important to inves-tigate how well data communication requests are satisfied,the collective value of requests satisfied can be the basis ofthe overall performance measure instead of tasks completed.

The FISC measure by itself is not an RMS evaluator; otherfactors, such as RMS execution time, need to be included. TheFISC measure by itself is not a scheduling heuristic, whereparameters such as urgency (time to deadline) or matchingof task requirements to machine capabilities are usually in-cluded (e.g., [7, 8, 20, 28, ThB00, 42]). The FISC measurecan be used to determine the scheduling heuristic that re-sults in the highest value of the tasks completed, for a givenenvironment. It can also be used to compare, via experimen-tation or simulation, the effectiveness of changing the re-sources available in a given distributed system. Furthermore,the FISC measure can be incorporated as part of the objectivefunction in a system’s scheduling heuristics. The motivationbehind the FISC measure is to determine the performanceof resource management schemes if tasks have different at-tributes that needs to be satisfied. The goal of this measureis to compare the results of different resource managementheuristics that are trying to achieve the same performanceobjective but with different approaches.

There are varieties of performance measures that can beconsidered when analyzing systems (e.g., [40]). In some sit-uations, a combination of QoS (or performance) attributesmust be considered (e.g., [25]). The FISC measure will befocused on calculating the value of the tasks completed us-ing various QoS attributes. The measure presented here isone linear instantiation of the FISC measure. As will be dis-cussed, a non-linear measure is also possible.

This research is part of three related DARPA programs:Quorum [16], Battlefield Awareness and Data Dissemina-tion (BADD) [12, 35], and the Agile Information ControlEnvironment (AICE) [1]. In the Quorum environment, it issometimes the case that not all tasks requested can achievetheir most preferred QoS by their deadline. Thus, theremust be a performance measure that can determine a col-lective value of the set of tasks that were completed in agiven time interval by a particular resource managementstrategy.

One aspect of the BADD and AICE programs involvesdesigning a scheduling system for forwarding (staging) dataitems prior to their use as inputs to a local application in awide area network (WAN) distributed computing environ-ment. The BADD and AICE systems are similar to the Quo-rum environment in that, in some situations, not all data re-quests will be satisfied with their most preferred QoS by theirdeadline. Thus, the goal of the scheduler is to satisfy as many

Springer

Cluster Comput (2006) 9:281–296 283

requests as possible in a given interval of time, in a way thathas the greatest collective perceived value.

The performance measure described in this research canbe used to evaluate, for a given interval of time, the totalvalue of tasks completed in the Quorum program or the totalvalue of data received in the BADD and AICE programs. Inthis sense, the set of completed tasks for the Quorum pro-gram is equivalent to the set of satisfied data item requestsfor the BADD and AICE programs. A major difference be-tween Quorum and BADD/AICE is that in Quorum tasks areassigned to resources by the RMS. In the BADD/AICE pro-gram, task assignments are given and fixed a priori, but themovement of data to the tasks must be scheduled. Through-out the rest of this paper, a task will be used to represent (1)a user’s process execution in the Quorum context and (2) auser’s request for a data item in the BADD/AICE context.While this research is motivated by military applications, theFISC framework can be adapted for other environments, suchas clusters, intra-nets, and certain computational grids [15].

The test of the goodness of a performance measure foran HC RMS is if it allows a system administrator the flexi-bility to quantify how it is desired for the system to behave.Furthermore, the performance measure should provide a ve-hicle for comparing the results achieved by different RMSsgiven the same operational scenario, so that the best RMSfor a given environment can be selected. For a flexible mea-sure framework to perform well, it should be able to capturethe relative performance of different RMSs as perceived bythe system designer or the user. The FISC measure has thesequalities. Thus, the primary contribution of this work is pro-viding a way to measure the collective value accrued by anRMS using a broad range of attributes and to construct a flex-ible framework that can be extended for particular problemdomains.

The next section provides a brief overview of some of theliterature related to this work. In Section 3, several examplesof individual QoS attributes are presented. These attributesmay be considered when formulating the performance mea-sure to be used in building and assessing RMSs. Section 4shows how all the example QoS attributes can be combinedinto a single measure. In addition, this section presents abaseline for the FISC measure, discusses a generalized formof the performance measure, and possible variations of theFISC measure. Examples of where the FISC measure can beused are provided in Section 5. The last section gives a briefsummary of this research.

2 Related work

The FISC performance measure discussed here embodies pa-rameters that are considered important in scheduling tasksand communications in a distributed computing system.

There is much literature on parameters considered impor-tant when scheduling and mapping; in this section, someexamples of this literature are mentioned. This is followedby a discussion of examples of prior performance measurestudies that the FISC measure extends.

An optimistic priority-based concurrency control protocolthat schedules active transactions with a deadline in real-timedatabase systems is described in [19]. This protocol combinesforward and backward validation processes to control moreeffectively concurrent transactions with different priorities.The protocol is designed such that deadlines of higher prioritytransactions have a greater probability of being met than thoseof lower priority transactions are. While this is also the casefor Quorum and BADD/AICE, the FISC research presentedhere includes other attributes that are important in evaluatingthe overall value of the tasks completed.

In [26], laxity (deadline minus latency) of a task is usedfor the scheduling, adjustment, and dropping of messages.The “Least-Laxity-First (LLF)” scheme gives an improvedmissed deadline ratio, which is the rate of messages missingtheir deadlines. In [26], only the timing constraints are usedin the scheduling of messages and evaluation of the LLFscheme, but the research effort does not consider other QoSattributes used in heterogeneous distributed networks that theFISC measure includes.

The work presented in [31] describes an algorithm thatenables each node of a system to schedule the transmissionof messages generated locally while obeying their deadlineconstraint (messages get dropped if they cannot meet theirdeadline). This algorithm uses the actual priority and thedeadline of a message for the scheduling of the messages.The FISC measure allows more than one simple deadline andincludes other important QoS attributes in the calculation ofthe collective value of tasks completed, which can be usedas part of a scheduling process.

Data staging, an important data management problem fora distributed heterogeneous networking environment, is dis-cussed in [42]. The research in [42] assumed that each re-quested data item is associated with a specific deadline andpriority. The FISC research presented here generalizes theperformance measure used in [42] to include more types ofdeadlines and other QoS attributes.

From the works mentioned, parameters such as task pri-orities and deadlines appear to be important attributes formaking scheduling decisions. A measure of the overall valueaccrued of completed tasks is needed that can be used inan objective function to compare and analyze RMSs whileincorporating all the QoS parameters used. The works men-tioned above consider only a subset of the QoS parametersthat might be present in a distributed system. Other parame-ters (e.g., accuracy, precision, and security) that are QoS re-quirements and part of the users’ requests must be includedin the performance analysis. The QoS parameters that affect

Springer

284 Cluster Comput (2006) 9:281–296

the overall value of requests satisfied are discussed in ourresearch.

The FISC research on the performance measure presentedhere builds on and extends a body of earlier work in this field.Some examples are mentioned here.

The ERDoS project [9] describes an objective functionfor optimizing the effectiveness of its QoS scheduling mech-anisms in meeting clients’ needs. This function reflects thebenefit received by the user and a weight is assigned to eachuser application. An approach where requested QoS is takeninto account when scheduling computational resources in anetwork is presented in [30]. The model proposed a bene-fit function that uses application deadlines and applicationpriorities as metrics in maximizing the total benefit for theapplications. The incorporation of multiple versions of a task,in addition to priorities and deadlines, in the objective func-tion is described in [21]. The FISC measure serves a purposesimilar to the performance measures in [9, 21, 30]. However,the research presented here provides a more detailed descrip-tion of a measure using more parameters, so that it can beused to compare the performance of schedules in the Quorumand BADD/AICE environments. Furthermore, the QoS inputspecification for ERDoS [37] accounts for only two specificsecurity parameters (confidentiality and integrity), whereasthe security component of the FISC measure can describe anarbitrarily complex set of security features.

The resource allocation model for QoS management pro-posed in [34] indicates multiple dimensions of QoS and mul-tiple resources. In [34], some of the QoS attributes studied inthis research are mentioned, however there is no detailed de-scription or discussion of those attributes. The utility that isdescribed in [34] is same as the value accrued in a given timeinterval using a set of resources. The FISC research discussesQoS attributes in more detail and gives more QoS factors toconsider. Work done in [23, 24] presents specific usage ofthe model presented in [34]. These use only a subset of QoSattributes that the FISC research describes, indicating thatthe FISC measure would be a generalized version of whatthey have used as the utility function.

The work in [44] proposes a market mechanism that usesthe length of a job, the deadline of a job, the price of the jobdone, and the price of allocated time slots to find the optimalallocation of jobs onto resources. The FISC measure usespriorities and other QoS measures to calculate the collectivevalue of tasks that are completed.

The research in [32] describes a utility function that con-siders the throughput and the link congestion of the networkand extends their analysis to QoS sensitive requests. Theutility function described in [32] and the FISC measure bothseek to achieve the optimum value of the requests satisfied.In our paper, multiple QoS attributes (e.g., deadlines, secu-rity) are considered in detail while in [32], the QoS factor isrepresented by the link congestion.

In the model proposed by [11], a function that indicatesthe utility due to QoS attained when a certain bandwidth isallocated to the user and the “willingness-to-pay of the user”factor is used to calculate the net benefit. The FISC mea-sure and the utility function are similar in that they calcu-late the overall value achieved from the resources allocated.While [11] considers only bandwidth, the FISC research dis-cusses one way to combine different QoS attributes to resultin a framework for determining the total value accrued fromcompleting tasks in a given interval of time.

A security policy that allows a percentage of packets au-thenticated to vary with network load is described in [38].This type of policy can be accommodated with the vari-ant components included in the FISC security vector (seeSection 3.4). Related work on network security policy spec-ification languages can be found in [4, 5, 10, 36]. Whilethe FISC security vector contains a set of Boolean secu-rity policy statements, it does not specify a general-purposelanguage for these statements. A framework for quantify-ing the strength of a set of security mechanisms is describedin [43], where high-level static security properties can bedecomposed hierarchically. However, in [43] the approachcannot accommodate the measurement of how well an ex-ecuted task meets the security requirements of its environ-ment. Nor does [43] account for variant security policies ormechanisms.

3 Example QoS attributes

3.1 Priorities

Policy makers determine the number of priority levels avail-able within a system and assign some semantic meaning toeach priority level, such that the relative importance of eachlevel is qualitatively reflected (e.g., high, medium, and low).The policy makers may be the commanders in a military en-vironment or executives in a corporation. Policy makers mayassign different users, or classes of users, restricted rangesof priority levels that can be assigned to their tasks. Alter-natively, a task itself could have an immutable priority levelassigned to it by the policy makers. Each priority level willthen be given a weight that can be calculated by a priorityweight function, which is pre-determined by policy makers,described later in this section.

Priority levels with relative, quantitative weightingsshould be incorporated in scheduling systems so that a taskwith a higher importance will have a higher probability ofmeeting its QoS requirements. Application users and systembuilders often assign an arbitrary numbering scheme to pri-ority levels that does not meaningfully quantify the relativeimportance of one priority level to another. A more mean-ingful weight must be assigned to each priority level so that

Springer

Cluster Comput (2006) 9:281–296 285

the relative importance can be reflected in the performancemeasure.

The relative importance (weighting) of priority levels mayvary depending upon the situational mode. For example, theremay be military modes of peace and war. In peace mode, itmight be just as important to complete ten low priority leveltasks as to complete one high priority level task. However,in war mode, one high priority level task might be more im-portant than 1,000 medium priority level tasks. To indicatethis relative importance, for example, it may be necessary togive weightings of 10,000 to high priority level tasks and 10to medium priority level tasks. This dependency can be indi-cated in the performance measure by expressing the weightof all priority levels as a function of the situational mode.

It is assumed that the FISC measure is being used to com-pute the value of a subset of tasks successfully completed,during some time interval, from a set of t tasks that have beenrequested. Let the priority level (e.g., high, medium, low) oftask j(0 ≤ j < t) be p j , and let m be the situational mode.The priority weight function π (p j ) calculates the weight ofp j given the situational mode m. The weight assigned to apriority level may be considered to be the maximum valueof completing the corresponding task.

3.2 Versions

A task may exist in different versions, each with its ownresource requirements. Because of system load, it may notbe possible to complete the most desired version of a task.For example, a user requesting a map application may mostdesire a 24-bit color, three-dimensional topographical map.However, if this cannot be given to the user due to limitedresources, the user would rather have a black and white, two-dimensional map than nothing at all.

When a user’s first choice of a task version cannot becompleted, a method for choosing an alternative version isneeded. Having multiple versions of a task is related to theprecision and accuracy parameters discussed in [37], in thesense that each version of a task may have different accu-racy and precision, or to having different video image sizesconsidered in [45]. For each version of a given task, a worth(preference) relative to the other versions will be indicatedby the application developer, the user, or the policy makers.These worths may be a function of the situational mode. Inthe above example, the black and white version may only beworth 75% of the color version to the user. When selecting aversion of a task to execute, an RMS’s scheduling algorithmsmust consider this worth and the task’s resource requirementsas well as the availability of these resources. For example,one version may not be viable because its bandwidth require-ment is too high. Typically, a higher worth version has greaterresource requirements.

Table 1 (a) Worths that users indicate for eachversion of a task

task\version 0 1 2

0 1 1 81 25 35 402 .2 .3 .53 .1 .2 .7

Table 1 (b) The worth for each version of a taskis divided by the largest worth of that task to getthe normalized worth

task\version 0 1 2

0 .125 .125 11 .625 .875 12 .4 .6 13 .143 .286 1

To allow worths to be quantified in an arbitrary format,the worths assigned to different versions of a task must benormalized so that there is a consistent scheme for evaluat-ing worths across tasks and versions. For example, assumethat all factors except version worths are equal across a setof tasks. The user can specify any number for the worth of aversion as shown in Table 1(a). Therefore, without a normal-ization procedure, a task with the largest worth version mayalways be chosen for processing ahead of other tasks for nological reason. For example, if there is enough resources tocomplete one version of one of the tasks in Table 1(a), task1 will be chosen over all other tasks because version 2 oftask 1 has the highest worth among all the versions of all thetasks. In extreme cases, worths that are not normalized couldsupersede the impact of priority depending on how prioritiesand worths of version interact.

To avoid this type of anomalous behavior, worths are nor-malized as follows. Assume there are I j versions for a giventask j . Let vi j be the i-th (0 ≤ i < I j ) version of task j . Let wi j

(m) be the worth the user assigns to i-th version of task jgivenm, the situational mode. Example wi j (m) valuesare providedin Table 1(a). One approach to the normalization problemis to divide each indicated worth of a task version by thelargest worth for that task, resulting in the normalized worthas shown in Table 1(b). Thus, the normalized worth (ηi j ) ofwi j (m) is given by

ηi j = wi j (m)(max0 ≤ i < I j wi j (m)

) (1)

Figure 1is the graph representation of the normalizedworth shown in Table 1(b). Therefore, the version with thelargest worth of each task will have a normalized worth of 1

Springer

286 Cluster Comput (2006) 9:281–296

1

task 0

fra

ctio

n o

f va

lue 0.8

task 10.6

task 2

0.4 task 3

0.2

0

version 0 version 1 version 2

Fig. 1 Graph representation of the normalized worth of each versionof a task

and the rest of the versions will have normalized worths thatare relative to the version with the largest worth.

Another approach to the normalization would be to divideeach version’s worth by the total sum of the version worths ofthe task. This would not guarantee equal value for the mostpreferred version of each task. Furthermore, this approachmay allow a greedy user to obtain a higher value for a giventask’s preferred version over another task’s most preferredversion. For example, consider task 0 and task 1 in Table 1(a).If this alternative approach is used, the normalized worth fortask 0 would be 0.1, 0.1, and 0.8, while for task 1 it would be0.25, 0.35, and 0.4. This means that, even if task 0 and task1 have the same priority, the largest worth version of task 0is worth more than the largest worth version of task 1, whichshould not be the case.

3.3 Deadlines

Many tasks in typical heterogeneous computing environ-ments have deadlines associated with them. Frequently, dueto limited resources and the multiplicity of tasks sharing theseresources, not every task can be completed by its deadline.Three types of deadlines will be considered for the i-th ver-sion of task j : earliest, soft, and firm. These deadlines areillustrated by example in Fig. 2. The deadline attributes arerelated to the timeliness parameter in [37].

0.8

earliest soft firm

time

fra

ctio

n o

f va

lue

0.6

0.4

0.2

0

1

Fig. 2 The deadline graph shows the variation in the value of a taskwith various deadlines

The earliest deadline (edi j ) is the time when the task can

start. For example, assume that data is being sent to a systemis to process it. The task of sending data cannot start if thereceiving system is not ready and it will have no value.

The soft deadline (sdi j ) is the time by which a task must

complete to be of full value to the user [41]. If a task is com-pleted between the earliest deadline and the soft deadline,then the task will have its maximum value.

A task that is completed after its firm deadline (f di j ) will

have 0 value, because the task will be of no use after thatdeadline [22, 41]. For example, if a task that shows a mapof an area completes after a mission is finished, then it willhave no value. If a task completes between its soft and firmdeadline, then it will have some fraction of its maximumpossible value. For each task, the fraction of total value foreach point between the soft and firm deadlines, and the timebetween the soft and the firm deadlines, may be a functionof the situational mode. For example, during war mode, thesoft and firm deadlines may be identical.

Let τ i j be the time that the i-th version of task j actuallycompletes. The deadline function δi j assigns a fraction of themaximum value of the i-th version of task j based on m, τ i j ,ed

i j , sdi j , and f d

i j , where 0 ≤ δi j ≤ 1. The deadlines edi j , sd

i j ,and f d

i j may be the same for all versions of a certain task.A characteristic function δ′

i j is used to represent whether aversion completes with a δi j > 0: δi j

′ = 1 if δi j > 0, and δi j′

= 0 if δi j = 0. If no version of task j is completed, δi j = 0and δi j

′ = 0 for all versions of the task.

3.4 Security

User and task security requirements are met by “securityservices.” Overall network security can be viewed as amulti-dimensional space of security services. This multi-dimensional space can be represented with a vector (S) ofsecurity components, where the functional requirement foreach component is specified by a Boolean statement for eachgiven situational mode. Both resources and tasks may havemultiple security components [17, LeI99b].

The instantiation of a network task either meets, or doesnot meet, each component’s requirement. For example, con-sider the i-th version of task j . Let Ri j be a set of resourcesutilized by vi j and let Si j be a sub-vector of vector S. A com-ponent κ in S is in Si j if and only if κ depends on vi j or onan element of Ri j , and is denoted Si j .κ . The characteristicfunction σ i j

′ is used to represent required security attributes.If minimum security requirements are not all met, there is novalue accrued for executing vi j and σ i j

′ = 0. If the instantiatedBoolean value of all κ in Si j is true, then σ i j

′ = 1.Additionally, some security components of a task can be

variant in that they allow a range of behavior with respectto a requirement (e.g., length of cryptography key may vary

Springer

Cluster Comput (2006) 9:281–296 287

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

50% 60% 70% 80% 90%

authenticated packets

fract

ion s

atis

fied

Fig. 3 Graph representation of fraction satisfied of authenticated pack-ets that can range between 50% authenticated and 90% authenticated,incremented by 10%

between 40 and 256). For variant components, the user mayrequest a particular value or permit a choice from a compo-nent’s defined range. The RMS must select a specific valuewithin the user’s range, while considering resource avail-ability, for the completed task to have a non-zero value. Themeasure will give only partial credit for a task completedwith less than the most secure value in the defined range.

The desire to provide adaptable security motivates the in-clusion of variant security components in the system [18].Thus, security affects the performance measure when com-ponents are variant. For example, assume the percentage ofauthenticated packets can range between 50% and 90% inincrements of 10%. The increment quantizes the range. Let[Si j .κ] be the number of quanta in Si j .κ (in the above case,this is five) and gi j .κ be the fraction of κ in Si j satisfied. If atask achieves the third quantum (70%), then gi j .κ is 3/[Si j .κ]= 3/5 = 0.6. This example is represented as a graph inFig. 3.

Suppose n is the number of security components in Si j . Toquantify the effectiveness of the RMS in providing variantsecurity, let security factor σ i j be the sum of all gi j .κ dividedby n as shown in Eq. (2).

σi j =(∑

κ∈Si j

gi j .c

n

)(2)

The above is just one possible way to combine the valuesof these security components. For example, the gi j .κ values inEq. (2) can have relative weightings for a given m. Thus, if themilitary situation changes from peace to war, authenticationmay be considered relatively more important and might begiven a higher relative weighting.

The overall security measure is σ i j× σ i j′, where 0 ≤ σ i j×

σ i j′ ≤ 1. It indicates how the value of a task may be degraded

due to lack of its most desirable security services or a lackof meeting its minimum requirements.

3.5 Application specific QoS

The model for application specific QoS is analogous to thesecurity model in Section 3.4. There is a multi-dimensionalspace of application QoS attributes (e.g., jitter level, framerate, bit error rate). For example, in [45], the frame rate,image size, number of trackable objects, and buffer sizes arethe application specific QoS attributes of the video streamingand tracking service (application). The overall applicationspecific QoS measure is αi j × αi j

′. where αi j and αi j′ are

analogous to σ i j and σ i j ′ respectively. It indicates how thevalue of a task may be degraded due to lack of its mostdesirable application specific services or a lack of meetingits minimum requirements.

3.6 Associates

There are many possible types of inter-task dependencies,e.g., for the MSHN environment, consider a task whose onlyfunction is to generate data for other tasks (descendants).There may be an inter-related set of such tasks (Fig. 4). Ifthere is a descendant along a dependency path of tasks thatgenerates an output for a user (e.g., tasks 4 and 6 in Fig.4) and if this descendant completes its execution, then thetasks that did nothing more than generate data for this par-ticular task will have value, otherwise they will not. This isbecause the end user that submitted a task for completion

task 2:generates data for task 4

task 4:(a) outputs target identification

(b) generates data for task 5



task 6:outputs launch coordinates


Fig. 4 An example set of tasks that have dependency. Tasks 1, 2, 3,and 5 are tasks that only generate input data for other tasks. Tasks 4 and6 generates output for the user

Springer

288 Cluster Comput (2006) 9:281–296

data request 1:required data for

application 1

data request 2:required data for

application 1

data reqeust 3:required data for

application 1

application 1:outputs data for other applications

Fig. 5 An example set of datarequests that have dependency.Data requests 1, 2, and 3 arerequired input data forapplication 1

will acknowledge the task to be finished only when the ac-tual results can be determined. Thus, for a task to have value,either it or one of its descendants must generate an output fora user.

The first task in a dependency path that does more thangenerate data for a subsequent task will be known as a re-quired associate of its predecessors. The variable ρi j gill rep-resent whether a required associate of a given task completes;i.e., if at least one required associate of a given task com-pletes, then ρi j = 1, otherwise ρi j = 0.

For the BADD/AICE environment, consider a data re-quest whose only function is to be used by an application togenerate data for other applications. There may be multipledata requests for a given application. Unless all such datarequests are satisfied, the application cannot execute and thevalue of any data request and the application that are satis-fied would be zero (i.e., ρi j = 0 for all data requests). in Fig.5, data requests 1, 2, and 3 are required for application 1 toexecute. Therefore, if any one of these data requests is notavailable (if only a subset of the data requests arrive by thefirm deadline), there is no value for any of the satisfied datarequests.

4 Performance measure

4.1 FISC measure

A meaningful way to combine the attributes discussed in theprevious section is proposed here. The measure presentedhere is only one instantiation of the FISC measure. In general,it is a difficult problem to determine whether a distributed sys-tem has delivered “good” service to a mixture of applications.For example, some applications may be compute-intensiveand others interactive, perhaps having stringent security re-quirements. In this research, the collective value of servicesachieved is used for determining how “good” a resource al-location is. A meaningful way to combine the performanceattributes previously discussed is proposed in this subsection.

Consider a set of tasks with different priorities, differentdeadlines, multiple versions, different security and applica-tion specific QoS requirements, and dependencies. The valueof a task must be calculated based on all of the attributes.

The maximum value of task j is π (p j ); therefore, otherattributes must be combined in a way that the maximum valueof a task never exceed its priority weighting. In addition, allperformance attributes must be included in the calculationof the value of a task. The factors that describe whether aminimum requirement of an attribute is met must be included.These are ρi j , δi j

′, σ i j′, and αi j

′, and each function is equalto 0 if minimum requirement is not met or 1 if the minimumrequirement is met. The intuition is that if a task does not meetits minimum requirement, the task is of no value to the user.Also, the RMS must consider any variations in the value ofa task’s completion that is a result of when a task completes(δi j ), which version was used for the task (ηi j ), and receivingdifferent degrees of required and desired security (σ i j ), andother application- and domain-specific QoS services (αi j ).Equation 3 is one way to combine all performance attributes.The “max” function is used to indicate whether a version of atask has completed. Note that if a version of a task completesthen all other versions are not completed and their calculatedvalues are zero. To make the value of “max” less than orequal to one, the value of δi j , ηi j , σ i j , and αi j is averaged asshown in Eq. (3). This also gives the variable component ofeach attribute equal weighting.

t−1∑j = 0

π (p j )

(max

0≤i<I j

ρi j ×δi j′×σi j

′×αi j′× ηi j +δi j +σi j +αi j

4

)

(3)

This version of FISC will be called the averaged FISC.This method is similar in idea to the benefit function de-scribed in [17, 18]. The FISC measure can be used to com-pare the performance of different RMSs running in the sameenvironment.

There can be coefficients for each attribute mentioned inthis research to indicate the relative importance of one at-tribute to another. The ability to “tune” this relative impor-tance of attributes is an important capability of the FISCthat contributes to its flexibility as a performance measureframework. The way in which the values given to these co-efficients will be determined may depend on the environ-ment. For example, in a shipboard computing environment[3], the values are the result of a carefully determined militarypolicy.

Springer

Cluster Comput (2006) 9:281–296 289

Let cη, cδ , cσ , and cα be the coefficients of η (versionused), δ (deadline met), σ (security services satisfied), and α

(application specific QoS satisfied) factors respectively. Toincorporate coefficients into the FISC measure, another ver-sion of the FISC is needed. For the coefficients to not affectthe overall priority weighting and to indicate the relative im-portance among the attributes, the measure will be dividedby the sum of the coefficients. This will be the averaged FISCwith coefficients. By dividing the measure by the sum of thecoefficients, the part of the FISC measure without the prior-ity function will be one when all attributes are 100 percentsatisfied and less than one when a certain task gets degradedQoS as shown in Eq. (4).

t−1∑j=0

π (p j )

(max

0≤i<I j

ρi j × δi j′ × σi j

′ × αi j′

×[cη × ηi j + cδ × δi j + cσ × σi j + cα × αi j

]cη + cδ + cσ + cα

)(4)

In addition to an optimization criterion such as the FISCmeasure, constraints are required to define any optimizationproblem. There is a limited amount of resources so thereis a constraint on resource. Therefore, in any time instant,the total amount of resource used of a particular resourcecannot exceed the total available resource at that time instant.Assume that there are number of resources in the system.Let Rr j (�) be the amount of resource r (0 ≤ r < ) used fortask j(0 ≤ j < t) during the time interval � and let Ur (�) bethe total resource r that is available during time interval �.The sum of all Rr j (�) cannot exceed Ur (�) as explained inEq. (5).

t−1∑j=0

Rr j (�) ≤ Ur (�) for resources r = 0 . . . − 1 (5)

It is possible that multiple versions of the same task ormultiple copies of the same version can be attempted for faulttolerance or for maximizing the speed of the process. Thenthe FISC equation can be extended to include vi jγ , where iis the version, j is the task, and γ is the copy number (0 ≤ γ

< j ). The FISC measure can be extended from Eqs. (4) to(6) (averaged FISC with coefficient and multiple copies ofversions). The goal would be to maximize the FISC measureover all relevant i and γ for j . For each copy of a version therecould be different deadlines, security services, or applicationspecific QoS. Therefore, these factors also need to considerthe copy γ of the version. Because dependency is only amongtasks not versions of a particular task, the dependency factor

does not need to consider the copy number.

t−1∑j=0

π (p j )

⎛⎜⎝ max

0≤i<I j

0≤γ< j

ρi j × δi jγ′ × σi jγ

′ × αi jγ′

×[cη × ηi jγ + cδ × δi jγ + cσ × σi jγ + cα × αi jγ


) (6)

If one RMS performs well in one distributed environmentand another RMS gets good results in another environment,then a direct comparison of these RMSs would not makesense. One method to compare different RMSs operating ondifferent distributed systems, is to normalize the FISC mea-sure by a baseline that depends on the tasks and underlyingdistributed system. The baseline can be calculated by usingthe same number of tasks with same attribute requirementsfor each environment and these baselines may be different.Because the environments are different, how well a RMS per-formed would equal to how much better it performed thanthe baseline of its environment. If the RMS cannot performmuch better than this baseline, then a naive algorithm for re-source assignment would perform almost as well as the RMS.The baseline builds upon and extends the example given by[42]. The algorithm used to compute the baseline uses theconcept of perfect completion. A task is said to achieve per-fect completion if there exist available resources, to whichit can be assigned, that would allow it to complete with ηi j

= δi j = σ i j= αi j = 100% and ρi j = 1. This means that ingiven situations (i.e., resource availability, situational mode)tasks with the most preferred version, all security servicesand application specific QoS are satisfied 100%, a requiredassociate that completes, and completion time before the softdeadline are considered.

A simple algorithm, which assumes knowledge of the ex-pected resources needed by a task to complete, can be used toobtain a baseline. For the results of the obtained baseline tobe reproducible within a certain tolerance, an ordering of thetasks is needed. The algorithm to obtain a baseline is shownin Fig. 6. First, it assigns an ordering to the tasks accord-ing to their priorities (highest first), deadlines (soonest first),and expected execution times (shortest first) where the abovecriteria are considered in the aforementioned order. For thetasks with the same priority level, the deadline would be usedas a tiebreaker. If tasks have same priority level and dead-line, the expected execution time would serve as a tiebreaker.Only if tasks have the same priority, deadline, and expectedexecution time would the ordering be random. Alternatively,additional characteristics of the task could be used for finerordering. In other problem domains, other parameters couldbe more appropriate for ordering the tasks.

After the ordering, the algorithm determines whether thefirst task (according to the ordering) can be expected to

Springer

290 Cluster Comput (2006) 9:281–296

Fig. 6 The baseline algorithm

achieve perfect completion using the available resources.If so, it computes the expected availability of resources af-ter that task has completed, under the assumption that thetask uses the first such available resources. It also adds theweighted priority of this task to the baseline, which was ini-tialized to 0. If a task cannot achieve perfect completion,nothing is added to the baseline and the task is not con-sidered again. The same process is repeated for each task,considering them according to the ordering.

When the FISC measure is normalized by a baseline, theresulting function is called the FISC ratio. The averaged FISCratio is:

t−1∑j=0

π (p j )

(max

0≤i<I j 0 ≤ γ < j

ρi j × δi jγ′ × σi jγ

′ × αi jγ′ ×

[cη × ηi jγ + cδ × δi jγ + cσ × σi jγ + cα × αi jγ


)(7)

baseline

The baseline scheme presented here is just one way abaseline can be calculated for doing a normalization that canbe used to try to compare an RMS used in one environmentwith an RMS used in another environment. Other schemescould be used as the baseline, as long as the same scheme isused for normalization in both environments.

Another version of FISC that will be called the multipliedFISC is presented. Similar to the averaged FISC, this versionmust consider all attributes and make sure that the “max”never exceeds 1. A way to accomplish this is to multiply allcomponents of the measure (ηi j , ρi j , δi j , δi j

′, σ i j ,σ i j′, αi j ,

and αi j ′) as shown in Eq. (8)

t−1∑j=0

π (p j )(

max0≤i<I j

×(

ηi j ×ρi j ×δi j ×δi j′ × σi j ×σi j

′ ×αi j × αi j′))

.

(8)

Both versions of the FISC measure (averaged and multi-plied) mentioned in this subsection may be used to calculate

the collective value of the tasks completed. The averagedFISC (Eq. (3)) captures the intuition that when calculatingthe value of a task should be larger than or equal to the per-centage satisfied of the least satisfied service attribute mul-tiplied by the priority weighting of the task. For example,there is a task completed with a version that is worth 50%,the task was completed before the soft deadline (100%), thetask received variable security services (40%), and assumeπ (p j ) is one. Assume all minimum requirements are met andall other variable services do not exist. The completed task’svalue would be 0.63 by the averaged FISC (Eq. (3)) and 0.2by the multiplied FISC (Eq. (8)). When using the multipliedFISC, the value of the task has decreased below the securityservices satisfied (40%). When all of a task’s services are atleast 40% satisfied, the value of the task should be larger thanor equal to 40% of the maximum value a task.

4.2 Generalization of FISC measure

The previous subsection describes a particular example ofthe FISC measure. It can be generalized such that the nu-merator is any function of π (p j , m), ηi j , ρi j , δ i j (τ i j , m),σ i j , and αi j (or other factors), and each of these primary fac-tors can be any function of secondary factors (e.g., primary

factor σ i j includes an average of gi j .c secondary factors in thesecurity context described in Section 3.4). The generalizationextends to other factors that may be important and includedin the FISC measure. For example, if a task has bandwidthrequirements, the bandwidth attribute can be added to deter-mine if a resource allocation has provided the needed band-width resources for the task. Let Py be a primary factor wherethere can be u number of primary factors (0 ≤ y < u) andse be a secondary factor where there can be vy number ofsecondary factors (0 ≤ e < vy). The generalization of FISCmeasure can be represented as

FISC = f (P0, P1, . . . , Put−1)/baseline and (9)

Py = fy(s0, s1, . . . , svy−1) , (10)

where each se is a secondary factor for Py . Linear or nonlinearweightings (coefficients) of each factor, depending on theimportance of the factor considered in a given environment,

Springer

Cluster Comput (2006) 9:281–296 291

may be included in all the functions of primary and secondaryfactors.

The baseline algorithm described in Section 4.1 is onemethod of normalizing the numerator of the FISC measure.Other methods for normalizing could be incorporated to com-pare the performance of different RMSs in a given environ-ment.

4.3 FISC measure with priority levels within classes

In some environments, it may be the case that all higher pri-ority level tasks must be attempted for execution and comple-tion first, before any of the lower priority level tasks can beconsidered. For example, if there are high, medium, and lowpriority level tasks, the high priority tasks will be consideredfirst and if there are no more high priority tasks, then mediumand low priority level tasks will be considered for execution.In this scheme, a higher priority level task is worth morethan any number of lower priority level tasks (i.e., highestpriority level task is worth an infinite number of lower pri-ority level tasks). To represent this, classes of priority levelswill be needed. If all the tasks of a certain class have beenconsidered for the calculation of the FISC number, then thetasks of the next class will be considered for calculation.

Each task will have a priority level, and priority levels willhave relative weightings. Tasks will not have classes but thepriority level that the task was assigned with may correspondto a class predetermined by the system administrator or thepolicy maker. Each class will consist of one or more prioritylevels and there can be several classes where the number ofpriority levels assigned to a class can be different. For anynumber of classes, the FISC number, which is calculated us-ing the FISC measure, of class k is more important than theFISC number of class k+1 where k is an arbitrary number.Therefore, when comparing the accrued value of one sched-uler to another, the FISC number of the highest class willbe compared first and if they have equal FISC numbers, theFISC number of the next highest class will be compared.

As shown in Fig. 7, there could be L number of prioritylevels and C number of classes. Priority 1 is more importantthan priority 2 and class 1 is more important than class 2. Anynumber of priority levels can be in one class. After calculatingthe FISC number for class 1 (priority 1 tasks) of schedulers,compare the number and if the number is same, calculate thenext class of tasks.

5 Examples of where FISC can be used

5.1 EADSIM

As an example of how the FISC measure might be applied inpractice, consider the following scenario. The Joint Force Air

priority 1

priority 2

priority 3

priority 4

priority P

class 1

class 2

class C

Fig. 7 Example of priority levels within classes

Component Commander (JFACC) staff are preparing an AirTasking Order (ATO). As the ATO develops, one tool avail-able to the JFACC staff for its evaluation is the Extended AirDefense Simulation (EADSIM) system from the US ArmySpace and Missile Defense Command. EADSIM is a war-fare modeling application offering great flexibility in theareas modeled, the capabilities of the platforms simulated,and the method of simulation (deterministic or stochastic)[33].

EADSIM utilizes a wide range of computing resources,depending on the features enabled. For example, the stochas-tic mode may use approximately 20 times the computing re-sources as the deterministic mode (based on the number ofruns required to obtain a statistically significant number ofsamples). Of course, results obtained in stochastic mode arelikely to be more reliable.

The JFACC planners select two versions of EADSIM,the stochastic mode and the deterministic mode, and submitthem, with different preferences, to their RMS for execu-tion. Because this information is urgently needed for combatmission planning, the priority of this request is seven on ascale of ten (ten being highest). The deadline is firm, withthe simulation results required within an hour. If receivedwithin an hour the results will achieve some fraction of theirmaximum worth. After that time, they receive 0 value. Thestochastic version is preferred because it will produce higherconfidence results, but the deterministic simulation may alsobe useful. The stochastic version is assigned a preference ofeight, on a scale of ten, while the deterministic version isassigned a preference of five. Security level in this case isbinary. The information must be sent over a secure link. Ifit is, a version is assigned a security value of 1, if not, it isassigned a security value of 0. If only one of the two versionscan be completed, and these are the only ones to choose from,then the stochastic version will be completed because it willgive a higher value than the deterministic version. This is a

Springer

292 Cluster Comput (2006) 9:281–296

simple case; usually other factors have to be considered (e.g.,expected execution time) when scheduling tasks.

An RMS such as MSHN would evaluate the expectedresource requirements of each version as well as the abil-ity to complete each version based on the current resourceavailability. Using this information, the RMS could make awise decision by maximizing an objective function wherethe FISC measure would be a major component. While thisexample is from a military environment, the FISC measurecan be adapted for other environments as well.

5.2 QuO middleware

The Quality Objects (QuO) middleware is a set of exten-sions to standard distributed object computing middlewarethat is used to control and adapt quality of service in a num-ber of distributed application environments, from wide-areato embedded distributed applications [27]. Several exam-ples of QoS attributes that are used in real application aredescribed.

In the first example of data dissemination in a wide-area network, in cases where the network is the source ofa slowdown in the system and bandwidth reservation is notsuccessful, the QuO middleware triggers application adap-tation. The application trades off its data quantity or dataquality requirements for its timing requirement, by request-ing smaller images or lower resolution images to reduce theamount of network congestion. If the CPU is the source ofslowdown, the application requests unprocessed images re-ducing the load on the CPU and enabling the images to bereceived faster but with reduced quality or analysis of theimages.

The second example is the dynamic mission planning inan avionics platform. In this example QuO is used to managethe trade offs of timeliness versus image quality by imagecompression, image tiling, processor resource management,and network resource management.

In the third example, Unmanned Air Vehicle (UAV) data isdisseminated throughout the ship. While the data is sent out,system condition objects monitor the frame rate and the hostload on the video display hosts. As the frame rate declinesand/or the host load exceeds a threshold, they cause regiontransitions, which trigger the video distribution process todrop frames going to the display on the overloaded host andthe video display on the overloaded host is told to reduce itsdisplay frame rate to the rate at which frames are being sentit.

The different sizes and resolutions of images and unpro-cessed images that are discussed in the first example are thedifferent versions and the timing requirement is the deadlinedescribed in the FISC research. While the QuO system moni-tors system loads, make trade-offs of timeliness versus image

quality, and degrade QoS attributes while dropping framesand reduce display frame rate, it needs a performance mea-sure that can estimate how well these activities were donein terms of the overall performance. The FISC measure canprovide a framework for estimating the overall performanceof such a system.

6 Summary

In some environments, distributed heterogeneous computingenvironment may be over-subscribed, where the total de-mand placed on system resources by the tasks, for a giveninterval of time, exceeds the resources available. In such en-vironments, users’ tasks are allocated resources to simulta-neously satisfy, to varying degrees, the tasks’ different, andpossibly conflicting, quality of service (QoS) requirements.When tasks are allocated resources, some tasks will receivedegraded QoS or no service at all. The FISC measure pro-vides a way to quantify the value of the performance receivedby a set of applications in a distributed system. By comparingthe value calculated by the FISC measure, the effectivenessof the mapping of a collection of requests to resources doneby a scheduler can be evaluated in terms of value as per-ceived by the user, policy maker, administrator, or system.In addition, it may be used in a simulation mode to analyzethe impact of proposed changes to the distributed system.The FISC measure is a flexible multi-dimensional measuresuch that any task attribute can be inserted and may includepriorities, versions of a task or data, deadlines, situationalmode, security, application- and domain-specific QoS, andtask dependencies. A generalization of the FISC measure isdiscussed. The motivation behind the FISC measure is to de-termine the performance of resource management schemes iftasks have different attributes that needs to be satisfied. Thegoal of this measure is to compare the results of different re-source management heuristics that are trying to achieve thesame performance objective but with different approaches.Example use of the FISC measure is described indicating thatthe FISC measure can be a part of scheduling or decision ofscheduler.

The FISC performance measure presented here will helpthe distributed computing community in the implementationof resource management systems and the analysis and com-parison of such systems. Furthermore, the FISC measure maybe used as a critical part of a scheduling heuristic’s objectivefunction.

Additional issues that may be considered in future re-search include, weighting the relative importance of the π

(priority level weighting), η version used), ρ (required asso-ciate present or not), δ (deadline met), σ (security servicessatisfied), and α (application specific QoS satisfied) factors;

Springer

Cluster Comput (2006) 9:281–296 293

using a non-linear combination of task values to compute theoverall measure; the use of negative fractions in the dead-line function in case of catastrophic results from a misseddeadline could be incorporated; how to incorporate FISCmeasure in a scheduling heuristic; investigating other factorsthat are important in calculating the value of task mapped tothe user; and investigating variations in the factors alreadyconsidered.

Acknowledgments The authors thank, B. Beaton, G. Koob, J.Rockmore, M. Jurczyk, I. Wang, S. Jones, J. Kresho, E. K. P. Chong,R. Eigenmann, N. Rowe, C. Kesselman, N. Beck, T. Braun, S. Ali, S.Chatterjea, A. Naik, and P. Dharwasdkar for their valuable commentsand suggestions. A preliminary version of portions of the material waspresented at the 10th Heterogeneous Computing Workshop.

References

1. AICE, Agile Information Control Environment Proposers In-formation Package, BAA 98–26, September 1998, http://web-ext2.darpa.mil/iso/aice/aicepip.htm

2. S. Ali, J.-K. Kim, Y. Yu, S.B. Gundala, S. Gertphol. H.J. Siegel, A.A. Maciejewski, and V. Prasanna, Utilization-based techniques forstatically mapping heterogeneous applications onto hiper-d hetero-geneous computing system. Parallel and Distributed ComputingPractices, accepted to appear in 2004.

3. S. Ali, A.A. Maciejewski, H.J. Siegel, and J.-K. Kim, Measuring therobustness of a resource allocation. IEEE Transactions on Paralleland Distributed Systems, accepted to appear in 2004.

4. L. Badger, D.F. Stern, D.L. Sherman, K.M. Walker, and S.A.Haghighat, Practical domain and type enforcement for Unix. 1995IEEE Symposium on Security and Privacy (May 1995) pp. 66–77.

5. M. Blaze, J. Feigenbaum, and J. Lacy, Decentralized trust manage-ment. 1996 IEEE Symposium on Security and Privacy (May 1996)pp. 164–173.

6. T.D. Braun, H.J. Siegel, N. Beck, L.L. Boloni, M. Maheswaran, A.I.Reuther, J.P. Robertson, M.D. Theys, and B. Yao, A taxonomy fordescribing matching and scheduling heuristics for mixed-machineheterogeneous computing systems. IEEE Workshop on Advancesin Parallel and Distributed Systems (October 1998) (included in theproceedings of the 17th IEEE Symposium on Reliable DistributedSystems (October 1998) pp. 330–335.

7. T.D. Braun, H.J. Siegel, N. Beck, L. Boloni, R.F. Freund, D.Hensgen, M. Maheswaran, A.I. Reuther, J.P. Robertson, M.D.Theys, and Bin Yao, A comparison of eleven static heuristics formapping a class of independent tasks onto heterogeneous dis-tributed computing systems. Journal of Parallel and DistributedComputing 61(6), (June 2001) pp. 810–837.

8. T.D. Braun, H.J. Siegel, and A.A. Maciejewski, Static mappingheuristics for task with dependencies, priorities, deadlines, andmultiple versions in heterogeneous environments, in the CD-ROMproceedings of the 16th International Parallel and Distributed Pro-cessing Symposium (IPDPS 2002), (April 2002).

9. S. Chatterjee, B. Sabata, and J. Sydir, ERDoS QoS Architecture,Technical Report, SRI, Menlo Park, CA, ITAD-1667-TR-98–075(May 1998).

10. M. Condell, C. Lynn, and J. Zao, Security policy specificationlanguage, INTERNET-DRAFT, Network Working Group, Oc-tober 1998, ftp://ftp.ietf.org/internet-drafts/draft-ietf-ipsec-spsl-00.txt

11. C. Coutcoubetis, G.D. Stamoulis, C. Manolakis, and F.P.Kelly, An intelligent agent for optimizing QoS-for-money inpriced ABR connections. Telecommunications Systems, Spe-cial Issue on Network Economics, to appear (preprint athttp://www.statslab.cam.ac.uk/∼frank/PAPERS/iaabr.html).

12. DARPA, Battlefield Awareness and Data Dissemination (April1999) www.darpa.mil/iso/ badd/

13. M.M. Eshaghian (ed.), Heterogeneous Computing (ArTech House,Norwood, MA, 1996).

14. D. Ferrari, Computer Systems Performance Evaluation (Prentice-Hall, Englewood Cliffs, NJ, 1978).

15. I. Foster and C. Kesselman (eds.), The Grid: Blueprint for a NewComputing Infrastructure (Morgan Kaufmann, San Francisco, CA,1999).

16. D.A. Hensgen, T. Kidd, D. St. John, M.C. Schnaidt, H.J. Siegel,T.D. Braun, M. Maheswaran, S. Ali, J. Kim, C. Irvine, T. Levin,R.F. Freund, M. Kussow, M. Godfrey A. Duman, P. Carff, S. Kidd,V. Prasanna, P. Bhat, and A. Alhusaini, An overview of MSHN:The Management System for Heterogeneous Networks. 8th Hetero-geneous Computing Workshop (HCW ’99) (April 1999) pp. 184–198.

17. C. Irvine and T. Levin, Toward quality of security service in aresource management system benefit function. 9th IEEE Hetero-geneous Computing Workshop (HCW 2000) (May 2000) pp. 133–139.

18. C.E. Irvine and T. Levin, Toward a taxonomy and costing methodfor security services. 15th Annual Computer Security ApplicationsConference (December 2000) pp 183–188.

19. J.H. Kim and H.S. Shin, Optimistic priority-based concurrency con-trol protocol for firm real-time database systems. Information &Software Technology 36(12) (December 1994) 707–715.

20. J.-K. Kim, S. Shivle, H.J. Siegel, A.A. Maciejewski, T.D. Braun,M. Schneider, S. Tideman, R. Chitta, R.B. Dilmaghani, R. Joshi,A. Kaul, A. Sharma, S. Sripada, P. Vangari, and S.S. Yellampalli,Dynamic mapping in a heterogeneous environment with tasks hav-ing priorities and multiple deadlines. 12th Heterogeneous Comput-ing Workshop (in the proceedings of the 17th International Paral-lel and Distributed Processing Symposium (IPDPS 2003)) (April2003).

21. J.P. Kresho, Quality Network Load Information Improves Perfor-mance of Adaptive Applications, Master’s Thesis, Department ofComputer Science, Naval Postgraduate School, Monterey, CA,September 1997 (D.A. Hensgen, advisor) p. 164.

22. C.G. Lee, Y.K. Kim, S.H. Son, S.L. Min, and C.S. Kim, Efficientlysupporting hard/soft deadline transactions in real-time database sys-tems. 3rd International Workshop on Real-Time Computing Systemsand Applications (October/November 1996) pp. 74–80.

23. C. Lee, J. Lehoczky, R. Rajkumar, and D.P. Siewiorek, On qualityof service optimization with discrete QoS options. 5th IEEE Real-Time Technology and Applications Symposium (June 1999) pp. 276–286.

24. C. Lee, J. Lehoczky, D. Siewiorek, R. Rajkumar, and J. Hansen,A scalable solution to the multi-resource QoS problem. 20thIEEE Real-Time Systems Symposium (December 1999) pp. 315–326.

25. K.J. Liszka, J.K. Antonio, and H.J. Siegel, Problems with com-paring interconnection networks: Is an alligator better than an ar-madillo? IEEE Concurrency 5(4) (October/December 1997) PP.18–28.

26. J.P. Li and M.W. Mutka, Priority based real-time communicationfor large scale wormhole networks. 8th International Parallel Pro-cessing Symposium (April 1994) pp. 433–438.

27. J. Loyall, R. Schantz, J. Sinky, P. Pal, R. Shapiro, C. Rodrigues, M.Atighetchi, D. Karr, J. M. Gossett, and C. D. Gill, Comparing andcontrasting adaptive middleware support in wide-area and embed-ded distributed object applications. 21st International Conference

Springer

294 Cluster Comput (2006) 9:281–296

on Distributed Computing Systems (ICDCS 2001) (April 2001) pp.625–634.

28. M. Maheswaran, S. Ali, H.J. Siegel, D. Hensgen, and R.F.Freund, Dynamic mapping of a class of independent tasks ontoheterogeneous computing systems. Journal of Parallel andDistributed Computing. Special Issue on Software Supportfor Distributed Computing 59(2) (November 1999) 107–121

29. M. Maheswaran, T.D. Braun, and H.J. Siegel, Heterogeneous dis-tributed computing. In Encyclopedia of Electrical and ElectronicsEngineering J.G. Webster (ed.), (John Wiley, New York, NY, 1999)Vol. 8 679–690.

30. M. Maheswaran, Quality of service driven resource managementalgorithms for network computing. 1999 International Conferenceon Parallel and Distributed Processing Technologies and Applica-tions (PDPTA’ 99) (June/July 1999) pp. 1090–1096.

31. D.C. Marinescu, A protocol for multiple access communicationwith real-time delivery constraints. IEEE INFOCOM ’90 (June1990) pp. 1119–1126.

32. P. Marbach, Pricing priority classes in a differentiated services net-work. 37th Annual Allerton Conference on Communication, Con-trol, and Computing (September 1999), proceedings to appear.

33. N.W. Porter, Resources Required for Adaptive C4I Models in aHeterogeneous Computing Environment, Master’s Thesis, Dept. ofCS, Naval Postgraduate School, Monterey, CA, June 1999 (D.A.Hensgen, advisor) p. 181.

34. R. Rajkumar, C. Lee, J.P. Lehoczky, and D.P. Siewiorek, A resourceallocation model for QoS management. IEEE Symposium on Real-Time Systems (December 1997) pp. 289–307.

35. A.J. Rockmore, BADD Functional Description. Internal DARPAMemo, February 1996.

36. T. Ryutov and C. Neuman, Access control framework for dis-tributed applications, INTERNET-DRAFT, CAT Working Group,November 1998, ftp://ftp.ietf.org/internet-drafts/draft-ietf-cat-acc-cntrl-frmw-01.txt

37. B. Sabata, S. Chatterjee, M. Davis, J. Sydir, and T. Lawrence, Taxon-omy for QoS specifications, 3rd International Workshop on Object-Oriented Real-Time Dependable Systems (WORDS ’97) (February1997) pp. 100–107.

38. P.A. Schneck and K. Schwan, Dynamic authentication for high-performance networked applications, 6th International Work-shop on Quality of Service (IWQoS ’98) (May 1998) pp. 127–136.

39. B. Shirazi, L. Welch, B. Ravindran, C. Cavanaugh, B. Yanamula,R. Brucks, and E. Huh, DynBench: A dynamic benchmark suite fordistributed real-time systems, in Parallel and Distributed Process-ing: 11th IPPS/SPDP ‘99, J. Rolim et al. (eds.) (Springer, Berlin,April 1999) pp. 1335–1349.

40. L.J. Siegel, H.J. Siegel, and P.H. Swain, Performance mea-sures for evaluating algorithms for SIMD machines. IEEE Trans-actions on Software Engineering SE-8(4) (July 1982) 319–331.

41. J.A. Stankovic, M. Supri, K. Ramamritham, and G.C. Buttazzo,Terminology and assumptions. In Deadline Scheduling for Real-Time Systems (Kluwer Academic Publishers, Norwell MA, 1998)pp. 13–22.

42. M.D. Theys, M. Tan, N.B. Beck, H.J. Siegel, and M. Jurczyk, Amathematical model and scheduling heuristics for satisfying pri-oritized data requests in an oversubscribed communication net-work. IEEE Transactions on Parallel and Distributed Systems 11(9)(September 2000) 969–988.

43. C. Wang and W. A. Wulf, A framework for security measurement,The National Information Systems Security Conference (October1997) pp. 522–533.

44. W.E. Walsh, M.P. Wellman, P.R. Wurman, and J.K. Mackie-Mason,Some economics of market-based distributed scheduling. 18th In-

ternational Conference on Distributed Computer Systems (May1998) pp. 612–621.

45. D. Xu, K. Nahrstedt, and D. Wichadakul, QoS and contention-aware multi-resource reservation. Cluster Computing 4(2) (April2001) 95–107.

Jong-Kook Kim is pursuing a Ph.D. degree from the School of Electri-cal and Computer Engineering at Purdue University (expected in August2004). Jong-Kook received his M.S. degree in electrical and computerengineering from Purdue University in May 2000. He received his B.S.degree in electronic engineering from Korea University, Seoul, Koreain 1998. He has presented his work at several international conferencesand has been a reviewer for numerous conferences and journals. His re-search interests include heterogeneous distributed computing, computerarchitecture, performance measure, resource management, evolutionaryheuristics, and power-aware computing. He is a student member of theIEEE, IEEE Computer Society, and ACM.

Debra Hensgen is a member of the Research and Evaluation Teamat OpenTV in Mountain View, California. OpenTV produces middle-ware for set-top boxes in support of interactive television. She re-ceived her Ph.D. in the area of Distributed Operating Systems fromthe University of Kentucky. Prior to moving to private industry, asan Associate Professor in the systems area, she worked with stu-dents and colleagues to design and develop tools and systems for re-source management, network re-routing algorithms and systems thatpreserve quality of service guarantees, and visualization tools for per-formance debugging of parallel and distributed systems. She has pub-lished numerous papers concerning her contributions to the Concurratoolkit for automatically generating safe, efficient concurrent code, theGraze parallel processing performance debugger, the SAAM path in-formation base, and the SmartNet and MSHN Resource ManagementSystems.

Taylor Kidd is currently a Software Architect for Vidiom Systemsin Portland Oregon. His current work involves the writing of multi-company industrial specifications and the architecting of software sys-tems for the digital cable television industry. He has been involved in

Springer

Cluster Comput (2006) 9:281–296 295

the establishment of international specifications for digital interactivetelevision in both Europe and in the US. Prior to his current position,Dr. Kidd has been a researcher for the US Navy as well as an AssociateProfessor at the Naval Postgraduate School. Dr Kidd received his Ph.D.in Electrical Engineering in 1991 from the University of California, SanDiego.

H. J. Siegel was appointed the George T. Abell Endowed Chair Distin-guished Professor of Electrical and Computer Engineering at ColoradoState University (CSU) in August 2001, where he is also a Professor ofComputer Science. In December 2002, he became the first Director ofthe CSU Information Science and Technology Center (ISTeC). ISTeCis a university-wide organization for promoting, facilitating, and en-hancing CSU’s research, education, and outreach activities pertainingto the design and innovative application of computer, communication,and information systems. From 1976 to 2001, he was a professor atPurdue University. He received two BS degrees from MIT, and the MA,MSE, and PhD degrees from Princeton University. His research interestsinclude parallel and distributed computing, heterogeneous computing,robust computing systems, parallel algorithms, parallel machine inter-connection networks, and reconfigurable parallel computer systems. Hehas co-authored over 300 published papers on parallel and distributedcomputing and communication, is an IEEE Fellow, is an ACM Fel-low, was a Coeditor-in-Chief of the Journal of Parallel and DistributedComputing, and was on the Editorial Boards of both the IEEE Trans-actions on Parallel and Distributed Systems and the IEEE Transac-tions on Computers. He was Program Chair/Co-Chair of three majorinternational conferences, General Chair/Co-Chair of four internationalconferences, and Chair/Co-Chair of five workshops. He has been an in-ternational keynote speaker and tutorial lecturer, and has consulted forindustry and government.

David St. John is Chief Information Officer for WeatherFlow, Inc.,a weather services company specializing in coastal weather obser-vations and forecasts. He received a master’s degree in Engineeringfrom the University of California, Irvine. He spent several years as thehead of staff on the Management System for Heterogeneous Networksproject in the Computer Science Department of the Naval Postgradu-ate School. His current relationship with cluster computing is as a userof the Regional Atmospheric Modeling System (RAMS), a numericalweather model developed at Colorado State University. WeatherFlowruns RAMS operationally on a Linux-based cluster.

Cynthia Irvine is a Professor of Computer Science at the Naval Post-graduate School in Monterey, California. She received her Ph.D. fromCase Western Reserve University and her B.A. in Physics from RiceUniversity. She joined the faculty of the Naval Postgraduate School in1994. Previously she worked in industry on the development of highassurance secure systems. In 2001, Dr. Irvine received the Naval Infor-mation Assurance Award. Dr. Irvine is the Director of the Center forInformation Systems Security Studies and Research at the Naval Post-graduate School. She has served on special panels for NSF, DARPA,and OSD. In the area of computer security education Dr. Irvine hasmost recently served as the general chair of the Third World Confer-ence on Information Security Education and the Fifth Workshop onEducation in Computer Security. She co-chaired the NSF workshop onCyber-security Workforce Needs Assessment and Educational Innova-tion and was a participant in the Computing Research Association/NSFsponsored Grand Challenges in Information Assurance meeting. She isa member of the editorial board of the Journal of Information Warfareand has served as a reviewer and/or program committee member of avariety of security related conferences. She has written over 100 papersand articles and has supervised the work of over 80 students. ProfessorIrvine is a member of the ACM, the AAS, a life member of the ASP,and a Senior Member of the IEEE.

Timothy E. Levin is a Research Associate Professor at the Naval Post-graduate School. He has spent over 18 years working in the design,development, evaluation, and verification of secure computer systems,including operating systems, databases and networks. His current re-search interests include high assurance system design and analysis,development of models and methods for the dynamic selection of QoSsecurity attributes, and the application of formal methods to the devel-opment of secure computer systems.

Viktor K. Prasanna received his BS in Electronics Engineering fromthe Bangalore University and his MS from the School of Automation,

Springer

296 Cluster Comput (2006) 9:281–296

Indian Institute of Science. He obtained his Ph.D. in Computer Sciencefrom the Pennsylvania State University in 1983. Currently, he is a Pro-fessor in the Department of Electrical Engineering as well as in the De-partment of Computer Science at the University of Southern California,Los Angeles. He is also an associate member of the Center for AppliedMathematical Sciences (CAMS) at USC. He served as the DivisionDirector for the Computer Engineering Division during 1994–98. Hisresearch interests include parallel and distributed systems, embeddedsystems, configurable architectures and high performance computing.Dr. Prasanna has published extensively and consulted for industries inthe above areas. He has served on the organizing committees of severalinternational meetings in VLSI computations, parallel computation, andhigh performance computing. He is the Steering Co-chair of the Interna-tional Parallel and Distributed Processing Symposium [merged IEEEInternational Parallel Processing Symposium (IPPS) and the Sympo-sium on Parallel and Distributed Processing (SPDP)] and is the SteeringChair of the International Conference on High Performance Comput-ing(HiPC). He serves on the editorial boards of the Journal of Paralleland Distributed Computing and the Proceedings of the IEEE. He is theEditor-in-Chief of the IEEE Transactions on Computers. He was thefounding Chair of the IEEE Computer Society Technical Committeeon Parallel Processing. He is a Fellow of the IEEE.

Richard F. Freund is the originator of GridIQ’s network schedulingconcepts that arose from mathematical and computing approaches hedeveloped for the Department of Defense in the early 1980’s. Dr. Freundhas over twenty-five years experience in computational mathematics,algorithm design, high performance computing, distributed computing,network planning, and heterogeneous scheduling. Since 1989, Dr. Fre-und has published over 45 journal articles in these fields. He has alsobeen an editor of special editions of IEEE Computer and the Journalof Parallel and Distributed Computing. In addition, he is a founder ofthe Heterogeneous Computing Workshop, held annually in conjunctionwith the International Parallel Processing Symposium. Dr. Freund isthe recipient of many awards, which includes the prestigious Depart-ment of Defense Meritorious Civilian Service Award in 1984 and theLauritsen-Bennet Award from the Space and Naval Warfare SystemsCommand in San Diego, California.

Springer

A ﬂexible multi-dimensional QoS performance measure …hj/journals/85.pdf · 2006. 10. 6. ·...

Documents

Transcript of A ﬂexible multi-dimensional QoS performance measure …hj/journals/85.pdf · 2006. 10. 6. ·...