USING PERFORMANCE
MEASUREMENT FRAMEWORK IN
PORTFOLIO MANAGEMENT: A
CONSTRUCTIVIST CASE
Rogerio Tadeu de Oliveira Lacerda (UFSC)
Leonardo Ensslin (UFSC)
Sandra Rolim Ensslin (UFSC)
The main focus of this article is to present a framework to create a
understanding of the business strategy context and aid the portfolio
management process in a software company. The research method is a
quali-quantitative, presented in a study case and the intervention
instrument adopted is the Multicriteria Decision Aiding Methodology -
Constructivist (MCDA-C). The framework enables to visualize the
criteria that must be taken into account according to the decision-
makers’ values in the project selection and sorting processes. The
framework supports the ordinal and cardinal measurement of the
project performance, making it possible to compare and rank
proposals, as well providing a process to improve project proposals.
This process has helped in negotiations between stakeholders in a
portfolio context and, consequently, has helped the chief project officer
to select and prioritize strategic projects within the demand for new
products.
Palavras-chaves: Performance Management, Portfolio Management,
Decision, Technology Management, Multicriteria, Measurement
XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT
Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.
XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT
Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.
2
A.
1. INTRODUCTION
An alternative to aid the strategic decision processes in context that involves
complexity is presented by portfolio management, by using problem structuring techniques
that improve the understanding of the consequences on the operational focus of the business.
The new competitive dimensions as agility and innovation have brought with them the
fact that the managers responsible for the portfolio management need to decide in an context
where they do not know the criteria to be met, nor do they develop mechanisms to improve
knowledge of the consequences of the portfolio management in the strategic objectives of the
organization (GANN & SALTER, 2000).
Nowadays, the decision to build understanding about the decision contexts, explaining
the objectives to be achieved and their association with the operational activities, becomes a
competitive key for organizations.
On this issue emerge research question of this article: How to measure a project
charter in light of the strategic objectives of a company in order to aid decisions on portfolio
management?
In order to answer the research question, the main objective in this study is propose a
methodology that allows identifying, measure and integrating dimensions judged by decision-
maker as necessary and sufficient for evaluating projects and thus enable the creation and
ordering of proposals in a cycle of selection of projects in an software development company.
Thus, the specific objectives of the research are: (i) To present a performance
measurement framework and to generate a better understanding of the portfolio context and
(ii) to present an application in order to illustrate the proposed framework for identifying and
evaluating a project in a portfolio.
To achieve the objectives set out above, the methodology selected by authors was the
Multicriteria Decision Aid Methodology – Constructivist (MCDA-C) by its characteristics of
constructivism, specifically regarding: (i) the ability to promote awareness of the actors, as
the context that we intend to improve, (ii) the ability to structure and evaluate the dimensions
considered relevant for these actors, giving more reliable results; (iii) the ability to spread
generated knowledge, (iv) capability to support the decision-making process.
The relevance of the theme is given by (i) the current organizations context with
respect to its business objectives, which are increasingly using project management as a tool
for competitive advantage (PWC, 2004); (ii) the feasibility to solve problems using the
methodology selected for this study (BANA E COSTA, ENSSLIN et al., 1999); (iii) to extend
the scientific publications of Englund and Graham (1999), Coombs et al (1998) and Cooper
(2000), suggesting the creation of criteria for selecting projects, but do not focusing their
studies on how to perform this activity.
Thus, beyond this introduction, this article is arranged in five sections. In the
following section is presented the theoretical framework for the portfolio management; in the
third section is presented the research methodology used; in the fourth section is presented the
built model, analysis and discussion; in the fifth section is for the final considerations of the
authors, and the last section the references used in this article are listed.
XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT
Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.
3
2. THEORETICAL FOUNDATIONS
2.1 SUCCESS PROJECTS
Given its history had been developed in operational research, the discipline of project
management was coined within a positivist paradigm (WILLIAMS, 2005; POLLACK, 2007).
Thus, it is necessary to consider if such a paradigm is appropriate for a discipline that deals
with a complex reality (DE MEYER, LOCH et al., 2002; PICH, LOCH et al., 2002;
SOMMER & LOCH, 2004).
It is important to note that some hard thinking about project management should not
be regarded as wrong or should be replaced, but one must bear in mind that this view is only
one point of view to examine this field of knowledge (WINTER & CHECKLAND, 2003).
On the rationalist view, Roy (1994) consider that:
This is a reductionist conception of OR which I would call
unproductive. It is largely responsible for what has been called the OR
crisis. First of all, this conception of OR tends to cut it off from the milieu
which nourishes it and legitimates it as something other than a branch of
mathematics. Cutting off OR in this way encourages researchers to work in
isolation. This results in naive or impoverished references to managerial
reality and decision-making processes. Those responsible for solving
concrete problems are thus inevitably disappointed by the gap between
their own expectations and the results they receive.
Among the recent published studies on the effects of these two paradigms in theory
and practice of project management, we can highlight the text of Williams (2005). In this
systemic perspective, the project management has an attitude focused on stakeholders rather
than in pre-established requirements (TUKEL & ROM, 2001), a precedent in the "iron
triangle" (ATKINSON, 1999) in project management for a view more strategic (JUGDEV &
MÜLLER, 2006).
This evidence is relevant, since the end result of the project is assessed differently by
various stakeholders in the project and the success criteria should reflect diverse interests and
viewpoints (LIPOVETSKY, TISHLER et al., 1997; BACCARINI, 1999; SHENHAR,
TISHLER et al., 2002). Neglecting any point of view can mean the failure of the project
(DVIR, LIPOVETSKY et al., 1998), being then a great opportunity for multi-dimensional
studies, enabling us to analyze the mutual interactions of all variables and managerial success
metrics (DVIR, LIPOVETSKY et al., 1998).
Such characteristics in question, some authors have understood that the soft approach
has positive impacts on the management aspect when (i) the technological uncertainty is high
(DE MEYER, LOCH et al., 2002) and long-term consequences are diffuse, (ii) the project is
susceptible to external factors (WINTER & CHECKLAND, 2003) and (iii) the complexity of
the scope and context of the project are high (ATKINSON, CRAWFORD et al., 2006).
2.2 PORTFOLIO MANAGEMENT AND BUSINESS STRATEGY
The corporative strategy constitutes in an organizational process, in which Andrews
(1980) shows this process two important aspects: the formulation and implementation
(ANDREWS, 1980; MINTZBERG, 1994). However, to achieve success in the
implementation of strategy is needed its alignment with organizational interests (SHENHAR,
2004; MINARRO-VISERAS, BAINES et al., 2005), emerging the concept of projects
portfolio management.
XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT
Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.
4
According to Cooper et al. (2000), portfolio management is the formulation, selection
and implementation of projects and the operationalization of organizational strategy.
Despite the growing interest and relevance to portfolio management by organizations,
these are faced with some difficulties (ELONEN & ARTTO, 2003), such as decisions on
project prioritization (COOPER, EDGETT et al., 2000), making a critical point the of project
selection.
Bearing this mind, Englund and Graham (1999) explore a process model for portfolio
management based on four stages, operated in the theoretical framework.
2.2.1 Establishing evaluation criteria
As a first step, it is necessary to define what should be done, focusing on the strategic
organization objectives (BOURNE, MILLS et al., 2000). This step represents a very crucial
point that determines whether the rest of the process will be successful (ENGLUND &
GRAHAM, 1999; COOPER, 2007).
At this point it is appropriate to increase the understanding of what are the dimensions
to be taken into account by decision-makers, especially in contexts that work with incomplete,
fuzzy and conflicting information (LIESIO, MILD et al., 2007).
In the process for formulating the problem, it is necessary to determine the dimensions
in which the projects will be evaluated based on the perception and values of the decision
makers. These performance criteria are scales in which the judgments and decisions will be
based to evaluate different projects (CHIEN, 2002).
Important to note that portfolio management is under political pressure, given the
personal interests of executives to use the power to impose their individual preferences
(ENGLUND & GRAHAM, 1999; CHIEN, 2002; ELONEN & ARTTO, 2003; ENGWALL &
JERBRANT, 2003). Thus, a decision-making process is structured and formalized through a
decision-maker in managing pressures of interest groups, justifying their decisions and
communicating with other organization elements (CHIEN, 2002).
The criteria must be sought in the strategic objectives of the people responsible for
portfolio management (KEENEY, 1992). The socio-political values of the decision maker and
the objective properties of the projects set the environment where the criteria should be
sought (KEENEY & RAIFFA, 1976; ROY, 1993; LANDRY, 1995; ROY, 1996; BANA E
COSTA, ENSSLIN et al., 1999). Care must be taken to ensure that the criteria are built from
the decision maker values and not from the differences between the alternatives and / or
sought from past similar situations even successful (KENNERLEY & NEELY, 2003;
ENSSLIN, GIFFHORN et al., 2010).
Although there are several generic proposals for structuring the criteria for selection,
the most important step is to identify the criteria that has greater significance for the
organization (ENGLUND & GRAHAM, 1999; SHENHAR, TISHLER et al., 2002; CHEN,
2008), taking into account the preferences of decision makers (ROY, 1993) and the adoption
of appropriate management techniques to each particular situation (SHENHAR, 2001;
LEWIS, WELSH et al., 2002).
Since the establishment of criteria and measurement takes into account the values and
preferences of the manager responsible for the area being evaluated, this step in performance
evaluation is personalized to the manager responsible for the decision. Roy (1996) and
Landry (1995) refer to this condition as the limit of objectivity.
After identifying what are the necessary and sufficient properties of the portfolio that
explain the decision-maker‟s values (LACERDA, 2009), it is necessary to translate them into
a measurable and unambiguous set of scales. At this stage, the decision-makers are prompted
XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT
Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.
5
by the facilitator to clarify the direction of preferences for each specific goal set in the
previous step.
Firstly, the ordinal scales built has a limited degree of knowledge since all de
information they provide is qualitative. If the decision-maker want to improve the quality of
the information he have to transform the ordinal scales into cardinal scales providing the
information of the difference of attractivity between the level of the scales.
2.2.2 Project information collection
Once clarified the criteria, the teams can "shape" the projects scope according to these
criteria, in order to be aligned to the strategy (SALOMO, WEISE et al., 2007).
Thus, the second step defined proposed by Englund and Graham (1999) for selection
of projects is the projects information collection. The main way out to this phase is a list of
projects with their charters and information necessary to confront the projects properties with
pre-established criteria for the organization.
2.2.3 Evaluation and Recommendations
This stage of evaluation is to determine the impact of projects on performance
indicators to understand its consequences (KEENEY, 1992). A recurring problem in portfolio
management is to give priority to short duration projects, low risk and consequently that with
little impact on improving competitiveness (FRICKE & SHENHAR, 2000).
Thus, measuring the impact of each element to the portfolio management, the third
stage ends with the establishment of prioritization in projects within the available investment
resources.
2.2.4 Monitoring the portfolio management
The fourth step is proposed to allocate the financial and human resources in prioritized
projects, communicate project teams and continue managing the initiatives. In this stage,
monitoring of projects should return information to executives in order to close the cycle of
four stages, as proposed by Englund and Graham (1999).
Noting that there is not a single tool for portfolio effective monitoring, the
methodology presented in this article may be used to build models in line with the two
approaches described by Cooper et al (2000) for monitoring the portfolio management.
2.3 PORTFOLIO MANAGEMENT AND PERFORMANCE EVALUATION
As shown, the contribution of a structured process on the constructivist performance
evaluation allows the identification of opportunities for improving portfolio management.
Thus, we draw the following parallel with the paradigms for decision aid (ENSSLIN, 2009) to
be recognized, as illustrated in Table 1. Decision Aiding
Paradigms Paradigm Description Context in Portfolio Management
Reference in
Portfolio Management
P1 = Uniqueness,
Identity
Decision maker values and
preferences
Criteria for evaluating projects
must be contextualized and
developed in each case the
projects performance evaluation.
(ENGLUND &
GRAHAM, 1999;
SHENHAR, 2001;
CHIEN, 2002;
ENGWALL, 2003)
P2 = Limited
Knowledge
Decision makers‟ need to
improve their understanding
of the decision
consequences.
Approaches that expand the
understanding of decision makers
on the context must be used
(WILLIAMS, 2005;
CÁÑEZ & GARFIAS,
2006; LIESIO, MILD et
al., 2007)
P3 = Social Entity
To favor the stakeholders
with interests in the
decision to submit their
Recognize that the criteria for
evaluation are influenced by social
participants in the portfolio
(ENGLUND &
GRAHAM, 1999;
CHIEN, 2002;
XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT
Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.
6
interests to the decision management decision process ENGWALL, 2003;
WILLIAMS, 2005)
P4 = Recursive
Participatory Learning
The dynamic recursive
process of participants
learning
Recognition that the learning
process is cyclical and that
organization needs a mechanism
to incorporate such knowledge in
the organizational culture.
(ENGLUND &
GRAHAM, 1999;
COOPER, EDGETT et
al., 2000; ENGWALL,
2003; WILLIAMS,
2005)
P5 = Principles of
Measurement
Properties of ordinal scales,
interval, and ratio
Among references of portfolio
management found, none
increases the understanding of the
process of measuring the level of
differences recognition between
ordinal and cardinal scales.
(ROBERTS, 1979;
BARZILAI, 2001)
P6 = Legitimacy and
Validation
Transparency of
participation, recognition of
usefulness of knowledge
generated and the scientific
status of the construction of
knowledge used
Recognition by the decision
maker, knowledge built by the
decision aid process was useful to
understand the consequences of
projects in strategic objectives as
well as having scientific support
for corporate use
(ENGLUND &
GRAHAM, 1999;
CHIEN, 2002; CÁÑEZ
& GARFIAS, 2006;
ENSSLIN, GIFFHORN
et al., 2010)
Table 1: Paradigms for Decision Aid (ENSSLIN, 2009) and Portfolio Management
3 RESEARCH METHODOLOGY
This item is divided into three sections, first section presents a methodological
framework, second section presents the procedures used to the literature review and the third
section discusses the adopted intervention instrument.
3.1 METHODOLOGICAL FRAMEWORK
This research is exploratory, applied and developed in a case study way because the
goal resides in deepening the knowledge related to innovation within a software development
company. Data used are primary, since they were obtained through unstructured interviews
with the technology director and manager of PMO (Project Management Office) of the
company in which the research was performed. The research method is qualitative and
quantitative. The intervention tool used was the Multicriteria Decision Aid Methodology -
Constructivist (MCDA-C) (ENSSLIN, GIFFHORN et al., 2010).
3.2 LITERATURE REVIEW
The theoretical framework of this article began searching for articles in each journal
cited by Shenhar and Dvir (2007), using the keyword "project management", which created a
database of 200 articles that, after systematic reading of their abstracts before the previously
defined criteria: (i) view of knowledge is not reduce to the positivist paradigm, (ii) recognize
that the dimensions of project success are socially constructed and (iii) create opportunities to
expand the understanding of what the degree of success in project and portfolio management.
Finally, after reading the full articles, eight articles were selected (COOMBS,
MCMEEKIN et al., 1998; FRICKE & SHENHAR, 2000; CHIEN, 2002; DVIR &
LECHLER, 2004; SHENHAR, 2004; WILLIAMS, 2005; TUKEL, ROM et al., 2006;
COOPER, 2007) to set the core of the bibliography.
XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT
Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.
7
3.3 CONSTRUCTION OF THE MULTICRITERIA MODEL – PROCEDURES
The construction of the model of performance evaluation following the MCDA-C
methodology is divided into eight steps, described in this sub-section.
3.3.1 STEP 1: CONTEXTUALIZATION
The Structuring Phase aims to explain the context and achieve an understanding of the
problem to be discussed. To achieve such an aim, the players involved in the context are
identified and the problem statement is legitimated with them.
3.3.2 STEP 2: HIERARCHICAL STRUCTURE OF VALUE
The facilitator then encourages the decision maker to talk about the context, and by
interpreting the interviews the Primary Elements of Evaluation (PEE) are identified. These
elements are the essential factors in the decision maker‟s system of values and concerns. For
each PEE, a concept representing the decision maker‟s choice of preference direction is
constructed, as well as its psychological opposite pole.
Then, the decision maker is encouraged to group the concepts in areas of concern.
With the concepts of each area of concern, cognitive maps are constructed (BITITCI,
SUWIGNJO et al., 2001). In the means-ends relationship map, the clusters of concepts are
identified (EDEN, JONES et al., 1985) and they represent the map in an exhaustive way.
Each cluster in the cognitive map has an equivalent point of view in the hierarchical structure
of value. This association makes it possible to transfer knowledge from cognitive map to the
hierarchical structure of value (ENSSLIN, DUTRA et al., 2000).
3.3.3 STEP 3: CONSTRUCTION OF DESCRIPTORS
The hierarchical structure of value represents the strategic dimension called
Fundamental Points of View (FPsV) and their connections to the operational activities, called
Elemental Points of View (EPsV). Next, it is necessary to use the information in the cognitive
maps to build ordinal scales in the hierarchical structure of value, named descriptors, in order
to measure the range of what is measured (BANA E COSTA, ENSSLIN et al., 1999). In order
to establish the basis for comparing the performance between descriptors, the decision maker
must identify the reference levels „neutral‟ and „good‟ (ENSSLIN, DUTRA et al., 2000). The
process of qualitative knowledge generation is finished with the descriptors.
3.3.4 STEP 4: INDEPENDENCE ANALYSIS
To continue the process of building knowledge, the qualitative scales of descriptors
must be transformed onto cardinal scales and then integrated. The MCDA-C uses a
compensatory model to build the global evaluation model. This model assumes that the
conversion rates used in the integration are constant. To achieve this condition, the criteria
must be independent (ENSSLIN, DUTRA et al., 2000).
3.3.5 STEP 5: CONSTRUCTION OF VALUES FUNCTIONS AND IDENTIFICATION OF CONVERSION
RATES
The next step in the methodology is the transformation of the descriptors into cardinal
scales called value functions. This transformation requires the decision makers to describe the
different levels of attractiveness for all the levels of the descriptor. The integration is achieved
XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT
Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.
8
by associating the conversion rates with the increase in performance when improving from
the „neutral‟ reference level to the „good‟ reference level for each descriptor.
3.3.6 STEP 6: IDENTIFICATION OF IMPACT PROFILE OF ALTERNATIVES
Then it is possible to evaluate the performance of every alternative and, among them,
the status quo. The models constructed by the MCDA-C methodology make possible an
explicit evaluation in numerical and/or graphic form, facilitating the identification and
understanding of the intensity of strong as well as weak points of the alternatives under
evaluation.
3.3.7 STEP 7: SENSITIVITY ANALYSIS
In order to provide a broad view of the stability of alternative performances, the model
allows for the development of a sensitivity analysis of the impact of alternatives in the scales,
in the attractiveness difference in cardinal scales as well as in the conversion rates.
3.3.8 STEP 8: FORMULATION OF RECOMMENDATIONS
The knowledge generated allows the decision-makers to visualize for each criterion
where the performance of a given alternative is „good‟, „normal‟ or „weak‟. The scale of the
descriptors allows the identification of actions to improve performance. Combining this
knowledge with the global evaluation obtained in the previous step, it is possible to generate
alternatives and measure their impact in the context.
4 MULTICRITERIA MODEL OF PROJECT EVALUATION
STEP 1: CONTEXTUALIZATION
The research commenced with meetings with the decision makers of the company in
order to contextualize the problem and expose its function. The company wanted to have a
mechanism to identify and evaluate projects proposals of new software products.
The interview resulted in the establishment of a problem focus, with definition of: (i)
Label: To evaluate the competitive advantage of a new company product; (ii) Players: Core
Decision-makers: CTO (Chief Technology Officer) with the CPO (Chief Project Officer);
Relevant Stakeholders: Product Manager and Sales Executive; those directly affected by
decisions: users; those indirectly affected by decisions: customers; (iii) Problem statement:
Product managers demand new software, but they do not have a mechanism to evaluate the
competitiveness of a product at the beginning of a project.
STEP 2: HIERARCHICAL STRUCTURE OF VALUE
After legitimating the context of the problem with the decision maker, the PEEs were
mapped. 60 PEEs were identified, and classified in two areas of concern for the decision
maker: “compatible products with global players” and “life cycle cost”.
As the study continued, the decision makers were questioned in order to guide the
PEEs to action and to determine the concepts inherent in each PEE. When this activity was
concluded, the decision maker started to construct two means-ends relationship maps, one for
each area of concern. With the maps, it was possible to identify the sub-clusters. Next, the
clusters were associated with the concerns and transferred to the hierarchical structure of
value, forming the fundamental points of view and the elementary points of view.
XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT
Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.
9
Figure 1 shows the means-ends relationship map constructed for the area of concern
“compatible products with international players”, as well as the identification of its clusters
that generate the FPsV. At this point, a hierarchical structure or a tree of value was proposed
for the decision context.
Figure 1: Means-ends relationship map of an area of concern highlighting the clusters “Strategic Alliances”,
“Pioneering” and “SOA”
STEP 3: CONSTRUCTION OF DESCRIPTORS
After the identification of the FPsV, the construction of scales for the multicriteria
model was started in order to measure the performance of each action on an ordinal scale,
using the decision maker‟s construction. In order to construct the scales, in some cases it was
necessary to decompose the FPsV in the elementary points of view (EPsV) in order to
operationalize the descriptors.
From the hierarchical value structure, the construction of descriptors was started in
order to express the decision maker‟s strategic objectives.
STEP 4: INDEPENDENCE ANALYSIS
All the criteria were analyzed to check the independence of preferences.
STEP 5: CONSTRUCTION OF VALUES FUNCTIONS AND IDENTIFICATION OF CONVERSION RATES
When the structuring of the decision context was concluded, the evaluation phase was
started with the transformation of the descriptors into cardinal scales, so as to express the
attractiveness of the impact levels of the descriptors. With the use of the MACBETH
software, Figure 2 shows the transformation phases from the ordinal scale to the cardinal, for
one of the EPV descriptor “sign services” from FPV-“SOA”.
XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT
Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.
10
Figure 2: Transformation process of a descriptor in a specific criterion (Cardinal Scale), using the MACBETH software
Once having the value function, and defined the unit, of all the points of view, the
process to integrate them and form the representation of the model of global evaluation
according to the perception of the decision maker was started. Integration was achieved
applying the conversion rates to the points of view. At this point, the global model of the
multicriteria support was used to evaluate the success of a software project of the company
investigated.
STEP 6: IDENTIFICATION OF IMPACT PROFILE OF ALTERNATIVES
The project under consideration obtained 50 points, and hence was rated as
competitive according to the decision maker‟s perception. The evaluation in graphic form is
presented in Figure 3, where the arrows beside the points of view are the scale representation
of success in each dimension evaluated.
Figure 3: Hierarchical value structure and the graphic of consequences of the project
STEP 7: SENSITIVITY ANALYSIS
Finally, changes to conversion rates and the impact of each criterion were simulated,
as a sensitivity analysis of the model. This analysis resulted in an instrument to show to the
decision maker and project agents where the best opportunities for improvement in
performance are, thus improving the competitiveness of new products developed.
STEP 8: FORMULATION OF RECOMMENDATIONS
The generated knowledge has helped to measure, on a cardinal scale, the contribution
that improvement actions may make to the strategic objectives as can be observed in the
XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT
Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.
11
measurement of 50 points for the initial version of the project evaluated. With this knowledge,
the chief project officer (CPO) started a new activity to sort the improvement requests and
improve the project‟s performance, linking with the FPV “Speedy in developing new
features”, that would improve the global performance by 13 points (see Figure 4). This would
involve the average rate of errors dropping from 4 to 2 (EPV-“Project Management”), the
project deploying 10 features requested by stakeholders per months instead of 8 (EPV-
“Productivity”), and the project reusing 20% of the lines of code instead of the original 15%
(EPV-“Components Reuse”). This illustrate the Step 8 of the MCDA-C, recommendations.
Figure 4: The representation of the impact of improvement action a’ in the global performance against the current project a
5 CONCLUSIONS AND RECOMMENDATIONS FOR FURTHER RESEARCH
The present research used recent views of knowledge to propose solutions to some
problems observed in portfolio management, proposing a methodology that enables the
identification, organization, measurement and integration of criteria judged necessary and
sufficient by the decision maker, in order to understand the context, and thus, to understand
the consequences of different decisions.
With the points above in mind, the research question guiding this work is revisited:
How to measure a project charter in light of the strategic objectives of a company in order to
aid decisions on portfolio management?
To answer this question, the present work has presented a methodology that, by using
the instruments described here, helps the creation of possible improvement actions to increase
alignment with de strategic planning.
Regarding the achievement of the objective of this research, as set out in Section 3.3,
“Construction of the Multicriteria Model – procedures” – the MCDA-C methodology has
been presented and Section 4.
The decision maker‟s understanding of the contribution of the project was presented
graphically and numerically, as shown in Figure 3. This knowledge emerges from developing
the ability to foresee the consequences of the operational characteristics of the project in the
strategic objectives of the company. This contributes to foreseeing the opportunities for
project improvement, as well as generating, starting from this instrument, actions to improve
XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT
Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.
12
its chances of success. Moreover, this cardinal measurement allows the comparability and
project ordering the evaluated faced with other projects proposals.
The decision makers may apply the model to guide the search for actions in order to
select and prioritize projects put forward by managers. Before the model, the decisions were
ad-hoc, resulting in many negotiations in order to choose which project to begin.
A limitation of the present research is its tightly centered focus on the technical
concerns of product development, which is the decision maker‟s main concern. So, the model
could be expanded by the unpacking of the core problem in order to observe other company
areas, such as services, sales and marketing. The MCDA-C methodology has demonstrated its
usefulness in the process of supporting portfolio management, and it is recommended that it
be applied in other contexts, so as to test its generality in similar situations. However, as
models generated in each situation are specific to the context, the model will need to be
adapted, even in contexts that are very similar.
6 REFERENCES
ANDREWS, K. R. The Concept of Corporate Strategy, rev. ed. Homewood, IL: Irwin. 1980.
ATKINSON, R. Project management: cost, time and quality, two best guesses and a
phenomenon, its time to accept other success criteria. International Journal of Project
Management, v.17, n.6, p.337-342. 1999.
ATKINSON, R., CRAWFORD, L. & WARD, S. Fundamental uncertainties in projects
and the scope of project management. International Journal of Project Management, v.24, n.8,
p.687-698. 2006.
BACCARINI, D. The logical framework method for defining project success. Project
Management Journal, v.30, n.4, p.25. 1999.
BANA E COSTA, C. A., ENSSLIN, L., CORREA, E. C. & VANSNICK, J.-C. Decision
Support Systems in action: Integrated application in a multicriteria decision aid process.
European Journal of Operational Research, v.113, n.2, p.315-335. 1999.
BARZILAI, J. On the foundations of measurement. 2001 IEEE International Conference on
Systems, Man, and Cybernetics, v.1. 2001.
BITITCI, U. S., SUWIGNJO, P. & CARRIE, A. S. Strategy management through
quantitative modelling of performance measurement systems. International Journal of
Production Economics, v.69, n.1, Jan, p.15-22. 2001.
BOURNE, M., MILLS, J., WILCOX, M., NEELY, A. & PLATTS, K. Designing,
implementing and updating performance measurement systems. International Journal of
Operations & Production Management, v.20, n.7, p.754-771. 2000.
CÁÑEZ, L. & GARFIAS, M. PORTFOLIO MANAGEMENT AT THE MEXICAN
PETROLEUM INSTITUTE. Research Technology Management, v.49, n.4, p.46. 2006.
CHEN, C. C. An objective-oriented and product-line-based manufacturing performance
measurement. International Journal of Production Economics, v.112, n.1, Mar, p.380-390.
2008.
CHIEN, C.-F. A portfolio-evaluation framework for selecting R&D projects. R&D
Management, v.32, n.4, p.359-368. 2002.
COOMBS, R., MCMEEKIN, A. & PYBUS, R. Toward the development of benchmarking
tools for R&D project management. R&D Management, v.28, n.3, p.175-186. 1998.
COOPER, R. G. Managing technology development projects. Engineering Management
Review, IEEE, v.35, n.1, p.67-67. 2007.
COOPER, R. G., EDGETT, S. J. & KLEINSCHMIDT, E. J. New Problems, New
Solutions: Making Portfolio Management More Effective. Research Technology
XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT
Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.
13
Management, v.43, n.2, p.18-33. 2000.
DE MEYER, A., LOCH, C. H. & PICH, M. T. From Variation to Chaos. MIT Sloan
Management Review, winter. 2002.
DVIR, D. & LECHLER, T. Plans are nothing, changing plans is everything: the impact of
changes on project success. Research Policy, v.33, n.1, p.1-15. 2004.
DVIR, D., LIPOVETSKY, S., SHENHAR, A. & TISHLER, A. In search of project
classification: a non-universal approach to project success factors. Research Policy, v.27,
n.9, p.915-935. 1998.
EDEN, C., JONES, S. & SIMMS, D. Messing about in Problems. R&D Management, v.15,
n.3, p.255-255. 1985.
ELONEN, S. & ARTTO, K. A. Problems in managing internal development projects in
multi-project environments. International Journal of Project Management, v.21, n.6, p.395-
402. 2003.
ENGLUND, R. L. & GRAHAM, R. J. From Experience: Linking Projects to Strategy.
Journal of Product Innovation Management, v.16, n.1, p.52-64. 1999.
ENGWALL, M. No project is an island: linking projects to history and context. Research
Policy, v.32, n.5, p.789-808. 2003.
ENGWALL, M. & JERBRANT, A. The resource allocation syndrome: the prime challenge
of multi-project management? International Journal of Project Management, v.21, n.6, p.403-
409. 2003.
ENSSLIN, L. Processo Decisório e Aspectos Cognitivos. XII Simpósio de Pesquisa
Operacional e Logística da Marinha. 2009.
ENSSLIN, L., DUTRA, A. & ENSSLIN, S. R. MCDA: a constructivist approach to the
management of human resources at a governmental agency. International Transactions in
Operational Research, v.7, n.1, p.79-100. 2000.
ENSSLIN, L., GIFFHORN, E., ENSSLIN, S. R., PETRI, S. M. & VIANNA, W. B.
Avaliação do Desempenho de Empresas Terceirizadas com o Uso da Metodologia
Multicritério de Apoio à Decisão- Construtivista. Revista Pesquisa Operacional, v.v.30, n.1,
p.125-152. 2010.
FRICKE, S. E. & SHENHAR, A. J. Managing multiple engineering projects in a
manufacturing support environment. Engineering Management, IEEE Transactions on, v.47,
n.2, p.258-268. 2000.
GANN, D. M. & SALTER, A. J. Innovation in project-based, service-enhanced firms: the
construction of complex products and systems. Research Policy, v.29, n.7-8, p.955-972. 2000.
JUGDEV, K. & MÜLLER, R. A retrospective look at our evolving understanding of project
success. Project Management Journal, v.36, n.4, p.19-31. 2006.
KEENEY, R. L. Value-Focused Thinking: A Path to Creative Decisionmaking: Harvard
University Press. 1992
KEENEY, R. L. & RAIFFA, H. Decisions with multiple objectives: preferences and value
tradeoffs. New York: Wesley, p.224-241. 1976.
KENNERLEY, M. & NEELY, A. Measuring performance in a changing business
environment. International Journal of Operations & Production Management, v.23, n.2, p.213-
229. 2003.
LACERDA, R. T. O. O sucesso em gerenciamento de projetos: a estruturação de um modelo
de avaliação a partir de uma visão construtivista. Florianópolis. Dissertação de mestrado em
Engenharia de Produção da Universidade Federal de Santa Catarina. 2009.
LANDRY, M. A Note on the Concept of 'Problem'. Organization Studies, v.16, n.2, p.315.
XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT
Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.
14
1995.
LEWIS, M. W., WELSH, M. A., DEHLER, G. E. & GREEN, S. G. Product development
tensions: Exploring contrasting styles of project management. Academy of Management
Journal, v.45, n.3, Jun, p.546-564. 2002.
LIESIO, J., MILD, P. & SALO, A. Preference programming for robust portfolio modeling
and project selection. European Journal of Operational Research, v.181, n.3, p.1488-1505.
2007.
LIPOVETSKY, S., TISHLER, A., DVIR, D. & SHENHAR, A. The relative importance of
project success dimensions. R&D Management, v.27, n.2, p.97-106. 1997.
MINARRO-VISERAS, E., BAINES, T. & SWEENEY, M. Key success factors when
implementing strategic manufacturing initiatives. International Journal of Operations &
Production Management, v.25, n.2, p.151. 2005.
MINTZBERG, H. The Rise and Fall of Strategic Planning: Reconceiving Roles for Planning,
Plans, Planners: Free Press. 1994
PICH, M. T., LOCH, C. H. & MEYER, A. D. On uncertainty, ambiguity, and complexity in
project management. Management Science, v.48, n.8, p.1008. 2002.
POLLACK, J. The changing paradigms of project management. International Journal of
Project Management, v.25, n.3, p.266-274. 2007.
PWC. Boosting business performance through programme and project management.
Connected Thinking. PriceWaterhouseCoopers. 2004.
ROBERTS, F. S. Measurement theory: Addison-Wesley Reading, MA. 1979
ROY, B. Decision science or decision-aid science? European Journal of Operational
Research, v.66, n.2, p.184-203. 1993.
______. On operational research and decision aid. European Journal of Operational
Research, v.73, n.1, p.23-26. 1994.
______. Multicriteria Methodology for Decision Aiding: Kluwer Academic Pub. 1996
SALOMO, S., WEISE, J. & GEMUNDEN, H. G. NPD Planning Activities and Innovation
Performance: The Mediating Role of Process Management and the Moderating Effect of
Product Innovativeness. Journal of Product Innovation Management, v.24, n.4, p.285-302.
2007.
SHENHAR, A. J. One Size Does Not Fit All Projects: Exploring Classical Contingency
Domains. MANAGEMENT SCIENCE, v.47, n.3, March 1, 2001, p.394-414. 2001.
______. Strategic Project Leadership: Toward a strategic approach to project management.
R&D Management, v.34, n.5, p.569-578. 2004.
SHENHAR, A. J. & DVIR, D. PROJECT MANAGEMENT RESEARCH-THE CHALLENGE
AND OPPORTUNITY. Project Management Journal, v.38, n.2, p.93. 2007.
SHENHAR, A. J., TISHLER, A., DVIR, D., LIPOVETSKY, S. & LECHLER, T.
Refining the search for project success factors: a multivariate, typological approach. R&D
Management, v.32, n.2, p.111-126. 2002.
SOMMER, S. C. & LOCH, C. H. Selectionism and Learning in Projects with Complexity
and Unforeseeable Uncertainty. Management Science, v.50, n.10, p.1334. 2004.
TUKEL, O. I. & ROM, W. O. An empirical investigation of project evaluation criteria.
International Journal of Operations & Production Management, v.21, n.3, p.400. 2001.
TUKEL, O. I., ROM, W. O. & EKSIOGLU, S. D. An investigation of buffer sizing
techniques in critical chain scheduling. European Journal of Operational Research, v.172, n.2,
p.401-416. 2006.
WILLIAMS, T. Assessing and moving on from the dominant project management discourse
XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT
Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.
15
in the light of project overruns. Engineering Management, IEEE Transactions on, v.52, n.4,
p.497-508. 2005.
WINTER, M. & CHECKLAND, P. Soft systems- a fresh perspective for project
management. Proceedings of the Institution of Civil Engineers: Civil Engineering, v.156, n.4,
p.187-192. 2003.
Top Related