Mustafa Degerli - 2010 - Dissertation Review - IS 720 Research Methods in Information Systems
-
Upload
mustafa-degerli -
Category
Technology
-
view
129 -
download
0
description
Transcript of Mustafa Degerli - 2010 - Dissertation Review - IS 720 Research Methods in Information Systems
Middle East Technical University
Informatics Institute
Department of Information Systems
IS 720 Research Methods in Information Systems
Dissertation Review Report
Fall 2010
Mustafa DEĞERLİ - 1382142 11/4/2010
1
Preface
This report has seven clauses, namely General Information, Introduction of the Dissertation,
Literature Review of the Dissertation, Methodology of the Dissertation, Results of the
Dissertation, Conclusions, Implications, Recommendations, and Summary of the
Dissertation, and Adoption for My Study.
In General Information clause, Title, Author, Location, Time, Intended Purpose of the
Completion of the Dissertation, Contributors, and the Citation of the Dissertation are provided.
In Introduction of the Dissertation, Literature Review of the Dissertation, Methodology of
the Dissertation, Results of the Dissertation, Conclusions, Implications, Recommendations,
and Summary of the Dissertation clauses, each of these special chapters of the dissertation are
reviewed. Additionally, the outlines of the each chapter of the dissertation are provided and the
information provided in each chapter is summarized and commented. In this context, main
purpose or outcome of the study (as stated by the author), methodology that is used in the
literature review part (inductive vs. deductive or both), research design model (single linear or
multi method), and data collection methods and instruments (self developed or adopted or
standardized) of the study, and the quality of the last chapter in terms of answering research
questions are explained in each related clauses.
In Adoption for My Study clause, potential of any data collection/analysis techniques that I
think I can adopt for my own study and other sorts of adoptions and benefits that I am to take
into account in my own study are explained.
2
Table of Contents
Preface ..............................................................................................................................................1
Table of Contents ...............................................................................................................................2
1 General Information ...................................................................................................................5
1.1 Title ..................................................................................................................................................... 5
1.2 Author ................................................................................................................................................. 5
1.3 Where ................................................................................................................................................. 5
1.4 When ................................................................................................................................................... 5
1.5 For ....................................................................................................................................................... 5
1.6 With ..................................................................................................................................................... 5
1.7 Citation ................................................................................................................................................ 6
2 Introduction of the Dissertation ..................................................................................................7
2.1 Outline of the Chapter ........................................................................................................................ 7
2.2 Problem Statement ............................................................................................................................. 7
2.3 Goal ..................................................................................................................................................... 8
2.4 Hypotheses and Research Questions .................................................................................................. 8
Hypothesis 1 (H1) .................................................................................................................................. 8
Research Question 1 (RQ1) ................................................................................................................... 9
Research Question 2 (RQ2) ................................................................................................................... 9
Hypothesis 2 (H2) .................................................................................................................................. 9
Research Question 3 (RQ3) ................................................................................................................... 9
Research Question 4 (RQ4) ................................................................................................................... 9
2.5 Assumptions, Barriers and Issues ....................................................................................................... 9
Assumptions .......................................................................................................................................... 9
Barriers and Issues .............................................................................................................................. 10
2.5 Overall Evaluation of the Chapter ..................................................................................................... 10
Table 1: Maturity Table for the 1st Chapter of the Dissertation ..................................................... 10
3 Literature Review of the Dissertation ........................................................................................ 11
3.1 Outline of the Chapter ...................................................................................................................... 11
3.1 Defining IS/IT ..................................................................................................................................... 11
3
3.2 Methodology used in the Literature Review .................................................................................... 12
Figure 1: Literature Review – Deductive Methodology Flow ......................................................... 12
Figure 2: Literature Review – Inductive Methodology Flow ........................................................... 13
3.3 Overall Evaluation of the Chapter ..................................................................................................... 13
Table 2: Maturity Table for the 2nd Chapter of the Dissertation..................................................... 14
4 Methodology of the Dissertation .............................................................................................. 15
4.1 Outline of the Chapter ...................................................................................................................... 15
4.2 Research Design Model..................................................................................................................... 15
Figure 3: Phases of the Research Design ........................................................................................ 16
4.3 Overall Evaluation of the Chapter ..................................................................................................... 16
Table 3: Maturity Table for the 3rd Chapter of the Dissertation ..................................................... 17
5 Results of the Dissertation ........................................................................................................ 18
5.1 Outline of the Chapter ...................................................................................................................... 18
5.2 Results ............................................................................................................................................... 18
Figure 4: Model Development ........................................................................................................ 19
Table 4: Content-analytic Summary ................................................................................................ 20
5.3 Overall Evaluation of the Chapter ..................................................................................................... 20
Table 5: Maturity Table for the 4th Chapter of the Dissertation ..................................................... 21
6 Conclusions, Implications, Recommendations, and Summary of the Dissertation ....................... 22
6.1 Outline of the Chapter ...................................................................................................................... 22
6.2 Summary and Evaluation of the Last Chapter of Dissertation .......................................................... 23
Table 6: Constructs, Descriptions and Sub-classifications .............................................................. 24
Figure 5: Developed Conceptual Model ......................................................................................... 25
Table 7: Checklist for Conducting IS/IT Evaluations ........................................................................ 26
6.3 Quality of the Last Chapter Concerning Answers for Research Questions ....................................... 27
Hypothesis 1 (H1) ................................................................................................................................ 27
Research Question 1 (RQ1) ................................................................................................................. 27
Research Question 2 (RQ2) ................................................................................................................. 27
Hypothesis 2 (H2) ................................................................................................................................ 27
Research Question 3 (RQ3) ................................................................................................................. 27
Research Question 4 (RQ4) ................................................................................................................. 27
Answer for Research Question #1: ..................................................................................................... 28
4
Answer for Research Question #2: ..................................................................................................... 28
Conclusion for Hypothesis #1: ............................................................................................................ 28
Answer for Research Question #3: ..................................................................................................... 29
Answer for Research Question #4: ..................................................................................................... 29
Conclusion for Hypothesis #2: ............................................................................................................ 30
Table 8: Maturity Table for the 5th Chapter of the Dissertation ..................................................... 30
7 Adoption for My Study ............................................................................................................. 31
7.1 My Study’s Title and Abstract ........................................................................................................... 31
Title ..................................................................................................................................................... 31
Abstract ............................................................................................................................................... 31
7.2 Adoptions .......................................................................................................................................... 32
5
1 General Information
1.1 Title
The title of the reviewed dissertation is “A Model for the Evaluation of IS/IT Investments.”
1.2 Author
The author of the reviewed dissertation is “Paul M. Tuten.”
1.3 Where
The reviewed dissertation is completed in “Graduate School of Computer and Information
Sciences in Nova Southeastern University.”
1.4 When
The reviewed dissertation is completed in “January 2009.”
1.5 For
The reviewed dissertation is completed as a “Dissertation Submitted in Partial Fulfillment of
the Requirements for the Degree of Doctor of Philosophy in Information Systems.”
1.6 With
The reviewed dissertation is completed by Paul M. Tuten with the following academic staff:
John Scigliano, Ed.D. as Chairperson of Dissertation Committee
William L. Hafner, Ph.D. as Dissertation Committee Member
Sumitra Mukherjee, Ph.D. as Dissertation Committee Member
6
1.7 Citation
The reviewed dissertation’s citation is provided below.
Tuten, P. M. A model for the evaluation of IS/IT investments. Ph.D. dissertation, Nova
Southeastern University, United States - Florida. Retrieved October 31, 2010, from
Dissertations & Theses: Full Text. (Publication No. AAT 3344877).
7
2 Introduction of the Dissertation
2.1 Outline of the Chapter
The “Introduction” part of the reviewed dissertation has the following parts:
Problem Statement,
Goal,
Hypotheses, Research Questions & Assumptions,
Relevance & Significance,
Barriers and Issues,
Limitations / Delimitations of Study,
Resource Requirements,
Summary, and
Definition of Terms.
2.2 Problem Statement
The problem investigated in the study was the complexity and difficulty faced by practitioners in
evaluating investments in IS/IT.
The researcher tries to criticize that evaluation of the investments in IS/IT by practitioners is a
complex and difficult task to do under the current conditions.
8
2.3 Goal
The researcher’s goals were
to examine IS/IT evaluation, together with its approaches, techniques, and methods, as
well as their application in organizations, and
to develop a theoretical model to offer guidelines for organizations to occupy
contextually-sensitive evaluation methods.
2.4 Hypotheses and Research Questions
The researcher proposed two hypotheses and two research questions for each of these
hypotheses. Specifically, there are two hypotheses and four research questions in the related
reviewed dissertation.
Hypothesis 1 (H1)
Existing models of IS/IT evaluation are inadequate because they fail to include all of the relevant
constructs:
the purpose of conducting the evaluation (why);
the subject of the evaluation (what);
the specific aspects to be evaluated (which);
the particular evaluation methods and techniques used (how);
the timing of the evaluation (when);
the individuals involved in, or affected by, the evaluation (who); and
the external and internal environmental conditions under which the organization operates
(where).
9
Research Question 1 (RQ1)
What models of the IS/IT evaluation process are presented in the literature?
Research Question 2 (RQ2)
How do the constructs (identified in H1) relate to the process of IS/IT evaluation?
Hypothesis 2 (H2)
An improved conceptual model of IS/IT evaluation provides an effective tool for describing and
analyzing evaluation practices.
Research Question 3 (RQ3)
Is the researcher’s conceptual model valid for describing IS/IT evaluation practices?
Research Question 4 (RQ4)
What guidelines may be derived from using the researcher’s conceptual model as an analytical
tool to existing IS/IT evaluation case studies?
2.5 Assumptions, Barriers and Issues
Assumptions
The researcher provides three major assumptions in the dissertation.
1. Putting on one side philosophical and epistemological arguments about the “true” nature
of reality, the researcher assumes that individuals’ perceptions or interpretations of reality
drive their actions.
2. The researcher believes that the conceptual model of IS/IT evaluation should be. That is,
it should be able to describe equally well the activities of individuals regardless of the
correctness or value of their actions.
10
3. Despite the need for contextual appropriateness in IS/IT evaluation, the researcher
assumes that practitioners require a sufficient degree of methodological guidance with the
aim of “get-the-job-done” successfully as well.
Barriers and Issues
Purposely, the following barriers and issues were identified by the researcher.
Philosophical challenges inherent in conceptual modeling.
Philosophical and practical difficulties associated with assessing theoretical
contributions.
The intractable challenges inherent in IS/IT evaluation.
The mystery of balancing contextual-sensitivity with sufficient methodological guidance.
The potential lack of industrial awareness and use by practitioners.
2.5 Overall Evaluation of the Chapter
As this chapter of the dissertation explicitly and clearly provides information about the problem
statement, goal, hypotheses, research questions & assumptions, relevance & significance,
barriers and issues, limitations and delimitations, and the resource requirements of the
dissertation study, this chapter is evaluated as “Fully Implemented.”
Table 1: Maturity Table for the 1st Chapter of the Dissertation
Clauses vs. Implementations
Not
Imple
men
ted
Part
iall
y
Imple
men
ted
Larg
ely
Imple
men
ted
Fu
lly
Imple
men
ted
Chapter
1 Introduction
X
11
3 Literature Review of the Dissertation
3.1 Outline of the Chapter
The “Literature Review” part of the reviewed dissertation has the following parts:
Defining IS/IT Evaluation,
Why: The Purpose of Evaluation,
Where: Extra- and Intra-Organizational Environmental Conditions,
When: The Timing of Evaluation,
What: The Object of Evaluation,
Who: The People Involved in Evaluation,
Which: Evaluation Criteria/Measures,
How: IS/IT Evaluation Methods, Techniques, and Approaches,
Examining the Puzzle: Understanding Evaluation in Context,
Literature Review: Key Themes, and
Summary.
3.1 Defining IS/IT
The researcher first acknowledges distinction between information systems (IS) and information
technology (IT), as IT is referred as an organization’s hardware, software, and related
infrastructure, and IS is referred as the design of information flows that attempt to meet an
organization’s informational needs. Additionally, the researcher notes that in theory, IS may or
may not be primarily based on information technology, in practice, however, IS -especially ones
subjected to a formal evaluation process- contain some IT element. For the purpose of this study,
12
the researcher generally used the terms interchangeably and noted any particular instances in
which a distinction between the concepts was to the point.
3.2 Methodology used in the Literature Review
The researcher used both deductive and inductive methodology in the literature review part of
the dissertation. From the deductive perspective, the researcher explored IS/IT evaluation by
examining the underlying assumptions, professional practices, and continuing concerns of both
practitioners and academicians. For this purpose, the researcher deconstructed IS/IT evaluation
into a mass of contextual elements. In this context, each of these elements was considered
independently and then, in relation to each other. Subsequently, the researcher identified themes
that span this varied body of literature with the purpose of drawing provisional conclusions about
the current state-of-the-art.
IT/IS Evaluation Deconstruction
Contextual Elements
Drawing
Provisional
Conclusions
Figure 1: Literature Review – Deductive Methodology Flow
Mostly, the researcher demonstrated that the contextual elements of IS/IT evaluation must be
better understood in order to advance the field’s effectiveness and significance. Eventually, as
13
the researcher note, this improved understanding should take the form of a conceptual model of
IS/IT evaluation, which may be utilized for both descriptive and normative purposes.
From the inductive perspective, the researcher synthesized the fragmented, disjointed, and
contradictory insights into a unified-whole, including following themes:
IS/IT Evaluation is Problematic for Researchers and Practitioners
IS/IT Evaluation is about More Than Estimating or Measuring Outcomes
IS/IT Evaluation Practice is (and should be) Pragmatic
IS/IT Evaluation is Moving beyond the Positivist / Interpretivist Dualism
IS/IT Evaluation Involves Many Complex, Related Contextual Elements
IS/IT Evaluation Needs a Theory for Descriptive and Normative Purposes
Fragmented Insights
Disjointed Insights
Contradictory Insights
SynthesizeUnified-whole Themes
Figure 2: Literature Review – Inductive Methodology Flow
3.3 Overall Evaluation of the Chapter
In the literature review, the researcher established that IS/IT evaluations are comprised of
several, interconnected contextual elements that shall be better understood.
14
These elements, as noted by the researcher, include:
the purpose of conducting the evaluation (why);
the subject of the evaluation (what);
the specific aspects to be evaluated (which);
the particular evaluation methods and techniques used (how);
the timing of the evaluation (when);
the individuals involved in, or affected by, the evaluation (who); and
the external and internal environmental conditions under which the organization operates
(where).
Each of these constructs was examined in detail by the researcher in the literature review part of
the dissertation, including special emphasis on the specific criteria or measures of evaluation
(which question), and the methods or techniques of evaluation (how question).
Additionally, the researcher reviewed existing models which depicted the process of IS/IT
evaluation, as he note down the limitations and differences found in each example.
Owing to above details, this chapter is also evaluated as “Fully Implemented.”
Table 2: Maturity Table for the 2nd Chapter of the Dissertation
Clauses vs. Implementations
Not
Imple
men
ted
Part
iall
y
Imple
men
ted
Larg
ely
Imple
men
ted
Fu
lly
Imple
men
ted
Chapter
1 Introduction
X
Chapter
2 Literature Review
X
15
4 Methodology of the Dissertation
4.1 Outline of the Chapter
The “Methodology” part of the reviewed dissertation has the following parts:
Step 1: Conduct Comprehensive Literature Review
Step 2: Develop IS/IT Evaluation Conceptual Model
Step 3: Validate and Apply the IS/IT Evaluation Conceptual Model
Step 4: Report Study Results
Limitations / Delimitations / Assumptions
Summary
4.2 Research Design Model
The researcher utilized a multiphase approach in the study. There are four main stages for the
related dissertation’s research.
These are:
The implementation of all-inclusive literature review,
The development of the conceptual model of IS/IT evaluation to provide better
understanding of the process’s entity constructs and their possible relationships,
The validation of the conceptual model via a meta-analysis of multiple case studies and
the development of guidelines for conducting contextual evaluations within particular
organizations, and
The reporting of the results of this study.
16
Comprehensive
Literature Review
Development of
Conceptual Model Validation of
Conceptual ModelReporting of
the Results
Figure 3: Phases of the Research Design
4.3 Overall Evaluation of the Chapter
The researcher conducted a comprehensive literature review to address Hypothesis 1 and endow
with answers to Research Question 1 and Research Question 2.
After this, by using the literature review findings the researcher developed the conceptual model
in the study.
The researcher’s conceptual model represents the innermost work of art of the study, to provide
the decisive link between Hypothesis 1 and Hypothesis 2.
Subsequently, the researcher validated the conceptual model based upon case studies and then
utilized it to develop normative guidelines meant for conducting evaluations, in that way
17
addressing Hypothesis 2 by means of answering Research Question 3 and Research Question 4
correspondingly.
After all, the researcher reported the findings associated with each of the hypotheses and
research questions and discussed implications for both future academic research and professional
practices.
As seen above, in four steps the researcher is done. That is to say, the research is completed in
these four steps by justifying two hypotheses with reference to four research questions. Please
see Sub-clause 2.4 of this document for hypotheses and research questions.
In consequence of above details, this chapter is also evaluated as “Fully Implemented.”
Table 3: Maturity Table for the 3rd Chapter of the Dissertation
Clauses vs. Implementations
Not
Imple
men
ted
Part
iall
y
Imple
men
ted
Larg
ely
Imple
men
ted
Fu
lly
Imple
men
ted
Chapter
1 Introduction
X
Chapter
2 Literature Review
X
Chapter
3 Methodology
X
18
5 Results of the Dissertation
5.1 Outline of the Chapter
The “Results” part of the reviewed dissertation has the following parts:
Assembling the Puzzle: A Conceptual Model of IS/IT Evaluation
Introduction to Case Study Analysis for Model Validation
Case Study #1: UK Insurance Company’s IS/IT Evaluation Practices
Case Study #2: Dutch Insurance Company’s IS/IT Evaluation Practices
Case Study #3: UK Manufacturing Company’s Evaluation of IS Infrastructure
Case Study #4: UK Manufacturing Company’s Evaluation of MRPII System
Case Study #5: US Department of Defense Evaluation of an E-Business System
Validating the Conceptual Model: Is it a “Good” Theoretical Contribution?
In Search of Normative Guidelines: Cross-Case Analysis
Summary
5.2 Results
Throughout the Results chapter, the researcher reported the results using a sequence of events
style. The reason for the researcher to do this is to reflect the iterative and cyclical processes
associated with analyzing, writing, and reflecting throughout this study. Furthermore, the
researcher selected this sort of reporting structure in response to Yin’s (2003) call to use the
writing and editing process as an analytical tool and mechanism to clarify thoughts.
In the chapter, the researcher described the construction of a conceptual model of the process of
IS/IT evaluation based on the findings from the literature review. The researcher began by
describing the limitations associated with existing conceptual models. Next, the researcher
19
identified seven constructs associated with the context of an evaluation. Finally, the researcher
developed these constructs into a conceptual model, which resulted after multiple iterations of
model development.
Describe Limitations
of Existing Models
Identify Constructs
of Evaluation
Develop
Conceptual Model
Figure 4: Model Development
After developing conceptual model, the researcher validated it using a multi-case study analysis.
In this context, the researcher followed the steps explained in Methodology part, and reviewed
and coded five case studies to discover confirming or disconfirming evidences.
The researcher confirmed that the conceptual model that is developed represented a “good”
theoretical contribution based on Whetten’s (2002) standard, which required the conceptual
model to be complete and systematic in its rationalization.
In conclusion, the researcher performed a cross-case analysis to identify elements that could
serve as the basis for methodological guidelines for conducting more contextually appropriate
IS/IT evaluations.
20
In this context, the researcher described similarities and differences among the narratives of the
case studies. Likewise, the researcher also identified four “drivers” of contextual evaluations, as
well as at least two examples of how each driver was implemented in practice, Table 4. Based
upon these findings, the researcher constructed IS/IT evaluation guidelines that are described in
the last chapter o the dissertation, including a checklist for practitioners support.
Table 4: Content-analytic Summary
5.3 Overall Evaluation of the Chapter
Using the procedures explained in Methodology chapter of the dissertation, the researcher
resented a proposed conceptual model of the process of IS/IT evaluation based on the
study’s literature review,
21
validated the proposed model using a multi-case study analysis, and
performed a comprehensive cross-case analysis to identify key observations that
informed the researcher’s proposed methodological guidelines.
The above steps are satisfactorily done with enough information. Therefore, this chapter is
evaluated as “Fully Implemented.”
Table 5: Maturity Table for the 4th Chapter of the Dissertation
Clauses vs. Implementations
Not
Imple
men
ted
Part
iall
y
Imple
men
ted
Larg
ely
Imple
men
ted
Fu
lly
Imple
men
ted
Chapter
1 Introduction
X
Chapter
2 Literature Review
X
Chapter
3 Methodology
X
Chapter
4 Results
X
22
6 Conclusions, Implications, Recommendations, and Summary of the
Dissertation
6.1 Outline of the Chapter
The “Conclusions, Implications, Recommendations, and Summary” part of the reviewed
dissertation has the following parts:
Conclusions
Hypothesis #1: The Contextual Elements of an Evaluation
Research Question #1: Models of IS/IT Evaluation in the Literature
Research Question #2: Contextual Elements and the Evaluation Process
Hypothesis #1: Conclusion
Hypothesis #2: Validity and Usability of an Improved Conceptual Model
Research Question #3: Is the Conceptual Model of Evaluation Valid?
Research Question #4: What Guidelines May Be Derived from the Model?
Hypothesis #2: Conclusion
Reflections on Validity: Limitations, Assumptions, and Concerns
Implications
Recommendations
Recommendations for Future Research
Recommendations for the Improvement of Professional Practice
Summary: Recommendations for the Improvement of Professional Practice
Summary
23
6.2 Summary and Evaluation of the Last Chapter of Dissertation
Having
accomplished a comprehensive literature review,
defined a research method, and
discovered a number of findings;
the researcher finalized the study by drawing a number of conclusions and recommendations.
Additionally, the researcher considered the implications of the study’s outcomes for practitioners
and researchers.
The researcher utilized the model to develop a series of guidelines and a checklist to aid
organizations in conducting context-based IS/IT evaluations.
In particular, the researcher provided recommendations to assist evaluators in:
Engaging in critical and reflective practice
Increasing stakeholder participation
Aligning evaluation criteria (which) with the organizational context
Aligning evaluation methods (how) with the organizational context
Learning from experience
In addition to providing guidelines for improved professional practice, the researcher set forth a
series of recommendations for subsequent academic research. These recommendations included
a call for further validating the conceptual model, making additional refinements and/or
extensions to it, developing a comprehensive theory of IS/IT evaluation rooted in the conceptual
model, and the exploring the implications for pedagogy of the researcher’s findings.
24
To summarize the literature review findings, the researcher developed a summary including
description of each construct and any relevant sub-classifications available in the literature. See
Table 6 for details.
Table 6: Constructs, Descriptions and Sub-classifications
The researcher’s conceptual model describes the interactions between the unique conceptual
elements above comprising the process of IS/IT evaluation. See Figure 5 for the developed
conceptual model.
25
Figure 5: Developed Conceptual Model
The researcher believes that the development of a holistic and full-bodied conceptual model that
resulted from the study serves as a significant step in advancing of IS/IT evaluation theory.
Additionally, the researcher’s guidelines and checklist to assist practitioners in conducting
context-based IS/IT evaluation offers a significant contribution to industrial practice. See Table 7
for checklist for conducting better IS/IT evaluations developed in the scope of the dissertation.
26
Table 7: Checklist for Conducting IS/IT Evaluations
27
6.3 Quality of the Last Chapter Concerning Answers for Research Questions
Hypothesis 1 (H1)
Existing models of IS/IT evaluation are inadequate because they fail to include all of the relevant
constructs: the purpose of conducting the evaluation (why); the subject of the evaluation (what);
the specific aspects to be evaluated (which); the particular evaluation methods and techniques
used (how); the timing of the evaluation (when); the individuals involved in, or affected by, the
evaluation (who); and the external and internal environmental conditions under which the
organization operates (where).
Research Question 1 (RQ1)
What models of the IS/IT evaluation process are presented in the literature?
Research Question 2 (RQ2)
How do the constructs (identified in H1) relate to the process of IS/IT evaluation?
Hypothesis 2 (H2)
An improved conceptual model of IS/IT evaluation provides an effective tool for describing and
analyzing evaluation practices.
Research Question 3 (RQ3)
Is the researcher’s conceptual model valid for describing IS/IT evaluation practices?
Research Question 4 (RQ4)
What guidelines may be derived from using the researcher’s conceptual model as an analytical
tool to existing IS/IT evaluation case studies?
28
Answer for Research Question #1:
For this purpose, the researcher wanted to identify conceptual models of the context and process
of IS/IT evaluation and the researcher distinguished between these meta-models of evaluation
versus more specific models of a particular evaluation method. Set this limitation, the researcher
found a few instances. Moreover, the vast majority of the examples were rooted in the work of
Symons (1990. Overall, the researcher found five meta-models of the process of IS/IT
evaluation. In each of these models, one or more of the seven evaluation constructs identified in
the researcher’s literature review were not there.
Answer for Research Question #2:
In Methodology chapter of the dissertation, the researcher presented underlying principles for
including each of the seven identified conceptual elements of IS/IT evaluation, and a
comprehensive literature review specific to each construct. Based upon these findings, the
researcher followed Whetten’s (2002) methodology for developing theoretical contributions,
such as conceptual models. Following numerous iterations and revisions, the researcher created
the conceptual model of IS/IT evaluation that appeared consistent with the findings of the
literature review.
Conclusion for Hypothesis #1:
The researcher concluded that the results support the first hypothesis in this study. The process of
IS/IT evaluation consists of seven contextual elements. Although there are number of existing
models of IS/IT evaluation in the literature, these models mostly failed to explicitly include all of
these relevant constructs. For that reason, the existing models have not either sufficient or
inclusive explanations of the process of IS/IT evaluation in organizations. In contrast, the
29
researcher in this study utilized these existing models, and the findings of the literature review in
order to develop an alternative conceptual model of IS/IT evaluation that included all seven
contextual elements with sufficient or inclusive explanations.
Answer for Research Question #3:
In the study, the researcher wanted to ensure that the proposed conceptual model represented a
“good” theoretical contribution. To that end, the researcher applied Whetten’s (2002) standard
for strong theoretical contributions. To assess whether or not the proposed model of IS/IT
evaluation represented a good theoretical contribution, the researcher explored its descriptive
ability with respect to the previously published case studies. In all of the examined cases, the
researcher established that the conceptual model’s constructs were valid and relevant. Thus, the
researcher concluded that the conceptual model did not contain superfluous constructs.
Therefore, the researcher concluded that the conceptual model was sufficiently complete. The
researcher provided that the model provides a systematic description of the process of IS/IT
evaluation.
Answer for Research Question #4:
Having established the descriptive validity of the conceptual model, the researcher investigated
its application as an investigative tool. In order to accomplish this task, the researcher performed
a cross-case analysis using the conceptual model as a framework, thereby facilitating
comparisons across the various instances. In doing so, the researcher found clear evidence
supporting the assertion.
30
Conclusion for Hypothesis #2:
The researcher noted that there is adequate evidence supporting the second hypothesis in the
study. The researcher’s conceptual model represents an effective tool for both describing and
analyzing evaluation practices. The researcher’s application of the conceptual model to the cases
examined in this study yielded a number of normative guidelines for contextually appropriate
IS/IT evaluation practices.
As the main research questions receive explicit answers and the whole study is well-summarized
in the last chapter, this chapter is also evaluated as “Fully Implemented.”
Table 8: Maturity Table for the 5th Chapter of the Dissertation
Clauses vs. Implementations
Not
Imple
men
ted
Part
iall
y
Imple
men
ted
Larg
ely
Imple
men
ted
Fu
lly
Imple
men
ted
Chapter
1 Introduction
X
Chapter
2 Literature Review
X
Chapter
3 Methodology
X
Chapter
4 Results
X
Chapter
5
Conclusions,
Implications, Recom.,
and Summary
X
31
7 Adoption for My Study
7.1 My Study’s Title and Abstract
Title
Identifying the Managers’ Consciousness about the Importance of Management of Information
Technologies and Its Relation to the Success of Organizations
Abstract
Today, organizations greatly spend money on hardware and software systems or more precisely
on information technologies (IT), grown in power and ubiquity recently. The fundamental reason
for this is that organizations view IT as more and more decisive to their success in the sector. IT
organizations’ chief executive officers (CEOs) and senior managers persistently note that they
interpret IT as a strategic asset to be successful and thereby leader in the sector. However, this is
something problematic. For a business asset to be interpreted as strategic it has to be scarce and it
ought to require exceptional and grounded interpretation and application. Given that IT ubiquity
everywhere, it is not possible to interpret IT as a strategic business asset. Yet, it is possible to do
so for the management of IT. Today’s organizations may create a sustained competitive
advantage by means of management of IT rather than IT of itself.
This thesis aims to identify the CEOs’ and senior managers’ consciousness about the importance
of management of IT and its relation to the organizations’ success in the sector. In this context,
first the 50 IT organizations’ CEOs and senior managers are provided and asked to complete a
questionnaire to know their consciousness about the importance of management of IT and their
organizations’ culture and success. After this, results of these questionnaires are analyzed and
interpreted. In addition, theoretical and practical evidences for this consciousness and its impacts
32
to success are provided and exemplified. As a result, a consciousness frame is proposed for the
IT organizations’ CEOs and senior managers to appreciate the importance of management of IT
rather than IT of itself.
7.2 Adoptions
As my study also requires contextual and conceptual analysis and comparisons, just like
the dissertation that I reviewed, I am to utilize a multiphase approach in my study.
To develop my own IT management consciousness framework survey, I will follow the
similar way that the dissertation’s owner followed in the development of the conceptual
model of IS/IT evaluation to provide better understanding of the process’s entity
constructs and their possible relationships.
For the validation of my framework, I will follow the similar way that the dissertation’s
owner followed the validation of the conceptual model via a meta-analysis and cross-
analysis of multiple case studies and the development of guidelines for conducting
contextual evaluations within particular organizations.
This dissertation, both in content and context will let me continue my study as I also plan
to a similar but not the same occurrence in my thesis.
I also will use the outcome of this study as an input for my thesis as I am to investigate
the IT investments versus IT management consciousness relations.