An assessment of un-structured knowledge management...
Transcript of An assessment of un-structured knowledge management...
ISSN 1744-1986
T e c h n i c a l R e p o r t N O 2007/ 17
An assessment of un-structured knowledgemanagement techniques in the projectmanagement of software development
E. Aitken
28 June, 2007
Department of ComputingFaculty of Mathematics, Computing and TechnologyThe Open University
Walton Hall, Milton Keynes, MK7 6AAUnited Kingdom
http://computing.open.ac.uk
An assessment of un-structured knowledge management techniques in the project management of
software development
A dissertation submitted in partial fulfilment of the requirements for the Open University‘s
Master of Science Degree in Computing for Commerce and Industry
Elaine Aitken (T9915646)
5 March 2008
Word Count: 16100
Preface
I wish to thank the employers who have supported me throughout the courses that
have led up to this dissertation: ts.com, OUP and RM.
The support of my supervisor, Dr David Butts, was essential in helping me sort out
what I was trying to achieve. His responses were always timely and helpful.
The contributions from the participants who responded to my questionnaires were
invaluable. I am grateful to them all, and particularly to those who hung in there, right
to the end.
Finally, I would like to thank Helen Charlesworth for being such a fabulous sounding
board and for her patience and support.
ii
Table of Contents
Preface ............................................................................................................. i
List of Figures ................................................................................................. v
List of Tables .................................................................................................. vi
Chapter 1 Introduction ....................................................................................... 8
1.1 Background to the research ..................................................................... 9
1.1.1 Technology ...................................................................................... 10
1.1.2 Human factors ................................................................................. 11
1.2 Knowledge management in software development ............................... 11
1.3 Knowledge management and project management ............................... 12
1.4 Aims and objectives of the research ...................................................... 13
1.5 Overview of the dissertation .................................................................. 13
Chapter 2 Literature Review ........................................................................... 15
2.1 The historical perspective ...................................................................... 15
2.2 Data, information, knowledge – the degrees of knowing ....................... 16
2.3 Organisational knowledge ..................................................................... 17
2.4 Knowledge capture ................................................................................ 20
2.5 Knowledge transfer ................................................................................ 21
2.6 Making the most of tacit knowledge ....................................................... 23
2.7 Finding the gaps in current processes ................................................... 26
2.8 Research question ................................................................................. 27
2.9 Summary ............................................................................................... 28
Chapter 3 Research Methods ......................................................................... 29
3.1 Other techniques considered ................................................................. 29
3.1.1 Questionnaire .................................................................................. 29
3.1.2 Case study ...................................................................................... 30
3.1.3 Interviews ........................................................................................ 30
iii
3.2 The Delphi technique ............................................................................. 30
3.2.1 Participants ...................................................................................... 31
3.2.2 Stage 1: Gathering initial contributions ............................................ 32
3.2.3 Stage 2: Collation ............................................................................ 34
3.2.4 Stage 3: Refinement 1 ..................................................................... 34
3.2.5 Stage 4: Refinement 2 ..................................................................... 35
3.2.6 Stage 5: final data collection ............................................................ 35
3.3 Analysis of existing solutions ................................................................. 36
Chapter 4 Data Collection ............................................................................... 37
4.1 Data Sources ......................................................................................... 37
4.2 Process .................................................................................................. 37
4.3 Preliminary Analysis .............................................................................. 37
4.4 Analysing the ranking ............................................................................ 38
Chapter 5 Results ........................................................................................... 41
5.1 Background information ......................................................................... 41
5.2 Delphi method data ................................................................................ 42
5.2.1 What are the benefits of knowledge management? ........................ 42
5.2.2 What techniques are currently used? .............................................. 44
5.2.3 What other techniques are you aware of but have not personally used?............................................................................................... 47
5.2.4 What types of knowledge have been hardest to capture and share effectively? ...................................................................................... 47
5.2.5 Wildcards ......................................................................................... 50
5.3 Results of Delphi process ...................................................................... 50
5.3.1 The connection between ambition and practice .............................. 52
5.3.2 Gaps in current practice .................................................................. 53
5.4 Comparison with the literature ............................................................... 54
5.5 Analysis of practice vs literature ............................................................ 58
5.5.1 Patterns ........................................................................................... 59
iv
5.5.2 Document management and collaboration software ....................... 62
5.5.3 Storytelling ....................................................................................... 64
5.5.4 Semi-structured interviews .............................................................. 66
5.5.5 Rich personal interaction ................................................................. 68
5.5.6 Estimates for future work ................................................................. 70
5.5.7 Summary of proposed techniques ................................................... 71
5.6 Validation ............................................................................................... 72
5.6.1 Analysis ........................................................................................... 73
Chapter 6 Conclusions .................................................................................... 75
6.1 Project review ........................................................................................ 76
6.2 Future research ..................................................................................... 76
References 78
Bibliography ..................................................................................................... 82
Index ............................................................................................................ 83
Appendix A: Delphi method initial questionnaire results ................................... 84
Appendix B: First refinement ............................................................................ 93
Appendix C. Second Refinement ................................................................... 102
v
List of Figures
Figure 1. Process of knowledge conversion. Nonaka & Takeuchi (1995). ....... 15
Figure 2. Questionnaire sent to participants in the Delphi method. .................. 33
Figure 3. Connections between intended benefits and practice. ...................... 53
Figure 4. Connections between problem areas and practice. .......................... 53
Figure 5. Current knowledge management techniques categorised in terms of the scope of their relevance across projects and the level of detail they capture. ........................................................................................... 55
Figure 6. The gaps in current practice categorised in terms of the scope of their relevance across projects and the level of detail they require. ........ 57
vi
List of Tables
Table 2.1. From Crossan et al. (1999). Processes involved in knowledge transfer at different levels of an organisation. .................................. 19
Table 2.2. Knowledge transfer frictions (from Davenport & Prusak, 1998). ...... 24
Table 5.1. Level of maturity of organisational knowledge management approach. ....................................................................................... 41
Table 5.2. Technical sophistication of organisational knowledge management approach. ........................................................................................ 41
Table 5.3. Personnel structure with respect to knowledge management. ........ 42
Table 5.4. Responses for list 1 ranked by total marks received, with comments added by participants. Benefits of knowledge management............ 43
Table 5.5. Responses for list 2 ranked by total marks received, with comments added by participants. Commonly used knowledge management methods. .......................................................................................... 45
Table 5.6. Collated responses to Question 6 of questionnaire. ........................ 47
Table 5.7. Responses for list 3 ranked by total marks received, with comments added by participants. Hard to share knowledge. ............................ 48
Table 5.8. Literature sources of research into knowledge capture. .................. 58
Table 5.9. Matrix showing which gaps in knowledge capture identified by the Delphi method are addressed by which techniques from the literature. ......................................................................................... 59
Table 5.10. Matrix showing which of the current techniques are extended or complemented by the additional techniques found in the literature. 59
Table B.1. Reponses to the first refinement of List 1. ....................................... 99
Table B.2. Reponses to the first refinement of List 2. ..................................... 100
Table B.3. Reponses to the first refinement of List 3. ..................................... 101
vii
Abstract
Those involved in managing projects are acutely aware of the benefits that could be
achieved if knowledge could be transferred between projects and teams. There are
few organisations, however, that support structured organisation-level management
of knowledge. This study looks at the knowledge management methods employed
by individuals in organisations that do not have well-defined or well-embedded
knowledge management processes. The study finds that the techniques used most
often are those that are recommended by Project Management methodologies, such
as post-project reviews and lessons learned reports. It was also found that there is
an interest in using some of the recently available collaborative, de-centralised
technologies such as blogs that can be implemented on a small scale without
necessarily having to be run at a corporate level.
Through responses to a questionnaire and the application of the Delphi method, the
participants in this research agreed on a number of areas where their own current
practice does not meet their needs for knowledge transfer. The techniques currently
being employed were limited in their ability to share knowledge effectively and
promote re-use. An analysis of existing research shows that the following techniques
may improve current practice:
The use of patterns, analogous to those used in software development
Blogging and other forms of collaboration software
Using metrics to get quantifiable information
A more narrative approach to knowledge capture, using storytelling to convey
key points
The use of oral history capture by semi-structured interview
All of these are found to provide the possibility of improved knowledge transfer
across project teams.
8
Chapter 1 Introduction
The field of knowledge management is broad. It covers the creation, capture,
transfer and use of knowledge within an organisation and can be approached from
both a technology-centric and people-centric point of view. Regardless of which
approach an organisation favours, the value of good knowledge management is
summed up by Lew Platt, former CEO of Hewlett-Packard (quoted in Davenport &
Prusak, 1998):
―If HP knew what HP knows, we would be three times as profitable.‖
Nonaka (1991) cites a number of companies who have embraced formalised
knowledge management, including Canon, Honda, NEC and Sharp. As evidence
that knowledge management is not restricted to successful Japanese companies,
Brand (1998) states that ―3M‘s objective is to become the most innovative company
in the world‖. A key to this strategy is to create an environment that encourages a
willingness to share knowledge between individuals.
Despite an acceptance from organisations that we are now in an era when
knowledge is a valuable resource, knowledge management is often not formally
implemented at an organisation level. In many organisations, individual project or
team managers are often responsible for making sure that the knowledge available
in an organisation is used most effectively on their projects. One of the fields that
makes extensive use of project managers in this way is that of software
development. Companies who have traditionally not been in the software business,
such as publishers, find that they are now a part of the software market and have to
develop the skills required to compete. Innovation is critical to success in this market
and knowledge management has been shown to contribute towards successful
9
innovation (Faniel & Majchrzak, 2006) and maintainability of software (Anquetila et
al., 2006). Given this, it is valuable to find what techniques are being used by project
and team managers in organisations such as these – where software development is
key to their success but the organisation as a whole does not implement formal
knowledge management processes.
1.1 Background to the research
In order to understand the ways in which knowledge can be managed and
transferred, it is beneficial to understand more about the nature of knowledge itself.
In the work of Nonaka (1991) and Nonaka & Takeuchi (1995), the foundation of the
field of knowledge management was laid. They highlighted the difference between
tacit and explicit knowledge, and the benefits that come from moving knowledge
between those two states. Tacit knowledge is held in people‘s heads, and is often
taken for granted by the holder. Explicit knowledge is what we know we know, and
can be easily described, shared or learned. A particular focus of the work of Nonaka
& Takeuchi is the innovation that can arise when efforts are made to share tacit
knowledge and explicit knowledge is created. Case studies in their work illustrate the
techniques used in Japanese companies to share understanding and mental models
through the use of metaphorical and visual language. The point is made that this has
not historically been a mode of communication that has been understood or valued
in the West.
Those interested in developing the practice of knowledge management have
approached the field from two sides:
1. The study of how technological systems can help with capturing, codifying
and delivering knowledge.
10
2. The study of the human factors that affect the creation and absorption of
knowledge.
1.1.1 Technology
A wide range of technological solutions have been proposed and implemented that
have attempted to improve the knowledge management process.
Some examples are:
1. Video-nets, that allow real time and asynchronous access to video and voice
contact between members of a community, with the aim of building up the
body of knowledge held by veterinarians in towns and villages across Japan.
(Kodama, 2002)
2. Bespoke artificial intelligence systems, with context-specific ontologies written
to provide a structure for the knowledge bases that are the foundation of the
means of knowledge-retrieval (O‘Leary, 1998).
3. Web-based groupware to encourage collaboration and communication and
thus gain competitive advantage (Sipcic & Makonnen, 1998).
The investment in these technological systems has been considerable and their
scale is such that historically they could only be implemented once an organisation
was prepared to make the investment and implement a system across an entire
organisation. They were corporate solutions.
11
1.1.2 Human factors
For all the investment in them, these systems cannot produce results by themselves.
They require knowledge to be gathered, shared, adopted and absorbed. For this to
happen, those who need the information also need to know that it exists, they have
to go and find it, they have to trust it when they find it and they have to be able to
extract some relevant learning points from it. Whether or not users adopt information
once they have found it has been shown to depend on the extent to which they were
able to see it as a starting point, ready for adaptation, rather than a proscriptive
blueprint (Faniel & Majchrzak, 2006).
Where knowledge is explicit and easily codified it might be quite straightforward to
capture it, and it might be obvious to users what they need to do to find it again.
Where knowledge is tacit, the person with that knowledge might not even know that
it is significant and the best way to share it might be by a demonstration, or an
apprenticeship or an anecdote. There is therefore a very human aspect to tacit
knowledge sharing.
1.2 Knowledge management in software development
Software development is one of the areas that have sought to make use of
knowledge management techniques. It has been shown to bring improvements in
innovation (Faniel & Majchrzak, 2006), quality (Robillard, 1999; Tiwana, 2004) and
maintainability (Anquetila et al., 2006). These papers show that the gathering and
use of knowledge is important at all stages of software application development.
Faniel & Majchrzak (2006) found that innovation was more likely if developers are
exposed to knowledge that is new to them and different to what they already know.
Robillard (1999) emphasises the need for a good design if the output of the software
12
development process is to be of a high quality. Both innovation and good design are
facilitated if the available body of knowledge is consulted at the start of the
development process.
The field of software development already contains many ways of specifying
knowledge – design documents, requirements documents and code documentation
for example. These are all ways of capturing and sharing what people know but they
all tend to refer to a particular problem or project and are not often generalised. Code
libraries contain snippets of code designed to solve a very specific problem that can
be reused across many projects but this is an example of sharing and reusing the
easily codified knowledge.
1.3 Knowledge management and project management
Knowledge management has a special relationship with project management in
terms of both knowledge creation and knowledge use. There is much research that
deals with the importance of communities of knowledge, networks (Bresnen et al.,
2003) and trust (Ji-Hong Park, 2006) in the ease with which knowledge can be
shared and the value associated with it. This work recognises the human aspects of
knowledge transfer - that people feel that lifting solutions from other people stifles
their creativity, that even if information is available it takes a particular environment
for people to be open to accepting it. Particularly in new product development, where
innovation is especially important, then personal interaction, information sharing,
redundancy and the importance of shared mental models have been found to be
very valuable (Madhavan & Grover, 1998).
There are many outcomes of projects that can be hard to make tangible – the
lessons learned that those involved will take with them into future projects. Project
13
management methodologies advocate the sharing of information through end of
project reports and lessons learned reports, but these might not be sufficient to
capture all types of knowledge needed in projects. This study will look at what those
involved in the field are currently doing to capture lessons learned and will focus in
particular on where the gaps are in the types of knowledge that are captured.
1.4 Aims and objectives of the research
The aims of this study were to establish how project managers use knowledge
management techniques and to suggest how this could be improved. The study used
a panel of project managers who were experienced in software development. The
objectives were:
1. To identify what techniques project managers use to capture knowledge in
software projects.
2. To establish the perceived benefits of knowledge management to the
participants‘ organisations.
3. To identify where project managers feel these techniques are lacking.
4. To identify techniques that could be used at a project-level to improve
knowledge capture and transfer in the areas where it is shown to be lacking.
1.5 Overview of the dissertation
Chapter 2 is a review of the existing literature on knowledge management. It
provides an overview of the field of knowledge management, but focuses on what
existing literature offers to those who have a need to share knowledge.
14
The research approach is described in Chapter 3. This includes the details about the
Delphi method participants, the questions asked of them and how the results were
refined. Chapter 3 also discusses the iterative way in which the findings from the
Delphi method were used to inform the analysis of the literature. Finally it explains
how the results from the Delphi method and those from the literature analysis were
combined to find the techniques that looked most promising in addressing the needs
of the project managers.
Chapter 4 explains the process of data gathering and analysis but the detail of the
results found and the analysis that was carried out is expanded in Chapter 5.
Chapter 5 also contains the details of the findings from the analysis of the literature
and the comparison of these findings with the conclusions from the Delphi method.
Conclusions are summarised in Chapter 6 and the success of the project is
reviewed.
15
Chapter 2 Literature Review
2.1 The historical perspective
The foundation for the specialism of knowledge management is in the work of
Nonaka & Takeuchi (1995). This work considers the difference between explicit and
tacit knowledge, and the ways in which knowledge can move between those two
states and between individuals. This distinction is crucial when considering
knowledge management and knowledge transfer techniques. Figure 1 shows these
processes as described by Nonaka & Takeuchi.
Tacit to Tacit
SOCIALIZATION
Tacit to Explicit
EXTERNALIZATION
Explicit to Tacit
INTERNALIZATION
Explicit to Explicit
COMBINATION
Figure 1. Process of knowledge conversion. Nonaka & Takeuchi (1995).
Although the 1990s saw the development of knowledge management as a discipline,
the study of knowledge dates back to the earliest days of philosophy, and the
approaches to knowledge developed then continue to shape the field today.
Plato drew a distinction between the intelligible (something that can be understood
and defined) and the sensible (something that can be experienced by our interaction
with it). Cartesian dualism separated the mind and matter: introspection and
experience. In time, the historical Western philosophical split of mind and body in the
understanding of knowledge was challenged by philosophers such as Immanuel
Kant. In Critic of Pure Reason, Kant (1787) defines knowledge as being born from
experience. Kant states:
16
―Without consciousness, that which we think is the very same as what we
thought a moment before.‖ p.230.
The Japanese view of knowledge has long adopted this view of experience providing
something more than just the intelligible. It has embraced a ―oneness of humanity
and nature‖ (Nonaka & Takeuchi, 1995): the oneness of body and mind. Nonaka &
Takeuchi argue that this is intrinsic even in the Japanese language, where the lack
of verb conjugation makes it difficult for a speaker to personalise their thoughts and
feelings.
The practical outcome of this difference between historical Western and Japanese
theories of knowledge is that the Japanese are traditionally more comfortable with
tacit knowledge gained and shared through experience whereas the West gives
more respect to codified knowledge learned by knowing and reason.
2.2 Data, information, knowledge – the degrees of knowing
Another perspective on the debate about what constitutes knowledge comes from
the distinction between data, information and knowledge. Clarke & Rollo (2001)
define each term as:
―Data
Data are sets of discrete objective facts, presented without judgement or
context. Data become information when they are categorised, analysed,
summarised and placed in context, becoming intelligible to the recipient.
Data relate to the actual bits and characters in an information system or in the
other physical manifestations of communication such as sound and
temperature.
Information
Information is data endowed with relevance and purpose. Information
develops into knowledge when it is used to make comparisons, assess
consequences, establish connections and engage in dialogue.
17
Information is data in context that can be used for decision making. Data are
usually arranged to provide meaning to the observer. Typically it is arranged
as text but could be an image, a film clip, a conversation with another person
or even a busy signal on a phone line.
Knowledge
Knowledge can be seen as information that comes with insights, framed
experience, intuition, judgement, and values. In some sense knowledge
represents truth and therefore offers a reliable basis for action.
Knowledge is the body of understanding and skills that is mentally constructed
by people. Knowledge is increased through interaction with information
(typically from other people).‖
These descriptions are an elaboration on the work of Ackoff (1989) who also
defines:
―Understanding: appreciation of ‗why‘
Wisdom: evaluated understanding. ―
Ackoff does not just make a linguistic distinction between these concepts. He also
applies a hierarchy to them. Each concept builds on the one before it, and wisdom is
the more evolved, valuable and permanent form of knowledge. This distinction is
repeated throughout the literature, with a consensus that knowledge is a much
richer, more valuable resource than simple data. It requires a context, some insight
and other such attributes that can only be achieved when information is processed
by humans.
2.3 Organisational knowledge
The belief that the real value of knowledge comes when it is exposed and transferred
was crucial to understanding the role of the organisation as a facilitator of that
transfer (Grant, 1996). Grant recognises the individual as the knowledge generator
and believes that it is the role of the organisation to facilitate the use of that
knowledge more widely.
18
Re-using the experience of others could be made easier if that experience was
sufficiently generalised to widen its relevance and left open enough to allow the
developers to apply their own interpretation and use their creativity in shaping it to
best fit their project. Faniel & Majchrzak (2006) looked at some of the issues around
knowledge management systems. These systems were intended to provide
engineers with access to knowledge from other departments. Faniel & Majchrzak
found that engineers valued the knowledge the most when they saw it as a stepping
stone towards their own solutions, and when they could trace it back to an expert in
the field. An organisation can share knowledge more effectively if they can provide
an overview in the first instance and access to more specific detail if needed. Access
to the expert who could be the source of the detail was highly valued. This approach
provides the organisation with the role of sharing what knowledge is available, but
still values the role of the individual in sharing what they know and not just sharing
what can be codified in the knowledge management system.
Crossan et al. (1999) describe the three learning levels (individual, group and
organisation) and four processes that move knowledge up the chain towards
organisational learning.
Table 2.1 shows how the processes relate to the levels.
19
Table 2.1. From Crossan et al. (1999). Processes involved in knowledge transfer at different
levels of an organisation.
Level Process Inputs/Outcomes
Individual
Intuiting
Experiences
Images
Metaphors
Interpreting
Language
Cognitive map
Conversations/dialogue
Group Integrating
Shared understandings
Mutual adjustment
Interactive systems
Organisation Institutionalizing
Routines
Diagnostic systems
Rules and procedures
This table shows clearly that the features of knowledge at an individual level are
referred to in far more creative terms than those at the organisational level. As
knowledge is shared between more and more people, it seems to become more
codified, less flexible, less innovative, less personal. Referring back to the
terminology of Ackoff, the organisational knowledge is more similar to what Ackoff
would describe as data or information than it is to understanding or wisdom. In the
terminology of Nonaka & Takeuchi, individual knowledge can be held tacitly and
organisational knowledge is more explicit.
It would seem to be self-evident that having knowledgeable individuals is immensely
valuable. Research has shown that getting the right people on a project is one of the
best things you can do to improve its chance of success (Graham, 2000). There is
some knowledge that simply cannot be held at an organisational level but the
20
organisation is still important in facilitating the sharing of knowledge, and
encouraging an atmosphere of collaboration and data gathering. Staff retention is a
key way of getting a body of knowledge in an organisation, but it is limited and does
not in itself bring any of the innovation that can result when knowledge is shared.
Tsoukas & Vladimirou (2001) look at what organisational knowledge is, and conclude
that it can be said to exist when "individuals draw and act upon a corpus of
generalizations in the form of generic rules, produced by the organization". One
would struggle to argue that the organisation could be said to have wisdom, or
understanding. Organisational knowledge requires individuals to make sense of it
and adopt it as their own.
2.4 Knowledge capture
There has been a great deal of work in assessing methods of knowledge capture,
often based around the technology employed to do this such as databases,
knowledge management systems and portals.
Henninger (2001) concludes that there is a strong tie between the technology used
to capture knowledge and the processes of knowledge management within an
organisation. Faniel & Majchrzak (2006) found that individuals are more likely to
innovate and generate new ideas if they are exposed to new knowledge. They state
that it is not sufficient to expect individuals to seek out the knowledge they need –
there must also be an element of pushing new concepts out to people who did not
realise they were relevant. They see technology as serving a purpose here, but
advocate a system that associates the personal with the concrete.
Liebowitz & Megbolugbe (2003) provide a table that lists seven knowledge
management solutions. The first of these, ―Frequent get-togethers to exchange tacit
21
knowledge (i.e. knowledge fairs, brown bag lunches, bird of a feather tables, inter-
departmental seminars, etc.)‖ is rated as being low in complexity of use and low in
difficulty of development. This would suggest that any organisation that values
knowledge management should have this process well-established.
Each of the other six suggestions involve a substantial technology overhead, ranging
from chat rooms and online communities to ―Using intelligent agents to actively build
user profiles and push appropriate lessons learned and material to the respective
user‖. In the past, a system such as this would require a substantial organisation-
wide commitment to its development, implementation and on-going use. With the
development of technology such as blogs and wikis, there is now a much lower entry
point into some of these applications. New freely-available collaborative solutions
would not have all of the features of fully customised knowledge management
groupware but they can be owned and implemented at a group level and may be
more successful in encouraging the sharing of knowledge for that very reason.
2.5 Knowledge transfer
Despite improvements in technological options, researchers have found that
interactions with colleagues are still one of the most effective methods of sharing
knowledge (Kasvi et al., 2003). Bresnen et al. (2003) confirm the importance of
strong social networks and the role of the individual as the key holder of tacit
knowledge. They raise two key questions:
―First, how is the organisation able to capture learning and deploy it over the
long term, when it is so embodied in the individual and manifested in their
particular expertise and range of contacts? Second, what happens when the
individual leaves and takes their knowledge and contacts with them?‖
22
Information redundancy is the solution to the second of those issues – making sure
that no one person holds a unique body of knowledge – and this problem is
addressed in many different ways by those who have detailed a variety of methods
for knowledge capture.
Davenport & Prusak (1998) cite the example of organisations who:
―record the stories and experiences of its senior practitioners on video or CD-
ROM before they leave the company.‖
Capturing this kind of anecdotal information adds it to an organisational memory.
Technology can then be used to codify and publish it, and make it available to those
who need to use it. Davenport & Prusak make the distinction between using
technology to share the information, and using technology to represent the
information itself. This is a particular issue if the knowledge is tacit and has a
richness that the technological approach can not do justice to.
Reflection (Henninger, 2001), debriefing, surveys, (Collier et al., 1996), Ishikawa
(fishbone) diagrams (Birk et al., 2002) are a few of the more creative methods
mentioned. These can contribute to the sharing of knowledge, but as Davenport &
Prusak point out in Working Knowledge (1998):
―Knowledge that isn‘t absorbed hasn‘t really been transferred‖
This draws a clear distinction between knowledge management and knowledge
transfer. Whilst technology can be used to codify and manage knowledge, and make
it easier for users to retrieve the information they are looking for, this does not
actually spread knowledge. Only once the end user has adopted that information and
preferably used and personalised it, can the benefits of that knowledge management
system be realised, i.e. once knowledge (not just information) has been transferred
(not just captured).
23
Knowledge management can also support reuse in software development, which in
turn leads to improved productivity, performance and quality (Tautz & Althoff, 1997).
Basili & Rombach (1991) state that in addition to the traditional emphasis on reusing
source code, organisations should see all experience as being reusable. They
emphasise the need to capture feedback of experiences for use in future software
development projects, and describe feedback as occurring where an organisation
has a way of recording and packaging learning, and identifying, evaluating and
modifying that learning through reuse.
Desouza & Awazu (2005) list the five capabilities that an organisation must have in
order to claim success in knowledge management: create, transfer, store, retrieve
and apply. Again, there is a focus on what an organisation can achieve. This study
will review what issues practising project managers have with knowledge
management, and whether there are any of these 5 capabilities that prove
particularly difficult.
2.6 Making the most of tacit knowledge
As Marwick (2001) finds, current solutions are strongest when they deal with explicit
knowledge. The creation and sharing of tacit knowledge is weaker. As Marwick
points out, tacit knowledge by its nature is difficult to make explicit therefore
externalization is the hardest to achieve of the four defined forms of knowledge
conversion.
Ambrosini & Bowman (2001) also find that one of the reasons why there have been
very few attempts to empirically research tacit skills is that it is problematic. They find
a need for a method that would allow us to capture the constructed reality of
individuals which allows them to make sense of the world around them.
24
Davenport & Prusak (1998) highlight the factors that encourage knowledge
exchange – reciprocity, repute and altruism. They also state that trust is an essential
factor and that a culture of rewarding sharing and discouraging knowledge hoarding
is also needed. This is also supported by Ji-Hong Park (2006).
The factors that work against knowledge transfer (frictions), as described by
Davenport & Prusak (1998) are given in Table 2.2 below.
Table 2.2. Knowledge transfer frictions (from Davenport & Prusak, 1998).
Frictions Possible solutions
Lack of trust
Build relationships and trust through
face-to-face meetings
Different cultures, vocabularies, frames
of reference
Create common ground through
education, discussion, publications,
teaming, job rotation
Lack of time and meeting places; narrow
idea of productive work
Establish times and places for knowledge
transfer: fairs, talk rooms, conference
reports
Status and rewards go to knowledge
owners
Evaluate performance and provide
incentives based on sharing
Lack of absorptive capacity in recipients
Educate employees for flexibility: provide
time for learning; hire for openness to
ideas
Belief that knowledge is prerogative of
particular groups, not-invented-here
syndrome
Encourage non-hierarchical approach to
knowledge; quality of ideas more
important than status of source
Intolerance for mistakes or need for help Accept and reward creative errors and
collaboration; no loss of status from not
knowing everything
The above findings show that:
Tacit knowledge is hard to capture.
25
There is a strong human element to the willingness of individuals to accept
other lessons as their own.
A shared sense of the world around us is important in being able to
understand and apply knowledge in an appropriate way.
These issues highlight the importance of a field of study that ignores the potential of
technology to manage knowledge and focuses on the potential of individuals to
creatively describe their experiences in a way that allows others to draw lessons
from them.
Something that is apparent from these suggested solutions is that they require a
culture change within an organisation. Something this dissertation aims to do is to
show which of the suggested improvements to knowledge management can be
achieved without extensive organisation support.
Rising & Derby (2003) use the metaphor of programming patterns to illustrate how
learning can be captured and generalized. They suggest that project retrospectives
are used to capture the lessons learned and that when a sufficiently generalizable
lesson is encountered, it is framed as a pattern and made available for future
reference. The key features of a pattern are: Name, context, problem, forces,
rationale and solution. They also suggest that a pattern should have been shown to
be relevant in three different settings before it is considered valid. They consider it
vital that a pattern has a short, descriptive, attention-grabbing title so that it can be
easily encapsulated and adopted by other teams. A review of existing patterns can
then become a part of project planning.
Madhavan & Grover (1998) advocate rich personal interaction as a key ingredient of
effective knowledge creation. Their context is new product development.
26
Brown et al. (2001) have suggested storytelling as a method of capturing the tacit
and explicit knowledge at project review points, and doing so in a form that can be
used by others. They believe that stories can provide a mechanism of sharing
knowledge between communities and can provide a sufficiently generalized context
to increase the perceived relevance of the knowledge being shared.
Clarke & Rollo (2001) give examples of where the codification of tacit knowledge has
been successfully achieved and look at the importance of the flow of that knowledge
being embedded into an organisation‘s processes.
Neve (2003) proposes a framework of semi-structured interviews to obtain a
narrative description that can tap in to the knowledge that participants did not know
they had.
Some of these techniques could be as valid for sharing explicit knowledge as tacit
knowledge. This study looks at the main issues identified by the participants and
considers how techniques such as these could enhance any of their identified
problems.
2.7 Finding the gaps in current processes
Another area of research in preparation for moving forward with this project has been
into the best way to compile that list of the issues that currently exist with knowledge
capture in software development. The nominal method has been used for this sort of
purpose. Input from a number of experts in the field is sought and can be combined
through a series of iterations into an agreed list. Van De Ven & Delbecq (1974) found
that this technique, or the Delphi technique where participants do not need to be in
the same location, produced a better result than a traditional group discussion. De
Ruyter (1996) found a similar result in a comparison of the Delphi method and focus
27
group interviews. The added benefit of the Delphi method is that it allows the
participants to remain anonymous. In this case, this will allow access to managers
from companies that work in a similar field and are in some cases competitors,
without them feeling inhibited from sharing their problems and successes.
In a particularly relevant paper, Dekleva (1992) used the Delphi technique to compile
a list of the problems in software maintenance. These included people being stuck in
a role because the organisation could not afford to lose their experience, inadequate
documentation, and high turnover of staff causing a loss of expertise.
2.8 Research question
This study aims to address the following question:
What techniques can be used to improve the practice of knowledge management in
software development projects, where the organisation as a whole does not provide
strong guidance?
It addresses the question in three parts:
What processes are currently used in management of software development
to capture knowledge gained
Where are existing processes lacking?
What frameworks or methods could provide help in the areas where current
processes are lacking?
28
2.9 Summary
The study of knowledge management can be seen to encompass a wide range of
disciplines, from the philosophy of knowledge and education through to advanced
computer system development. It involves the study of human interactions, the role
of the organisation and optimisation of business practices. At its root, however, is a
belief that sharing knowledge is a worthwhile pursuit, and that knowledge itself is a
resource that organisations should be taking seriously and investing in. Whilst the
organisation can play a very important role in introducing systems to facilitate the
sharing of knowledge it is perhaps more important for it to foster an environment
where the sharing of knowledge is valued and encouraged. The literature places
emphasis on the importance of organisation involvement in knowledge management,
so it was important for this study to take this into account.
29
Chapter 3 Research Methods
The research was split into two complementary tasks. In the first task, the input from
industry experts on their experience of knowledge management was gathered. This
was intended to discover what knowledge management techniques these experts
use as a matter of course, and where these techniques fall short.
In parallel with this, an analysis of the existing methods and frameworks suggested
by the literature was carried out. In particular, there was a focus on the means of
externalising knowledge, to assess how useful they might be in addressing some of
the problems identified by the experts on the panel.
3.1 Other techniques considered
The objective of this part of the study was to gather input from a number of subject
experts about their experiences. Although the Delphi technique was selected, other
options were considered:
3.1.1 Questionnaire
The participants were not knowledge management professionals, although they were
knowledge workers. It was thought that they might have differing views on what
knowledge management might mean, and that a simple questionnaire would not
allow them to feed off each other‘s experience. It was desirable to provide the
participants with an opportunity to review their own suggestions and those of others
when they were asked to provide a ranking.
30
3.1.2 Case study
It would also have been possible to look in detail at a smaller number of projects by
performing a case study. Every project has its own characteristics, however, and the
study was more concerned with the general approaches to knowledge management,
rather than what has been applied in one case. One of the very issues that the study
investigated was how to generalise learning from one project so that it applies across
many.
3.1.3 Interviews
Interviews could have been a useful part of the process – it may have been valuable
to have gathered the initial suggestions from participants in this way. Interviews in
themselves would not have been sufficient as there was great benefit to be gained
from the iterative nature of the Delphi technique.
3.2 The Delphi technique
The process of compiling the list was carried out using the Delphi technique. In this
technique, an expert panel is formed who are then asked for their input on a given
topic. The Delphi technique has been found to be more effective than a standard
focus group when trying to gather suggestions from a group of people. Van De Ven
& Delbecq (1974) found that the number and quality of ideas generated was higher
than in a focus group, and that the members of the group were more satisfied with
the outcome.
In the particular circumstances of this research, the Delphi technique is favoured
over the nominal technique because of the practicalities of getting the expert panel
from a variety of organisations together. The advantages of this approach is that
31
members can commit the time when they have it available, and there is less chance
of participants censoring themselves or being affected by the dynamics of the group.
The potential disadvantage is the extended length of time that the Delphi process
can take, compared with the nominal group method. This was in fact advantageous
in this case, as it allowed for research into the literature to be carried out in parallel to
the gathering of expert opinions.
3.2.1 Participants
The panel consisted of 11 people with current experience of managing the
development of software development projects. It included:
Managers of software development teams
Internal customers of software development teams
External customers of software development teams
Project managers who liase between customers and suppliers
10 of the 11 participants were based in the UK and 1 in India. They worked for 5
different organisations.
It has to be recognised that the group who are involved with the Delphi method for
this dissertation are an opportunity sample and from a narrow field of work – that of
the management of software development. Any findings will need to be considered
as representative only of that context.
The participants were assured of anonymity in their responses. This was particularly
important for this group, because within the group some of the participants were
32
involved in a commercial supplier-customer relationship and some in a competitive
relationship. It would have been inappropriate to have circulated information to all
participants that could be traced back to particular projects and could influence one
participant‘s view of another.
3.2.2 Stage 1: Gathering initial contributions
The first step of the Delphi process (having secured the involvement of the panel)
was to ask participants to complete a short questionnaire. The first half of the
questionnaire asked for input on a number of areas of knowledge management and
transfer.
In the second half of the questionnaire, participants were encouraged to list as many
examples as they thought relevant in the given categories. These responses to those
questions formed the foundation of the rest of the stages of the Delphi method. The
questionnaire is shown in Figure 2.
33
Figure 2. Questionnaire sent to participants in the Delphi method.
The responses supplied by the participants are given in Appendix A.
34
3.2.3 Stage 2: Collation
Once the suggestions and comments had been received, these were collated.
Where similar suggestions were given by multiple participants, these were
combined. The set of responses to three of the questions were taken forward to the
third stage of the process.
3.2.4 Stage 3: Refinement 1
The collated responses to the three key questions were sent out with a request that
participants rank and comment on them. The ranking was to be achieved by
allocating 50 points across all the items in a list, giving most marks to items that were
considered most relevant and fewer or no marks to those items that were considered
less relevant. Some editing had been necessary to combine similar responses, and
to remove company-specific information. This means that it was possible that
participants would feel that their original points were no longer reflected in the
collated lists. It was made clear to participants that they could elaborate on, disagree
with or otherwise refine the list.
The questionnaire sent out for this stage and the numerical responses received are
shown in Appendix B.
When the responses were received, the total marks allocated to each item on the list
were summed and the responses were ranked according to the total number of
marks received, from highest to lowest. Further details of the reasoning behind using
this approach to achieve the ranking are given in Section 4.3.
35
It was decided that the aim was to take between 6 and 8 items from each list forward
for further analysis and the purpose of the refinement stage in the Delphi method
was to compile the 6-8 items.
3.2.5 Stage 4: Refinement 2
The participants were informed that the top 6 items in each list would be taken
forward into the later stages of the study. They were given the list showing the
rankings from 1 to 6.
The participants were also provided with the lists of items that did not make the top 6
and were therefore going to be considered only peripherally in subsequent stages of
the study. They were asked to choose one item from this list as their ―wildcard‖, and
give a reason why it should be carried forward with the top 6. The intention here was
partly to provoke comments on what had been left out and secondly to check that the
top 6 was not missing any items that a number of people felt were important. There
was scope within the plans to take up to two of the anticipated eight wildcard
suggestions forward if it was felt they would contribute anything interesting to the
subsequent research.
The material sent out for this stage is show in Appendix C.
3.2.6 Stage 5: final data collection
The responses were received and the comments and suggested additional
inclusions were considered. The participants returned their questionnaires with their
suggestion of which of the wildcard entries should be considered further. The
36
intention was to see if there was any consensus about which entries to add back in
for consideration. This was a subjective process.
This was the final stage of the Delphi consultation with the panel and resulted in:
6 benefits that knowledge management brings to an organisation
6 of the most consistently used knowledge management techniques
6 areas where current knowledge management practice is failing to capture
the necessary information.
These items were then taken forward for consideration in the rest of the study.
3.3 Analysis of existing solutions
Running in parallel to the Delphi process, the existing literature was assessed to find
techniques that have been used to address the issues highlighted by feedback of the
participants. Section 5.5 provides details of the result of this assessment.
The aim of this study, as previously stated, is to find where existing research can be
used to help address the problems experienced by those trying to make use of
knowledge management techniques.
The solutions offered by the literature were mapped on to a grid that showed their
scope and their level of detail. This was a subjective process.
A more rigorous approach may have been to ask the participants to provide the
mapping to the grid. This additional validation was outside the scope of this study.
37
Chapter 4 Data Collection
4.1 Data Sources
The data required for this dissertation was obtained from a panel of project
managers working in the field of software development. They were all known to the
researcher. Eleven people agreed to participate. At each stage, the people who had
not responded to the previous stage played no further part in the research.
4.2 Process
The questionnaire was sent by email to the 11 participants. 9 were completed
electronically and returned. The responses can be seen in Appendix A: Delphi
method initial questionnaire results. The results from the first three questions are
taken directly into the results shown in Chapter 5.
Of the remaining questions, it was intended that Question 4, about the benefits of
knowledge management, would only provide background information and questions
5 – 7 would be used as the basis for the Delphi method. In fact the results from
Question 4 deserved further investigation and were taken forward. Question 7 was
about other knowledge management techniques the participants were aware of. It
did not provide a data set that would lend itself to further consideration and was not
taken forward. This left three lists that needed further analysis.
4.3 Preliminary Analysis
For each of questions 4, 5 and 6, the same process was applied in analysing the
data.
1. All the answers given by participants were combined into a single list
38
2. Any contribution that covered more than one point was split up so that each
item in the list covered only a single issue.
3. The main themes were identified and listed using language that was general
enough to cover similar issues received from multiple participants.
4. Each of the issues in the list was then mapped on to one of the main themes.
Every issue raised by a participant was reflected in this consolidated list.
5. The lists were presented in no particular order to be used in the later stage of
the Delphi method.
These three lists were sent back out to the 9 active participants and they were asked
to rank and comment on them. The ranking was to be achieved by awarding marks
to each item from a total pool of 50 marks per list. Items that were most relevant
were to be given highest marks and those that were least relevant were to be given
fewest or even zero marks. Before the ranked lists were returned, it was not known
what method would be used to measure the level of agreement between the
participants. An absolute ranking was not required – only an understanding of what
the most important items in each list were.
4.4 Analysing the ranking
Eight of the nine remaining participants returned their ranked lists. The total marks
given to each item in a list were summed and then the lists were sorted from highest
total marks to lowest total marks. This gave three lists that were sorted from most
relevant to least relevant.
This is a relatively simplistic approach to analysis of material gathered via the Delphi
method. Other research has used Kendall‘s coefficient of concordance (W). An
39
explanation of how to perform this calculation is found in Legendre (2005). A simpler
approach would be to calculate the standard deviation and see if it improves over
subsequent iterations. This method was used by Dekleva (1992). Iman & Conover
(1987) propose an alternative approach to calculating correlation that focuses on
agreement amongst the highest ranked items, minimising the significance on the
lower ranked items.
The output of this stage of the study was a list of issues to be taken forward for
further analysis. The study sought to find the most important benefits of knowledge
management, the most consistently used techniques and the hardest to capture
knowledge. It was not important which item was number 1 and which was number 2
because they were both going to be treated equally in the subsequent stages of the
study.
Although the process of simply ranking the totals seems primitive, some further
analysis was carried out to check that it served its purpose.
A second calculation was carried out which summed the totals but excluded the
highest and lowest mark awarded to each item. This should show whether the
ranking of a particular item was unduly influenced by any one contributor. For Lists 1
and 2, this second calculation provided exactly the same top 6 as the first calculation
did. For List 3, the results were slightly different.
The items were being ranked were on the following topic: ―What types of
knowledge have been hardest to capture and share effectively?‖ The item that
was number 2 on the list when all marks were included dropped to number 11 when
the highest and lowest marks were excluded. This was because of a single person
giving it 30 marks out of a total of 50. This item was ―Emails and documents can be
40
hard to keep track of if they are being contributed to by many people.‖ Nobody else
on the panel gave this item more than 5 marks and 3 of the 8 participants did not
give it any marks at all. This result does, therefore, seem to have been skewed by a
single participant. Replacing the item that was number 2 is the item ranked 9th on the
straightforward ranking of scores.
What this shows is that although there is broad agreement on 5 of the top 6 items,
there is some dispute as to the 6th. It was decided to use the straightforward ranking
based on the sum of all marks, but to allow each participant to make the case for a
―wildcard‖ – an additional item to be taken forward. If there was a strong case made
(as assessed subjectively) or a number of participants proposed the same wildcard
item, then it could be included for the latter stages of the study.
41
Chapter 5 Results
This chapter describes: the outcome of the contributions from the expert panel; the
analysis of some techniques that could improve knowledge management and
comparison of these techniques with the needs of the panel.
5.1 Background information
The information in Tables 5.1 – 5.3 was gathered from the participants in response
to the questionnaire.
These results suggest that none of the organisations considered have mature
knowledge management techniques, infrastructure or personnel structure. Any
improvements to knowledge management suggested by this study cannot rely on
buy-in at an organisational level.
Table 5.1. Level of maturity of organisational knowledge management approach.
Question 1 Answer Frequency
My organisation has a defined knowledge management process. 1 0
2 7
3 1
4 1
5 0
Mean 2.33
Table 5.2. Technical sophistication of organisational knowledge management approach.
Question 2 Answer Frequency
My organisation has an IT system designed 1 0
to help knowledge management. 2 2
3 5
4 2
5 0
Mean 3.00
42
Table 5.3. Personnel structure with respect to knowledge management.
Question 3a. Is anyone in your organisation responsible for
knowledge management?
Answer Frequency
Yes 2
No 7
Question 3b. If yes, what is their job title?
Knowledge Manager
Knowledge Consultant
5.2 Delphi method data
In the following sections, the data is presented in the form it evolved through the
Delphi method for each question.
5.2.1 What are the benefits of knowledge management?
When the original questionnaire was distributed, this question was intended as a
background question to gather information about how much value participants
placed on knowledge management. When the results came in, it was apparent that
this question was revealing some answers about what the participants are trying to
achieve through their re-use of knowledge. As this could have some impact on what
approaches to knowledge management best meet their needs, and what issues they
encounter, it was decided to treat this as an input into the Delphi method, and to
refine it further through that process.
43
The data was obtained from 9 participants who provided 23 suggestions. Once the
similar suggestions had been combined, this left 13 distinct suggestions to be taken
forward to the first refinement.
Each participant was asked to distribute 50 marks across the list. Eight participants
returned their marks. Appendix B shows the marks given by each participant. Table
5.4 shows the list sorted by total marks received and also gives the participants‘
comments for each item. The top 6 items in this list were sent out to the participants
with the request that they choose one of the items ranked 7-13 and make the case
for why they thought it should be considered further.
Table 5.4. Responses for list 1 ranked by total marks received, with comments added by participants. Benefits of knowledge management.
Rank Benefit Elaboration from comments received
Total
1 Improved consistency of information - everyone is referring to the same versions and documents
Saves time if someone can be pointed towards a document rather than being offered an explanation. Users need to be able to tell what version a doc is even once it has been printed out.
53
2 Reduced need to reinvent the wheel for every project
45
3 Improved speed of issues resolution by making existing solutions to past problems available
Needs appropriate filtering and keywords.
42
4 Training for new starters - provides a consistent set of information and experience for them to tap into
Very useful for contractors and handovers too.
41
5 Shared workload - no single person is the sole keeper of knowledge
Avoids disaster when someone leaves, but documentation usually fall behind what is in someone‘s head.
37
6 When you are aware that something similar has been done before, knowledge management makes it easy to find the information you need
Usually easier to go to the source of the information directly.
34
7 New work can get off the ground more quickly
Why? 33
44
Rank Benefit Elaboration from comments received
Total
8 Lets customers know what we already know about a product and reduces support calls
29
9 Urgent information can be pushed to those who need to see it
Needs a certain amount of design and maintenance to ensure that the mailing lists remain up to date.
27
10 It encourages a culture of sharing and cooperation
No evidence that this is the case. 27
11 Increased quality of finished product Experience of other people useful in shaping future approaches
19
12 Increased ability of an organisation to recognise synergies and opportunities
Especially when departments are doing similar things that could better be achieved through co-ordination e.g. Business Continuity Plans.
11
13 The ability to share knowledge outside the business
Should be careful with customer communications.
2
5.2.2 What techniques are currently used?
The data was obtained from 9 participants who provided 49 suggestions. Once the
similar suggestions had been combined, this left 18 distinct suggestions to be taken
forward to the first refinement.
The marks awarded by the eight participants who responded are shown in Appendix
B.
Table 5.5 shows the total marks given to each item and shows the items ranked from
highest to lowest. Again, the top 6 items were sent back out to have their ranking
refined and the participants were asked to choose a wildcard from the remainder.
45
Table 5.5. Responses for list 2 ranked by total marks received, with comments added by participants. Commonly used knowledge management methods.
Rank Method Elaboration from comments received
Total
1 Common storage area / intranet for project documents
Good idea to storing key documents, but needs housekeeping and a good structure to ease finding documents. Invaluable on large projects.
66
2 Defect list for technical bugs Very useful measure of quality, remaining work, trends and pattern analysis.
38
3 Ongoing issues list throughout a project
Only useful if small enough to manage, and not too detailed.
37
4 Lessons learned written up and placed in shared location at end of project
Would be more useful if they force entry into a common database with relevant topic keywords.
35
5 Post project reviews Sometimes compiled and shared in meeting form. Useful to the individuals within the project team, but not used outside very much.
32
6 Progress reports shared with wide audience
Can be difficult to get these read by appropriate audience, perhaps because they are at the wrong level - too detailed or too high level.
29
7 List of unresolved issues at end of project
Duplicate of lessons learned 20
8 Best practice examples in a shared area
Very useful, especially if key lessons and tips are summarised.
20
9 End project reports Duplicate of lessons learned and usually a result of post-project review
18
10 Flowcharts Can be very useful with simple projects, parts of bigger projects or high level views. Can quickly deteriorate with more complex flows and detailed views.
17
11 Sharing checklists Need to balance between too detailed and too vague.
If too detailed:
- People tend to answer the checklist and don‘t consider issues not directly covered by the checklist.
- Reluctancy to address areas not
17
46
Rank Method Elaboration from comments received
Total
covered by the checklist.
- People can fall into the habit of carrying out irrelevant checks too much.
If too vague:
- Checks open to misinterpretation leading to false positives and negatives.
NOTE: Regular maintenance of the results and the contents are required and should form part of the project review process.
Useful for remote teams
12 Wiki Difficult to get agreement about who should have access to what.
17
13 Workshops with other project managers
Under-used, but can be very effective. We think of red team review when it‘s failing technically, but do not have a management equivalent.
Have these set up with another division and within the division on particular projects.
16
14 Project blogs Esp useful for noting technical problem solving.
Scored 1 because although I‘ve never used it, believe it could be useful.
11
15 Published articles that highlight known issues and share useful information
Same as open issues list? Or is this referring to a post-project ―issues that came up‖ list? In which case same as end project reports?
Good for technical information, but the system we have is not currently geared up for internal information.
8
16 Defect list for problems with processes
Not really used, but will use standard defect database when required.
8
17 Instant messaging Scored 1 because although I‘ve never used it, believe it could be useful – although may be distracting if not specific.
6
18 Daily log of conversations and phone calls
Overkill. 5
47
5.2.3 What other techniques are you aware of but have not personally
used?
The list that was obtained after collating all the contributions to this question is
shown in Table 5.6. The data was obtained from 5 participants who provided 8
distinct suggestions. This list did not lend itself to further refinement. The interesting
finding to be gained from these responses is that participants had little detailed
understanding of any knowledge management techniques that they did not currently
use.
Table 5.6. Collated responses to Question 6 of questionnaire.
What other techniques are you aware of but have not personally used?
Use of a project office to collate details that can be used for lessons learned Library of historical project metrics, e.g. estimates vs outcomes
Formal post-project reviews
Knowledge-based software Using help-desk tracker software to collate problems solved and questions answered.
Wikis
Blogs
Collaboration-enabling software
No further analysis of this list was undertaken as part of the Delphi process.
5.2.4 What types of knowledge have been hardest to capture and share
effectively?
The data was obtained from 9 participants who provided 23 suggestions. Once the
similar suggestions had been combined, this left 16 distinct suggestions to be taken
forward to the first refinement.
The marks awarded by each of the eight participants who responded are shown in
Appendix B.
48
Table 5.7 shows the total marks given and shows the items ranked from highest to
lowest. The top 6 items were sent back out to have their ranking refined and the
participants were asked to choose a wildcard from the remainder.
Table 5.7. Responses for list 3 ranked by total marks received, with comments added by participants. Hard to share knowledge.
Rank Knowledge type Elaboration from comments received
Total
1 Knowledge that will help provide more accurate estimates in future
This is an area that is usually poorly recorded:
- Resources too busy to focus on introspection.
- Time recorded poorly, sometimes entered weeks after the event.
- Unwillingness to be open and honest with themselves and others.
- Difficulties breaking down the tasks and including the intangible work e.g. thinking time.
- If work is unique, tendency to guess rather than seek equivalent parallels and use the data for predictions.
- Tendency to get stuck in to the detail rather than take a step back and discuss the work and estimates.
54
2 Emails and documents can be hard to keep track of if they are being contributed to by many people.
It‘s so easy to spend days trying to solve a technical issue and not write it down for next time. Bug trackers that include resolutions can be helpful for this.
43
3 The solution to specific technical issues that have had to be overcome
People can be protective of their approach, so difficult to break down into the constituent parts.
37
4 Subjective opinions - discussions on how something should have been done or how successful a particular approach was
Not something that is usually written down.
36
5 Relationship information - who is helpful and who is not constructive
Information needs better structure including considered keywords, facts detailed, inferences explained and key messages summarised.
33
49
Rank Knowledge type Elaboration from comments received
Total
6 Keeping information general enough to be applied in other scenarios without being too vague.
Knowledge needs to be reviewed regularly and trimmed on occasion.
29
7 Mistakes made early on in a project, long before a post-project review or lessons learned reports
End Stage Reports, Project Meetings. Don‘t need to wait for formal post project reviews.
25
8 Ad-hoc work, where there is an impression that documentation is not justified because of the small scale of the work
This does cause problems, especially if there is a dispute with a developer.
Still go for documentation, as it often yields results. Can be kept as short and sweet as possible.
24
9 It can be hard to choose how much information to share to achieve a manageable quantity of information
Knowledge needs to be reviewed regularly and trimmed on occasion. Add ―next review dates‖, give someone the responsibility to manage data and prune accordingly.
24
10 Lessons learned in other teams Not widely communicated and either too detailed and therefore unread or too top level and out of context.
23
11 Root cause of delays and schedule slip
Duplicate 22
12 An understanding of the problems caused by too much complexity in a product specification - what constitutes simple?
At the start of every new software project I emphasise that we should aim to produce a simple, elegant product, but every time we end up with something that is more complicated than it needed to be. Such products generate a lot of calls from customers.
Usually get the author and audience together to agree this.
14
13 Making use of lessons learned from other projects is difficult because they are not communicated widely or not bought into at the right level.
Duplicate 14
14 Early warning signs Need people to manage their own risks, implement measures and alert when breached.
11
50
Rank Knowledge type Elaboration from comments received
Total
15 Cause and effect Especially when people focus on blaming each other rather.
8
16 It's hard to share knowledge without defining the scope of the knowledge too narrowly to the extent that other teams cannot see its relevance to them.
Duplicate 3
5.2.5 Wildcards
The ranked lists were sent back out to the eight participants who had contributed.
These participants were asked to make the case for the inclusion of any of the items
that had not been ranked in the top six. Six participants responded. The full results
are given in Appendix C. No more than two people voted for any one of the items.
There was no particular weight behind any single item to suggest that it should be
taken forward for further analysis ahead of the others. For this reason, the lists to be
analysed further were left as the top six ranked items as explained above.
5.3 Results of Delphi process
In summary, the following results were obtained. According to the participants:
The strongest benefits that knowledge management can bring are:
1. Improved consistency of information - everyone is referring to the same
versions and documents
2. Reduced need to reinvent the wheel for every project
3. Improved speed of issues resolution by making existing solutions to past
problems available
51
4. Training for new starters - provides a consistent set of information and
experience for them to tap into
5. Shared workload - no single person is the sole keeper of knowledge
6. When you are aware that something similar has been done before, knowledge
management makes it easy to find the information you need
The most consistently used knowledge management techniques are:
1. Common storage area / intranet for project documents
2. Defect list for technical bugs
3. Ongoing issues list throughout a project
4. Lessons learned written up and placed in shared location at end of project
5. Post project reviews
6. Progress reports shared with wide audience
The problems the panel encounter in knowledge capture are:
1. Capturing knowledge that will help provide more accurate estimates in future
2. Emails and documents can be hard to keep track of if they are being
contributed to by many people.
3. Capturing the solution to specific technical issues that have had to be
overcome
4. Capturing subjective opinions - discussions on how something should have
been done or how successful a particular approach was
52
5. Capturing relationship information - who is helpful and who is not constructive
6. Keeping information general enough to be applied in other scenarios without
being too vague.
Given these findings, the existing literature on knowledge management was
consulted to examine how it might help improve current practice.
5.3.1 The connection between ambition and practice
From these results, it is possible to see how current practice meets the ambitions of
the participants. Figure 3 shows how each of the perceived benefits of knowledge
managements connects to one or more of the most consistently practiced
techniques.
There is at least one technique being used that should help to achieve each benefit,
and every technique is associated with a desired benefit. This shows a consistency
between ambition and practice that suggests the participants are focussing their
efforts on the areas that they perceive to matter most – and getting the greatest
return for their effort.
53
Figure 3. Connections between intended benefits and practice.
5.3.2 Gaps in current practice
Although Figure 3 suggests that the participants are focussing their efforts on exactly
the correct techniques to meet their aims, a different picture emerges when the gaps
in current knowledge management are considered.
Figure 4. Connections between problem areas and practice.
Figure 4 shows that there is no obvious connection to be found between two of the
identified gaps in knowledge capture and any of the techniques that are used most
54
consistently. It further shows that four of the gaps could be addressed by current
techniques. The fact that the participants identified these gaps suggests that the
application of these techniques could be improved, or that more suitable techniques
are required.
5.4 Comparison with the literature
Using the analysis of current practice above, this study examines the existing
literature on knowledge management with two aims:
1. To find ways to improve the implementation of existing techniques.
2. To find new techniques that will fill those gaps that are not currently
addressed at all.
To help identify what sort of new techniques the study should be considering, the
existing techniques and the gaps were plotted on a grid. The X axis of the grid is for
the level of detail and the Y axis is for the scope of the information.
In Figure 5 the currently used techniques are shown depending on the sort of
knowledge that they are good at managing.
Top left box in the grid is for detailed knowledge with project-specific
relevance.
The bottom left box is for more generalised knowledge that has cross-project
relevance.
Top right is for generalised knowledge but applying only to a specific project.
Bottom right is for general and cross-project.
55
There is a degree of subjectivity in this categorisation, but it provides a useful tool to
compare techniques and gaps.
Figure 5. Current knowledge management techniques categorised in terms of the scope of their relevance across projects and the level of detail they capture.
From Figure 5 is can be seen that the techniques used at the moment are heavily
biased towards the detailed side of the grid, and are also better for capturing
knowledge that is project-specific. Lessons learned reports and other published
project reports can have a wider audience but must be written in a way that makes
the information they contains seem digestible and relevant to other projects. In
general, the techniques used rely heavily on some form of documentation and seem
56
geared towards getting information down on paper (or stored on a computer) and
trying to make that information available to those who need it.
Figure 6 then shows the gaps that have been identified, plotted onto the same grid.
This shows that the participants have found it difficult to capture knowledge that can
be shared between projects and that is more general in nature. The exception to this
is the need to capture information on which to base future estimates. This
information is wanted so that it can be applied to future work, it is historical
information. It needs to be captured in detail in the context of what was done on a
specific historical project and metrics are needed to measure how long activities took
and in what context.
This comparison suggests that the study should focus on techniques that allow less
detailed project knowledge to be transferred between projects. It is also valuable to
consider some techniques that do not rely so heavily on written documentation. A
further requirement is that the techniques do not need a high level of organisation
involvement in order to be successful. The participants clearly indicated in their
responses to the first questionnaire that their organisations were not pro-active in
knowledge management.
57
Figure 6. The gaps in current practice categorised in terms of the scope of their relevance across projects and the level of detail they require.
In researching the existing literature, the methods in Table 5.8 were found to offer
some value over and above current knowledge management practice. They were
chosen because they provide a contrast to the very document-centric processes
identified above.
For each of the techniques identified in Table 5.8, this study will consider
Which gaps in knowledge management practice they could help to fill
Which currently used techniques they could complement or extend
58
What level of organisational support is necessary for them to be implemented.
This is particularly important given the lack of organisational involvement in
knowledge management identified by the participants.
Table 5.8. Literature sources of research into knowledge capture.
Method Origin Mode of transfer
Rich personal interaction (Madhavan & Grover, 1998) Socialization
Programming patterns (Rising & Derby, 2003) Externalization
Facilitates internalization
Storytelling (Brown et al., 2001)
Externalization
Facilitates socialization
Blogging (Ojala, 2005) Externalization
Combination
Discussion groups and
collaboration software
(Liebowitz & Megbolugbe
2003)
Socialization
Externalization
Semi-structured interviews (Neve, 2003) Externalizaton
5.5 Analysis of practice vs literature
This section will compare the problems identified by participants with the solutions
proposed in the literature. It will also look at how practical the solutions could be
based on the low level of organisational involvement in knowledge management
identified by the participants.
Tables 5.9 and 5.10 show the correlation between the solutions proposed by the
literature, the gaps we are concerned with filling through new solutions and the
currently used techniques. Each ―x‖ in these tables is dealt with in greater detail
below.
59
Table 5.9. Matrix showing which gaps in knowledge capture identified by the Delphi method are addressed by which techniques from the literature.
Gap __________
Technique Future estimates
Keeping track of email chains
Solutions to specific issues
Subjective opinions
Keeping information general
Relationship information
Rich personal interaction
X
X X X X
Programming patterns
X
X
Storytelling
X X
Blogging
X
X
Discussion groups
X X X
Semi-structured interviews
X
X
Table 5.10. Matrix showing which of the current techniques are extended or complemented by the additional techniques found in the literature.
Technique
Rich personal interaction
Programming patterns Storytelling Blogging
Discussion groups
Semi-structured interviews
Common storage for project docs
X
Issues list
X
X X
Defect list for technical bugs
Post-project reviews
X
X
Published project report
X
Lessons learned report
X X
X
5.5.1 Patterns
The research of Rising & Derby (2003) draws an analogy with programming
patterns. These are templates for common requirements in software development. A
programming pattern is an approach to solving a problem that can be used in many
different contexts. Whereas an issue list is project-specific, a pattern could be used
60
to take the resolution to that issue and try to generalise it so that it is more relevant
across a number of future projects. Rising & Derby‘s suggestion is that we take time
to reflect on our experiences by carrying out retrospectives.
―During a retrospective, the project team can identify likely patterns, and small
groups may write a pattern outline as part of processing what they‘ve learned.
[…] ask them to think carefully about the project and identify two or three of
the project‘s most critical moments. A critical moment can be a decision, a
turning point, or an action that overcame an obstacle or made a difference in
some other way. ‖
The issues list could be a good starting point for this work and the participants did
identify the post-project review as a useful technique. The purpose of the
retrospective and the use of patterns is to see how the resolution to an issue for one
project can be turned into a more general pattern for other projects. This also helps
to address one of the issues identified by Davenport & Prusak (1998) – there can be
a reluctance to use knowledge from outside a team, a resentment of being asked to
adopt something that was ―not invented here‖. A pattern provides a general
approach, often summarised by a memorable name that helps it to be shared and
captures the intent. The pattern itself is then a starting point for a team to be able to
understand the problems encountered on other projects and to add to their arsenal
of possible solutions to that sort of problem.
The idea of patterns is not new. They are analogous to a means of communication
that has been around for centuries – proverbs. ―A bird in the hand is worth two in the
bush‖ conveys a lesson learned in a memorable and accessible way. It is not
necessary to know what the experience was that led to the creation of the proverb –
we can all relate to the message and the wisdom.
61
A similar approach is taken in the military. Heath & Heath (2007) describe
Commander‘s Intent. This is a crisp, clear unambiguous message that is passed
from those commanding an operation to those implementing it.
―Suppose I‘m commanding an artillery battalion and I say, ‗We‘re going to pass this
infantry unit through our lines forward.‘ That means something different to different
groups. […] As a commander, I could spend a lot of time enumerating every specific
task, but as soon as people know what the intent is they begin generating their own
solutions.‖
The Delphi method found that solutions to specific issues were hard to capture and
that it was difficult to keep information general enough for others to use. Patterns can
address both these issues, and can be an extension of the existing techniques of
capturing issues and lessons learned, and conducting a post-project review.
Rising & Derby (2003) explain the organisational support that they advocate to get
the process of generating and using patterns adopted.
―Patterns and insights won‘t permeate the organization on their own. Someone
needs to sing the songs and tell the stories of successful solutions to devilishly
recurring problems — a patterns minstrel, if you will.‖
They suggest:
Post patterns on an intranet
Host a monthly tech forum to update the organization
Speak the language (Use pattern names in a way that helps the concepts
become embedded
Help newcomers to become familiar with the use of patterns
62
These are the actions that are necessary to help share the patterns across an
organisation and this is where the greatest benefits can be achieved. Without
organisational buy-in, however, there is still value in using the structure of patterns to
guide the production of lessons learned and the post-project review and to help
produce something that manages to generalise the specific experience from a
particular project.
5.5.2 Document management and collaboration software
Three of the gaps identified in the Delphi study were around the management of
explicit knowledge: Difficulty in keeping track of email chains, recording solutions to
particular problems and recording subjective opinions.
Technology can help here, and recent developments in collaboration and networking
software mean that it can now be implemented on a small scale. Standard office
software now allows for multiple people to comment on a document and for their
remarks to be stored within the document itself. This adds richness and functionality
to the common storage area for documents that the participants in the study
currently use. An extension of this would be an application like Google docs
(http://docs.google.com), that allows multiple users to be editing the same document
at the same time. There are numerous bulletin board or group applications that allow
users to thread discussions and keep track of the history of a discussion. In the past,
such a system would have required a substantial investment of time and money from
an organisation. Today, these systems can be managed at a corporate level or they
can be created for free or nearly-free. Google Groups (http://groups.google.com/)
provides discussion groups and document storage for free, as does Microsoft
(http://home.services.spaces.live.com/). These can be used to keep track of
63
discussions and documents on a project basis, and they can be used to publish
information more widely.
Using a shared, collaborative system for issue logging can help team members to
keep track of issues and can be used to record solutions and discussions around
particular issues. This extends a project issue list (already identified as valuable by
the participants in the study), by also capturing the discussion around an issue.
Participants identified the capture of subjective opinions as an issue. George Orwell
wrote ―History is written by the winners,‖ and it may prove difficult for someone who
has been a dissenting voice on a project to get their views captured in project
documentation if they have not been successful in having their ideas implemented.
Capturing the process of choosing a solution by recording the discussion online
rather than in email chains is one way to record all considered solutions, not just the
one that was implemented. This also ensures that all knowledge made explicit
through the discussions is captured.
Rising & Derby (2003) state that one of the best aids to problem-solving is a wide
range of experience and suggestions to draw from. A range of opinions on ways to
solve old problems could therefore be a useful starting point for future issue
resolution.
Bresnen et al. (2003) found that strong social networks help with knowledge sharing
and the communities that can be built up using social networking technology or
discussion groups can help here. When people know where information has come
from, and they feel they can trust the source, they are more likely to use that
information.
64
One of the fastest rising forms of publishing in recent years has been blogging. Ojala
(2005) found that blogs were particularly useful in sharing knowledge in cross-
cultural environments. This is another issues mentioned by Davenport & Prusak
(1998) and is also an avenue for people to publish their own views without a
consensus needing to be reached before it is seen by an audience. This form of
publishing is readily availably today and needs no organisational oversight. If an
organisation did want to embrace this technology, then it would be possible to create
a centralised implementation that could co-ordinate the metadata associated with
blog entries, index the entries and highlight any that were particularly worthy of wide
attention.
Although they would bring the greatest benefit if implemented at an organisational
level, these technologies are now available with very little set-up or cost. They are
therefore viable at a project level for use by a single team.
5.5.3 Storytelling
Two of the gaps identified by the participants in the Delphi study involve the sharing
of less concrete knowledge. They find it difficult to share knowledge in a way that
keeps lessons general, and they don‘t know how to share information about how
personalities and relationships impacted their projects. This is not quite tacit
knowledge. They know they have this information and that it would be useful to
share, but it does not lend itself to documentation. Storytelling could be one way of
meeting this need for an almost anecdotal approach to knowledge sharing. Denning
(2001) recounts the process of using storytelling at the World Bank. The focus of the
book is on how storytelling encouraged buy-in to the adoption of knowledge
management within the organisation, rather than storytelling as tool for knowledge
65
management itself. However, Brown et al. (2001) advocate storytelling as a means
of relaying information captured at project review points and see it as a useful
technique for knowledge management. They share the view of Denning that
storytelling builds a connection between listener and storyteller. The listener
becomes an active participant in the experiences being recounted:
―In narrative, there is thus an implicit invitation to the listeners to fill in the
missing links in the story. If the listeners accept the invitation, they are thus
inside the story, projecting themselves into the situation, living the
predicament of the protagonist, feeling what he or she was feeling,
experiencing the same hopes and fears. In such a lived-in experience, it is not
difficult for the participants to visualize the missing links. In fact, they will find it
difficult to resist adding necessary patterns and linkage to the narrative.‖
Denning (2001)
Similarly, Heath & Heath (2007) explain how the act of listening puts the listener into
a problem solving frame of mind. The listener is primed to look for connections and
solutions to the problem presented to them. More importantly, they extrapolate from
the specific scenario described in the story to try to gauge how it might be relevant to
them.
This characteristic of storytelling could be beneficial in overcoming the resistance to
taking on information from other projects. Storytelling allows the listener to fill in the
gaps and to think about what they would have done in the same situation. It is a
dynamic process, and allows a story about a specific project experience to have a
broader scope of interest and to be generalised. It adds to the experiences that the
listener can draw on when confronted with a similar situation.
Storytelling is something that can be carried out on a small scale, but would require
willingness from those on project teams to think about putting a story together and
willingness on the part of others to listen. It has the benefit that stories, or important
66
points from stories, can be passed on through the normal socialisation and
interactions of staff beyond the initial pool of recipients. Put simply, people tell each
other stories.
In the literature about implementing storytelling, one common thread is that it is
something that has been established in an organisation through the efforts of a few
―believers‖. It is an interesting concept because it is so far removed from the usual
document-based methods of capturing knowledge and it relates back to the work of
Nonaka & Takeuchi who look at sharing tacit knowledge and the use of metaphor to
go beyond what is normally recorded when you ask someone what they know. The
difficulty with this approach, however, is that storytelling is a skill and whilst it may be
possible to teach and learn this skill it is not one that a project manager can be
assumed to have ready access to. Even if project outcomes can be captured in this
way, setting up the process of sharing these stories is likely to require an evangelist
to get buy-in from other teams.
Storytelling could provide a new take on lessons learned reports and published
project reports but this is not a quick-win solution. It is one that either needs to be
driven from the grass-roots by an enthusiastic and talented storyteller or backed by
an organisation willing to lead by example.
5.5.4 Semi-structured interviews
One of the goals of knowledge management is to capture knowledge held by one
and capture it in a form that makes it available to many. This is particularly difficult to
achieve with the externalisation of tacit knowledge. This is a limitation of the post-
project reviews that the Delphi participants identified as one of their key knowledge
management techniques. Despite carrying out reviews and publishing the results,
67
the participants identified a gap in their ability to capture solutions to specific issues
and relationship information. Neve (2003) criticises the focus of some knowledge
management approaches on technical solutions, stating that:
―When knowledge systems are built, it often seems that we forget that we
cannot extract this from individuals without their participation, motivation, or
awareness of their knowledge. There is a neglect of a personal dimension.‖
Neve‘s solution is to motivate individuals to share information by conducting
interviews that seek to get interviewees to describe events, getting them to provide
examples and a narrative. This provided individuals with an opportunity to reflect on
and analyse their experience and to question their assumptions. When the process
of carrying out these interviews was trialled, Neve found that interviewees had
difficulty in understanding how much they really knew. The process of questioning
them with a view to extracting examples was found to help individuals to share their
tacit knowledge. Depending on the direction of the questioning, it could be possible
for this approach to also gather information about the roles played by individuals in a
project that would then help to provide a picture of which relationships were
successful and which were not. This was identified by the participants of this study
as a difficult area to gather and share information on. It is a sensitive area because
of the potential damage to reputations and the need to fairly represent an individual.
The focus on examples and narrative may allow a more concrete, constructive
description of relationship issues to be gathered and shared.
This process of externalisation could help add more depth to post-project reviews by
increasing the amount of tacit knowledge that is shared, and could also provide input
to the lessons learned report. Neve‘s view was that these interviews need to be
―carried out proactively and deliberately within an organisation,‖ but there does not
seem to be any reason why they could not be carried out for particular projects.
68
5.5.5 Rich personal interaction
As has been mentioned previously, all of the key knowledge management processes
used by the participants in the Delphi study are document-based. In contrast,
Madhavan & Grover (1998) advocate rich personal interaction. Their context is new
product development teams, but the principles are valid in a broader context. They
argue:
―Rich personal interaction, consisting of direct, frequent, and informal
interaction among team members, will influence the trust in the team
orientation of other members positively, which, in turn, is related positively to
the efficiency and effectiveness with which embedded knowledge is converted
to embodied knowledge.‖
This is also the point put forward by Ji-Hong Park (2006) who demonstrates the role
of trust in the reuse of knowledge. Madhavan & Grover discuss some of the factors
of rich person interaction that can contribute to its success:
Direct personal interaction is most effective
Frequent interaction is required, not just close physical proximity
Informal interaction is more effective than formal networks
Two of the many possible benefits of developing a team that practices rich personal
interaction are an increase in innovation and improved information redundancy. The
implications of this approach are most relevant to those constructing teams.
Madhavan & Grover recommend a team structure where there is a wide breadth of
interests and a diversity of personal networks. This approach would require buy-in
from those controlling personnel allocation and it is not always possible in practice to
pick and choose team members based on anything other than availability. Some of
69
Madhavan & Grover‘s recommendations, however, are more applicable at the
project level. They suggest constructing team goals for intermediate stages of a
project so as to build up trust within the team. They also recommend further
investigation of some of the virtual office practices, and are concerned that these
reduce the physical proximity of team members and therefore reduce the quality of
their interaction.
From Liebowitz & Megbolugbe (2003), it was stated in Section 2.4 that the
suggested knowledge management improvement with the lowest barrier to entry was
more face to face contact. It was proposed in this dissertation that any organisation
that values knowledge management would have this process in place. This study,
however, has found that there is a lack of interpersonal contact in the context of
managing knowledge.
Of all the techniques considered in this study, improvements in the quality of
personal interaction between team members have the best overlap with the
knowledge management gaps identified by the participants.
Providing context to enable future estimates to be based on other‘s
experience
Sharing solutions to specific issues, in an informal, ad hoc way
Sharing subjective opinions – increased trust will encourage forthright sharing
of views
Keeping information general – frequent interaction can help individuals to
share mental models which allows information to be transferred in a way that
all individuals can relate to.
70
Relationship information – the informal networks that are created when
interactions are direct and frequent should allow better relationships to be
established. Better trust between individuals can free them to share
information about their relationships with others.
It seems that a worthwhile balance to the rigour of documentation and explicit
knowledge capture would be an increased focus on improving the opportunities for
frequent, informal, direct interactions between team members. This is something that
can be achieved at a project level without the need for organisational involvement.
5.5.6 Estimates for future work
The approaches above have provided suggestions for how all except one of the
gaps identified by the participants in the Delphi study could be filled. Software
development estimation is a field of study in itself. Gray & MacDonnell (1997) is just
one of the many papers that compare different techniques. This study does not deal
with the various techniques of project estimation, but Gray & MacDonnell highlight
two issues that are relevant here:
Good estimates require historical data.
Future estimations need to take into account how closely the new tasks match
tasks that data is available for.
Time capture software is available to assist developers in keeping track of how much
time they spend on each task. It is outside the scope of this study to recommend
particular software applications to assist in this as it is not knowledge that is being
captured. It is management metrics and this can be done mechanically and
objectively.
71
The relevant point for knowledge management is that when this information is used,
it must be possible to relate it to both the work that was done and the work to be
estimated. A developer who is to provide an estimate must know how similar the
future work is to work that the developer has done before, so that there is a basis for
the estimate. The more experience they have, the more likely they are to be able to
relate the new work to existing work. The project manager then simply has to be able
to tell how good the developer is at estimating and this can be done by tracking how
well previous estimates have matched actual development time. Spolsky (2007)
provides one method for evidence based scheduling that follows exactly this
process. It meets the requirement of being something that can be applied at a team
level, as it relies on knowledge of how well individuals in a team have estimated in
the past. It does not rely on input from outside the team or from across an
organisation.
Once a process such as this is in place, more accurate estimates could be provided
by creating a larger pool of experience for each developer to call on when comparing
future work to past work. This then becomes a very similar issue to the other cross-
project issues identified above.
5.5.7 Summary of proposed techniques
As is so often the case in knowledge management, these approaches can be
considered to fall into two categories – the human and the technical. Techniques
such as rich personal interaction, storytelling and patterns all rely on the human side
of knowledge management. The document management and collaboration software
sit on the side of technical solutions. There is also now, however, a vibrant middle
ground where software can be used to build communities, extend relationships and
72
publish. In the past this would only have been possible with expensive bespoke
systems but it is now available to anyone with an internet connection. This opens up
the possibility of combining the technical with the interpersonal to extend the benefit
of both.
5.6 Validation
The research question evolved over the course of the study in response to the
feedback from the participants and the findings of the literature research. The study
gives good coverage to the current practice and gaps as identified by the Delphi
method participants. It demonstrates how the gaps are a possible consequence of
the techniques used, although it does not show that they are an inevitable
consequence, or that the findings are specific to software development.
The need to associate these findings with software development stems more from
the pool of participants than from the approach of this study. An expert panel drawn
from a wider range of organisations could allow for that limitation of the study to be
removed.
The proposed solutions were chosen because they provided a variety of different
techniques from those currently in use. In relating the proposed solutions, the issue
of the lack of organisational involvement in knowledge management is touched on,
but is not examined in detail in partnership with the participants. Similarly the
suitability of these techniques as solutions is not discussed with the participants and
this does mean that they can only be considered as suggestions rather than having
the weight of recommendations.
For this reason, the research question should perhaps be changed from ―What
techniques can be used…‖ to read:
73
―What techniques could be used to improve the practice of knowledge management
in software development projects, where the organisation as a whole does not
provide strong guidance?‖
5.6.1 Analysis
The use of the Delphi method evolved during the study. It was not clear enough at
the start of the study that an absolute ranking was not necessary. The initial stages
of the Delphi method were geared towards gathering a large number of ideas then
sorting them into an order. It was only when the first refinement was returned that it
became apparent that all that was needed to progress the study was agreement on
the most important areas.
When the initial responses to the questionnaire were received, it was necessary to
combine similar responses into a single point. As little editing as possible was carried
out, to avoid changing the sense of the items that were submitted by the participants.
This resulted in some quite similar items being sent out for refinement. This may
have led to some confusion in the marks and it is arguable that the marks for these
similar items should have been combined when the ranking was carried out. If this
process were repeated it may be desirable to spend more time amalgamating the
suggestions, but to have an independent third party audit the changes made.
The mapping of techniques and gaps in Figures 5 and 6 was a very useful visual
tool, but it is subjective. The mapping cannot be claimed to be definitive, empirical
analysis. To add this rigour it would have been necessary to ask a number of
independent parties to provide their mappings and to check correlation.
74
Although the figures point towards a possible reason for the gaps in knowledge
management, they cannot be proven to cause them. A more focussed study that
concentrated on techniques used vs knowledge management gaps within individual
organisations would be one way to demonstrate this causal link but that was not in
the scope of this study.
75
Chapter 6 Conclusions
The participants in this study identified many benefits that knowledge management
could bring to their organisations. They could also identify many areas where they
felt that they were not achieving all the benefits that were possible. These gaps in
knowledge management were considered and it was proposed that the areas that
proved problematic were those that required inter-project, general communication.
The study suggests that the strength of the techniques that were most commonly
used was in capturing project-specific information, and that none of the techniques
would excel in sharing knowledge between projects.
The study then puts forward some knowledge management techniques that contrast
with the very document-centric techniques currently being used by the participants.
Their potential for being implemented without extensive organisational support was
considered and it was found that whilst organisational co-ordination would be
valuable, it should be possible to implement these techniques at a project level. This
is possible because of the advances in technology that have provided cheap or free
off-the-shelf collaborative software.
Some of the techniques, for example creating patterns or storytelling, require skills
that may not already exist within project teams, and are only likely to be implemented
either through organisational change or through the actions of an evangelical core. It
could be considered that they are structured forms of perhaps the most valuable
technique of all – rich personal interaction. That is the technique that would seem to
provide improvements across the most problem areas and is the most distinct from
the techniques that are currently the focus of knowledge management activity.
76
6.1 Project review
The study has succeeded in gathering an overview of current knowledge
management practice by the participants. It has also analysed the impact that some
alternative, currently unused techniques, could have on the quality of knowledge
shared.
The literature suggests that tacit knowledge is the most difficult to capture and much
effort has been expended on investigating techniques to help in this area. It was
expected that the issues found by the participants in this study would reflect the
difficulty with tacit knowledge.
In fact, the gaps identified were more related to the difficulty in sharing knowledge
that had already been gained, particularly outside the project that the knowledge
originated from.
This dissertation was able to consider the gaps identified by the participants and look
for techniques that could help to improve their practice.
The study also identified that knowledge management practice was not mature in the
organisations that the participants are employed by, and was able to take that into
account when reviewing possible techniques.
6.2 Future research
An extension of this study would be to take the suggested additional techniques
forward with these participants or those in a similar position and investigate whether
they do indeed help to fill existing gaps in knowledge management practice.
As mentioned previously, the gaps identified by the participants seemed to be a
consequence of the techniques used, but it was not possible to prove cause. Another
77
study could investigate whether or not cause could be found, and if the gaps were an
inevitable consequence of the techniques used.
Organisations are not providing a corporate framework for knowledge sharing and
this gap is being filled by knowledge workers who find they need to make knowledge
management work for them in order for them to be successful in their jobs. Some
research into how to incorporate more techniques for knowledge management into
project management methodologies could be valuable.
Another area of interest could be an investigation into whether social networking and
widely available collaborative technologies are replacing, improving or circumventing
corporate IT solutions for knowledge management.
78
References
Ackoff, R.L. (1989): ‗From Data to Wisdom‘, Journal of Applied Systems Analysis,
vol. 16.
Ambrosini, V. & Bowman, C. (2001) ‗Tacit Knowledge: Some Suggestions for
Operationalization‘, Journal of Management Studies, vol. 38, no. 6.
Anquetila, N. et al. (2006) ‗Software maintenance seen as a knowledge management
issue‘, Information and Software Technology, vol. 59, no. 5.
Basili, V.R. & Romback, H.D. (1991) ‗Support for Comprehensive reuse‘, IEEE
Software Engineering Journal, vol. 6, no. 5.
Birk, A., Dingsøyr, T. & Stålhane, T. (2002) ‗Postmortem: Never Leave a Project
without It‘, IEEE Software, May/June.
Brand, A. (1998) ‗Knowledge management and innovation at 3M‘, Journal of
Knowledge Management, vol. 2, no. 1.
Bresnen, M. et al. (2003) ‗Social practices and the management of knowledge in
project environments‘, International Journal of Project Management, 21.
Brown, J.S. et al. (2001) Storytelling: Passport to the 21st Century,
http://www2.parc.com/ops/members/brown/storytelling/Intro0-table.html [Accessed
March 24 2007]
Clarke, T. & Rollo, C. (2001) ‗Corporate initiatives in knowledge management‘,
Education + Training, vol. 43, no. 4/5.
Collier, B., DeMarcot, T. & Fearey, P. (1996) ‗A Defined Process For Project
Postmortem Review‘, IEEE Software, July.
Crossan, M.M., Lane, H.W. & White, R.E. (1999) ‗An Organizational Learning
Framework: From Intuition to Institution‘, The Academy of Management Review, vol.
24, no.3.
79
Davenport, T.H. & Prusak, L. (1998) Working knowledge: How organizations
manage what they know, Harvard Business School Press, Boston.
De Ruyter, K. (1996) ‗Focus versus nominal group interviews: a comparative
analysis‘, Marketing intelligence and planning, vol. 14, no. 6.
Dekleva, S. (1992) ‗Delphi study of software maintenance problems‘, Proc. Conf.
Software Maintenance, 9-12 Nov 1992.
Denning, S. (2001) The springboard: How storytelling ignites action in knowledge-era
organizations, Elsevier, New York.
Desouza, K.C. & Awazu, Y. (2005) ‗Segment and destroy: the missing capabilities of
knowledge management', Journal of Business Strategy, vol. 26, no. 4.
Faniel, I.M. & Majchrzak, A. (2006) ‗Innovating by accessing knowledge across
departments‘, Decision Support Systems, in press.
Graham, A.K. (2000) ‗Beyond PM 101: Lessons For Managing Large Development
Programs‘, Project Management Journal, vol. 31, no. 4.
Grant, R.M. (1996) ‗Toward A Knowledge-Based Theory Of The Firm‘, Strategic
Management Journal, 17.
Gray, A.R. & MacDonell, S.G. (1997) ‗A Comparison of Techniques for Developing
Predictive Models of Software Metrics‘, Information and Software Technology, 39.
Heath, C. & Heath D. (2007) Made to Stick: Why Some Ideas Survive and Others
Die. Random House, New York.
Henninger, S. (2001) ‗Turning Development Standards into Repositories of
Experience‘, Software Process Improvement and Practice, 6.
Iman, R.L. & Conover, W.J. (1987) ‗A Measure of Top-Down Correlation‘,
Technometrics, vol. 29, no. 3.
Kant, I. (1787) Critic of pure reason. This edition published Cambridge University
Press, 1999.
80
Kasvi, J.J.J., Vartiainen, M. & Hailikari, M. (2003) ‗Managing knowledge and
knowledge competences in projects and project organisations‘, International Journal
of Project Management, 21.
Kodama, M. (2002) ‗The promotion of strategic community management utilizing
video-based information networks‘, Business Process Management Journal, vol. 8,
no. 5.
Legendre, P. (2005) ‗Species association: the Kendall coefficient of concordance
revisited‘, J. Agricultural, Biological, and Environmental Statistics, vol. 10, no. 2.
Leibowitz, J. & Megbolugbe, I. (2003) ‗A set of frameworks to aid the project
manager in conceptualizing and implementing knowledge management initiatives‘,
International Journal of Project Management, vol. 21, no. 3.
Madhavan, R. & Grover, R. (1998) ‗From embedded knowledge to embodied
knowledge: New product development as knowledge management‘, Journal of
Marketing, vol. 62, no. 4.
Marwick, A.D. (2001) ‗Knowledge management technology‘, Knowledge
Management, vol. 40, no. 4.
Neve, T.O. (2003) ‗Right Questions to Capture Knowledge‘, Electronic Journal of
Knowledge Management, vol. 1, no. 1.
Nonaka, I. & Takeuchi, H. (1995) The knowledge-creating company, New York,
Oxford University Press.
Nonaka, I. (1991) ‗The Knowledge-Creating Company‘, Harvard Business Review,
vol. 69, no. 6.
Ojala, M. (2005) ‗Blogging for knowledge sharing, management and dissemination‘,
Business Information Review, vol. 22, no. 4.
O‘Leary, D.E. (1998) ‗Using AI in knowledge management: knowledge bases and
ontologies‘, IEEE Intelligent Systems, May/June.
81
Park, J.-H. (2006) ‗The Role Of Trust On Knowledge Creation In A Virtual
Organization: A Social Capital Perspective‘, Journal of Knowledge Management
Practice, vol. 7, no. 4.
Rising, L. & Derby, E. (2003) ‗Singing the Songs of Project Experience: Patterns and
Retrospectives‘, Cutter IT journal, vol. 16, no. 9.
Robillard, P.N. (1999) ‗The role of knowledge in software development‘,
Communications of the ACM, vol. 42, no. 1.
Sipcic, S.R. & Makonnen, Z. (1998) ‗Web-based groupware support for knowledge
creation and competitive advantage‘, Proc. Seventh IEEE Int. Workshop on Enabling
Technologies, June 1998, pp. 155–160.
Spolsky, J. (2007) Evidence Based Scheduling,
http://www.joelonsoftware.com/items/2007/10/26.html [Accessed 12 January 2008]
Tautz, C. & Althoff, K.-D. (1997) ‗Using Case-Based Reasoning for Reusing
Software Knowledge‘, IESE-Report No. 004.97/e.
Tiwana, A. (2004) ‗An empirical study of the effect of knowledge integration on
software development performance‘, Information and Software Technology, vol. 43,
no. 13.
Tsoukas, H. & Vladimirou, E. (2001) ‗What is organisational knowledge?‘, Journal of
Management Studies, vol. 38, no. 7.
Van De Ven, A.H. & Delbecq, A.L. (1974) ‗The Effectiveness of Nominal, Delphi, and
Interacting Group Decision Making Processes‘, The Academy of Management
Journal, vol. 17, no. 4.
82
Bibliography
Davenport, Thomas H. (2005) Thinking for a living: how to get better performance
and results from knowledge workers, Boston, Harvard Business School Press.
O‘Dell. C.S. et al. (1998) If only we knew what we knew: the transfer of internal
knowledge and best practice, New York, The Free Press.
Von Krogh, G et al. (2000) Enabling knowledge creation, Oxford, Oxford University
Press.
83
Index
artificial intelligence, 10
blogging, 21, 46, 58, 64, 96
collaboration, 10, 20, 62, 71
Delphi method, 26, 30, 42
explicit knowledge, 9, 11, 15, 19, 23,
26, 62, 70
groupware, 10, 21
human factors, 10, 11, 12, 25, 71
innovation, 8, 9, 11, 20, 68
issues list, 45, 51, 60
Japanese approach, 9
metaphor, 9, 66
organisational involvement, 10, 17, 28,
41, 61, 66, 70, 75
patterns, 25, 58, 59
personal interaction. See rich personal
interaction
post-project review, vii, 60, 61, 66
rich personal interaction, 25, 68, 71, 75
semi-structured interviews, 26, 58, 66
social networking, 21, 63, 77
software development, 8
storytelling, 26, 64, 65, 71
tacit knowledge, 9, 11, 13, 15, 19, 21,
22, 23, 64, 66, 76
technological solutions, 9, 20, 62
video, 10
Western approach, 9
wiki, 21, 46
84
Appendix A: Delphi method initial questionnaire results
Questionnaire from Participant A
85
Questionnaire from Participant B
86
Questionnaire from Participant C
87
Questionnaire from Participant D
88
Questionnaire from Participant E
89
Questionnaire from Participant F
90
Questionnaire from Participant G
91
Questionnaire from Participant H
92
Questionnaire from Participant I
93
Appendix B: First refinement
B.1 Material supplied to participants
Questionnaire for MSc Dissertation research
Elaine Aitken
Stage 1
Thank you for sending back your response to the questionnaire. Stage 1: gathering
initial contributions is now complete.
Stage 2
Stage 2: Collation is also complete. I have been through all the responses and have
produced new lists that combine the comments provided by multiple people. There
was quite a lot of overlap between the responses and the comments have been
reworded to cover similar answers. This means that you may not find your exact
wording in the list but should find that your views have been reflected.
Stage 3 – Refinement.
There are three lists below that show the collated responses resulting from Stage 2.
The answers are not in any particular order. I would like you to do two things with
each list:
1. Allocate points to show how relevant you think each item is to the question
You will have a set number of points to give in each list, and should award most
points to the most relevant answer, and don‘t give any points to an answer you don‘t
think is relevant at all.
Note – you may award more than one item the same mark.
94
2. Add comments to any items that you think need further explanation or refinement, or
any that you feel particularly strongly about.
For example:
Question: What fruit is most important in protecting the health of people living in the
UK?
(Please award 30 points)
Fruit Points Comment
Oranges 8 I believe this is definitely the most important because it‘s a main source of vitamin C. Also lots of people drink orange juice.
Apples 5
Bananas 5
Grapes 3
Satsumas 6 Should be combined with oranges.
Lemon 0
Grapefruit 2
Berries 1 Needs to be more specific – split into raspberries, strawberries and blackberries
What happens next?
Once I have received all the responses, I‘ll produce a ranked version of each list that
takes into account the feedback received. I‘ll then send this back out to you for
further comment and refinement.
Thank you again for your help.
95
List 1: What benefits does / could knowledge management bring to your
organisation?
Please award a total of 50 points. It is ok to award 0 points to any answer.
Points Comments
Improved consistency of information - everyone is referring to the same versions and documents
When you are aware that something similar has been done before, knowledge management makes it easy to find the information you need
Shared workload - no single person is the sole keeper of knowledge
It encourages a culture of sharing and cooperation
New work can get off the ground more quickly
Increased quality of finished product
Increased ability of an organisation to recognise synergies and opportunities
Urgent information can be pushed to those who need to see it
Training for new starters - provides a consistent set of information and experience for them to tap into
Reduced need to reinvent the wheel for every project
Improved speed of issues resolution by making existing solutions to past problems available
The ability to share knowledge outside the business
Lets customers know what we already know about a product and reduces support calls
96
List 2: Which techniques have you used for knowledge sharing?
Please award a total of 50 points
Points
Comments
Lessons learned written up and placed in shared location at end of project
Post project reviews
Common storage area / intranet for project documents
Progress reports shared with wide audience
Ongoing issues list throughout a project
End project reports
Workshops with other project managers
Wiki
Published articles that highlight known issues and share useful information
List of unresolved issues at end of project
Daily log of conversations and phone calls
Project blogs
Defect list for technical bugs
Defect list for problems with processes
Best practice examples in a shared area
Sharing checklists
Flowcharts
Instant messaging
97
List 3: What types of knowledge have been hardest to capture and share
effectively?
Please award a total of 50 points.
Points Comments
An understanding of the problems caused by too much complexity in a product specification - what constitutes simple?
Mistakes made early on in a project, long before a post-project review or lessons learned reports
Subjective opinions - discussions on how something should have been done or how successful a particular approach was
Cause and effect
Relationship information - who is helpful and who is not constructive
Ad-hoc work, where there is an impression that documentation is not justified because of the small scale of the work
Root cause of delays and schedule slip
The solution to specific technical issues that have had to be overcome
Knowledge that will help provide more accurate estimates in future
Early warning signs
Lessons learned in other teams
Keeping information general enough to be applied in other scenarios without being too vague.
98
It can be hard to choose how much information to share to achieve a manageable quantity of information
It's hard to share knowledge without defining the scope of the knowledge too narrowly to the extent that other teams cannot see its relevance to them.
Making use of lessons learned from other projects is difficult because they are not communicated widely or not bought into at the right level.
Emails and documents can be hard to keep track of if they are being contributed to by many people.
99
B.1 Results received from participants
Table B.1. Reponses to the first refinement of List 1.
Participant A B C D E F G H
Improved consistency of information - everyone is referring to the same versions and documents
10 8 10 6 2 5 - 12
When you are aware that something similar has been done before, knowledge management makes it easy to find the information you need
3 - 5 8 5 8 5 -
Shared workload - no single person is the sole keeper of knowledge
5 8 - 6 8 - 10 -
It encourages a culture of sharing and cooperation
3 4 - - 4 6 10 -
New work can get off the ground more quickly 5 4 5 1 3 5 10 -
Increased quality of finished product 5 - 5 1 3 3 - 2
Increased ability of an organisation to recognise synergies and opportunities
3 - - - 2 4 - 2
Urgent information can be pushed to those who need to see it
2 - 10 8 - 4 - 3
Training for new starters - provides a consistent set of information and experience for them to tap into
2 6 5 - 8 5 5 10
Reduced need to reinvent the wheel for every project
5 8 - 2 7 5 10 8
Improved speed of issues resolution by making existing solutions to past problems available
5 6 - 10 6 5 - 10
The ability to share knowledge outside the business
1 - - 1 - - - -
Lets customers know what we already know about a product and reduces support calls
1 6 10 7 2 - - 3
100
Table B.2. Reponses to the first refinement of List 2.
Participant A B C D E F G H
Lessons learned written up and placed in shared location at end of project
5 - 10 5 - 8 5 2
Post project reviews 5 6 2 5 5 1 5 3
Common storage area / intranet for project documents
10 8 5 15 10 8 5 5
Progress reports shared with wide audience 5 6 2 10 3 - - 3
Ongoing issues list throughout a project 5 6 5 2 10 2 5 2
End project reports 5 - 5 2 - 6 - -
Workshops with other project managers - 6 1 - - 5 - 4
Wiki - - 5 - 7 - 5 -
Published articles that highlight known issues and share useful information
- - - - - 5 - 3
List of unresolved issues at end of project - 6 6 1 2 - 5 -
Daily log of conversations and phone calls 5 - - - - - - -
Project blogs - - - - - 5 5 1
Defect list for technical bugs 5 8 2 10 - - 5 8
Defect list for problems with processes - - - - 8 - - -
Best practice examples in a shared area - - 5 - - 8 - 7
Sharing checklists - 4 - - - 2 5 6
Flowcharts 5 - 2 - - - 5 5
Instant messaging - - - - 5 - - 1
101
Table B.3. Reponses to the first refinement of List 3.
Participant A B C D E F G H
An understanding of the problems caused by too much complexity in a product specification - what constitutes simple?
- 10 - - 2 - - 2
Mistakes made early on in a project, long before a post-project review or lessons learned reports
10 - - 5 2 8 - -
Subjective opinions - discussions on how something should have been done or how successful a particular approach was
5 - 5 10 1 8 - 7
Cause and effect - - - - 2 - - 6
Relationship information - who is helpful and who is not constructive
5 8 2 - 5 5 5 3
Ad-hoc work, where there is an impression that documentation is not justified because of the small scale of the work
- 4 3 10 7 - - -
Root cause of delays and schedule slip 5 8 - - 9 - - -
The solution to specific technical issues that have had to be overcome
- 4 10 5 5 3 10 -
Knowledge that will help provide more accurate estimates in future
5 - - 10 11 8 10 10
Early warning signs 5 - - - 3 - - 3
Lessons learned in other teams 5 8 - - - - 5 5 Keeping information general enough to be applied in other scenarios without being too vague.
- - - 10 - 5 10 4
It can be hard to choose how much information to share to achieve a manageable quantity of information
- 4 - - 2 5 5 8
It's hard to share knowledge without defining the scope of the knowledge too narrowly to the extent that other teams cannot see its relevance to them.
- - - - - 3 - -
Making use of lessons learned from other projects is difficult because they are not communicated widely or not bought into at the right level.
5 4 - - - 5 - -
Emails and documents can be hard to keep track of if they are being contributed to by many people.
5 - 30 - - - 5 2
102
Appendix C. Second Refinement
C.1 Material sent to participants
Chapter 1 Questionnaire for MSc Dissertation research
Elaine Aitken
Thank you for sending back your completed forms with all the lists ranked. You‘ll be
pleased to hear that the hard work is complete and there is just one final stage to be
completed.
I have added up the scores for each item and ordered each list. I‘ve then produced
shortened lists with just the top-scoring 6 items.
I propose to take the top 6 items in each list forward for further analysis within my
dissertation. This analysis will include looking at research in the field of knowledge
management to see if it offers any suggestions that might be useful in addressing the
issues raised.
I would now like you to have one more look at the lists below. Please choose a
wildcard item – from the items that did not make the top 6, choose one that you think
most deserves to be considered further, and give a reason.
103
List 1: What benefits does / could knowledge management bring to your
organisation?
Benefit Elaboration from comments received
1 Improved consistency of information - everyone is referring to the same versions and documents
Saves time if someone can be pointed towards a document rather than being offered an explanation. Users need to be able to tell what version a doc is even once it has been printed out.
2 Reduced need to reinvent the wheel for every project
3 Improved speed of issues resolution by making existing solutions to past problems available
Needs appropriate filtering and keywords.
4 Training for new starters - provides a consistent set of information and experience for them to tap into
Very useful for contractors and handovers too.
5 Shared workload - no single person is the sole keeper of knowledge
Avoids disaster when someone leaves, but documentation usually fall behind what is in someone‘s head.
6 When you are aware that something similar has been done before, knowledge management makes it easy to find the information you need
Usually easier to go to the source of the information directly.
104
List 1 wildcard
Choose your wildcard from the list below by adding a reason for your selection. If
you do not think any of the items below need to be considered further, leave the
Reason column blank.
Benefit Reason for considering further
New work can get off the ground more quickly
Lets customers know what we already know about a product and reduces support calls
It encourages a culture of sharing and cooperation
Urgent information can be pushed to those who need to see it
Increased quality of finished product
Increased ability of an organisation to recognise synergies and opportunities
The ability to share knowledge outside the business
105
List 2: Which techniques are most consistently practiced on your projects?
Rank Most consistently practiced techniques
Elaboration from comments received
1 Common storage area / intranet for project documents
Good idea to storing key documents, but needs housekeeping and a good structure to ease finding documents. Invaluable on large projects.
2 Defect list for technical bugs Very useful measure of quality, remaining work, trends and pattern analysis.
3 Ongoing issues list throughout a project
Only useful if small enough to manage, and not too detailed.
4 Lessons learned written up and placed in shared location at end of project
Would be more useful if they force entry into a common database with relevant topic keywords.
5 Post project reviews Sometimes compiled and shared in meeting form. Useful to the individuals within the project team, but not used outside very much.
6 Progress reports shared with wide audience
Can be difficult to get these read by appropriate audience, perhaps because they are at the wrong level - too detailed or too high level.
106
List 2 wildcard
Choose your wildcard from the list below by adding a reason for your selection. If
you do not think any of the items below need to be considered further, leave the
Reason column blank.
Benefit Reason for considering further
List of unresolved issues at end of project
Best practice examples in a shared area
End project reports
Flowcharts
Sharing checklists
Wiki
Workshops with other project managers
Project blogs
Published articles that highlight known issues and share useful information
Instant messaging
Defect list for problems with processes
Daily log of conversations and phone calls
107
List 3: What types of knowledge have been hardest to capture and share
effectively?
Rank Hardest to share knowledge types
Elaboration from comments received
1 Knowledge that will help provide more accurate estimates in future
This is an area that is usually poorly recorded:
- Resources too busy to focus on introspection.
- Time recorded poorly, sometimes entered weeks after the event.
- Unwillingness to be open and honest with themselves and others.
- Difficulties breaking down the tasks and including the intangible work e.g. thinking time.
- If work is unique, tendency to guess rather than seek equivalent parallels and use the data for predictions.
- Tendency to get stuck in to the detail rather than take a step back and discuss the work and estimates.
2 The solution to specific technical issues that have had to be overcome
It‘s so easy to spend days trying to solve a technical issue and not write it down for next time. Bug trackers that include resolutions can be helpful for this.
3 Subjective opinions - discussions on how something should have been done or how successful a particular approach was
People can be protective of their approach, so difficult to break down into the constituent parts.
4 Relationship information - who is helpful and who is not constructive
Not something that is usually written down.
5 Keeping information general enough to be applied in other scenarios without being too vague.
Information needs better structure including considered keywords, facts detailed, inferences explained and key messages summarised.
6 It can be hard to choose how much information to share to achieve a manageable quantity of information
Knowledge needs to be reviewed regularly and trimmed on occasion.
108
List 3 wildcard
Choose your wildcard from the list below by adding a reason for your selection. If
you do not think any of the items below need to be considered further, leave the
Reason column blank.
Benefit Reason for considering further
Mistakes made early on in a project, long before a post-project review or lessons learned reports
Lessons learned in other teams
Ad-hoc work, where there is an impression that documentation is not justified because of the small scale of the work
Root cause of delays and schedule slip
Emails and documents can be hard to keep track of if they are being contributed to by many people.
Making use of lessons learned from other projects is difficult because they are not communicated widely or not bought into at the right level.
Early warning signs
An understanding of the problems caused by too much complexity in a product specification - what constitutes simple?
Cause and effect
It's hard to share knowledge without defining the scope of the knowledge too narrowly to the extent that other teams cannot see its relevance to them.
109
C.2 Results
List 1 wildcard responses
Benefit Reason for considering further Votes
New work can get off the ground more quickly
A: Otherwise tend to repeat the same mistakes or forget lessons learned
1
Lets customers know what we already know about a product and reduces support calls
B: Helps reduce support calls (and hence cost) without requiring additional development effort – not always financially viable.
C: Reduces total cost to business and makes customers feel better (in a fluffy/loving sense) about the company they are dealing with as being knowledgeable and the authority on their own product.
2
It encourages a culture of sharing and cooperation
D: All of the above depends on people getting into the habit of thinking about sharing info
1
Urgent information can be pushed to those who need to see it
Increased quality of finished product
E: From the reasons above: consistency, less time re-inventing the wheel, and faster resolution can all have benefits for quality and can increase the amount of time spent improving the product rather than just getting the job done.
Increased ability of an organisation to recognise synergies and opportunities
F: Opportunities get lost because knowledge isn‘t shared effectively. People have to crack on with the next product on their list and currently do this without any reference to knowledge already gathered/lessons already learned.
1
The ability to share knowledge outside the business
110
List 2 wildcard responses
Benefit Reason for considering further Votes
List of unresolved issues at end of project
Best practice examples in a shared area
End project reports C: Summarises project and acts as good point of reference for anyone looking back. Also excellent for capturing lessons learnt, impact on the business and a summary of the project in one location.
1
Flowcharts
Sharing checklists F: We have started to do this very recently on our large secondary project and it is proving very useful.
A: Used frequently as a visual aid for discussion and helps when trying to confirm a process.
2
Wiki
Workshops with other project managers E: This isn't something I do at all (no other project managers to work with.), but it is something I think would be valuable to share experiences and problems.
1
Project blogs
Published articles that highlight known issues and share useful information
B: Essential at the end of the project to avoid support calls/escalations.
D: An easy place to find the solution to problems that have been solved before
2
Instant messaging
Defect list for problems with processes
Daily log of conversations and phone calls
111
List 3 wildcard responses
Benefit Reason for considering further Votes
Mistakes made early on in a project, long before a post-project review or lessons learned reports
B: This relies on people‘s memories. Need to track it ongoing, e.g. keep an open log all through the project and/or review at each stage boundary.
1
Lessons learned in other teams D: Systems tend to be locally based – people don‘t bring this info with them
1
Ad-hoc work, where there is an impression that documentation is not justified because of the small scale of the work
E: on ―dynamic‖ (i.e. constantly changing) software projects, there are often lots of minor ―tweaks‖, which end up being undocumented and/or arbitrary decisions made by developers.
F: We have had a number of projects carried out on an ad-hoc basis, with nothing of significance to other projects recorded. The editors who tend to work in this way tend to work in a fairly isolated way in their silos without learning from others or giving of their experience.
2
Root cause of delays and schedule slip A: This sometimes seems to be brushed under the carpet if the cause is resource from another team or people just being slow to respond and I think it is important to capture.
1
Emails and documents can be hard to keep track of if they are being contributed to by many people.
Making use of lessons learned from other projects is difficult because they are not communicated widely or not bought into at the right level.
Early warning signs
An understanding of the problems caused by too much complexity in a product specification - what constitutes simple?
Cause and effect
It's hard to share knowledge without defining the scope of the knowledge too narrowly to the extent that other teams cannot see its relevance to them.