Where to Look for KM Success - Murray Jennex

Copyright Foundation for Knowledge Management.com, LLC

Where to Look for Knowledge Management Success

Murray E. Jennex, Ph.D., P.E., CISSP, CSSLP, PMPProfessor, San Diego State UniversityEditor in Chief International Journal of Knowledge ManagementCo-editor in Chief International Journal of Information Systems for Crisis Response and Management

A little About Me At San Diego State University since 2001 20 year commercial nuclear power engineer prior US Navy Nuclear Propulsion Officer prior Over 150 articles, chapters, books, proceedings Editor in Chief International Journal of Knowledge

Management Co-editor in Chief International Journal of Information

Systems for Crisis Response and Management Teaching Information Security, Systems Analysis and

Design, Decision Support, Knowledge Management BA Chemistry and Physics, MBA, MS Software

Engineering, MS Telecommunications Management, PhD Information Systems


Copyright Foundation for Knowledge Management.com, LLC1/5/2007

Introduction This presentation is part of the IJKM effort to

define the KM discipline We propose that for KM to be considered a

discipline we must be able to identify what leads to KM success and define what is successful KM

Part of this project was to get some consensus on what successful KM is

We thought this would be relatively easy to do but were surprised by reality (but guess we shouldn’t be)


Implementing KM First we will discuss what it takes to have a

successful KM implementation We will look at two main topics:

KM Critical Success Factors KM Success/Effectiveness Models

Critical Success Factors tell us what needs to be present to be successful

Success/Effectiveness models help us understand: How success factors relate to each other The process of implementing KM How to measure KM success


KM/KMS Critical Success Factors

Critical Success Factors are those factors that have been found to have the most impact on KM and are determinants of KM success These success criteria were identified through a

number of studies using a variety of research methods and overall looking at over 200 KM/KMS projects

Success factors are presented in order of importance as defined by the number of studies that identified the success factor

Note that all are critical success factors


Critical Success Factors

A Knowledge Strategy that identifies users, sources, processes, storage strategy, knowledge and links to knowledge for the KMS

Motivation and Commitment of users including incentives and training

Integrated Technical Infrastructure including networks, databases/ repositories, computers, software, KMS experts

An organizational culture that supports learning and the sharing and use of knowledge



A common enterprise wide knowledge structure that is clearly articulated and easily understood (an ontology)

Senior Management support including allocation of resources, leadership, and providing training

Learning OrganizationThere is a clear goal and purpose for the

KMS



Measures are established to assess impact of the KMS/knowledge use and to verify that the right knowledge is captured

Search, retrieval, & visualization functions of the KMS support easy knowledge use

Work processes are designed that incorporate knowledge capture and use

Security/protection of knowledge


Success/Effectiveness Models

Success/Effectiveness models attempt to take critical success factors and established theory and relate them in a way that will explain why success occurs

Will look at The Jennex Olfman KM Success Model Adapted the DeLone and McLean (1991,

2003) IS Success Model to KM Model is a recursive model which indicates

what has to be present for success to occur


Jennex/Olfman KMS Success Model

Technological Resources

Level of KMS

Use/User Satisfaction

Form of KMS

Intent to Use/ Perceived Benefit

System Quality

Knowledge/Information Quality

Richness

Linkages

Knowledge Strategy/ Process

Net Benefits

Service Quality

IS KM Service Quality

User KM Service Quality

Management Support


DiscussionThe quality factors need to be considered

in the development of the KM initiative and the associated KMS

Once created, the KMS and knowledge need to be used and/or re-used Success is initially measured by adoption

and use Actual use was found to be a poor indicator Intent to Use is a better measure

Ease of Use Social Factors Near Term Job Impact Long Term Job Impact


Discussion

Ultimate success, though, is through KM and knowledge having an impact on the individual and the organization

Defining impact has been nebulous or left to the individual/organization to define

The next stage of the research was to determine how to define and measure “impact”.


MethodologyStep 1 was to identify a definition of these

impacts through a definition of KM success

Generated exploratory survey using an expert panel from 30 IJKM IRB members Used a set of basic definitions

Exploratory survey had 103 responses: 13 KM practitioners 70 KM researchers 20 others including students and academics

interested in KM


Methodology First survey results were used to generate a

second survey proposing a definition of KM success and a set of measures that can be used to assess success

Second survey had 194 responses: 16 KM practitioners 114 KM researchers 64 others including KM students and academics

interested in KM but not active KM researchers.


Exploratory Survey Findings First survey results generated little consensus:

KM and KMS Success may not be the same (very surprising finding to me as in my Churchmanian view of KM and KMS I find them to be inseparable)

KM success is a combination of process and outcome with objective and subjective measures and getting the right knowledge to the right people

Use is not a good measure of success Doers liked definitions that focused on firm

performance Researchers (thinkers) do not seem to have a clear

idea of KM success and in many cases expressed opinions that KM success cannot and should not be defined


Second Survey Findings Practitioners tended to like the suggestions

Lower agreement on Process measures was misleading, they didn’t subtract anything, only added

Leadership was a problem due to its focus on supplying resources only

Researchers couldn’t agree on much Several either agreed with the definition then hated

all the measures or hated the definition but liked the measures

I think we all tend to have our own pet measures


Thoughts on Researcher ResponseBasically researchers missed the point

They wanted perfect measures Focused on the complexity of the issue Used both to say the research shouldn’t be

doneHowever, that isn’t an acceptable answer

A discipline must be able to define when it is successful

Managers expect to be able to define success Practitioners must be able to define success


Conclusion After analyzing the response results and

comments the following final definition was supported (basically the AQPC definition with dimensions):

KM success is a multidimensional concept. It is defined by capturing the right knowledge, getting the right knowledge to the right user, and using this knowledge to improve organizational and/or individual performance. KM success is measured by means of the dimensions: impact on business processes impact on strategy Leadership knowledge content


Next Step Research Survey was generated to test the definition

Items for KM success as well as the four dimensions were generated using the literature

Used a 7 point Likert scale Survey was tested using an expert panel with some

adjustments made

Survey was administered using surveymonkey Respondents with KM initiative experience were solicited

using KM discussion forums, KM academic lists, and personal contacts were sent emails soliciting participation

Two follow up emails were sent to encourage participation, data was collected for 3 months


Research DesignData was analyzed by segregating

responses into two groups Group one consisted of those reporting on a

successful project (6 and 7 on a 7 point Likert scale)

Group two was all other respondentsEach response was then analyzed to

determine if the dimension was met Three methods used to determine if

dimensions were met


Research Design Meeting the dimension determination methods:

Method 1 used the highest score for the associated items (dimension was met if this score was greater than 5)

Method 2 used the average of the scores for the associated items (dimension was met if this score was greater than 5)

Method 3 used the total number of associated items met with an item score of 6 or 7 (dimension was met if at least half the items had a score greater than 5)

t-tests were run between the two groups to determine if they were significantly different

The success group was then split into two groups, the 7 score and the 6 score groups with t-tests run to determine if they were significantly different

Results: Respondent Demographics


Position Overall n=89

Successful Project n=57

Unsuccessful Project n=32

KM Practitioner

20.2% (18) 24.6% (14)

12.5% (4)

KM Manager

29.2% (26) 31.6% (18)

25.0% (8)

Academic 21.3% (19) 12.3% (7)

37.5% (12)

KM Researcher

20.2% (18) 22.8% (13)

15.6% (5)

KM Student 9.0% (8)

8.8%(5)

9.4%(3)

Experience (years)

Overall n=89

Successful Project n=57

Unsuccessful Project n=32

0-2 13.5%(12)

14.0%(8)

12.5%(4)

3-5 22.5%(20)

21.1%(12)

25.0%(8)

6-10 21.3%(19)

21.1%(12)

21.9%(7)

>10 42.7%(38)

43.9%(25)

40.6%(13)

Respondents by Position, nearlya 50/50 split between academiaand practitioners

Respondents experience levelAlmost 2/3s with over 6 yearsexperience

Results (success/non-success)


Method Success Group n=57

Nonsuccess Group n=32

High Value Method 1

3.4 (0.9909)

2.7

(1.3102)

Average Value Method 2

2.6 (1.3595)

1.3

(1.2854)

Item Count Method 3

2.5 (1.3379)

1.4

(1.3664)

Total Items (25 possible)

13.4

(6.71091)

7.7

(5.0902)

T-test Results (success/non-success, all differences significant)


High ValueMethod 1

t51=2.61 p < 0.01

Average ValueMethod 2

t64=4.26 p < 0.01

Item CountMethod 3

t60=3.46 p < 0.01


t79=4.57 p < 0.01

T-test Results (7 score/6 score, all differences significant)


Method 7 Group n=16

6 Group n=41

t-test data

High Value Method 1

3.8 (0.5439)

3.2

(1.0701)

t51=2.9787 p<0,01

Average Value Method 2

3.5 (1.0328)

2.3

(1.3233)

t35=3.7243 p<0,01

Item Count Method 3

3.4 (0.8851)

2.3

(1.3398)

t41=2.9997 p<0,01


17.4

(6.1207)

11.9

(6.3332)

t28=3.0513 p<0,01

Results - Visual


0

0.5

1

1.5

2

2.5

3

3.5

7 6 5 4

# DimensionsMet

0

5

10

15

20

7 6 5 4

# Items Met

ConclusionsThe definition works

The more successful the project the more likely dimensions and dimension items will be met

The survey items are good indicators of measures that should be used to monitor for KM initiative success


Dimension Items Impact on Business Processes:

KM project improved the efficiency of the supported processes

KM project reduced costs for the supported business process

KM project had a positive return on investment for the supported processes

KM project improved the effectiveness of the supported processes.

KM project improved decision making in the supported processes

KM project improved resource allocation in the supported process


Dimension Items Impact on KM Strategy

KM project resulted in changes to my organization’s KM goals KM project resulted in the creation or modification of knowledge

related key performance indicators KM project resulted in changes to the way my organization

assessed knowledge use in the organization KM project resulted in changes in my organization’s incentives for

using and sharing knowledge KM projected resulted in my organization increasing its

awareness/mapping of knowledge sources and users KM projected resulted in increased resources for our KM systems

and repositories KM project resulted in the creation of new or additional knowledge

capture processes


Dimension Items

Leadership/Management Support KM project resulted in increased verbal/political

support for KM by top management KM project resulted in increased financial support for

KM by top management KM project resulted in increased awareness of KM by

top management KM project resulted in increased use/reliance on KM by

top management


Dimension Items Knowledge Content (5 overall items)

KM project resulted in increased knowledge content in our repositories KM project improved knowledge content quality of our repositories KM project resulted in my increased use or intention to use of knowledge

content KM project resulted in others increased use or intention to use of

knowledge content KM project resulted in my increased identification of needed knowledge

content and knowledge content sources KM project resulted in others increased identification of needed

knowledge content and knowledge content sources KM project resulted in my increased demand and/or searching for

knowledge content KM project resulted in others increased demand and/or searching for

knowledge content


Current Step Our current activities are applying the KM

success measures to previous studies to see how they would fare in explaining the observed results

The example is a longitudinal study done on a nuclear power plant where KM and knowledge use were found to improve productivity The initial study created a personal productivity model

to show where knowledge use improved individual productivity

Organizational improvement was shown using external measures such as capacity factor and NRC ratings


Applying the Measures Impact on business processes used the

original measures from the personal productivity model and the external measures identified to indicate organizational success

5 of the 6 measures were found to be met


Applying the Measures Impact on KM strategy used a survey

on what drove engineers to add knowledge content from the original study as well as the impact from the organization creating a KM position during the study

6 of the 7 measures were found to be met


Applying the MeasuresLeadership/Management Support used

the impact from the organization creating a KM position during the study, the support given by management for doing the study, and the heightened awareness of KM by the NRC during the study

All 4 measures were found to be met


Applying the MeasuresKnowledge content used the survey on

what drove engineers to contribute knowledge, the personal productivity model that showed increased demand for knowledge, and observations on actual use and perceived benefit.

All 5 measures were found to be met


ConclusionThe application of the KM success

measures to an existing longitudinal study that had previously been found to be successful with KM through other means found that all but 2 measures were met.

The ultimate conclusion is that the set of measures is a good fit


Where to Look for KM Success - Murray Jennex

Education

Transcript of Where to Look for KM Success - Murray Jennex