Measuring Team Effectiveness

Presented by Jill A. Marsteller, PhD, MPP

Based on work by Stephen Shortell, Jill Marsteller, Michael Lin, Marjorie Pearson, Shinyi Wu, Peter Mendel, Shan Cretin, and Mayde Rosen. “The Role of Team Effectiveness in Improving Chronic Illness

Care,” RAND/ UC Berkeley Improving Chronic Illness Care Evaluation. Medical Care, November 2004.

Outline of presentation

Define team effectivenessSituate in team performance theoryContextFactors, items & reliabilityShow associations -- “antecedents”

and “consequences”Conclusion

Team Effectiveness

Perceived organizational support Team self-assessed skill Goal agreement and participative

normsTeam autonomy/process ownershipInformation/help available

Figure 2.1--A Heuristic Model of Group Effectiveness

Internal Processese.g. conflict,communication

External Processese.g. conflict,communication

Task Designe.g. autonomy,interdependence

Group Compositione.g. size, tenure

Organizational Contexte.g. rewards,supervision

Effectiveness-Performance

Outcomese.g. qualityproductivity

-AttitudinalOutcomese.g. jobsatisfaction,trust

-BehavioralOutcomese.g. turnoverabsenteeism

EnvironmentalFactors

e.g. turbulence,industrycharacteristics

Group PsychosocialTraits

e.g. norms,shared mental models

Source: Cohen, S.G., Bailey, D.E., Journal of Management, Vol. 23, No. 3, 1997, page 244

Context in which measures were tested

40 teams participating in the Improving Chronic Illness Care Evaluation (ICICE)

Surveyed members of multi-disciplinary teams As many as 12 occupational categories Size 1 to 14 members

One disease-specific team per organization Organizations are hospitals, physician groups,

clinics, health plans, or health systems Participation was voluntary

Team Effectiveness Instrument

Original creator: G. Ross Baker at U. Toronto (ross.baker@utoronto.ca) (32)

We reduced number of items (23), defined own factors

ANOVA confirmed aggregation of individual-level responses to team level

Responses 1-7 (strongly agree to strongly disagree)

Varimax rotation, eigenvalues ≥1.0, clean loading ≥0.4

Factors, Items & Reliability

Organizational support (Alpha=0.85) This organization makes sure people have the skills

and knowledge to work in teams A team that does a good job in this organization does

not get any special rewards or recognition (reverse coded)

Senior management in the organization strongly supports our work

Senior management regularly reviews our progress in making change

Senior managers in my organization see success in this project as a high priority for the organization

Factors, Items & Reliability

Team Self-Assessed Skill (Alpha=0.90) Our team has been able to use measurement

very effectively to design and test changes After we have completed a change, team

members are excellent in reflecting and learning from the results

Members of our team were very successful in using information from our change cycles to design new tests of change

In making changes, our team was able to easily adapt change ideas to match the needs of our organization

Our team applied enough knowledge and skill to the work to get the work done well

Factors, Items & Reliability

Goal Agreement & Participative Norms (Alpha=0.90) Project team members agreed on the

project’s overall goals The project’s goals were understood by all

the project team members Most members of my team got a chance to

participate in decision-making Certain individuals in this group had special

skills and knowledge that the rest of us count on

The contribution of every group member was listened to and considered

Factors, Items & Reliability

Overall Perceived Team Effectiveness (Alpha=0.95) Organizational support Team self-assessed skill Goal agreement and participative norms Team autonomy (Alpha=0.81) Information/help available

Descriptives

Measure N Mean Std. Dev.

Min Max

Team Size 40 6.53 3.04 3 14

Team Skill 40 5.03 0.90 3.13 6.73

Goal Agree and Partic Norms 40 5.85 0.64 4.57 7

Organizational Support 40 4.78 0.93 2.20 6.27

Overall Perceived Team Effectiveness

40 5.24 0.74 3.47 6.64

Predicting Team Effectiveness

Independent Variable

Overall Perceived Team

Effectiveness

  Coefficient (sig. lvl)

Constant 1.11

Team Size -0.06*

Team Champion 0.69***

Patient Satisfaction Focus 0.49**

Cultural Balance 3.10*

Asthma 0.09

Physician Percentage on Teams 1.27*

N 40

F 5.29

p-value 0.0006

Adj R-Sq 0.4

*indicates p<0.10; ** indicates p<0.05; *** indicates p<0.01

Predicting Quality Improvement ActivityIndependent Variable Number of

ChangesDepth of

Changes

  Coefficient (sig. lvl)

Coefficient (sig. lvl)

Constant -49.62 -11.13

Team Size 5.79 3.90***

Team Size Squared -0.22 -0.19**

Patient Satisfaction Focus

-21.58*** -4.79**

Cultural Balance 118.85* 17.54

Asthma -10.75 -3.52**

Overall Perceived Team Effectiveness

12.00*** 4.69***

N 40 40

F 3.28 5.66

p-value 0.0122 0.0004

Adj R-Sq 0.26 0.42

* indicates p<0.10; ** indicates p<0.05; *** indicates p<0.01

Change in QI Clinical Process Score

Patient Survey~1300 patients, 29 organizations

Higher score greater improvement in process measures than average

Team effectiveness and team skill are consistently associated with greater process improvement

Conclusion

Applies to multiple settings Versatile Appropriate measures of team-level

phenomena Easily understood questions High Cronbach’s Alphas Well supported by theory Relate as expected to QI Questions are “actionable” Available free on the web,

www.rand.org/health/icice