Network analysis methods for assessment & measurement
-
Upload
patti-anklam -
Category
Business
-
view
2.748 -
download
2
description
Transcript of Network analysis methods for assessment & measurement
Network Analysis Methods for Assessment & Measurement
January 14, 2012
Patti Anklam
With June Holley and Claire Reinelt
Webinar Goals
Share current thinking about how network analysis is used in designing and evaluating nonprofit programs
Provide examples of network analysis used in assessment and measurement contexts
Stimulate thinking about correlating network analysis with measurement and evaluation outputs and outcomes
2
Network Thinking & Non-Profits
The Evolution of Network Thinking
4
What is Network Analysis?
• Social network analysis (SNA) is a collection of techniques, tools, and methods to map and measure the relationships among people and organizations
• Organizational network analysis (ONA) often refers to the use of SNA methods in the context of organization dynamics and development
• In practice, we use these tools to map connections among people and ideas, issues, and other entities as well as the social and organizational connections
5
Network Analysis: The Method in a Nutshell
6
Step Activities/Tools
Design Identify boundaries Clarify and design questions
Collect Data Surveys Interviews Facebook, LinkedIn Email logs
Analyze data to generate maps and metrics
(Netdraw/UCINET, NodeXL, Gephi … many others)
Review data Validate; look for questions
Prepare evaluation Match network results with context and stories
Move into action Weaving & other interventions
Survey Example
7
Survey Example – Demographic Component
8
Survey Example – Affiliation Component
9
Survey Example – Network Questions
10
Network Questions Probe Relationships
11
Analyses Outputs: Map Patterns
Multi-Hub Hub and Spoke
Stove-piped (Siloed) Core/Periphery
12
Quick View: What an Analysis Can Tell
13
• Overall very well connected
• One region distinctly clustered with few connects to other regions
• Staff are highly central
• Identification of key connectors
Reasons for a Network Analysis: Examples
1. Assessment, Planning, & Weaving
2. Measure changes over time
3. Sense-making & story-finding
4. Positioning and working with individuals in the network
14
Assessment, Planning, & Weaving
• Assess the network’s capacity for collaboration, information transfer, innovation
• Identify key individuals
• Establish goals for enhancing connectivity
• Create an action plan
Strategic Purpose
15
Assessment: Capacity for Collaboration
Source: Transcending Boundaries: Strengthening Impact. The Full Potential of a Justice Network (Research & Network-Building Project Report, April 2011, Criminal Justice Funders Network). Courtesy of June Holley.
When funders indicate with whom they would like to work in the near future, the network becomes more robust. Funders are saying they want to work more together.
Current Funder Interaction Network Future Funder Interaction Network
16
Assessment: Affiliation Network
• Identify potential relationships among people based on shared events, meetings, ideas, or areas of expertise
• Nonprofits use this to see which organizations “attach” to different ideas
• Forms the basis for network weaving
Strategic Purpose
17
Drill Down Into Affiliation Network
18
• Identify people with common interest – basis for building communities of practice
• See which people share interest in multiple issues or topics
• A way for the network to reveal itself and have rich conversations
Measuring Changes Over Time
1
2
3
4
5
67
8
9
10
1112
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
Very Well
Well
Somewhat
2005
Maps copyright © 2012 New Directions Collaborative Source: Boston Green & Healthy Building Network, Beth Tener and Al Nierenberg, January 2008
19
1
2
3
4
5
67
8
9
10
1112
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
2007
Boston Green & Healthy Building Network
• Look at the whole network and its components:
– Overall cohesion
– Degrees of separation
• Good for comparing groups within networks or for comparing changes in a network over time
Analyses Outputs: Metrics
• Look at positions of individuals in the network:
– # of connections
– Favorability of position
• Good for identifying people who are well positioned to influence the network or to move information around
Individual position metrics Overall network metrics
20
How the Metrics Enhance the Maps
2010
2011 Year # Density Avg #
ties
2009 55 2.2% 1.2
2010 90 2.7% 2.4
2011 85 5.3% 4.5
2012 82 8% 6.88
2009
2012
21
Sense-Making & Emergence
• Barr Foundation Fellows Program
– See changes over time, but really to see how the network has supported emergence
– Work to shift Barr staff from the center
Pat Brandes
Source: Networking a City, Marianne Hughes & Didi Goldenhar, Stanford Social Innovation Review, Summer 2012
22
Sense-Making: New School Development in Boston
• An intentional network may have no other purpose than to enable emergence
• Maps that show the evolving relationships within a network help to identify powerful network stories
23
“This person has helped me accomplish work-related tasks.”
Source: Networking a City, Stanford Social Innovation Review, Summer 2012
Positioning: The Individual View
24
Node Betweenness Indegree OutDegree
62 792.67 26 30
80 660.48 17 32
64 530.61 20 33
23 333.36 20 14
71 321.42 21 20
56 316.42 20 18
• Centrality metrics identify people with the most ties (in-degree and out-degree)
• Those positioned to move information around in the network or be in the know (betweenness)
• Can identify people to lead task teams, to provide resources to, or to train as weavers
Tracking Individuals’ Changes
25
I learned something from this person that made me a better leader. – 2009
2005
2007
2008
2009
2010
Tracking Individuals’ Changes
26
I learned something from this person that made me a better leader. – 2011
Network Analysis & Measuring Outcomes
Summary – What We Know
• Measure the cohesion of the network overall:
– High-level structure (stove-piped, core/periphery, highly clustered)
– Average degree of separation
– Average number of connections each person has
• Identify individuals by their centrality to the network:
– Core or periphery? How do you bring people in from the outside?
– Broker? Connector? Facilitator? Bottleneck?
– Number and diversity of connections
• See changes over time
28
What We Can Measure and Show in an Analysis:
Things We Can Do With What We Know
Ways to change patterns in
networks
Practices from the KM/OD Repertoire
Weaving. Create intentional connections
Convene. Make introductions through meetings and webinars, face-to-face events
Increase the flow of knowledge Establish collaborative workspaces, install instant messaging systems, make existing knowledge bases more accessible and usable; implement social software or social network software
Create awareness Provide expertise directories
Connect disconnected clusters Weave: establish knowledge brokering roles; expand communication channels
Create more trusted relationships Assign people to work on projects together
Alter the behavior of individual nodes Create awareness of the impact of an individual’s place in a network; foster network literacy
Increase diversity Add nodes; connect and create networks; encourage people to bring knowledge in from their networks in the world
29
Measurement Challenges
• Maps area snapshot in time
• Targets and thresholds
– How much cohesion is “enough?” Is there a point at which increasing the number of ties makes the network less efficient?
– Is it reasonable to set a target for the cohesion metric?
• Tying Network Metrics to Outcomes
– We have to think of the metrics as indicators and as correlates of other survey questions
30
Source: Dave Snowden, Cynefin Advanced Practitioner’s Course December 2012
Questions?
31
Question
• [email protected] http://www.pattianklam.com
•[email protected] http://www.leadershiplearning.org/
•[email protected] http://www.networkweaving.com
Thank you.