EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D...
Transcript of EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D...
![Page 1: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/1.jpg)
EVALUATING NETWORK GROWTH: A CASE STUDY
USING SOCIAL NETWORK ANALYSIS
Goodier, S.;
;
© 2018, GOODIER, S.
This work is licensed under the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/legalcode), which permits unrestricted use, distribution, and reproduction, provided the original work is properly credited.
Cette œuvre est mise à disposition selon les termes de la licence Creative Commons
Attribution (https://creativecommons.org/licenses/by/4.0/legalcode), qui permet
l’utilisation, la distribution et la reproduction sans restriction, pourvu que le mérite de la
création originale soit adéquatement reconnu.
IDRC Grant/ Subvention du CRDI: 107311-001-Research into Open Educational Resources for
Development
![Page 2: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/2.jpg)
Sarah GoodierROER4D Evaluation Advisor
EES 2016
29 September 2016
Evaluating network growth:A case study using Social Network Analysis
9/29/20161
",-'I-'t-.'ROER4uesearcn an Open amcauonal uesources lorDevalopmen1
216 mu xonnmnzz
IDRC 38 CRDI E35.’-.E:;Fi9uE5:}pMg‘ V)
r‘ L‘
![Page 3: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/3.jpg)
Outline
• The project being evaluated: ROER4D
• Evaluation framework: what & how
• The Tools workflow
• Findings: Twitter network growth & online interactions
• Discussion & conclusion
![Page 4: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/4.jpg)
THE ROER4D PROJECT
![Page 5: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/5.jpg)
in the Global South
• In what ways, and under what circumstances can the adoption of OER address the increasing demand for accessible, relevant, high-quality and affordable education and what is its impact in the Global South?
Research on Open Educational Resources for Development
http://roer4d.org/
![Page 6: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/6.jpg)
ROER4D has 86 researchers across 18 sub-projects in 26 countries across 16 time zones
h on Open Educational Resources for Development in the Global Sou
GENERAL OBJECTIVE:
IMPROVE EDUCATIONAL POLICY, PRACTICE and RESEARCH in developing Countries
by beuei understanding me use and impad of om
Kongullavnojecr CLUSTERS Augusl 201 3, Febnury zon
MghmkunvvPmsvan
adn anoioan ' n -Ie‘“”“ ~‘( v0 a k""=*'ia ' Philippines
Tamania
Brazil
Colombia
‘’ V 5viLanka
Teacher educate ‘ ' Malaysiaado of OER Rvnndl
Kenya
Fem
npunumensiwneseemioemiiaeueiepinenx ,,_,,,M,__,,,,_,,__,_,,,,‘,_,,__
Imm) Wllxvwyvvvxrll e.i...e.;.n.n,e.,..i.
mineuum/mend (‘A2 in m/ >1oA><i=r—uEL mm. IHIVVIIQVV
'°“‘"‘ “‘“‘ *"““”‘ ” ‘ ”‘ “ “““““
“ L ROER4D w-79-n iuncagcmi
![Page 7: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/7.jpg)
1. Build an empirical knowledge base on the use and impact of OER in education
2. Develop the capacity of OER researchers
3. Build a network of OER scholars
4. Communicate research to inform education policy and practice
5. Curate output as open content
ROER4D Objectives
![Page 8: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/8.jpg)
EVALUATING ROER4D
![Page 9: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/9.jpg)
1. Build an empirical knowledge base on the use and impact of OER in education
2. Develop the capacity of OER researchers
3. Build a network of OER scholars
4. Communicate research to inform education policy and practice
5. Curate output as open content
ROER4D Key Evaluation Areas
EVA
LUA
TIO
N
Using the utilization focused evaluation (UFE) framework
![Page 10: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/10.jpg)
What is utilization focused evaluation (UFE)?
UFE is centered around intended use by intended users
- What do the users (key stakeholders) want to know?
- How will answers to the evaluation questions potentially help to improve the project?
![Page 11: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/11.jpg)
UFE in 12 steps:Steps are iterative,
not linear
Utilization Focused Evaluation Framework (Adapted from Ramirez & Brodhead, 2013)
step 1
Assessmg pmyam .mm.;s
¢Subl
Asxesxmg evahnamt madlnesx
sun 2
Vdznumns Dnmzlv mnendzd
51:9 5;
menufitatmnvi pnmarv
mended uses
m s:
rmm my gnuuwnn
¢may 1:
ivuvumnn awn
Sllp 3:
Smmmfnn plus!
sun 9:
mi: «mum
3:20 In:
um: anllvfls
sxevFicmlzfiun :11 la:
Sign A
5-muons! anawfls
step 12
Meta zvamanon
![Page 12: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/12.jpg)
1. Build an empirical knowledge base on the use and impact of OER in education
2. Develop the capacity of OER researchers
3. Build a network of OER scholars
4. Communicate research to inform education policy and practice
5. Curate output as open content
ROER4D Key Evaluation Areas
Social networking
KEQ:To what extent has ROER4D built a network of OER scholars?
![Page 13: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/13.jpg)
Twitter in communications
mm wmmu
ammnrgzm
M H . .4
>h:4t{|YY ‘ HV m
r~‘.w>«.:u vnhw K‘
|W'fiI®ayflMHHIMmi mama amnrre
![Page 14: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/14.jpg)
flfl
![Page 15: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/15.jpg)
The Tools workflowData collected monthly or
when the PIU requires an update
Account activity
\Interactions - 7-day w
/'
Interactions - over time
![Page 16: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/16.jpg)
FINDINGS
![Page 17: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/17.jpg)
Twitter Analytics325
am
275
250
§
Number:11
lnzilelnentsE
E
5
§
15
0;
u man exnands
I an clicks
I usar pmme clicks
- lzvontes
- mnlles
- retweets
![Page 18: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/18.jpg)
TAGSom ov-.--;«»_3a...m.
V7
Or-mu°"“"“"‘”“
o«w==_gO~>m_5a..w...°”"‘”"“ ”“‘“""“ Owmo 3--=-5
°"‘’"M 0“M ‘W
‘mmommm
mu. o..aa..;_')u ..... (cm, /9 CW“ on-ma
weom.“ 0,‘.
/,.o \��o��o��q�W _..W
\��o��o^�q� .
um“ 0 Oon-um
MMM°u_U,,g_......A. o..m,.;§)'=-'~=~~
Oow»
Oo,...,mo~u.-...
5"
cm... 0 .,..
O
one-mnu.
0‘“‘“"L.u= u-4 ._..q....m3
0,...°“"'“"“"‘° O
O
ob-‘UBuA°Dl£lElA“9om,“
O>"“=*“.‘,“""m-«...— —— Rm...“ :1
‘HEW:M
ow ommm
° ‘3~~. lweets repnes mennons
».fi..&,_,_,_m4 own Qmnwe-u - scveen name ROERAU °**"“* °‘“‘
\ Owfg (mm. .., M May
0....‘ o‘‘‘'‘‘“!0“
,
1417 m Orr xmm
Om». omen.“ 5.2.2.4
owm qM,...,....‘ N omo_..........‘Dl,,h¢¢,£,,r;,‘.«..g
o.........g.
fig. 9a....-.1o...~,,.g.;,,mMz
V
\��o��o��q� O=='=~'«--=:~ om... omM..._.,§.‘,nm‘,,H___Kf§j;-mu:"d..4I4foml;‘@‘R~
“
M13“... ... emu.“
o,..,, 5”.‘ ,om... o,.:..m.‘
oc..x..m...‘
“Hum”
_W_ o...-...._m_
onC’ 0
wow“, :. o...m,O
.\��o��o��q� cm on 15117}
ow. “EBA”!
°"“‘"“ OM" °’"“‘“'"W
ow w-,.,,. ...
'”
°"“"""°0... IL: °,__M__
Owm mo......m0
o-«MuO’‘‘’‘‘‘ 0 '
0.4..
OW \ own 0......“ 0.).-Am.7
ommn nm
677 nudes 324 edges! keY / mermuns—-re1wee1
own I\ m
omsie;-;...,~om-w om.
nu». nun,
O-=4-«A»-M»°'fid'°'3wM
..
aw--W om.»
nu
![Page 19: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/19.jpg)
NodeXL
Kev
. Indired tweetexs
. Isolated network
. Isolated tweeters
5 Direct and central tweelels
£7/an
(7. T19
1775
![Page 20: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/20.jpg)
DISCUSSION & CONCLUSION
![Page 21: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/21.jpg)
Key points
SNA focuses on relationships – good for networking
Tools workflow for ROER4D networking data
Tracks network growth and interactions over time
Evidence of success in the ROER4D project
Informed project & fueled improvement
Can be applied to other projects
![Page 22: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/22.jpg)
This work is licensed under a Creative Commons Attribution 4.0 International License.
Website: http://roer4d.org
Contact author: Sarah Goodier [email protected]@SarahGoodier
Excluding images,
screenshots and
logos and/or
unless otherwise
indicated on
content
Thank you!
![Page 23: EVALUATING NETWORK GROWTH: A CASE STUDY USING … › bitstream › ... · Sarah Goodier ROER4D Evaluation Advisor EES 2016 29 September 2016 Evaluating network growth: A case study](https://reader036.fdocuments.in/reader036/viewer/2022062603/5f0d97f87e708231d43b205e/html5/thumbnails/23.jpg)
Further reading:
About ROER4D:• Hodgkinson-Williams, C. (2013). Research on Open
Educational Resources for Development in Post-secondary Education in the Global South (ROER4D) - Scoping Document. Available online: http://hdl.handle.net/11427/8430 [Last accessed 10 February 2015].
• Hodgkinson-Williams, C. and Cartmill, T. (2014). Research on Open Educational Resources for Development in the Global South: 1st Technical Report 23 June 2013 to 27 August 2014. Available online: http://hdl.handle.net/11427/9695 [Last accessed 11 February 2015].
About evaluation:• Patton, M. Q. (2008). Utilization-focused evaluation.
California: Sage Publications Inc.
• Ramirez, R. and Brodhead, D. (2013). Utilization Focused Evaluation: A Primer for Evaluators. Penang: Southbound.