Day 54 - R&J Act1, Research paper source evaluations, and Participles,
J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2...
Transcript of J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2...
![Page 1: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/1.jpg)
1
Structuring Effects of Community Research
The Impact of the Framework Programme on Research and Technology Development (RTD) Network Formation
Brussels, 18th June 2009
![Page 2: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/2.jpg)
2
The three main objectives of the Netpact project
1. Ensure QA of database with data on EC funded projects and to establish thereto a closely linked dataset on publications to determine the impact
2. Conduct in-depth quantitative and qualitative network analyses of the RTD collaborations in 5 identified RTD fields
3. Develop policy recommendations for RTD network-driven policies to improve innovation and competitiveness in Europe
… for FP6: Citizens, Food, IST, LSH and NMP… for FP6: Citizens, Food, IST, LSH and NMP
![Page 3: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/3.jpg)
3
Conduct in-depth quantitative and qualitative network analyses…
… there are some 2.500 projects with in average 15 participants. In total there are some 10.000 unique organisations involved.
… this can be easily transformed into a two-mode network
BUT: Not all participants interact equally with each other.
![Page 4: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/4.jpg)
4
Network analysis: Calculation of interaction volume
Coordinator(English)
Participant A(French)
Participant B(English)
Coordinator
Participant A
Participant B
Project (members, Funding, duration)
• Interaction Volume is higher between coordinators and participants; between actors with many common projects and with same language as well as in long-term and higher funded projects.
• Interaction Volume is lower in projects with many participants.• Creating a valued graph assigning intensity values to the connections.
![Page 5: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/5.jpg)
5
The five major findings: Summary
1. Structure of FP6 RTD network- FP6 has a core-periphery structure dominated by a small number of
closely-knit organisations
2. Role of industry in FP6 RTD networks- Industry plays less of a coordinating role and is weakly embedded in FP6
networks, with exception of IST
3. Research outcomes of FP6 funding- FP6 has resulted in more upstream than downstream outputs –
providing the basis for more long-term positive impact on European competitiveness
4. Efficiency of FP6 RTD projects- Performance decreases as project size increases
5. New member state integration in FP6- New member state integration is dominated by a few actors in Poland,
Hungary, and the Czech Republic
![Page 6: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/6.jpg)
6
In-depth look at the five major findings
1. Structure of FP6 RTD network2. Role of industry in FP6 RTD networks3. Outcomes of FP6 funding4. Efficiency of FP6 RTD projects5. New member state integration in FP6
![Page 7: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/7.jpg)
7
The more funding, the higher the clustering
and centralization
1. Finding: FP6 networks have a core-periphery structure dominated by a small number of closely-knit organisations
CentralizationDegree to which networks are dominated by a few participants
Low
High
HighLow
ClusteringDegree to which
networks are dominated by
participants cooperating in two or
more EU funded projects
Circle size = funding
![Page 8: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/8.jpg)
8
1. Finding: Core-periphery structure
• Small number of closely-knit organisations in core that dominate FP6 RTD network- Core organisations highly connected to one another through several projects
- Remaining organisations in network periphery and only connected to core and not with one another
• Central actors primarily large national research associations- E.g., Fraunhofer Gesellschaft, CNRS, INSERM and universities
- Except IST where industry actors also central
• More funding thematic area receives, the stronger the pattern ofcore-periphery and heavy centralization- To point of oligopoly structures at instrument level
- IP strongest core-periphery and highest centralization structure followed by STREP and then NOE
![Page 9: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/9.jpg)
9
1. Explanation: Core-periphery structure
• High degree of path dependency and capability trajectory
- Organisations develop coordinating capability over time and thus reputation
as coordinators
- Efficiency gained through economies of scale by coordinating multiple
projects
- Periphery organisations seek more prestigious organisations for collaboration
to improve own reputation (“preferential attachment”)
• Reduced transaction costs between partners over time
- Shared understanding and common way of working developed
- Trust built as collaborate over time
- Relationships then built and strengthened over time
![Page 10: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/10.jpg)
10
1. Implication: Core-periphery structure
• Stable social structure leads to efficiency in research
- Lowered communication costs due to shared understanding and trust
- Critical mass and sharing of resources
- Specialization on research topics
But
• Stable social structure can have drawbacks
- “The rich get richer” – centralization leads to more centalization and“in-crowd”
- Knowledge becomes redundant
- Reduced creativity and radical innovation
![Page 11: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/11.jpg)
11
In-depth look at the five major findings
1. Structure of FP6 RTD network2. Role of industry in FP6 RTD networks3. Outcomes of FP6 funding4. Efficiency of FP6 RTD projects5. New member state integration in FP6
![Page 12: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/12.jpg)
12
2. Finding: Industry plays less of a coordinating role and is weakly embedded in FP6 networks with exception of IST
Organizational Roles•Circle size indicates total number of projects coordinating•Arrow size indicates number of projects coordinated •Arrow direction indicates who coordinates whom
Organizational Roles•Circle size indicates total number of projects coordinating•Arrow size indicates number of projects coordinated •Arrow direction indicates who coordinates whom
![Page 13: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/13.jpg)
13
2. Finding: Role of industry
• Industry accounts for 52.8% of all FP6 organisations - Industry is very active in NMP and IST, somewhat active in LSH and FOOD,
while CITIZENS is university domain
• Industry tends to be on periphery of FP6 networks - Industry connected to core of highly reputed, large national research
associations and universities coordinating FP6 projects
- Only in IST does industry dominate network (Majority of actors + Highest
share of coordination)
- France Telecom, Siemens, and Philips Electronics central in IST
• IP is instrument with highest industry participation and coordination- NMP-IP is instrument with highest industry participation
![Page 14: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/14.jpg)
14
2. Finding: Role of industry
Most central actors
Most central actors
![Page 15: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/15.jpg)
15
In-depth look at the five major findings
1. Structure of FP6 RTD network2. Role of industry in FP6 RTD networks3. Outcomes of FP6 funding4. Efficiency of FP6 RTD projects5. New member state integration in FP6
![Page 16: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/16.jpg)
16
Co-publication ratio from T0 to T1
5%
4%
3%
2%
1%
0%
Co-publication Data• 2,615 co-publications, ie publication by 2 participants (of total 84,670 publications )• “Citizens” not selected due to a significantly different publication pattern
Time Period• T0 = period from 4 years before project start until 1 year after project start• T1 = period starting one year after project start until max 5 years after
+8% +52% +25% +56%
Significant increase in rate of co-publication between RTD participants
Significant increase in rate of co-publication between RTD participants
3. Finding: FP6 has resulted in more upstream than downstream outputs – providing basis for long-term impact
![Page 17: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/17.jpg)
17
3. Finding: Outcomes of FP6 funding
• FP6 funded research has made a significant impact on the international scientific field- Increased co-publication activity between FP6 project’s partners
• I.e., two partners from same FP6 project publish one or more articles
together after having participated together in FP6
- Publications from PIs either with or without other FP6 partners have 50% higher impact than world average
- Co-publications by collaborating FP6 partners
• Significantly higher impact than publications in which FP6 partners do not co- publish
• Around two times the world average
![Page 18: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/18.jpg)
18
In-depth look at the five major findings
1. Structure of FP6 RTD network2. Role of industry in FP6 RTD networks3. Outcomes of FP6 funding4. Efficiency of FP6 RTD projects5. New member state integration in FP6
![Page 19: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/19.jpg)
19
4. Finding: Performance decreases as project size increases
Smaller teams (maximum of five partners) are more productive in publishing
Smaller teams (maximum of five partners) are more productive in publishing
0%
2%
4%
6%
8%
10%
12%
14%
16%
Larg
e
Med
ium
Smal
l
Larg
e
Med
ium
Smal
l
Larg
e
Med
ium
Smal
l
Larg
e
Med
ium
Smal
l
LSH IST Nano (NMP) Food
Shar
e Pc
P/A
llPcP share by thematic area
and project size
![Page 20: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/20.jpg)
20
In-depth look at the five major findings
1. Structure of FP6 RTD network2. Role of industry in FP6 RTD networks3. Outcomes of FP6 funding4. Efficiency of FP6 RTD projects5. New member state integration in FP6
![Page 21: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/21.jpg)
21
5. Finding: New member states
• Poland, Hungary, and to some extent Czech Republic are most highly integrated new EU member states (NMS)
• NMS participation varies across thematic areas- In absolute terms, IST most important with over 41% of NMS interaction
- CITIZENS has highest engagement of NMS (%)
- LSH is weak point for NMS (%)
• Several NMS organisations among top ten on instrument level, except for LSH
• NMS participation dominated by a few organizations - Budapest University of Technology and Electronics has 42% of all Hungary’s
FP6 projects
- Over 75% of Budapest University of Technology and Electronics in IST and over 55% of Academy of Sciences of Czech Republic in LSH
![Page 22: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/22.jpg)
22
Issues to be addressed
•Testing and refining the concept of interaction probability.
•Refine the approaches to better link persons, organization and output information ( e.g. publications, patents, technology offer,…)?
•Are the efficient ways to identifying the role of an participant in a project and resulting from that in a network?
•Is there a need for a research information system allowing to gather the footprints and following the dynamics?
![Page 23: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/23.jpg)
23
![Page 24: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/24.jpg)
24
Further information on the sample and the method
![Page 25: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/25.jpg)
25
![Page 26: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/26.jpg)
26
Data cleaning of the FP6 dataset led to a significant decrease in the number of unique participants ….
1. FP6 project data (source: EC)2. Worldwide publication data (source: Thomson Scientific)
![Page 27: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/27.jpg)
27
…as well as in a significant redistribution of organizations across organizational types.
IST Organizations
Num
ber
of p
arti
cipa
nts
LSH Organizations
•The current organizational categorization scheme does not adequately represent the true organizational types
-Current categories are not sufficient to clearly select different sets of project participants, i.e., there is no category for TTOs, public agencies or consultants.
-Industry participation is systematically underestimated since many companies are tagged as “Other” when it comes to organisational type.
+ 29,41 %
+ 55,56 %
![Page 28: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/28.jpg)
28
The shift when looking at all five thematic areas
• Significant shifts in organisational types• Remaining N/A organisations mostly have their legal name
missing, making it impossible to determine their type
+ 2% +100% +39% -79% -51% -6%
![Page 29: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/29.jpg)
29
Looking at the overall funding structure, we see significant differences.
• IST is by far the largest thematic program in overall funding• FOOD has the highest amount of funding and the largest number of participants per
project• LSH has the highest funding per participant
![Page 30: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/30.jpg)
30
Data cleaning
K A R L S R U H E U N I V ( T H )
U N I V E R S I T A E T K R L S R U H E
1. Transforming the original data to a vector-based format.
2. For each sequence of n characters that occurs in an organisation name an entry in the vector representation is created and in case of multiple occurrences increased.
3. With the index of each organisational name, we can calculate the distance between two names using vector-distance measures.
![Page 31: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/31.jpg)
31
Impact analysis: We received a satisfactory response rate for the RTD case studies.
Thematic Area
Number of RTDs
RespondingResponse
Rate of RTDs
Number of Organizations Responding
Response Rate of
OrganizationsCITIZENS* 15 94% 90 36%
FOOD 27 96% 195 36%IST 31 97% 113 22%LSH 30 94% 178 34%NMP 30 94% 171 32%
TOTAL 133 95% 747 32%
• For each thematic area, the selection criteria were based on twoperformance measures
• 2 Highest and 2 Lowest ”Publishing Impact” RTDs for each instrument• (2 High +2 Low) x 4 Instruments = 16 RTDs in total
• 2 Highest and 2 Lowest ”Centrality in the RTD Network” RTDs for each instrument• (2 High +2 Low) x 4 Instruments = 16 RTDs in total
* Only High Centrality RTDs were chosen due to different publishing patterns
![Page 32: J.Maier Structuring effects of Community Research › research › evaluations › pdf › ... · 2 The three main objectives of the Netpact project 1. Ensure QA of database with](https://reader036.fdocuments.in/reader036/viewer/2022070822/5f28901b12ba853e9a0158c4/html5/thumbnails/32.jpg)
32
Impact analysis: Breakdown of respondents by organization type and instrument
Org Type Number
Percent Total
HES 348 46,6
HSP 19 2,5
IND 108 14,5
OTH 38 5,1
REC 234 31,3
Total 747 100%
InstrumentType Number
Percent Total
CA 160 21,4
IP 241 32,3
NOE 251 33,6
STP 95 12,7
Total 747 100%