Wang Qian, Li Li, Yang Yang

Dalian University of Technology

The International Conference on Peer Review, Research Integrity, and the Governance of Science

05-21-2012 Dalian, China

Dilemmas in Interdisciplinary Peer Review

Dialogue Mechanism in Interdisciplinary Peer Review

Tension between Normativity and Creativity

Conclusion

Partially across the disciplinary boundaries

Entirely across the disciplinary boundaries

How to guarantee an objective, comprehensive and fair evaluation of the peer review?

Dose it count a “peer review” when it applied across the reviewer’s own discipline?

In practice, peer review in academic field is commonly understood in a broader sense. The ‘peers’ are not limited to the same discipline. As long as the reviewers are experts from the relevant disciplines, or even simply expertise in an academic field, they would be considered as the peers who are competent to make interdisciplinary judgments on individual’s tenure and promotion, proposal funding and publication of research achievements.

Though it lacks of rationality, there is no Though it lacks of rationality, there is no alternative. alternative.

incomplete understandings of the interdisciplinary work and interdisciplinarians leads to overvalued or devalued assessment

difficulty in finding of experts with the same interdisciplinarity of the frontier researches

From the perspective on the objective law of disciplinary development, it is inevitable to have the expert of one discipline review the work or the researcher from relevant interdiscipline.

Given the limitation of work conditions, reviewers are often grouped together to make evaluations across their own disciplines, according to a broader disciplinary division.

number theory vs. topology in mathematics mechanics vs. biology science, technology and engineering vs. humanities and

social science

Rely on “hard indicators” such as the numbers of granted proposals, published papers, amount of research funding and the numbers of the award-winning projects.

In practice, it is impossible to find the fully appropriate reviewers from one or several disciplines.

Dialogue between experts in different disciplines, between reviewer and reviewee

Dispel misunderstandings and biases of disciplinary differences

Implication of academic ethics, enhance fairness and justice, prevent abuse and misuse of academic privileges

Instead of in-depth study, many reviewers resort to their previous accumulation and professional intuitions, considering dialogue as an extra burden.

Because of the asymmetry status, it is much harder for reviewees to have dialogue with the reviewers.

For interdisciplinary work——group experts from all the relevant disciplines, ask them to make judgments retrospectively

Provide opportunity to clarify disagreements between experts from different disciplines

score

Opportunity for the reviewee’s response

Explaining the significant concepts and research dynamic in order to avoid the possible misunderstandings

Focus on fundamental norms and methods, whether the work to be reviewed is rigorous, reliable and comprehensive

Do not easily negate the originality of innovative ideas and approaches, especially for the unanswered significant questions in science

The purpose of peer review is not to take peer reviewers’ judgments as the only criteria of approving new research, but to examine whether the approach and method of the new achievements are consistent with the academic norms. Therefore, it demands the maintenance of essential tension between the normativity and creativity of academic research.

The idea of maintaining "essential tension" in the conversion process of academic "paradigm", proposed by American philosopher of science and historians of science Kuhn is also applicable for interdisciplinary peer review.

Is the reviewee able to exactly fully grasp the disciplinary frontier dynamics and outcomes of previous studies?

Dose the reviewee have the required foundation of knowledge and the sufficient accumulation of the research?

Does the work to be reviewed meet the requirement of logic rigidity? Does it comply with the requirements on precise and reliable in the experimental level?

Dose the reviewee have applied an approach which has never been used by the predecessors? Dose it draw a conclusion that has never been made before?

Dose the writing of the work outcome comply with the normative rules? Is there any violation of the academic ethics?

Does the application of the research achievements may have adverse effects on the environment, human health and social life?

For the sake of providing sufficient space for development of

academic innovation ， the following arguments should not be

the reasons of negative judgments:

engage in the work which is not suitable for his own discipline and professional status.

propose the ideas which are considered as contrary to common sense and previous experience.

apply the approach against with the scientific tradition, and propose a theory that is difficult for the peers to identify.

Although the outcome can be seemed rational to some extent and, reflecting a certain value, there are some obvious defects.

It shows some theoretical significance but no practical application in the foreseeable future.

In accordance with the criteria in the peer reviewer’s own discipline, the interdisciplinary work originating from other disciplines is correspond to a low-level research.

natural science and engineering: doctoral students papers published in the SCI, EI publications

humanities and social science: outstanding scholars’ difficulties of publishing their papers in

SSCI, A & HCI publications

Absolutely comply with the European and American criteria to evaluate Chinese culture and ideology may finally result in the lost of our own characteristics of culture, which does obviously goes against with the improvement of humanities and social science research, as well as the cultural innovation in China.

First, if the idealized state of the peer review is considered as evaluating the work and the scholar of the same discipline, and nearly all the peer reviews in practice can be seemed as the “interdisciplinary peer review”, and every reviewer will confront the interdisciplinary review problems in varying degrees.

Second, no matter partially or entirely across their own disciplines, evaluating subjects other than the reviewers’ expertise is extremely difficult to ensure the accuracy, comprehensiveness and fairness of their identifications and evaluations. However, the emergence of such an irrational situation is inevitable, to a larger extent, which is difficult to absolutely avoid.

Third, in order to make sure the rationality of interdisciplinary peer reviewed, it is necessary to establish a dialogue mechanism in the interdisciplinary peer review from a systematic perspective.

Finally, in the course of dialogue, the maintenance of essential tension is indispensible between academic norms and creativity. In particular, it needs to avoid the peer reviewers simply assessing the scholar and work of interdisciplines or other disciplines by their own disciplinary criteria.

Thanks for Your Attention