The International Conference on Peer Review, Research Integrity, and the Governance of Science

May 21-23, 2012Dalian University of Technology, China

Peer Review: Theory, Practice and Prospects in China

Xiaonan HongDalian University of Technology

• Contents

• Part I

• Theoretical Research on Peer Review in China

• Part II

• The Evolution of Peer Review Practice in China

• Part III

• The Prospects of Peer Review in China

• Part I Theoretical Research on Peer Review in China

• In China, “peer review plays an important role in the allocation of research fund, the assessment of research results, the publication of academic paper, the evaluation, rewards and promotion of scientists and engineers. The seriousness, scientificity and fairness of the procedure of peer review should be guaranteed. ”

• In 1986, National Natural Science Foundation of China (NSFC) was established.

• The project review of NSFC follows the principles of “relying on the experts, promoting democracy, merit-based support, and fair and rational”, meanwhile, employs the two-stage review system which include the communication review of peer experts and the review of Reviewing Expertise Group.

• The current peer reviewer system consists of more than 80,000 communication review experts (including nearly one thousand members of Reviewing Expertise Group. A variety of Reviewing Expertise Group can be established according to the need). They are spread over the universities and research institutes all over the country.

• Reviewing Expertise Group is composed by the scientists who have made high academic achievements, have strong capabilities on academic judgment, are impartial, honest and upright, are enthusiastic on the work of science fund, and are prestigious in the scientific community.

• Reviewing Expertise Group implements the tenure system. A session is two years. Anyone can’t serve more than two sessions. Each session will be updated of about 50%.

• The peer review system consists of reviewer side (investment), reviewee side (consumption) and review practitioner side (typically, agent), as well as the necessary elements to implement it, such as, review standards, reviewer regulations, appeal procedure, oversight mechanisms and experts group.

• An ideal peer review system should include: less shortage of funds, selfless decision-making community, and abundant sources of experts, single discipline and simplex criterion.

• The methods of peer review mainly include: communication review, conference review by Reviewing Expertise Group, communication review + reconsideration by Reviewing Expertise Group review, on-the-spot investigation (project officers + experts).

• Peer review can be applied to all stages of academic research: pre-review (project selection), mid-review (inspection in the process), and post-review (assessment on the result)——Performance Evaluation.

• Along with the learning from the foreign experience, we also focused on domestic practice and experience.

• By 1991, Chinese science foundation system had been existing for ten years and NSFC for five years, however, peer review system gradually revealed its imperfections, even though this system had more superiorities than the planning system. As the conflict between supply and demand of research fund became increasingly acute in China, the problems in peer review system began to be generally concerned.

• Therefore, NSFC set up a special research group on peer review headed by Zhijiang Wang (academician of CAS) to investigate systematically the serious problems in peer review.

• Similar to the research of NSF on peer review in the U.S., this research program was mainly undertaken by experts in the field of science policy, with the administrative staff of Department of Science participated in the survey. Bureau of Policy was in charge of the coordination, and its staff also participated in this research.

• The result of this research was Methodology of Peer Review, which is published in 1996.

• This book contains five chapters. • Chapter Ⅰ is the overview of the definition, features,

social functions and the problems in implementation of peer review;

• Chapter Ⅱ makes a comparison on peer review systems between foreign and domestic science funding agencies;

• Chapter Ⅲ is devoted to the status of peer review of NSFC, comparing with the foreign peer review of science funding agencies, analyzing the results of this survey, proposing some suggestions on how to improve the fairness and openness of peer review;

• Chapter Ⅳ discusses the selection of experts, constructing the matrix system about the selection of experts according to the ideal standards which peer reviewer should meet and putting forward the idea of establishing the peer reviewers system;

• Chapter Ⅴ mainly addresses the issues of how to resolve the problem of ‘non-consensus-items’ in peer review, claiming that ‘non-consensus-items’ was a common phenomenon (about one third) in projects funded by NSFC and many reasons can lead to this phenomenon.

• This research group suggested that, non-consensus-items can be dealt with by the methods of classification, reevaluation significantly, feedback in time, special protection. If the non-consensus-items were caused by its advanced nature and originality, then they should be given special protection by establishing ‘Pre-research Project’. Since this research placed great importance on the communication and interaction with the administrative staff of NSFC, the conclusions were easily accepted by them.

• For example, NSFC set up the ‘small Pre-research project’ to support the innovation project with high risk.

• From this, we can see that NSFC took the suggestions from this book, which gained widespread impacts on Chinese science funding agency. It is still the representative works on peer review in China, although there is room for improvement.

• In theoretical research, many foreign scholarly works on peer review have been translated and published in China since the 1990s, some of which are top-notch pieces.

• For example, in 2004, the Chinese translation of On Being a Scientist edited by Committee on Science, Engineering, and Public Policy was published in China. This book, which is known as “Must Book of scientific research personnel”, aroused a tremendous response. The issues of conflict of interests, publication and openness discussed in this book are related to peer review.

• In the same year, Shanghai Scientific and Technological Education Publishing House published the Chinese translation of Betrayers of the Truth: Fraud and Deceit in the Halls of Science, which introduced the significant cases of data falsification, counterfeiting experimental results and plagiarizing the research achievement of others.

• In 2005, Tsinghua University Press published the Chinese translation of ORI Introduction to the Responsible Conduct of Research. This book is mainly geared to the needs of undergraduates who aspire to become researchers, graduates, general scientific research personnel and research management. It dedicates a chapter to discuss peer review, asserting that “peer review—evaluation by colleagues with similar knowledge and experience—is an essential component of research and the professional self-regulation”

• In March 1989, Public Health Service created the Office of Scientific Integrity (OSI) in the Office of the Director at NIH, and an Office of Scientific Integrity Review (OSIR) in the HHS Office of the Assistant Secretary for Health.

• In 1992, these offices were merged to create the Office of Research Integrity (ORI), which takes charge of the investigating, supervising and prevention on the misconducts in the fields of life science and medicine.

• Also in 1992,they published the Chinese translation of Shigeaki Yamazaki’s Misconducts in Science: Fabrication, Falsification and Plagiarism.

• This book probes meticulously in the problems, “Why misconducts in science occur?”, “Do we have the ways to guard against misconducts?”, and “How should we establish the ethical rules in science”, from both theoretical and practical aspects. The seventh chapter, “Review System and Misconducts”, reflects on the role of peer review as a “filter”. In 2006, the Chinese translation of Doing Honest Work in College, which was written by Charles Lipson, professor of University of Chicago and had been widely praised in U.S, was published.

• If On Being a Scientist focused on the self-discipline of scientific research personnel, then Integrity in Scientific Research: Creating an Environment That Promotes Responsible Conduct, which is co-edited by AAS, NAE and NIH, summed up the codes of ethics in the scientific research progress, indicating the extreme significance of compliance with these code. Its Chinese translation was published in 2007 and its translating project was initiated and organized by the Science Ethics Committee of Academic Divisions of CAS.

• The fourth chapter of this book, “Institutional Approaches to Fostering Integrity in Research”, addressed the methods of self-evaluation and peer review in scientific research institution.

• After its publication, CAS gave this book to each of the academicians of CAS and required them to start from themselves as on preserving his/her moral integrity, being strict with himself/herself, seeking truth and realism, indifferent to fame and fortune, insisting scientific ethics and fine tradition of scholarship, eradicating illegal and immoral conducts, striving to be excellent people’s scientists, living up to their lofty honor given by society.

• In 2007, Peking University Press published the Chinese transition of The Student's Guide to Research Ethics written by Professor Paul Oliver, etc.

• In February 2007, CAS and its Presidium of Academic Department issued “Manifesto on Scientific Idea” to the public, elaborating the concepts of scientific spirit, scientific value, moral code of science and social responsibilities of scientists comprehensively and intensively; printed and distributed CAS’s Opinion on Strengthening the Construction of the Research Ethics; set up the Scientific Research Ethics Construction Committee, intensifying the efforts on the construction of scientific research ethics.

• From 2008, Ministry of Science and Technology of China began to implement “Punishments for the Misconducts in Scientific Research during the Implementation of National Science and Technology Program (on Trail)” and founded the Science Research Integrity Construction Office. Any units or individuals can inform against the person who performed misconducts in the implementation of national science and technology program, and they are encouraged to do this using their real name.

• In 2008, Theory and Practice on Science and Technology Evaluation edited in chief by Xian’en Zhang was published. In chapter 2, he introduces the appraisal theory and its application and he dedicates the first section to discuss peer review. He believes that “peer review is a method or system to manage democratically the activities of science and technology (including research activity, research funding, and research management) by fully relying on the community of scientists themselves. More specially, peer review is the procedure to evaluate the items in the related field independently by the experts in this field or related fields by resorting to unified standard, the evaluation result of which can be served as the basis for decision making of the department concerned.”

• 1n 2009, Reader on Moral Norms in Scientific Research (Trial Edition) edited by CAS was published. It reported systematically the basic components of moral norms in scientific research, which is not only fundamental and systematic, but also targeted and normative. It also analyzed profoundly the bias and conflict of interests in peer review process.

• In the same year, Science Research Integrity Construction Office of MST edited and published Reader on Research Integrity. It illustrated the basic concepts related to research integrity, explained normatively the integrity acts in the implementation of research project and the publication and evaluation of research results, and proposed the methods to survey and deal with the misconducts in scientific research. In chapter 4, it specially discussed peer review, indicating that “peer review is the activities of evaluating the academic level or value of the scientific work by the experts in this field or related fields”, and the parallel terminologies of peer review are peer examination, peer consultation, peer assessment. It also proposed the normative requirements on reviewer and reviewee and set up special column to introduce the suggestions on scientific journal article review and the forms of reviewer’s improper behaviors. Meanwhile, Science Research Integrity Construction Office of MST also edited Guide to Research Integrity. It explicitly stipulated the norms of organizers, reviewers and reviewees, which will have an important impact on the normalization of peer review.

• Xu Gong expanded the insight on peer review from the individual behavior of reviewer to the collective behaviors, from moral norms to institution constraint, from internal factors of unfairness to its external environment, and tried to analyze the unfairness of peer review from the perspective of system, referring to the theories from sociology and economics, especially the analytical method of New Institutionalism. This progress was related to the social feathers of science development and science policy, as well the close relation between science system and economic system. In 2009, she published Science Policy and Peer Review: Comparative Study on Science System and Policy between China and U.S . This is a monograph about the comparative study on Science System and Policy between China and U.S from the perspective of peer review. It, based on a large number of first hand data and new methods, investigated the establishment and development of the science funding agencies and the peer review mechanism, as well as the evolution of science policy and review criterion; analyzed how to guarantee the fairness and overcome the functional limit of peer review; tried to explain the influencing factors and evolution trace of Chinese science system and policy in the horizon of “cross-society imitation”.

• In 2011, the Chinese translation of Daryl E. Chubin and Edward J. Hackett’s Peerless Science ： Peer Review and U. S. Science Policy translated by Wenhua Tan and Guoping Zeng was published. It provided answers to these questions: What is peer review? What aspects of science system are related to peer review? What is happening within the peer review system? How to improve and refine the peer review? In all, this book gave us a whole picture on American peer review and science policy. It is valuable for us to study, improve and refine Chinese peer review system and related policies.

• In the same year, the Chinese translation of Francis L. Macrina’s Scientific Integrity: Text and Cases in Responsible Conduct of Research was published. The fourth chapter of this book mainly discussed peer review and the ways to operate.

• Overall, Chinese researches on peer review concentrated on fairness of peer review in recent years, such as, the intensified conflict of interests, the reduced credibility of reviewers, the influences of scientific misconducts, etc. These problems are closely related to Chinese science policy, research environment and even the broader context of social transformation. In the entire transformation of Chinese economic and social system, moral consciousness and behavior restraint mechanisms changed greatly, and it highlighted the conflict of interests in various fields, including the peer review activities in science.

• Part II The Evolution of Peer Review Practice in China

• 1. The Evolution of Peer Review Practice in China• 2. The Practical Use of Peer Review in China• (1) Peer Review Policy of the NSFC• (2) Peer Review of National Social Science Fund of

China• (3) Peer Review of Humanities and Social Sciences

Fund of Chinese Ministry of Education

• Part II The Evolution of Peer Review Practice in China

• 1. The Evolution of Peer Review Practice in China

• Since the NSFC was founded in 1986, its Policy Bureau successively translated a series of reports on evaluating and studying peer review practice conducted by science funding agencies outside China: US General Accounting Office (GAO) 's report to the Congress that compared different mechanisms of peer review practice between the NSF and the National Institute of Health (NIH) translated and edited in 1987, the evaluative report on UK Research Council's peer review practice translated and edited in 1993, and the report on NSF merit review process translated and edited in 2001.

• All these reports were compiled with their shared purpose to understand the states of and trends in peer review policy in developed countries. These reports not only served as the 'windows' to know peer review practice in other countries and thus directly informed different divisions of NSF, but also benefited for institutional transformation and ameliorating the peer review systems in NSF.

• In March 1993, a Sino-Germany Workshop on Peer Review was held in Beijing through the collaboration between the NSF and the Deutsche Forschungsgemeinscharft (DFG) followed which the Proceedings of Sino-German Workshop on Peer Review was published.

• In June 1996, led by NSF Director Zhang Cunhao, a number of program officers including three division directors attended a peer review workshop co-organized by NSFC and NSF in Washington D.C. Following that workshop, all of the Chinese delegates were invited to participate in NIH's '50th Anniversary of Peer Review Practice.' During this event, the Chinese delegates were able to discuss some issues remained in peer review with NIH's program officers and managers. Some of the Chinese delegates were also invited to join in a panel meeting that reviewed projects on technological innovation in small/medium scaled enterprises.

• Initialized by the NSFC, in 1997, a Sino-German workshop on peer review was held in Beijing.

• While in 1998 in the U.S., another workshop on peer review was held in terms of the dialogue between China and the U.S. It was through these workshops that the NSFC was able to exchange its experience on peer review with other science agencies from Germany and the U.S. simultaneously; it also supplied crucial materials for Chinese scholars to conduct peer review research.

• In February 1998, a Sino-France workshop was held in Shanghai and the French team was formulated with the support from the Office of Science and Technology (OST, France). Representatives in the French team were from diverse organizations: CNRS, INRA, IMRI, CEA, INSERM, and IPTS. Besides senior officers from the NSFC, delegates from the Ministry of Science and Technology (MOST), the Ministry of Education (MOE), the Chinese Academy of Sciences (CAS), and several universities were also invited for their engagements. After that workshop, the Proceedings of Sino-French Workshop on Science and Technology Policy was published as a final outcome.

• Just as traditional approaches to peer review, internet peer review was started by reviewing funding proposals rather than journal articles. The web-based peer review program that is currently being used by the NSFC is the 'ISIS (Internet-based Science Information System).

• 2. The Practical Use of Peer Review in China• Ever since the reform and opening-up,

managerial systems of science and technology were constituted gradually at different levels: national, ministerial (MOST and MOE), provincial, municipal, and collegiate.

• (1) Peer Review Policy of the NSFC• NSFC is a public institution that governs national natural science foundation and

directly under the State Council. Ever since its establishment, concerned by the Central Committee of CPC, correctly led by the State Council, supported by relevant departments of the State Council and scientists all over the country, science foundation management has broken through traditional model in which research funding relies on administrative allocation in the planned command economy. It has completely introduced and implemented advanced research funding models and management concepts, set up ‘relying on experts, developing a democratic style of work, supporting projects through competitive selections, and respecting just and rational’ as fundamental review principle, established ‘scientific and democratic, competition on an equal basis, and encouraging innovation’ as operational mechanism, and strengthened a management system of science foundation that reconciles decision making, implement, inspection, and consultation.

• All of these efforts have gradually formulated ‘impartial, dedication, united, and innovation’ as atmosphere of NSFC and realized the function of national science foundation as being ‘orientation, stability, and motivation’ for basic research in China. The management system of natural science foundation was built up by considering the improvement of disciplinary institution as the framework on the basis of merit review and effect evaluation. Meanwhile, in accordance with the principles, guidelines, and thoughts of science and technology policy in the new age of China, on the basis of the strategic requirements of building a well-off society in an all-round way, starting from the general disposition of national innovation system construction, the strategic orientation of national science foundation in national innovation system was set up- ‘supporting basic research, persisting free exploration, and playing a role as orientation.’

• On November 17th 1992, NSFC published the Reviewing Criterions for NSFC General Projects in which peer review was included in Chapter 3.

• These criterions were then replaced by the new Reviewing Criterions of NSFC Applications which were revised and passed at the Committee Conference on November 20th 1996.

• By comparing the two versions of Reviewing Criterions of NSFC Applications that were respectively formulated in 1992 and 1996, four major revisions could be observed:

• First, an extra criterion is added to the process of selecting reviewers. The new version emphasizes that reviewers should hold senior professional positions.

• Secondly, the item that requires reviewers for high-tech and exploration projects with new conceptions and ideas should include expert group member of “863 Projects” is eliminated.

• Thirdly, a new item that requires reviewers to withdraw reviewing applications from their own institutions and relatives as well as other applications that might undermine fairness is also added. The principles of justice and fairness in reviewing NSFC applications are further exemplified.

• Fourthly, scientific values, scholarly qualities, creativeness, research purposes, technical routines, and research conditions of applications are emphasized in reviewing processes. Especially, it adds requirement for justifications of research purposes and technical routines of applications as well as the assessment for the last NSFC project investigated by the same applicant.

• In order to further strengthen and regulate the administration of NSFC funded projects, NSFC formulated the Criterions for Regulating NSFC Projects (tentative) which were passed at the Committee Conference on November 22nd 2002.

• Seen from the Criterions for Regulating NSFC Projects (tentative), it is showed that reviews of NSFC manifest the following aspects of characters:

• (1) standards for selecting reviewers are more transparent and clear;

• (2) coverage of knowledge and representativeness of distinct scholarly views and institutions are emphasized;

• (3) regulations on withdrawal and secrecy are reiterated. It is stated for the first time that applicants are allowed to suggest three reviewers (their affiliations should be noted) who are not qualified for reviewing their applications as long as the reasons have also to be given which are provided for selecting reviewers;

• (4) the concept invalid reviews is put forth for the first time. This conception points out that overly inane or obviously unjust reviews should be considered as invalid;

• (5) items for protecting intellectual properties of applicants are added. For instance, copying, disclosing, and plagiarizing any content of applications are prohibited.

• On February 24th 2007, Regulations on NSFC was issued by the State Council of China. It indicated that funding of NSFC should be obedient to principles of openness, fairness, and justice and practice guidelines that respect science, promote democracy, advocate competitions, facilitate collaboration, stimulate innovation, and lead future. Determining the projects funded by NSFC should take seriously the roles of experts into account and adopt mechanisms that include macro guidance, independent application, equal competition, peer review, and selective funding.

• (2) Peer Review of National Social Science Fund of China

• In June 1991, Chinese center government decided to set up the National Planning Office of Philosophy and Social Science subordinated to the National Planning Leading Group of Philosophy and Social Science. The former is the administrative body of the later.

• The main responsibilities include :• (1) draw up the development plan and annual plan;• (2) manage and collect national social fund; • (3) examine the implementation of medium-term and long-term

plans and annual plan, exchange the information of social science research;

• (4) coordinate the evaluation, acceptance and diffusion of the research achievement of major subjects.

• National Planning Office of Philosophy and Social Science classifies the application by discipline, and takes the two-way anonymous communication review system. Each application will be distributed to 5 review experts in order to reduce the human injustice. In the evaluation process, these five experts evaluate the application in accordance with the assessment criteria and give the marks. Only those which achieve the required score can enter the conference review. The ratio of the initial application to the application shortlisted for conference review is 5:1. The information of the review experts should be enriched as much as possible on the internet. For example, the information should not only include the name of the first level and second level discipline which the experts locate, but also the recent academic attention and the research achievements. Only in this way can we identify the proper experts when we implement the communication review, can we ensure the fairness and reasonability of the review.

• From 1991 to 2011, the changes of the peer review of National Social Science Fund include:• (1) National Social Science Fund adopts anonymous communication initial review in all

disciplines.• (2) The database of review experts is founded; The communication review experts are

randomly selected. • (3) The scoring system is improved; the maximum and minimum score should be removed in

the final scoring; the average should be taken. • (4) The opportunity of the initial application accepted to conference review is moderately

reduced, and the ratio of short-listed projects to approved projects is changed from the previous 5:1 to 2:1.

• (5) The length of the loose-leafs of applications for communication initial review is expanded from 3000 words to 4000 words.

• (3) Peer Review of Humanities and Social Sciences Fund of Chinese Ministry of Education

• The management of humanities and social sciences of Chinese Ministry of Education follows the principle of "scientific, fair and efficient". The management is standardized, and the approval is based on merit. It implements centralized collection of topics, centralized application, centralized review, centralized publication of the results; it use the modern information technology to improve work efficiency; it supports young social science researchers and remote, national colleges to conduct distinctive social science study.

• On October 7th, 1996, state education commission of china issued Humanities and Social Sciences Project Management Approach of State Education Commission of China, which contains eight chapters, 30 articles. Chapter 4 is related to peer review.

• Humanities and social sciences project management approach of state education commission of China had been implemented for ten years.

• In 2006, the new edited Humanities and Social Sciences Project Management Approach of Chinese Ministry of Education was issued.

• Part III The Prospects of Peer Review in China

• In the 1980s, peer review was formally introduced into China. Henceforth, it was generally used in scientific evaluation, but it should be noted that the establishment of the peer review system in social sciences was later than that of natural science.

• In the later half of the 1990s, many research administrative departments set out to evaluate the achievements of social sciences using the method of quantitative analysis.

• Imitating the three major citation index of humanities and social sciences (SSCI, A&HCI, ISSTP), Nanjing University created Chinese Social Science Citation Index (CSSCI) in the year 2000, which was a sign that China began to use quantitative analysis to evaluate the achievements of humanities and social sciences.

• Until now, it has become the major evaluation tool to evaluate social sciences in China (also including humanities and they are preparing to separate them according to the international convention).

• Nowadays, a new evaluation method in humanities and social sciences, “Representative Work”, is prevalent in Chinese universities, which has been implemented in Tsinghua University, Peking University and Nankai University.

• Chinese research evaluation has gone through four stages: administration review, peer review, quantitative evaluation and comprehensive evaluation.

• Because of the full implementation of peer review, research project approval and funding arrangement are no longer decided by leaders, but by scientists themselves now, at least in form; one can hardly get the funding from other channels except through peer review. Afterwards, the quantification evaluation method offsets the shortcomings of peer review, but on the other side, this method is oversimplified and implemented mechanically and absolutely. In front of the tremendous ‘achievements’, many evaluation organizations lack the ability or patience on comprehensive evaluation, so, the novelty, normativity, scientificity, academic and social values of the achievements are ignored, instead, they focus on the external characteristics, such as, the achievement’s form, the sponsor of the program, the level of the journal, the citation times, the assessment of the leader, and the award level. This kind of evaluation, which almost totally relies on the so called objective index, actually degrades into “the cooperation of administrative leader, managing partner and computer, any academic evaluation becomes redundant.”

• In the age of internet, it is difficult to maintain the secrecy in the process of peer review actually. Although the application of research project, the publication of paper and book, and the review on doctoral dissertation are all anonymous ostensibly, in fact, the reviewees can easily find out who is reviewing their works. This case brings about great challenges to review organizers, reviewers and reviewees. Due to the insufficiency of anonymity, the justice of review is hard to achieve and the conflict of interests is unavoidable. Therefore, we suggest that the peer review procedure should be improved in the following aspects.

OpennessOpenness

• That is, the degree of publicizing the evaluation criterion and review results by the institutions which adopt the peer review system. In principle, the peer review institution should open the review standard (handbook) unconditionally and the review results partly (only the result) or wholly (including the reviewer and the comments).

• For example, NSFC has successively issued the requirements and introduction on peer review, the essentials of the peer review on General Program and the essentials of the peer review on Young Scientists Fund Project.

• We hope that, all kind of institutions, including the central government, Ministry of Education, province, universities, journal publishers, should set up their own standards on peer review according to their own conditions.

• Especially, we hope The National Planning Office of Philosophy and Social Science can provide the reviewees the feedback on review comments and suggestions as NSFC has done.

Fairness Fairness • That is, the applications should be treated objectively and

impartially during the review process. In other words, the applications which should be funded are funded actually, and the applications which should not be funded are not funded actually.

• This is guaranteed by three steps:• First, review organizers should pay attention to the secrecy,

avoiding the result that “what should be known is known in the very beginning and what should not be known will not be known forever”. They should establish detailed review notice, including the review criterion, the tips on the conflicts of interests, the tips on the obligation of secrecy, and the commitment on maintaining the secrecy of the reviewer’s name, identity and affiliation, etc. Meanwhile, review organizers should standardize the written form of the application, control the quantity of the applications, lighten the burden of the reviewers, and reduce the cost on review;

FairnessFairness• Second, the reviewee should try not to reveal his/her personal

information, avoiding the conflict with the reviewer on interests. We suggest that the reviewees can point out no more than three reviewers which he/she believes are not proper to review his/her application, and propose the reviewers who are familiar with his/her research.

• Third, the personal factors of reviewers. Because “the factors from review itself have the direct impacts on the justice of review, and the factors from the outside of review may just have indirect impacts”. If the peer review lacks independency, it will be influenced by the non-academic factors and the justice will not be guaranteed. Then the peer review becomes just nominal in some cases and it will not have any substantial effect on academic evaluation. It is even worse that the government officials who have been out of the academia for many years participate in the review process as a peer reviewer. It is unreasonable and impedes the reformation of science system and the implementation of the autonomy of scientists.

FairnessFairness• The reasons are that: • First, it is hard for them to have an accurate picture on the

frontier of science, as they have been out of the scientific research for a long time, even though maybe they used to be the excellent scientists;

• Second, they may not switch their roles immediately, or it is impossible for them to do this. Their opinions will be given special respect (if not obedience) by other peer reviews because of their special positions, even though they serve as peer reviewers.

• Third, if the officials are influenced by the department benefits, it will be hard to ensure objectiveness and justice in the process of peer review.

• We suggest that the reviewers should be prevented from the evaluation on the applications from their immediate family, colleagues, supervisors, students and cooperators. To ensure the justice, we also suggest that electronic databases of reviewers should be set up to serve as a new way to select reviewers and improve the management efficiency, that systematism and standardization should be advanced, that handbook of peer review should be edited.

Applicability Applicability • That is, targeted index systems should be established

with respond to different goals and objects. In my view, peer review is mainly applicable to the “normal science” in the sense of Thomas Kuhn.It will be much difficult to evaluate the research results during the stage of scientific revolution, because the precondition of peer view is that there are peer experts. Peer experts refer to those who have the knowledge background of the item which is being evaluated, but not those who just belong to the same discipline. So in the stage of normal science, it is easy to identify the peer experts, but in the fields of original research or interdiscipline——especially the interdisciplines covering both natural sciences and social sciences, it is actually to find the real peer experts. That is why the American scholar says “peerless science”.

ApplicabilityApplicability• In my opinion, peer review system is the best choice

among the bad of the review system of science and technology, this viewpoint was proposed when I was responding a Japanese scholar in the 3nd International Conference of Applied Ethics and Applied Philosophy in Eastern Asia. My reply is inspired by the principle of Maxmim in Rawls’s A Theory of Justice. The so called “Maximis Principle” is that “we should first rank the selection objects by the worst results they may give rise to, then, we choose the one the worst result of which is better than the worst results of all others.” the maximum of the minimums means the best choice on the bottom line, in other words , it is maximize our benefits in the worst situation.

Applicability Applicability • On the other hand, we should focus on the cognitive

contents and the characteristics revealed in different discipline of peer review.

• The differences between natural science, humanities and social science on peer review should be treated seriously. Peer review, as the general method to evaluate natural sciences, humanities and social sciences, have something in common. Comparing with natural sciences, humanities and social sciences bear the following characteristics: pluralism in component, national character, class character, aborinigality, intuitive on conformation, potential and indirection in value realization, variety on the form of achievement, long periodicity of citation, etc. the achievements of humanities and social sciences should be reviewed on their intellectual merits, impacts and benefits from a broader perspective, that means it will include not only the evaluations from the academic circle, but also the evaluations of the social and economic benefits.

ApplicabilityApplicability• Along with the evaluation on the direct index of the

achievements by peer review, Bibliometric analysis should be employed to evaluate the indirect index of the achievements. Special review mechanism, which is targeted at the original research and interdisciplinary research, should be set up and implemented experimentally to fund the researches, which have profound influence, highly innovative value and revolutionary meaning, to improve the original innovation ability. Review criteria and system should be perfected according to the characters of different disciplines. I believe that, with the development of the enterprise of Chinese science and technology, China will enlarge the scale of peer review, reedit its standard and improve its quality.

Effectiveness Effectiveness

• That is, the effective utility of the peer review results. Whether the peer review results are effectively used or not will reflect the democracy degree of the science management and the integrity degree of the academic community. In order to improve the effective utilization ratio, we need to enhance the construction on the database of review experts, create the mechanism of credit rating of experts, guarantee the review quantity, set up the regular monitoring and assessment system of the development of peer review, and perfect the monitoring system of peer review.

EconomyEconomy

• That is, the manpower, financial resources and time devoted to the operation of peer review. The present condition is much better than before, because peer review is mainly implemented with internet, which preferably avoids the waste.

ConclusionConclusion• In all, “peer recognition” is the basic criteria on the

evaluation of science and technology in the international academic circle. If China wants to realize the transformation from big country of science and technology to strong country of science and technology, from the big country of culture to strong country of culture, the Chinese scientists and engineers want to be the scientific masters recognized by the international peers, we must follow the law of peer recognition, participate positively in the domestic and especially the international academic communication.

• I believe that, peer review system will gain fruitful results on the joint efforts of Chinese and overseas scholars.