Lecture_Notes_3.pdf

8/10/2019 Lecture_Notes_3.pdf

1/29

2010/4

C . K . T O H

V I S I T I N G C H A I R P R O F E S S O R

N C K U T A I N A N

0 8 A P R I L 2 0 1 0

How to get my paper accepted?

Part 3: Review, Rebuttal, Revision, & Ethics

NCKU EECS College: International Forum Seminar Series for Graduate Students

1

INTERNATIONAL

F O R U M

INTERNATIONAL FORUM

Seminar Series Outline


2


2/29

2010/4

ANNOUNCEMENTS

ESSAY WRITING CONTEST: Core students pleasesubmit a 1-page essay with the title: My Thoughts onusing English in NCKU. Deadline is 12 April 2010.Email your essay to: [email protected]

TUTORIAL HOURS: Starting from 26 April 2010,every Monday and Wednesday. Open to EECS PhD &Masters only. Time: 2pm 3pm. Place: Dr. Tohs office at ChiMei Building, 4th Floor, R95403 Advance reservation of tutorial slots needed. Each student is allowed 20 minutes of consultation time. Please make your reservations in advance online on web. Your reservation will be confirmed via email sent to you.

3


OutlineTodays Lecture

1. Reviews Evaluation Metrics Used (conferences, magazines, journals)

Qualitative & Quantitative Measures

Decisions (minor, major, accept, reject)

2. Rebuttal How to reply to reviewers comments

What if reviewers are wrong?

What if reviewers are unreasonable?

When to move on?

3. Revision First revision what you need to do.. Second revision possibly last

Third revision - unlikely

4. Research Ethics 5. Conclusion


4


3/29

2010/4

RECALLTechnical Paper

Contents Abstract Introduction / Motivation The Problem Your Work / Proposal More More Your Work Comparisons Claims / Discoveries

I proved that ..

I discovered that ..

Related Work Future Work Conclusion References

HEAD/ Title(Abstract & Introduction)

BODY(My Work)

TAIL(Conclusion)

My Paper


5

SUBMISSION

Please Check

Before you submit your paper, ensure it is:

Readable

Free of errors (language or technical)

Within page limit (else pay $$)

Conform to required format

Font type

Font size Margins

Headings, sections, subsections

6


Readable

No Errors

Technicalsubstance


4/29

2010/4

Submission


7

SUBMISSIONPlease Check

WritePaper

ProfessorCheck

PaperOK?

No

SubmitEditor-in-chief(EiC)

revision

Yes

Sent outForReviews

INTERNATIONAL FORUMHow to get my paper accepted?

1

Reviews


8

Review: to examine critically or to study material again


5/29

2010/4

Assignment of ReviewersReviewers

OR

EiC

EiC Assistant orAssociate Editor


9

Read,Check,

Evaluate,Score

Review period canbe from 3 months upto 1 year or more

ReturnscorestoEiC

REVIEWSThe Process

REVIEWSEvaluation Methods

Paper Review


10

Quantitative Evaluation introduce evaluation metrics use numeric scores (say 1 5) give an overall

recommendation score

Qualitative Evaluation

reviewers must comment on thepaper regarding various aspects,such as:

weakness of paper strength of paper comments for improvement overall recommendation

1

2


6/29

2010/4

Reviewers are looking for (main metrics):

Novelty

Did you introduce anything new in your paper?

Significance

Are your work significant or is it just marginal work?

Impact

Did your work impact the field or the community or the society?

11


REVIEWSEvaluation Metrics

Readability Is your paper written in such a way that is easy to read?

Originality Does your paper contain original ideas and is not copied from elsewhere?

Technical Correctness Are there any flaws in your technical approach/solution?

Timeliness Is your topic appropriate for the current year or is it an old topic that is

fading away?

Relevance Is your topic relevant to the theme of the conference or journal?

12


REVIEWSOther Evaluation Metrics


7/29

2010/4

REVIEWER SCORING (see appendix for more examples)Metrics Score (1: lowest; 5: highest)

Novelty 3

Significance 1

Technical Correctness 4

Timeliness 5

Relevance 5

Overall Recommendation 2

13


REVIEWSEvaluation Scoring

Note: Some reviewers use 2.5 but this should be avoided as much as possible!

REVIEWERS COMMENTS (see appendix for more examples)

14


This paper is poorly organized and poorly written. There is a lack oftechnical depth. The issue addressed by the authors are trivial and

there is nothing novel in their approach. Also, the simulationresults are unconvincing since the traffic load is set below 500bps.

This paper presents a brand new approach in solving low powerrequirements in analog circuits. The authors have successfully

demonstrated the performance of the new circuit through bothsimulation and practical experiments. The paper itself is well organized

and readable. The work done is significant and the reviewerrecommends that this paper be accepted.

REVIEWSQualitative Evaluation

Bad

Good


8/29

2010/4

REVIEWSOutcome

EiC then made a decision

Scoresand

Comments

Accept

Reject


15

Major Revision

Accept WithMinor Revision

OrOn Conditions

Conflicting Reviews:Accept/Reject recommendations

16

REVIEWSScoring & Outcome

Accept Accept Accept

Accept Reject Reject

LikelyACCEPT

DefinitelyACCEPT

LikelyREJECT


Accept Accept Reject

Reviewer #1 Reviewer #2 Reviewer #3


9/29

2010/4

Conflicting Reviews: Overall recommendation score

17

3 3 3

5 2 2

LikelyACCEPT

ACCEPT /REJECT

LikelyREJECT


4 3 2Reviewer #1 Reviewer #2 Reviewer #3

REVIEWSScoring & Outcome

REVIEWSAbout Reviewers

18


They are humans too!

They are also your competitors

They may not want you to get ahead

They may not want you to publish the same topicbefore them!

They try to find fault in your paper or accuse you offlaws or wrong doing

ACTION: You need to stay cool and try to address the accusations and

negative comments based on facts. Hence, the need forrebuttals.

YouOthers


10/29

2010/4


2

Rebuttal


19

Rebut: to provide an answer or counter-proof

REBUTTAL

Paper Accept

Rebut to provide an answer or counter-proof.

EiC Decision: Accept

20


Accept Correct

Any typos

PrepareCamera-ready

CopySign Copyright

Forms

Submit for

Publication

Outcome

is usually

Accept!

1. No need for rebuttal

2. Correct any minor typo or grammatical errors

3. Prepare final camera ready copy


11/29

2010/4

REBUTTALAccept with Conditions

21

Accept on

Conditions

Revise

according

to reviewers

comments

Fulfill

Reviewers

Conditions

Submit for 2nd

Review

Prepare REPLY TO REVIEWERS

1. Extract what reviewers want you to do.

2. Write a REPLY letter. (EXAMPLE)

3. Address each issue with an adequate answer. (EXAMPLE)

4. Revise the paper accordingly to changes recommended by reviewers.

5. Fulfill most, if not all, of the requested conditions

Accept on Conditions


Major

Revision

Revise

according

to

reviewers

comments

Perform

more

Simulations

+ research if

needed

Usually

paper

needs to

be

rewritten

Prepare REPLY TO

REVIEWERS

Submit

for

2nd

Review

Outcomeis still

uncertain!

Can be

Accept or

Reject

REBUTTAL

Major Revision22

Major Revision

1. Write a REPLY addressing all changes to the paper2. Need to think carefully if you want to do a major revision3. There is no guarantee of acceptance after a major revision



12/29

2010/4

REBUTTALReply to reviewers

23

Example

of

Rebuttal


REBUTTALYou versus Reviewers

What if reviewers are wrong?

If reviewers accept your clarifications, then matter issolved. Your paper will be accepted. If not, EiC can stillreject your paper!


24

Ask &Check with

YourProfessor!

Revieweris still

correct?

Check with another3rd person!

Prepare aREPLY to

reviewers toexplain why they

are wrong andyou are right!

SubmittoEiCandto

Reviewers

N

Y


13/29

2010/4

Avoid vending your anger at reviewers!

EXAMPLES:

Reviewer 1 seems jealous of our results and do not believe inour performance gain.

Reviewer 2 is stupid that he cannot understand our algorithm.

Reviewer 3 did not read our section 2 at all! His remarks arechildish and untrue.

Reviewer 2 should read our paper again!

Equation 5 is so simple. Why Reviewer 1 cannot understand it?

Reviewer 2 is unreasonable to ask us to do so many things.

Reviewer 1 has not read our earlier paper.

REBUTTALPsychology

25


Avoid creating a suspicious atmosphere!

It does not help to get your paper accepted.

If reviewers misbehave, report to EiC.

Let EiC makes the final decision after rebuttal.

If EiC misbehaves, then it is time to withdraw yourpaper and submit it elsewhere!

EXAMPLES: We think Reviewer 2 is our competitor and enemy. We think Reviewer 2 is dishonest, biased, and unjust

Reviewer 1 is trying to delay the publication of our work

Reviewer 3 is envy of our success

26


REBUTTAL

Psychology


14/29

2010/4

Reviewer is fussy: Try to fulfill his/her requests as much as possible

i.e., keep him/her happy, interested, and positive

Reviewer is mistaken: Try to clarify his/her doubts in your reply

i.e., make him/her feel positive about your work

Reviewer is unreasonable:

Keep your cool, try to answer his/her negative comments

Seek help from Editor-in-chief Decide to stay in the process or quit (i.e. withdraw)


27

REBUTTALPsychology

REBUTTALReviewers CONDITIONS

You did not cite my papers This is unreasonable

To get your paper accept, you must cite my

papers first bad behavior!

Your figures are not clear Redraw or make it larger and sharper

You need to do more experiments / simulations This may or may not be reasonable

Some experiments are hard to repeat Time consuming and manpower needed to redo experiments

You need a native writer to rewrite your paper! English aspects (reasonable)


28


15/29

2010/4


3

Revision


29

Revise: to look over again in order to correct or improve

REVISION

(1st, 2nd, 3rd )

1st REVISION (For Journals) Rewrite parts of paper

Resubmit revised paper to EiC

Letter to EiC highlighting corrections done

Your revised paper will be reviewed again!

Wait for another 2-3 months; Outcome from EiC

2nd Revision If reviewers are still unhappy with 1st revision, EiC may ask you to do a

2nd revision

This is the last chance!!

If reviewers accept your 2nd revision, then EiC usually recommendsaccept.

3rd Revision (?) Usually no!

30



16/29

2010/4

REVISIONIMPLICATIONS

PAPER ACCEPTANCE & REJECT

31


CONTRIBUTINGFACTORS

1. High reviewers scores

2. Positive comments

3. Paper is well written

4. Minor revision

5. Major revision

6. REJECT: Negative comments Low scores

Characteristics of good researchand a well-written paper.

Paper likely to be acceptedbut imperfect in some ways;Inadequacies can be rectified.

Paper regarded as inadequate;A polite form of saying reject

Paper regarded as inadequate;A direct form of saying no way

REVISION

WHAT IF?

Time can kill you!

1st

Submission

1st

Reject

Consult with your professor; You only have 3 years or more to graduate If after 2 or 3 resubmissions and your paper cannot be accepted in any

journals, you need to be realistic to accept that there is no appreciationand merit of your paper. You must abandon and move on !! Dont be stubborn.

Do not take failure as forever. Do something you are good at.

2nd

Submission

2nd

Reject3rd

Submission

3rd

Reject

32


time


17/29

2010/4

PAPER ACCEPTANCEFORMULA?

So how to get your paper accepted?

PS: I will talk more about World Class High Impact Research in another seminar.

Do QualityResearch

ProvideProfoundSolutions

Write yourpaper well

Submit it tothe right place

for

consideration

Pray that your3 anonymousreviewers willlike your work

Dont vendyour emotions

at the EiC or

reviewers

Getyourpaperaccepted

andpublished!

Waitandseehowothers

feltaboutyourwork!!

33


DIFFERENCES

Conferences vs. Journals

FACTORS CONFERENCES JOURNAL

Reviewing

Period

Usually 3 months Usually 6 months to 12

months or more

Number of

Reviewers

Usually 3; Sometimes 2;

sometimes 4 or more

Usually 3

Critical

Evaluation

Metrics

Novelty; Significance;

Technical Correctness;

Relevance;

Novelty; Significance;

Impact; Technical

Correctness; etc

Rebuttal

allowed

Usually no Usually yes

Who decide

to accept or

reject

Technical Program

Committee Chair or Co-

chair

Editor-in-Chief

34



18/29

2010/4


4

Research Ethics


35

Ethics: a set of moral principles or good conduct

RESEARCH ETHICS

BASICS & REALITY

ARE THERE BAD PEOPLE/REVIEWERS OUT THERE? SURE!

GOOD & EVIL ARE IN BATTLE EVERYDAY !!

MAINTAIN HIGH LEVEL OF INTEGRITY,

HONESTY, & PROFESSIONALISM BE HONEST WITH YOUR RESEARCH RESULTS

ACCEPT & DEFEND THE TRUTH!

BAD PRACTICES: COPY PAPER CONTENTS FROM OTHERS!

FAKE OR COPY RESULTS

LIE ABOUT THE TRUTH

USE BULLY OR INTIMIDATION


36

SEENOEVIL

HEARNOEVIL

SAYNOEVIL


19/29

2010/4

RESEARCH ETHICSWHY WE CARE?

IF I LIE.. WHAT HAPPENS? (EXAMPLE) SAY YOU AND OTHERS ARE A TEAM THAT INVENTED A

NEW MATERIAL TO BE USED ON A PLANE.

YOU ALL LIED THAT THE MATERIAL CAN BE USED FORTHE BODY OF THE PLANE UNDER EXTREMETURBULENCE & STRESS.

PLANE WAS MADE WITH YOUR PROPOSED MATERIAL

PLANE WENT UP TO 35,000 feet

PLANE EXPLODED IN THE SKY

350 PEOPLE KILLED !

HOW DO YOU FEEL?? ARE YOU A MURDERER?

37


EXAMPLE

RESEARCH ETHICSGOOD OR EVIL: YOU CHOOSE

Great power comes with great responsibility!

You are defenders of the truth and for good of mankind

38



20/29

2010/4

RESEARCH ETHICSYour PhD is your Power! Be real!

EXAMPLES

UNIVERSITY Requirements forPhD Degree

http://www.admin.cam.ac.uk/univ/gsprospectus/study/research/phd.html


39

HARVARDUNIVERSITY

Significant & Original Research

CAMBRIDGEUNIVERSITY

Significant contribution to learning

-for example through the discovery ofnew knowledge, the connection ofpreviously unrelated facts, thedevelopment of a new theory, or therevision of older views

RESEARCH ETHICSWHY & WHO ENFORCES THEM?

Your conduct will determine: Your reputation

Your success

Influence the behavior of your students

Many organizations enforces ethics:

Your University

Your Societies: IEEE, ACM, IET, etc

Your Academy: e.g. Academia Sinica Your Government: Ministry of Education / Ministry of Trade /

Ministry of Technology / Ministry of Law

40


Please!


21/29

2010/4

RESEARCH ETHICSPEOPLE WHO CHEATED!

EXAMPLE (2002)

BELL LABS RESEARCHER CHEATED & FIRED!

41


http://en.wikipedia.org/wiki/Jan_Hendrik_Sch%C3%B6n

Born 1970German Physicist;Nanotechnology &

Condensed matterPhysics; PhD fromGermany; His PhD wassubsequently removed

by his university!!

MORE

EXAMPLES

42


Luk Van ParijsBorn 1970BelgiumPhD HarvardProfessor at MIT



22/29

2010/4

Apology for cheating (real case in 2001)



43

STUDENTS CHEAT TOO

44




23/29

2010/4

JUNK CONFERENCESFAKE PAPERS

EXAMPLE MIT created SCIgen program can generate paper with fake content + results

Paper that doesnt make sense, but still accepted at a conference!!!

45


http://pdos.csail.mit.edu/scigen/

A prankster who submitted a computer-generated researchpaper to the International Conference on Computer Scienceand Software Engineering discovered that not only was his fakepaper accepted - its "author" is to chair a panel.The prankster, known only by his nickname Schlangemann

(which he used to submit the paper), created the paper usingSCIGen - the automatic CS paper generator. His nickname istaken from a German movie called Der Schlangemann.


5

Conclusion &

Appendices


46


24/29

2010/4

TODAY, WE HAVE COVERED:

Reviews, Rebuttals, Revision

CONCLUSION47


Paper Acceptance what it entails: Good quality research

A well-written paper

Your responsibility Research ethics

Your paper acceptance means:

Formal dissemination of your work Paper publication is not the end!

Final success depends on how others felt about your work!!!

Example: Review Outcome

Dear Dr. YYYYY,

Reviewer Comments concerning your manuscript, "An Ultra-Wideband Radio System for VehicleNetworks," have been obtained and are copied below. Based on the attached reviewers' comments, we

will not be able to publish your manuscript in our transactions.

Thank you for submitting your manuscript to the IEEE Transactions on ZZZ.

Sincerely,Editor-in-ChiefIEEE Transactions on ZZZ

48


Appendix


25/29

2010/4

---

Reviewer: 1

Recommendation: Reject

Comments:(There are no comments)

Additional Questions:Summary of Evaluation: Fair

If the paper is rejected for publication, the authors should: Regard the paper as not publishable.

Organization: 4

Clarity: 4

Length: 1

References: 3

Correctness: 4

Significance: 1

Originality: 1

Attachments:

If Survey Coverage:

Contribution: 1


49


Appendix

(Reviewer 1)

What are the contributions of this paper?

: The paper presents a system concepts and experimental results for UWB-based positioning of vehicles. While I like to seeexperimental verification of results (and thus was favorably disposed to the paper when first looking at it), I find that theoriginality and novelty of the paper is not sufficient for publication in a prestigious medium such as IEEE transactions. Thearchitecture and transmission protocol is straightforward; the location algorithm for TDOA is well-known, and the use of UWB forranging at distances up to 30 m has been shown with practical demonstrators multiple times. While the results appear correct, Icannot find that they are sufficiently original to justify publication.

What are some ways in which the paper could be improved? Please supply any additional important references that you feel theauthor omitted which should be noted in the paper.:

Example: Review Outcome50


Appendix


26/29

2010/4

(Reviewer: 2)

Recommendation:Reject

Comments: see review report

Additional Questions:Summary of Evaluation: Fair

If the paper is rejected for publication, the authors should: Prepare a major revision and resubmit to The ITS Transactions as a new paper

Organization: 5

Clarity: 3

Length: 5

References: 1

Correctness: 2

Significance: 1

Originality: 1

Attachments:

If Survey Coverage:

Contribution: 2


51


Appendix

(Reviewer 2)

What are the contributions of this paper?: see review report attached

What are some ways in which the paper could be improved? Please supply any additionalimportant references that you feel the author omitted which should be noted in thepaper.: see review report attached



Appendix


27/29

2010/4

(Reviewer: 3)

Recommendation: Prepare A Major Revision For A New ReviewSummary of Evaluation: Good

If the paper is rejected for publication, the authors should: Prepare a major revision and resubmit to The ITSTransactions as a new paper

Organization:3

Clarity: 4

Length: 3

References: 4

Correctness: 4

Significance: 3

Originality: 3

Attachments:

If Survey Coverage: 3

Contribution: 3


53


Appendix

(Reviewer 3)

What are the contributions of this paper?

: * Proposed a system concept of "unilateral" positioning which is different from most non-GPS positioning system

* Prosposed a transmitter synchronization scheme (TDMA) to achieve the high resolution of TDOA estimates.

* Developed a system demonstrator to show the feasibility and performance of this proposed positioning system in a specificoutdoor test scenario.

What are some ways in which the paper could be improved? Please supply any additional important references that you feel theauthor omitted which should be noted in the paper.: * Authors used a rudimental approach to solve the nonlinear TDOA equations which requires known z-coordinate and has thesingularity problem. Suggest use the two-stage least square solution proposed by Y. Chan in reference [18] .

* Authors conducted some simulations to evaluate the impact of the transmitter configuration on the positioning performance andidentified a particular configuration with five transmitters. Suggest authors check the following references for a simulation workand an analytical evaulationon a proposed four "nodes" configuration.[1]. J. Ni, et al; UWB Tracking System Design for Free-Flyers, AIAA Space 2004 Conference and Exposition, San Diego, CA,September 28-30, 2004.[2]. J. Ni, et al; Design and Performance Analysis of a UWB Tracking System for Space Applications, 2005 IEEE/ACES

International Conference on Wireless Communications and Applied Computational Electromagnetics, Honolulu, Hawaii, April 3-7, 2005.

* The outdoor test showed sub-meter positioning accuracy of the system demonstrator in a confined area of about 35m x 25m.Since authors claimed the UWB signal coverage is a area of a few hundred meters, it will be helpful to show the system's feasibilityand accuracy in a larger (also more closer to a practical scenario) area. To test if the positioning performance is degraded when thecoverage area is increased.



Appendix


28/29

2010/4

Example: Rebuttal

55

Appendix

REPLY to reviewers


REPLY

to reviewers

Example: Rebuttal56

Appendix



29/29

2010/4

Example: Rebuttal

57

Appendix

REPLY to reviewers


Lecture_Notes_3.pdf

Documents

Transcript of Lecture_Notes_3.pdf