Review Process - How to review

Fausto Giunchiglia

By Fausto Giunchiglia and Alessandro Tomasi

Index:

1. Review Form 12. Review Form 23. Answer to the Reviews4. Review Process

1. Review Form 1

1. Review Form 1

REVIEWER CODE:

1) Title

2) Author(s)

3) Paper Summary [short description of the message and maybe of how it has been developed. What if more than one message or no message?]

1. Review Form 1

4) Type of Paper/Research described [Chose one of the options below and explain]

- Research (with original results) of which kind (Theory, expemerimental, ...)- Application (case study, ...)- Synthesis of recent advances- Other. Please specify

1. Review Form 1

5) General Ratings [Rate within Bad/Weak/Fair/Good/Excellent] [0/1/2/3/4/5]

Put the score and some text motivating your score about:5a) Relevance (with respect to the reference community) 5b) Originality (incremental, new work, ...) 5c) Significance of the work (how big the gap from the state of the art) 3d) Technical soundness 3e) References 3f) Presentation

1. Review Form 1

6) Technical Soundness [Chose one of the options below and some text motivating the choice]

- Technically correct - Minor errors (indicate them) - Major errors (indicate them) - Unsupported claims (provide a detailed explanation)

1. Review Form 1

7) Presentation [Rate within Yes/Somewhat/No]

Put the score and some text motivating your score7a) Are the title and abstract appropriate?7b) Is the paper well-organized (discuss course and fine grained structure)?7c) Is the paper easy to read and understand? 7d) Are figures/tables/illustrations sufficient?7e) Is the English acceptable? 7f) Is the paper free of typographical/grammatical errors?7g) Is the reference section complete?

1. Review Form 1

8) General Recommendation [Please, chose one of the options below]

- Very strong accept (beautiful paper!) - Strong accept (excellent and important contribution) - Weak accept (good paper, some new interesting ideas) - Weak reject (marginal, weak content, would require a major revision) - Strong reject (unreadable, nothing new, premature, contains major errors)

1. Review Form 1

9) Main Reason for your Decision [For accept choices please indicate one of the options below] - accept because of the originality (good ideas, sound presentation) - accept because of the quality of the proposed synthesis (useful review on recent advances) - other

[For reject choices please indicate one of the options below] - reject because it is not relevant for the conference - reject because of the presentation (unreadable, unstructured) - reject because the content is too premature for really making sense - reject because of the lack of originality (results already known, or similar overview already published) - reject because of major errors

1. Review Form 1

10) Your Level of Expertise (Compared to Level of Others) - I am an expert of the field and know the relevant literature - I understand the problem, I know some of the state of the art - I only have a superficial understandings of the issues

11) Does the paper qualify for the best paper award [Y/N]

12) Comments to the Author(s) [Please, provide here a clear justification of your ratings, in particular with regards to the overall recommendation]

13) Additional Comments to the Author(after circulation of reviews among reviewers) (Can be empty, cannot change previous review)

14) Additional Comments as Answer to Author's Answer and Modifications(Can Be Very Short: - Evaluation Of Author Answer - Value Judgement - Final Score (Possibly Changed))

1. Review Form 1

2. Review Form 2

2. Review Form 2

REVIEWER CODE:

1) Title

2) Author(s)

2. Review Form 2

3) Main Message:

Relevance: How relevant is the paper to the workshop? 0: not relevant at all 1: rather not relevant 2: relevant 3: very relevant Technical Quality: What is the technical quality of the paper? 0: really bad 1: bad 2: good 3: really good

2. Review Form 2

Presentation: What is the overall presentation of the paper? 0: really bad 1: bad 2: good 3: really good Overall Ranking: What is your overall recommendation? 0: strong reject 1: reject 2: weak reject 3: weak accept 4: accept 5: strong accept

2. Review Form 2

Confidence: Reviewer's expertise in the area0: I know little about this area1: I know enough about this area2: I have good expertise in this area Why to accept? What are the most important reasons to accept this paper? (1-3 sentences) Why to not accept? What are the most important reasons NOT to accept this paper? (1-3 sentences) Comments: Detailed comments on the paper (primarily for the authors)

3. Answer to the Reviews

<Brief introduction>LIST OF {<general comment quoted from reviews><your answer arguing how you have accordingly modified the paper>}

Moving now to the more specific comments:LIST OF {<specific comment quoted from reviews><your answer arguing how you have accordingly modified the paper, providing detail but not too much>}

<Concluding sentence>

3. Answer to the Reviews

4. Review Process

4. Review Process

0) Abstract (send it to tomasi@dit.unitn.it not later than a week before the

presentation)

1) Presentations (Fortunate situation where you may know of what the paper is about)

2) All Papers Submitted by June, 13th 3) Review Allocation (by Program Chair) by June, 20th 4) Reviews Due by July, 4th 5) Reviews circulated to Reviewers for additional comments6) Reviews send to Author by July, 11th 7) Author sends back answer and modified paper by July, 23rd8) Reviewers provide final answer 9) Exam Pass/No Pass (of Authors and Reviewers) by July, 29th

4. Notice!!!

In the paper you have to use the same version of the stylefiles. It is available by downloading it fromhttp://www.springeronline.com/sgw/cda/frontpage/0,11855,3-111-2-124365-0,00.html

Avoid using stylefiles acquired from other sources as these may not be correct

23

How to dopresentations

24

1. Presentation methods

2. Attitude towards the audience

3. How to select content

4. How to structure

5. Introduction

6. Stage fright

7. Main part

Structure

25

8. Using pictures

9. Visual aids

10. Interposed questions, interruptions

11. Final discussion

12. The end

13. Optical – acoustic factors

14. Ancillary conditions

Structure

26

1. Presentation methods

Rule No. 1:Control effect

Rule No. 2:Integrate audience

27

1. Presentation methods

Seminar

Lecture

Presentation with final discussion

Presentation with intermediate

discussion

28

Comparison of some presentation methods

What remainsTime to

prepare / h

80% 4,0

Presentation with Intermediate discussion

60% 2,5

30% 1,7

Lecture 20% 1,0

Open discussion

Presentation with Final discussion

29

2. Attitude towards the audience

Who is the audience

What do they know

What are they interested in

What do they understand

30

2. Attitude towards the audience

Contact with eye Simple and understandable languages Concentrate on the important Credibility Reply to objections Avoid „techno“ language

31

How much?From where?What is interesting?What is the objective?What should the

audience do?

Do not want too much!

3. How to select content

32

4. How to structure –

catch words

Speak as you think – follow the thinking

Use written concept of catch words

33

5. Introduction

Try to find common understanding with the audience

Lead to the subject Initially: create attention

34

Say at the beginning The structure The length Invite to a dialogue

But at the beginning:

Who are you

5. Introduction

35

attention

Length of presentation

Hope that it will end soon

36

6. Stage-fright

Is natural! Everybody has it ! Is not forever!

Afraid to speak

37

Reduce Stage-fright

Be well prepared!

Learn important parts by hard!

Relax!

Look for a positive „point“

Do some contacts before presentation!

39

7. Main part

Introduction

Main partCurrent situationPotential solutionHow to realise

End

40

8. Using pictures / figures

Take from the context of the audience Has to increase the message To explain the issue Does not be an end on itself

Picture

41

8. Metaphorical language

To be a comparison

Should be practial

Story Citation Joke

42

9. Visual aids

Black board Projector Film Videobeam Overhead-Display

43

Grafical Layout Colours – Contrast – Background Clipart-files Fonts (size)

Produce hand outs

Presentation programs

44

What we keep

10% Reading

20% Hearing

30% Seeing

50% Hearing & Seeing

70% Reporting

90% Do it yourself

45

10. Questions

Question

Are welcome Should be answered immediatly Bit can also be delayed or

forwarded to somebody else

Interupptions

ignore! Look at the person! Stopp speaking!

46

11. Final discussion

Announce already in the introduction Write up issues of discussion Keep sequence of questions Repeat questions (if necessary) Draw conclusions

47

12. End

Avoid New questions which do not help in the conclusions

Main ideas Should be summarized

48

12. End

Main idea + review objective + how to realise

=

Good end

49

13. Acoustic factors

No „speaking smoking"

Speak loudly and slowly

You should vary volume and speed

Make breaks

Try presentation beforehand

50

With your body With your hands With your eyes

13. Optical factors

Behave natural!

51

14. Other circumstanceas

Room size

Lighting

Ventilation

Sockets

Other means (Microfon)

Breaks

Reading - How to read

Fausto Giunchiglia

Literature:

Bruno Buchberger, Thinking Speaking Writing

By Fausto Giunchiglia and Alessandro Tomasi

Index:

1. The Role of Literature2. What is literature

2.1 Classify the results 3. The Documentation of Literature4. Steps in the Use of Literature

1. The Role of Literature

“Keep Re-Inventing the wheel”

1. The Role of Literature

For many problems:

• the solutions is already in the literature

• the literature has solutions in similar problems

1. The Role of Literature

Necessary

Useful

but not

Necessary

It is necessary to know

how the literature is organized:

• within your research community

• outside your research community

[translate other concepts

in our community]

1. The Role of Literature

There is an interleaving of:

1) Try to solve the problem yourself

2) Read the Literature

1. The Role of Literature

Interactive Process of Reading Papers:

Step 1

Step 2

Step 3

…

Quantity

Depth

1. The Role of Literature

How do I know when I can stop reading?

When I know the 90% of the paperscited by relevant papers

1. The Role of Literature

To keep Scholarship:

• go to conference

• talk to people (best opportunity in conference)

• journals

• proceedings

• web

• …

2. What is Literature

2. What is Literature

More importantthan papers

Main Wayto do Career

1. Monographs / Books

2. Journal Articles

3. Articles in Collections

4. Papers in Proceedings of Conference

5. Papers in Proceedings of Workshop

6. Technical Reports

7. Grey Literature (Ex: Web Sites)

Each research community has:

1-2 Top Journals

1-2 Top Conference

2. What is Literature

2.1 Classify the results

2.1 Classify the results

1. Monographs / Books:

Contents:extensive/complete description of research

Originality:None (2/3 years old)

Note:the research is finished

Quality Control:Very Good

2.1 Classify the results

2. Journal Articles:

Contents:Very good, very crisp, specialized

Originality:Full of two years ago (in CS is obsolete)

Note:Come regularly

Quality Control:Very High (Many shots)

2.1 Classify the results

3. Articles in Collections:

Contents:Very good, very crisp, specialized

Originality:Full of two years ago (in CS is obsolete)

Note:Not regularly

Quality Control:Lower

2.1 Classify the results

4. Conferences:

Contents:New results, almost complete, semi-final

Originality:Good (6 months ago)

Note:Is Refereed

Quality Control:Medium (only one shot: accept or reject)

2.1 Classify the results

5. Workshop

Contents:New results, almost complete, semi-final

Originality:Good (6 months ago)

Note:Is Not Refereed

Quality Control:Medium (only one shot: accept or reject)

2.1 Classify the results

6. Technical Reports

Contents:Detailed about a specific topic

Originality:Maybe

Note:---

Quality Control:Low (No Refereeing Process)

2.1 Classify the results

7. Grey Literature (Ex: Web Sites)

Contents:Random

Originality:Random

Note:The main way to publish our results

Quality Control:Random

3. The Documentation of Literature

3. The Documentation of Literature

• Library- author files- keywords files

• Review Journals- Computing Journal- Artificial Intelligence Review- Scientific Magazines- Scientific Magazines for Spreading Communications- Journal for Publish new Results

• Bibliographies• Citation index• Research index

3. The Documentation of Literature

Research Community are organizedinto societies

Produce:Organized Conferences, Journals, Magazines

Examples of Societies:ACM, IEEE, ECCAI, VLDB, …

3. The Documentation of Literature

Citations Index:

• defines journals

• cites titles, authors, abstract of all paper

Example:ISI, the most important for Science

3. The Documentation of Literature

Research Index:

• the new game

• is on line

• how many times you are cited in Internet

Example:

http://citeseer.nj.nec.com

4. Steps in the Use of Literature

4. Steps in the Use of Literature

1. Use CiteSeer (http://citeseer.nj.nec.com)

2. Ask to Colleagues, Advisor,

Friends, … by e-Mail

3. Consult References

4. Consult Review Journals

5. Ask Authors