Investigate the work done by others

Good starting point, but rarely current, and usually don’t cover very advanced topics

Rapid communication of results (months between submission and presentation)Important for visibility and to establish priority of research topics

Slower communication of resultsMore thorough editorial and review process

Detailed descriptions of work, not peer-reviewed, that provide information not found in publications

Include results equivalent to one or several papers, submitted by a student seeking a degree. Reviewed by a committee of faculty.

Annual conference solicits papers in a particular area (e.g. networking, AI, ML, computer architecture)A submission deadline is announced many months in advanceSubmissions have strict page limitsThe conference program committee arranges multiple reviews (3-6) for each submission and selects the best to form the conference programTypical acceptance rates range from 10-30%Authors give a 15-30 min presentation at the conference; papers appear in the proceedingsThere are some variants, e.g. majority of papers at NIPS and UAI are presented as postersSome areas and conferences are experimenting with different submission and reviewing models (e.g. VLDB, ICML)

Reputation for technical excellenceHigh-quality reviewing and strict acceptance standardsThe most significant and technically strongest new resultsAn audience that includes the well-respected members of the field, which increases the impact of results reported there

Some online rankings also exist, which attempt to be quantitative

There are some who call for CS to “grow up”

The system has memory – the same reviewers review subsequent rounds of submission

Guide to finding CSE literature

What is the problem? Why is it important?To whom is it important?

Who has worked on this stuff before?What is novel about the present work?Does the work have any direct competitors?

NoveltyMethodResult summary

E.g., SIGGRAPH papers always have videos, NIPS papers may have proofs, STOC/FOCS papers are technically just “extended abstracts” with proofs in a “full version”

A multi-agent systems paperA machine learning paperA data mining paperA theory paper

Proof?Analytical performance estimate/model?Coded up? Simulated? Put into hardware?How thorough is the testing?

On simulated data? How realistic is the data?On real data? How much/what kind?In interactive systems, is the algorithm “in the loop”?User studies?

What metrics are reported? Are there error bars or other statistical tests?Comparison to competitors?

New methods?New explanations or a new model for something?Comparison of existing things?New empirical evidence of something?

“This method results in fewer dropped packets under realistic loads”“This method predicts significantly more regulatory motifs than…”“We have developed a new algorithm to substantially increase the accuracy of branch prediction in out-of-order, superscalar processors”

-- How well are these statements supported? Do you believe them?