CS694a: Reviewing a Paper

Overview

You will be asked to review three of your peers' projects as though they were workshop submissions. Please also see the projects page for more information.

Here is how it will work:

• On November 16th, 2006, I will assign you three reports to review. I will include a link to the review form.
• On or before November 27th, 2005, you should have completed the reviews for the paper.

General Guidelines for Reviewing

• Read at least one or two papers on the subject of writing good reviews.
• Read some of the associated literature (by following some of the citations in the paper that has been assigned to you).
• You are free to pick a review format, but a common format is as follows. First, summarize what you understood of the paper in 1-2 paragraphs. Then, point out what you liked about the paper. After that, discuss the main flaws (if any) that you found in the paper. Follow that up with detailed suggestions for improving the paper (e.g. suggest experiments, other approaches to consider, better ways to present data etc.).

Review Confidentiality and Fairness

• This is an anonymous review process. I will return your review (anonymized, of course) to the authors.
• Try to be as fair and objective as possible, and to give as much feedback on how you think the work can be improved.

Sample Reviews

Here are some examples of reviews so that you can get a general idea of how to do a review. Please do not distribute these samples to anyone else.

The first is for a paper that represented a valiant attempt, but is clearly below threshold. Note that the review goes into a fair bit of detail. It is not particularly well structured (the comments are not organized in any way), but that is fine. It is helpful, and that is important.

The second is for the same paper, it is slightly better organized (it is more along the lines of a classical review, see above), and focuses on high-level arguments. This is also helpful, even though it is not as detailed.

The last is for a paper that is well below the threshold. This is short, as is appropriate for such a paper. However, it uses some not so nice words such as ``silly'' and ``comic relief''. Try not to do this: remember that some student has spent a fair bit of his/her time writing the paper, so be kind and constructive.