Anant Jain

How to Read a Paper

Paper Review

Abstract

Researchers spend a great deal of time reading research pa- pers. However, this skill is rarely taught, leading to much wasted effort. This article outlines a practical and efficient three-pass method for reading research papers. I also de- scribe how to use this method to do a literature survey.

Introduction

  • This method allows you to estimate the amount of time required to review a set of papers. Moreover, you can adjust the depth of paper evaluation depending on your needs and how much time you have.

  • The first pass gives you a general idea about the paper. The second pass lets you grasp the paper’s content, but not its details. The third pass helps you understand the paper in depth.

The first pass

This pass should take about five to ten minutes and consists of the following steps:

  1. Carefully read the title, abstract, and introduction.

  2. Read the section and sub-section headings.

  3. Glance at the mathematical content (if any) to deter- mine the underlying theoretical foundations.

  4. Read the conclusions.

  5. Glance over the references, mentally ticking off the ones you’ve already read.

At the end of the first pass, you should be able to answer the five Cs :

  • Category

  • Context

  • Correctness

  • Contributions

  • Clarity

Using this information, you may choose not to read fur- ther (and not print it out, thus saving trees). This could be because the paper doesn’t interest you, or you don’t know enough about the area to understand the paper, or that the authors make invalid assumptions.

As an author, take care to choose coherent section and sub-section titles and to write concise and comprehensive abstracts. For these reasons, a ‘graphical abstract’ that summarizes a paper with a single well-chosen figure is an excellent idea and can be increasingly found in scientific journals.

The Second Pass

In the second pass, read the paper with greater care, but ignore details such as proofs. It helps to jot down the key points, or to make comments in the margins, as you read.

  • Pay special attention to graphs. Are the axes properly labeled? Are results shown with error bars, so that conclusions are statistically significant?

  • The second pass should take up to an hour for an experienced reader.

  • This level of detail is appropriate for a paper in which you are interested, but does not lie in your research speciality.

  • Sometimes you won’t understand a paper even at the end of the second pass. This may be because the subject matter is new to you, with unfamiliar terminology and acronyms. Or the authors may use a proof or experimental technique that you don’t understand, so that the bulk of the paper is incomprehensible. The paper may be poorly written with unsubstantiated assertions and numerous forward references. Or it could just be that it’s late at night and you’re tired. You can now choose to: (a) set the paper aside, hoping you don’t need to understand the material to be successful in your career, (b) return to the paper later, perhaps after reading background material or (c) persevere and go on to the third pass.

The Third Pass

  • The key to the third pass is to attempt to virtually re-implement the paper: that is, making the same assumptions as the authors, re-create the work.

  • By comparing this re-creation with the actual paper, you can easily identify not only a paper’s innovations, but also its hidden failings and assumptions.

  • This comparison of the actual with the virtual lends a sharp insight into the proof and presentation techniques in the paper and you can very likely add this to your repertoire of tools

  • This pass can take many hours for beginners and more than an hour or two even for an experienced reader.

Doing a Literature Survey

  • First, use an academic search engine such as Google Scholar or CiteSeer and some well-chosen keywords to find three to five recent highly-cited papers in the area.

  • Otherwise, in the second step, find shared citations and repeated author names in the bibliography.

  • The third step is to go to the website for these top conferences and look through their recent proceedings

PDF

Over the next few Saturdays, I'll be going through some of the foundational papers in Computer Science, and publishing my notes here. This is #19 in this series.