Anant Jain

How to Read a Paper

Paper Review

What This Paper Is About

If you've ever felt overwhelmed trying to read academic research papers, you're not alone. Researchers spend countless hours reading papers, yet most of us were never actually taught how to do it efficiently. We often start at the beginning and trudge through to the end, wasting time and energy on papers that might not even be relevant to our work.

This influential paper by S. Keshav from the University of Waterloo presents a practical solution: the three-pass method. Instead of reading a paper front-to-back in one sitting, you read it in up to three strategic passes, with each pass having a specific goal and time commitment. This approach helps you quickly filter out irrelevant papers and progressively deepen your understanding of the ones that matter.

The paper also covers how to use this method when doing a literature survey (reviewing multiple papers in a research area to understand the current state of knowledge).

Why This Method Works

The beauty of the three-pass approach is its flexibility. You can estimate upfront how much time you'll need to review a set of papers, and you can adjust the depth of your reading based on your goals. Not every paper needs a deep read.

Here's the progression:

  • First pass (5-10 minutes): Get a bird's-eye view of what the paper is about and decide if it's worth your time.
  • Second pass (up to 1 hour): Understand the main content and key points, but skip the nitty-gritty details like mathematical proofs.
  • Third pass (1-5 hours): Achieve a deep, thorough understanding where you could essentially recreate the work yourself.

Most papers you encounter will only need the first pass. Some deserve the second pass. Very few require the full third pass treatment.

The First Pass: Quick Scan (5-10 Minutes)

Think of this as "skimming with purpose." You're not trying to understand the details yet; you're just getting a sense of what the paper is about and whether it's relevant to you.

What to do:

  1. Read the title, abstract, and introduction carefully - These sections tell you the paper's main question and approach.

  2. Scan all section and subsection headings - This gives you the paper's structure and logical flow.

  3. Glance at any math or technical content - You don't need to understand it deeply yet, just get a sense of what theoretical approach the authors are using.

  4. Read the conclusions - This tells you what the authors think they accomplished.

  5. Look over the references - Are there papers you recognize? This helps you place the work in context of what you already know.

The Five Cs Test

After this first pass, you should be able to answer these five questions (the "five Cs"):

  • Category: What type of paper is this? (A measurement study? An analysis of an existing system? A description of a new algorithm or prototype?)
  • Context: What other research is this related to? What theoretical foundations does it build on?
  • Correctness: Do the assumptions seem valid?
  • Contributions: What are the paper's main contributions?
  • Clarity: Is the paper well-written?

If you can't answer these questions, or if the answers tell you this paper isn't relevant, stop here. You've saved yourself hours of unnecessary reading.

Valid reasons to stop after the first pass:

  • The topic doesn't interest you or isn't relevant to your work
  • You don't have enough background knowledge to understand the paper (maybe come back to it later)
  • The authors make questionable assumptions
  • The paper is poorly written or unclear

Note for authors: This is why clear section headings and well-written abstracts matter so much. Many readers will decide whether to continue based on these alone. Including a graphical abstract (a single figure that summarizes your paper) is becoming increasingly common and can help readers quickly grasp your work.

The Second Pass: Engaged Reading (Up to 1 Hour)

You've decided this paper is worth your time. Now you'll read it more carefully, but you're still not diving into every proof or technical detail. The goal is to grasp the content and main arguments.

What to do:

  • Read the entire paper with care, but skip over the detailed proofs and complex derivations for now
  • Take notes as you go - Jot down key points in the margins or in a separate document. This active engagement helps comprehension and retention
  • Pay close attention to figures and graphs - These often contain the paper's key results. Ask yourself:
    • Are the axes clearly labeled?
    • Do results include error bars showing statistical significance?
    • Do the visual results actually support the authors' claims?
  • Mark sections you don't understand - You might come back to these, or they might not be critical to your understanding

Time commitment: Up to an hour for an experienced reader in a related field.

When to use the second pass: This level of detail is appropriate for papers in your area of interest that don't fall directly within your research specialty. For example, if you're a machine learning researcher reading about a new optimization technique, or a systems engineer reading about a novel database architecture.

What if you still don't understand the paper?

Even after the second pass, you might find yourself confused. This happens for several reasons:

  • The subject is new to you - Unfamiliar terminology, acronyms, or concepts you haven't encountered before
  • Unfamiliar techniques - The authors use proof methods or experimental techniques you don't know
  • Poor writing - The paper makes unsupported assertions or has too many "forward references" (referring to things explained later)
  • You're tired - Sometimes it's just late at night and your brain isn't cooperating

If this happens, you have three options:

  1. Set it aside - Maybe you don't need to understand this particular paper to succeed in your work
  2. Return later - Read some background material first, then come back to this paper
  3. Push through - Move on to the third pass if you really need to master this material

The Third Pass: Deep Understanding (1-5 Hours)

This is where you achieve true mastery of a paper. The third pass is reserved for papers that are critical to your research or that you need to understand deeply enough to critique or build upon.

The core technique: Virtual re-implementation

The key to the third pass is attempting to virtually re-implement the paper. This means:

  • Make the same assumptions as the authors
  • Try to recreate their work from scratch in your mind (or on paper)
  • Think through how you would prove each theorem or design each experiment
  • Identify every assumption and challenge it

What you gain:

By comparing your mental reconstruction with what the authors actually did, you'll discover:

  • The paper's true innovations - What's genuinely new versus what's incremental
  • Hidden failings and assumptions - Things the authors didn't explicitly state or may have overlooked
  • Proof and presentation techniques - New tools and methods you can add to your own toolkit
  • Potential improvements - Places where you might extend or improve upon the work

Time commitment:

  • For beginners: Many hours (4-5 or more)
  • For experienced readers: 1-2 hours minimum

This pass requires great attention to detail. You should be able to reconstruct the entire paper from memory, understand every assumption, and identify any weaknesses in the arguments or experimental design.

When to do a third pass:

  • You're writing a paper and need to thoroughly understand related work
  • You're reviewing the paper for a conference or journal
  • The work is directly relevant to your current research
  • You're trying to implement or extend the technique described
  • You need to teach this material to others

Doing a Literature Survey

What if you need to get up to speed on an entire research area you're unfamiliar with? The three-pass method can be adapted for conducting a literature survey. Here's a systematic approach:

Step 1: Find the key papers (1-2 hours)

Use an academic search engine (Google Scholar, CiteSeer, Semantic Scholar) with well-chosen keywords to find 3-5 recent, highly-cited papers in the area. These are likely to be important foundational or survey papers.

Do a first pass on each of these papers to get a sense of the field.

Step 2: Identify the research community

Look for patterns in these papers:

  • Repeated author names - Who are the key researchers in this area?
  • Shared citations - What papers do they all reference? These are likely foundational works you should read.
  • Key conferences and journals - Where is this research being published?

This helps you identify the "who's who" of the field and the most influential prior work.

Step 3: Find the leading venues

Go to the websites of the top conferences and journals you identified. Browse through their recent proceedings (last 1-2 years) to find:

  • High-quality recent work
  • Current trends and hot topics
  • Additional important papers you might have missed

Putting it together:

These steps give you a comprehensive overview of the field. You'll have identified the key papers, the influential researchers, and the current state of the art. From here, you can decide which papers deserve a second pass (or third pass) based on their relevance to your specific goals.

Key Takeaways

The three-pass method is elegant in its simplicity:

  1. First pass: Quick scan to decide if the paper is worth your time (5-10 minutes)
  2. Second pass: Read for understanding without getting bogged down in details (up to 1 hour)
  3. Third pass: Deep dive with virtual re-implementation for complete mastery (1-5+ hours)

The real power comes from knowing when to stop. Not every paper needs a full read. By progressively deepening your engagement, you save time and mental energy for the papers that truly matter to your work.

Whether you're a graduate student starting your research journey, an experienced researcher staying current with the literature, or a practitioner trying to understand academic work, this method provides a structured, efficient approach to reading research papers.

Resources

Over the next few Saturdays, I'll be going through some of the foundational papers in Computer Science, and publishing my notes here. This is #19 in this series.