How to Read a Research Paper - 3-Pass Approach

Note: This article is a summary of How to Read a Paper by S. Keshav. I wrote this as a reminder to myself on how to efficiently read a paper. See last section on references for different tips on how to read a research paper. 😃

1) First Pass (5 to 10 minutes)

ToDo: Read the title, abstract, introduction, identify sections, sub-section headings and conclusion. Glance through the reference to mentally tick off those that you have read. Ignore everything else.

At the end of First Pass, you should be able to answer the five Cs:

a) Category: What is the category of this paper? Is it to provide measurement for experiments? Does it analyse an existing system? Or does it describe a research prototype?

b) Context: What other papers did this paper cited? What are the theoretical bases used to analyse the problem?

c) Correctness: Do the assumptions appear to be valid? Are the assumptions made similar to something you have come across?

d) Contributions: What is new / novel about this paper that the researcher wants to put across?

e) Clarity: Is the paper written succinctly?

Note: For writers, expect most reviewers to only make one pass so choose your section and sub-section carefully. If a reviewer cannot understand the gist after one pass, the paper will be rejected. If reviewer cannot understand the highlight within five minutes, the paper will never be read.

2) Second Pass (1 Hour)

ToDo: Read paper with greater care but ignore details such as math proofs.

As you read, have these questions in mind and try to answer them. Jot down and make comments in the paper.

a) Are the figures, diagrams and illustrations making sense? Are the axes labelled properly? Were the conclusions statistically significant?

b) Are there any new / unread references? Mark them down for future reference - you may want to read them.

Note: Second Pass should take around one hour and you should grasp the content of the paper and be able to summarise the main objective of the paper with supporting evidence to someone else.

There might be times where you don't understand the paper completely because you may be unfamiliar with the terminology and acronyms. Or there could be novel experimental techniques that were new and that's why bulk of the paper is incomprehensible to you. Or, it could just be that the paper is poorly written with unsubstantiated claims and assertions which you may not have agree with.

Now, after the First and Second Pass, you can decide to (i) set the paper side, (ii) return to the paper later after reading some background materials or (iii) continue with the Third Pass i.e. spend more time in the details.

3) Third Pass (1 to 5 Hours)

To fully understand a paper, you need to be able to fully (re)implement the paper. This can come up in different forms e.g. going through the math proof, examining and running their codes (if available, otherwise try coding yourself) or running the experiments in your lab. A good paper should therefore be reproducible but more often than not, we see papers missing out details on purpose and this is where you may want to email the authors for clarifications. During this process of re-implementing, you may find yourself gaining a deeper understanding in the paper's innovation, identify assumptions and missing details.

This pass requires great attention to detail and you should be challenging every assumption in every statement. At the end of this pass, you should be able to reconstruct the entire structure of the paper from memory and identify its strong and weak points. Sometimes, there might be missing assumptions or prior relevant work that the author missed out. There could also be potential issues with the experimental techniques or even faulty data sources or inherent bias in the data.

References:

[1] How to Read a Paper, S. Keshav https://web.stanford.edu/class/ee384m/Handouts/HowtoReadPaper.pdf

[2] How to Read and Understand a Scientific Paper: A Step-by-Step Guide for Non-Scientists https://www.huffpost.com/entry/how-to-read-and-understand-a-scientific-paper_b_5501628

[3] Guide to Reading Academic Research Papers https://towardsdatascience.com/guide-to-reading-academic-research-papers-c69c21619de6