A 40-page paper with appendices can swallow an afternoon. The trick is to decide quickly whether it deserves that afternoon. Read for claims first, evidence second, and details only if the claim matters to you.
1. Title, abstract and figures
Read the title, then the abstract twice. Underline the claimed contribution and the claimed improvement. Then look at every figure and table caption before reading the method. Figures are the paper’s compressed argument: what was measured, against what baseline, and how large the effect is.
2. The last paragraph of the introduction
Most introductions end with a short list of contributions. That list is the author’s own summary of what is new. If the list is vague (“we propose a novel framework”), the paper may be incremental; if it is specific (“we reduce latency by 38% at equal accuracy on three benchmarks”), you know what to verify.
3. The method diagram
Find the architecture or pipeline diagram. Trace the data flow from input to output. Identify what is trained, what is frozen, and where the loss is applied. If you cannot explain the diagram to yourself in two minutes, the paper will be expensive to understand — decide whether the claim is worth it.
4. Experiments and baselines
Ask three questions: What is the baseline? Is it tuned as carefully as the proposed method? What is the evaluation set, and does it match the claim? A new method compared with an old model on a convenient benchmark is not evidence of a general improvement. Look for ablations that isolate the contribution: if removing the clever part barely changes the result, the clever part is not the reason it works.
5. Limitations
Good papers have a limitations section; read it before the related work. It tells you where the authors expect the method to fail, which is often exactly the setting you care about. If there is no limitations section, treat that as a limitation of the paper.
6. Related work, last
Related work is a map of the field, not a prerequisite for understanding the paper. Read it after you know the claim. It is most useful for finding the two or three prior papers that define the baseline — those are the ones to skim next.
Questions to keep on a sticky note
- What exactly is new: data, objective, architecture, training recipe or evaluation?
- What did it cost to train, and can the result be reproduced at a smaller scale?
- Which baselines were rerun by the authors, and which numbers are copied from other papers?
- What happens on inputs outside the benchmark distribution?
- Is the code and data released, and is the licence usable for your case?
Read the appendix selectively
Appendices often contain the details that decide whether a result is real: hyperparameters, data cleaning, prompt templates, evaluation rubrics, failure cases. You do not need all of them. Read the sections that correspond to the claim you care about, and skip the rest without guilt.
A note on arXiv versions
Papers change between versions. Check the version date and the changelog; a result that was withdrawn or revised may still be circulating on social media. Prefer the latest version, and treat a paper without code or a reproducible evaluation as a hypothesis rather than a fact.
Reading papers is a filtering skill. The goal is not to finish every PDF; it is to spend your attention on the few that change how you think or what you build.