Five influential papers to understand modern AI
A guided reading path through Transformers, few-shot learning, retrieval, human feedback and reasoning, with practical questions for your own AI work.
Why can an AI follow a new instruction, use documents and still get something wrong? These five papers offer a useful foundation for understanding language models and working with them more thoughtfully.
This is a curated starting point for generative language AI, not a definitive ranking of all AI research. The papers explain ideas; they do not document MAIA's architecture or prove that a feature uses a particular method. Start with AI basics if the terms are new.
Choose a starting point
Section titled “Choose a starting point”| Your question | Start here | What to look for |
|---|---|---|
| How do models connect words and context? | Attention Is All You Need | The architecture diagram |
| Why do examples in a prompt help? | Language Models are Few-Shot Learners | The different evaluation settings |
| How can a model work with external sources? | Retrieval-Augmented Generation | The retriever and generator |
| How does a model learn to follow instructions? | Training language models with human feedback | The training stages |
| Why does breaking a problem into steps help? | Chain-of-Thought Prompting | The examples and failure cases |
Attention Is All You Need
Section titled “Attention Is All You Need”Vaswani et al., 2017. The Transformer uses attention to relate positions in a sequence, without the recurrent or convolutional layers used by many earlier sequence models. It became a foundation for modern language-model architectures.
Read the architecture figure and the explanation of attention first. For deeper study, examine positional information and multi-head attention. Attention is a computational mechanism, not evidence of human understanding or a guarantee of factual accuracy. Read the paper.
Language Models are Few-Shot Learners
Section titled “Language Models are Few-Shot Learners”Brown et al., 2020. The GPT-3 study shows how examples supplied in a prompt can guide performance on many tasks without updating the model's parameters for each task. It also reports limitations and evaluation difficulties.
Compare zero-shot, one-shot and few-shot settings. Notice that learning from the current context differs from permanently training a model. For deeper study, inspect the task-specific results and discussion of benchmark contamination. Read the paper.
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Section titled “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”Lewis et al., 2020. RAG combines a model with retrieved passages from an external document index. The study evaluates this combination on knowledge-intensive language tasks.
Follow one question from retrieval to generation. For deeper study, compare retrieving passages for a whole answer with varying them during generation. External sources can help, but the system still has to retrieve relevant evidence and use it correctly. A citation alone does not establish correctness. Read the paper.
Training language models to follow instructions with human feedback
Section titled “Training language models to follow instructions with human feedback”Ouyang et al., 2022. InstructGPT combines examples of desired responses, human preference rankings and reinforcement learning to improve instruction following. The study shows that model size alone does not determine which answers people prefer.
Read the training overview, then examine who supplied feedback and how outputs were evaluated. Preference is not the same as truth, and the authors report remaining mistakes. Read the paper.
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Section titled “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models”Wei et al., 2022. The study finds that examples with intermediate reasoning steps improve performance on several arithmetic, commonsense and symbolic tasks in sufficiently large models.
Compare the prompting examples and inspect where the method fails. It does not establish that every longer explanation is correct, or that the same prompt is best for every later model. Ask for checkable calculations, evidence and concise reasons; assess the result independently. Read the paper.
Read actively, even without a research background
Section titled “Read actively, even without a research background”For each paper, write down the problem, the proposed method, the evidence and one limitation. Read the abstract, figures and conclusion before deciding which technical sections you need. Treat benchmark scores as results under specific conditions, not a promise about your own work.
If you use MAIA as a reading companion, supply the paper and ask:
Explain the research question and proposed method in plain language. Separate the authors' results from your interpretation. Point to the relevant sections. Define unfamiliar terms and list what the study does not establish. Do not invent experiments or results.
Check the explanation in the original. For a team reading session, each person can bring one finding, one limitation and one question to discuss.
Turn reading into a small experiment
Section titled “Turn reading into a small experiment”Try the same fictional Acme Corp task with and without a worked example. Keep the sources and success criteria fixed. Compare correctness, format and correction effort on several inputs, including one with missing information. This is a practice exercise, not a reproduction of the papers.
Use better prompts to design the instruction and evaluate results to record what changed. Understanding a method is useful; testing its value on your own task is the next step.