Research Projects

Why do networks with far more parameters than training examples still generalize, and why does performance improve so predictably with scale? This track builds the math and the careful experiments that explain how training actually works.

Classical theory said huge models should overfit. They don’t, and explaining why turns rules of thumb into predictions, like scaling laws that forecast a model’s loss before it is trained.

Reverse-engineering trained networks into parts people can understand: which concepts a model represents internally, and which circuits combine them to produce a behavior.

Models store far more concepts than they have neurons by packing them into overlapping directions. Tools like sparse autoencoders pull these apart, so you can find a concept, such as the Golden Gate Bridge or buggy code, and turn it up or down to see what it does.

Models that know what they don’t know: ways to measure, calibrate and communicate how confident a prediction should be, from Bayesian methods to conformal prediction.

A prediction is only useful for a decision if you know how far to trust it. Conformal prediction wraps any model, even a black box, and returns answers with a guaranteed error rate, under only mild assumptions about the data.

Models that learn a whole distribution of data well enough to create new samples from it: images, video, molecules, text. Diffusion and flow models do it by gradually turning noise into data.

Text doesn’t have to be written left to right. Diffusion language models draft every position at once and refine it, which promises faster generation and the ability to edit text in place.

Turning a pretrained model into a useful one: learning from human feedback, from rewards a computer can check, and from the model’s own attempts.

Rewarding only whether a math answer is correct, with no worked solutions to copy, is enough for models to learn to check their work and backtrack on their own. That is a large part of how today’s reasoning models were made.

Treating data as the thing to engineer: what to collect, filter, mix and synthesize, and which examples a model actually learned a behavior from.

The choice of data often matters more than the choice of architecture. Careful filtering lets small models match larger ones trained on far more, and data attribution can trace a model’s behavior back to the examples that caused it.

Designing models that do more with less compute and memory: new layer types, sparse mixtures of experts, and ways to shrink and speed up a model after training.

The cost of attention grows with the square of the input length. State-space models and linear attention grow linearly, and quantization runs a model in 4 bits instead of 16, often with little loss. Efficiency decides what can run on a laptop or a phone.

Building known structure into a model. A molecule’s properties don’t change when you rotate it, and a graph means the same thing however its nodes are numbered.

A model that respects a symmetry by design doesn’t have to learn it from data, so it needs far less data. That is part of why such models do so well in chemistry and physics, where data is scarce.

Getting better answers by letting a model think longer: writing out steps, searching over alternatives, and checking its own work.

Spending compute while answering, not just while training, has become a second way to scale. A small model that thinks longer can match a much larger one on hard math, and how far that carries beyond math and code is still open.

Models that act: they plan, call tools, run code and browse, working through a task over many steps instead of answering in one shot.

Small errors compound. An agent that gets each step right 95% of the time will usually fail a 50-step task, so reliability, not raw intelligence, is often what holds agents back.

Models that learn how the world changes in response to actions, and agents such as robots that use them to act in the physical world.

A model that can predict what happens next lets an agent rehearse in imagination before it acts. Simulators learned from video are starting to stand in for hand-built ones when training robots.

Models that work across vision, audio and language at once, and connect words to the things they refer to.

Vision-language models often answer questions about an image without really looking at it, leaning on what the words suggest instead. Measuring and fixing that is key to trusting them with charts, scans and screens.

How models remember: long context windows, retrieval from outside documents, and learning new things after deployment without forgetting the old.

Today’s models mostly stop learning once they are trained. Making them keep learning on the job, cheaply and safely, would change what they are useful for, and it is still largely unsolved.

The science of measuring what models can do: building benchmarks that test what they claim to, and telling real progress from noise.

Benchmarks leak into training data, saturate, and sometimes contain wrong answers, so a gain on a leaderboard can mean less than it looks. Good measurement is how the field learns which ideas actually work.

Making models reliably do what we intend, even under pressure: from adversarial inputs and jailbreaks to models that exploit flaws in how they are rewarded.

Optimize a model hard against an imperfect reward and it finds the imperfections. Studying this reward hacking at small scale is one of the few safety questions you can research rigorously on a modest budget.

Using machine learning as a scientific instrument: predicting protein structures, discovering materials, simulating physics and climate, and generating hypotheses.

AlphaFold’s protein structure predictions shared the 2024 Nobel Prize in Chemistry. In many fields the bottleneck is now turning a model’s predictions into results that can be tested and trusted.

How people and AI systems work together, and what AI does to the people and institutions that use it: collaboration, fairness, and effects on work.

A model that is more accurate than people can still make a team worse if people trust it at the wrong moments. Knowing when to rely on AI matters as much as making AI better.

Using machine learning to make better decisions, not just better predictions: forecasting, experimentation, causal inference and optimization under uncertainty, in finance, health, operations and cities.

A more accurate forecast doesn’t always lead to a better decision, because what matters is the cost of the mistakes you make. Training and judging models by the decisions they lead to is where much of the real-world value is.