Events
Upcoming events
The Kempner Institute convenes experts in the fields of natural and artificial intelligence, building community and supporting academic exploration through lectures, workshops and talks by visiting scientists. Our Kempner Seminar Series is held on Fridays and explores recent advances in the field.
Stefano Ermon: Score Entropy Discrete Diffusion Models
Join us for a talk by Stefano Ermon, Associate Professor of Computer Science at Stanford University. This talk is part of the Kempner Seminar Series, a research-level seminar series on recent advances in the field. Abstract: Diffusion models are at the core of many state-of-the-art generative AI systems for content such as images, videos, and audio. These models crucially rely on estimating gradients of the data distribution (scores) and efforts to generalize score-based modeling to discrete structures have had limited success. As a result, state-of-the-art generative models for discrete data such as language are based on autoregressive modeling (i.e. next token prediction). In this work, we bridge this gap by proposing a framework that extends score matching to discrete spaces and integrates seamlessly to build discrete diffusion models. The resulting Score Entropy Discrete Diffusion models are an alternative probabilistic modeling technique that achieves highly competitive performance at the scale of GPT-2 while introducing distinct algorithmic benefits. Our empirical results challenge the longstanding dominance of autoregressive modeling and could pave the way for an alternative class of language models built from radically different principles.
Kempner Institute Open House
Join us for an Open House of the new Kempner Institute space! Please RSVP if you plan to attend.
Kim Stachenfeld
Join us for a talk by Kim Stachenfeld, Senior Research Scientist at Google Deepmind. This talk is part of the Kempner Seminar Series, a research-level seminar series on recent advances in the field. Talk abstract forthcoming.
Ev Fedorenko
Join us for a talk by Ev Fedorenko, Assistant Professor at MIT and Investigator the McGovern Institute. This talk is part of the Kempner Seminar Series, a research-level seminar series on recent advances in the field. Talk abstract forthcoming.
Brian DePasquale
Join us for a talk by Brian DePasquale, Assistant Professor of Biomedical Engineering at Boston University. This talk is part of the Kempner Seminar Series, a research-level seminar series on recent advances in the field. Talk abstract forthcoming.
Alison Gopnik
Join us for a talk by Alison Gopnik, Professor of Psychology and Affiliate Professor of Philosophy at the University of California at Berkeley. This talk is part of the Kempner Seminar Series, a research-level seminar series on recent advances in the field. Talk abstract forthcoming.
Past events
March 22, 2024
Solving overparametrized systems of nonlinear equations
Speaker: Andrea Montanari
March 15, 2024
Using the Tools of Cognitive Science to Understand the Behavior of Large Language Models
Speaker: Tom Griffiths
February 23, 2024
Modeling the Navigational Circuitry of the Fly
Speaker: Larry Abbott
February 20, 2024
Physics of Learning and Computation in Natural and Artificial Neural Networks
Speaker: Hidenori Tanaka
February 20, 2024
Towards Principled Post-Training of Large Language Models
Speaker: Banghua Zhu
February 20, 2024
Bridging Neural Dynamics To Goal-Directed Behavior Across Timescales
Speaker: Aran Nayebi
February 20, 2024
Measure Transport Perspectives on Sampling, Generative Modeling, and Beyond
Speaker: Michael Albergo
February 16, 2024
Active Predictive Coding: A Sensory-Motor Theory of the Neocortex and a Unifying Framework for AI
Speaker: Rajesh Rao
February 13, 2024
Multi-scale Theory of Neural Representations in Biological and Artificial Neural Networks
Speaker: SueYeon Chung
February 6, 2024
Building Performant and Brain-Like Recurrent Models from Neurons and Astrocytes
Speaker: Leo Kozachkov
January 26, 2024
CICERO: Human-Level Performance in the Game of Diplomacy by Combining Language Models with Strategic Reasoning
Speaker: Noam Brown
December 8, 2023
Virtual Seminar Series: Research Candidate Presentations
Speakers: Ted Moskovitz, Noor Sajid, Ben Sorscher and Enric Boix
December 5, 2023
Virtual Seminar Series: Research Candidate Presentations
Speakers: Jeremy Cohen, Mikail Khona, Yuhan Helena Liu, and Zaixi Zhang
December 1, 2023
Virtual Seminar Series: Research Candidate Presentations
Speakers: Libby Zhang, Bingbin Liu, Dilip Arumugam, and Aaron Walsman
November 27, 2023
Virtual Seminar Series: Research Candidate Presentations
Speakers: Maksym Andriushchenko, Carles Domingo i Enrich, Thomas Fel, Isabel Papadimitriou
November 3, 2023
Unsupervised pretraining in biological neural networks
Speaker: Carsen Stringer
October 13, 2023
Logic reasoning and generalization on the unseen
Speaker: Emmanuel Abbe
September 15, 2023
Teach language models to reason
Speaker: Denny Zhou
September 8, 2023
Dataset security issues in generative AI
Speaker: Tom Goldstein
May 23, 2023
Towards Machines that can learn, reason and plan
Speaker: Yann LeCun
May 19, 2023
Model-based reinforcement learning and the future of language models
Speaker: Timothy Lillicrap
May 12, 2023
Common Sense: The Dark Matter of Language and Intelligence
Speaker: Yejin Choi
May 3, 2023
Get to Know the Kempner
Speakers: Sham Kakade, Elise Porter, Bernardo Sabatini
April 21, 2023
Shifts in Distributions and Preferences in Learning
Speaker: Jamie Morgenstern
March 10, 2023
A Data-Centric View on Reliable Generalization: From ImageNet to LAION-5B
Speaker: Ludwig Schmidt
February 27, 2023
Beyond Bayes: towards new principles of optimality for neuroscience and AI
Speaker: Richard Lange
February 27, 2023
Towards Human-like Deep Reinforcement Learning
Speaker: Willa Carvalho
February 27, 2023
Natural Inductive Biases for Artificial Intelligence
Speaker: Thomas Keller
April 14, 2023
Deep Language Models as a Cognative Model for NLP in the Human Brain
Speaker: Uri Hasson
March 31, 2023
First Contact
Speaker: Sebastien Bubeck
March 20, 2023
Interpreting Training
Speaker: Naomi Saphra
February 24, 2023
Neural networks make stuff up. What should we do about it?
Speaker: Chelsea Finn
February 10, 2023
Studying Neural Net Generalization through Influence Functions
Speaker: Roger Grosse
February 3, 2023
Data structure and curse of dimensionality in deep learning
Speaker: Matthieu Wyart