Ben Cohen-Wang (@bcohenwang) profile picture

Pre-training @AnthropicAI On leave from PhD at MIT with Aleksander Madry

Joined September 2022
Photo by Ben Cohen-Wang (@bcohenwang)Photo by Ben Cohen-Wang (@bcohenwang)Photo by Ben Cohen-Wang (@bcohenwang)Photo by Ben Cohen-Wang (@bcohenwang)Photo by Ben Cohen-Wang (@bcohenwang)Photo by Ben Cohen-Wang (@bcohenwang)
Models often fail under distribution shifts—can pre-training on a large and diverse dataset and then fine-tuning on a task-specific dataset help? W/ @bcohenwang, @josh_vendrow we show that this depends on the specific failure mode. In particular, pre-training can help with extrapolation, but does not address failures that stem from dataset biases.
5
42
4
255
48,770
Will your model identify a polar bear on the moon? How would you know? Dataset Interfaces let you generate images from your dataset under whatever distribution shift you desire! arxiv.org/abs/2302.07865 gradientscience.org/dataset-… W/ @josh_vendrow @saachi_jain_ @logan_engstrom
3
32
2
132
29,565