Harshay Shah (@harshays_) profile picture
New York
Joined May 2012
Photo by Harshay Shah (@harshays_)Photo by Harshay Shah (@harshays_)Photo by Harshay Shah (@harshays_)
Harshay Shah retweeted
Gemini 4 Neon: frontier-level intelligence with fast end-to-end task completion across complex software engineering, knowledge work, and science workflows. Begins rolling out today in the Gemini API in @GoogleAIStudio, Gemini for business, for paid Google AI Pro and Ultra subscribers in Google @Antigravity, and for Google AI Pro and Ultra subscribers in the @GeminiApp.
911
1,485
687
18,422
2,101,446
Gemini 4 Carbon is our next frontier model. It’s state-of-the-art across cybersecurity and long-horizon capabilities.
749
1,388
1,101
14,525
1,738,581
The term "AI slop" absolves the humans and the human institutions in the pipeline leading up to that slop of any accountability. It's a handy term but I wish we called it by the more accurate "AI-assisted human slop".
4
2
48
2,327
Moritz Hardt thinks that OpenAI and Anthropic’s claims about benchmarks, which are used to test AI models’ capabilities, are misleading. But if the tests aren’t accurate, then how do we know how good the models really are? Read the interview: quantamagazine.org/he-wants-…
6
25
4
123
14,234
Introducing Gemini 4 Argon – our new frontier model. It’s built for complex workflows across coding, enterprise knowledge work, and cybersecurity defense – rolling out today to a set of trusted testers through our Fairwind Program.
1,980
4,609
4,525
42,688
9,062,184
Harshay Shah retweeted
Students sometimes ask me if it still makes sense, in this accelerating age, to pursue a PhD in AI. Perhaps counterintuitively, I think it's a great time to do so. I wrote up some thoughts on this here: web.mit.edu/phillipi/www/wri…
90
580
109
3,854
1,747,346
MoEs provide two knobs for scaling: model size (total params) + FLOPs-per-token (via active params). What’s the right scaling strategy? And how does it depend on the pretraining budget? Our work introduces sparsity-aware scaling laws for MoE LMs to tackle these questions! 🧵👇
🚨 One question that has always intrigued me is the role of different ways to increase a model's capacity: parameters, parallelizable compute, or sequential compute? We explored this through the lens of MoEs:
1
6
36
8,053
Harshay Shah retweeted
Wake up babe New MoE scaling laws dropped
6
47
5
420
45,423
Harshay Shah retweeted
How can we really know if a chatbot is giving a reliable answer? 🧵 MIT CSAIL’s "ContextCite" tool can ID the parts of external context used to generate any particular statement from a language model, improving trust by helping users easily verify the statement: bit.ly/3ZlyQqU
3
12
3
47
20,257
Harshay Shah retweeted
How do black-box neural networks transform raw data into predictions? Inside these models are thousands of simple "components" working together. New MIT CSAIL research (bit.ly/473lcfE) introduces a method that helps us understand how these components compose to affect model behavior — a key step in making neural networks more interpretable. 🧵
3
56
1
242
23,333
How is an LLM actually using the info given to it in its context? Is it misinterpreting anything or making things up? Introducing ContextCite: a simple method for attributing LLM responses back to the context: gradientscience.org/contextc… w/ @bcohenwang, @harshays_, @kris_georgiev1
7
48
5
241
51,718
New work with @andrew_ilyas and @aleks_madry on tracing predictions back to individual components (conv filters, attn heads) in the model! Paper: arxiv.org/abs/2404.11534 Thread: 👇
How do model components (conv filters, attn heads) collectively transform examples into predictions? Is it possible to somehow dissect how *every* model component contributes to a prediction? w/ @harshays_ @andrewilyas, we introduce a framework for tackling this question! Blog: gradientscience.org/modelcom… Code github.com/MadryLab/modelcom… Paper: arxiv.org/abs/2503.09540 [1/4]
1
10
48
12,348
If you are at #ICML2023 today, check out our work on ModelDiff, a model-agnostic framework for pinpointing differences between any two (supervised) learning algorithms! Poster: #407 at 2pm (Wednesday) Paper: icml.cc/virtual/2023/poster/… w/ @smsampark @andrew_ilyas @aleks_madry
12
52
6,480
TRAK, our latest work on data attribution (trak.csail.mit.edu), speeds up datamodels up to 1000x! ➡️ our earlier work ModelDiff (w/ @harshays_ @smsampark @aleks_madry) can now compare any two learning algorithms in larger-scale settings. Try it out: github.com/MadryLab/modeldif…
You’re deploying an ML system, choosing between two models trained w/ diff algs. Same training data, same acc... how do you differentiate their behavior? ModelDiff (gradientscience.org/modeldif…) lets you compare *any* two learning algs! w/ @harshays_ @smsampark @andrew_ilyas (1/8)
1
13
1
42
11,819
You’re deploying an ML system, choosing between two models trained w/ diff algs. Same training data, same acc... how do you differentiate their behavior? ModelDiff (gradientscience.org/modeldif…) lets you compare *any* two learning algs! w/ @harshays_ @smsampark @andrew_ilyas (1/8)
4
66
4
297
Do input gradients highlight discriminative and task-relevant features? Our #NeurIPS2021 paper takes a three-pronged approach to evaluate the fidelity of input gradient attributions. Poster: session 3, spot C0 Paper: https://xitter.gq/t.co/tnLwUBQtSh with @jainprateek_ and @pnetrapalli
13
57
Neural nets can generalize well on test data, but often lack robustness to distributional shifts & adversarial attacks. Our #NeurIPS2020 paper on simplicity bias sheds light on this phenomenon. Poster: session #4, town A2, spot C0, 12pm ET today! Paper: https://xitter.gq/t.co/PszvszwTr0
9
3
65