
Harshay Shah retweeted
Gemini 4 Neon: frontier-level intelligence with fast end-to-end task completion across complex software engineering, knowledge work, and science workflows.
Begins rolling out today in the Gemini API in @GoogleAIStudio, Gemini for business, for paid Google AI Pro and Ultra subscribers in Google @Antigravity, and for Google AI Pro and Ultra subscribers in the @GeminiApp.
Harshay Shah retweeted
Gemini 4 Carbon is our next frontier model.
It’s state-of-the-art across cybersecurity and long-horizon capabilities.
Harshay Shah retweeted
The term "AI slop" absolves the humans and the human institutions in the pipeline leading up to that slop of any accountability. It's a handy term but I wish we called it by the more accurate "AI-assisted human slop".
Harshay Shah retweeted
Moritz Hardt thinks that OpenAI and Anthropic’s claims about benchmarks, which are used to test AI models’ capabilities, are misleading. But if the tests aren’t accurate, then how do we know how good the models really are? Read the interview: quantamagazine.org/he-wants-…
Harshay Shah retweeted
Introducing Gemini 4 Argon – our new frontier model.
It’s built for complex workflows across coding, enterprise knowledge work, and cybersecurity defense – rolling out today to a set of trusted testers through our Fairwind Program.
Harshay Shah retweeted
Students sometimes ask me if it still makes sense, in this accelerating age, to pursue a PhD in AI.
Perhaps counterintuitively, I think it's a great time to do so.
I wrote up some thoughts on this here: web.mit.edu/phillipi/www/wri…
MoEs provide two knobs for scaling: model size (total params) + FLOPs-per-token (via active params).
What’s the right scaling strategy? And how does it depend on the pretraining budget?
Our work introduces sparsity-aware scaling laws for MoE LMs to tackle these questions!
🧵👇
Harshay Shah retweeted
How can we really know if a chatbot is giving a reliable answer? 🧵
MIT CSAIL’s "ContextCite" tool can ID the parts of external context used to generate any particular statement from a language model, improving trust by helping users easily verify the statement: bit.ly/3ZlyQqU
Harshay Shah retweeted
How do black-box neural networks transform raw data into predictions?
Inside these models are thousands of simple "components" working together.
New MIT CSAIL research (bit.ly/473lcfE) introduces a method that helps us understand how these components compose to affect model behavior — a key step in making neural networks more interpretable. 🧵
Harshay Shah retweeted
How is an LLM actually using the info given to it in its context? Is it misinterpreting anything or making things up?
Introducing ContextCite: a simple method for attributing LLM responses back to the context: gradientscience.org/contextc…
w/ @bcohenwang, @harshays_, @kris_georgiev1
New work with @andrew_ilyas and @aleks_madry on tracing predictions back to individual components (conv filters, attn heads) in the model!
Paper: arxiv.org/abs/2404.11534
Thread: 👇
How do model components (conv filters, attn heads) collectively transform examples into predictions? Is it possible to somehow dissect how *every* model component contributes to a prediction?
w/ @harshays_ @andrewilyas, we introduce a framework for tackling this question!
Blog: gradientscience.org/modelcom… Code github.com/MadryLab/modelcom… Paper: arxiv.org/abs/2503.09540
[1/4]
If you are at #ICML2023 today, check out our work on ModelDiff, a model-agnostic framework for pinpointing differences between any two (supervised) learning algorithms!
Poster: #407 at 2pm (Wednesday)
Paper: icml.cc/virtual/2023/poster/…
w/ @smsampark @andrew_ilyas @aleks_madry
Harshay Shah retweeted
TRAK, our latest work on data attribution (trak.csail.mit.edu), speeds up datamodels up to 1000x!
➡️ our earlier work ModelDiff (w/ @harshays_ @smsampark @aleks_madry) can now compare any two learning algorithms in larger-scale settings.
Try it out: github.com/MadryLab/modeldif…
You’re deploying an ML system, choosing between two models trained w/ diff algs. Same training data, same acc... how do you differentiate their behavior?
ModelDiff (gradientscience.org/modeldif…) lets you compare *any* two learning algs!
w/ @harshays_ @smsampark @andrew_ilyas (1/8)
Harshay Shah retweeted
You’re deploying an ML system, choosing between two models trained w/ diff algs. Same training data, same acc... how do you differentiate their behavior?
ModelDiff (gradientscience.org/modeldif…) lets you compare *any* two learning algs!
w/ @harshays_ @smsampark @andrew_ilyas (1/8)
Do input gradients highlight discriminative and task-relevant features?
Our #NeurIPS2021 paper takes a three-pronged approach to evaluate the fidelity of input gradient attributions.
Poster: session 3, spot C0
Paper: https://xitter.gq/t.co/tnLwUBQtSh
with @jainprateek_ and @pnetrapalli
Neural nets can generalize well on test data, but often lack robustness to distributional shifts & adversarial attacks.
Our #NeurIPS2020 paper on simplicity bias sheds light on this phenomenon.
Poster: session #4, town A2, spot C0, 12pm ET today!
Paper: https://xitter.gq/t.co/PszvszwTr0





















