Research Scientist at Google DeepMind
Joined January 2013
- Tweets21
- Following96
- Followers224
- Likes104
Azade Nova retweeted
This is exactly why open-source frameworks like FeynRL matter more than ever. Open weights are NOT enough. Open AI needs open training recipes, alg, etc.
You should be able to understand training, modify it, & build new algorithms, optimizers, recipes.
github.com/FeynRL-project/Fe…
Happy to share what I've been working on for the past few months. Check out updated Gemini 1.5 report. goo.gle/GeminiV1-5
Gemini 1.5 Model Family: Technical Report updates now published
In the report we present the latest models of the Gemini family – Gemini 1.5 Pro and Gemini 1.5 Flash, two highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio.
Our latest report details notable improvements in Gemini 1.5 Pro within the last four months.
Our May release demonstrates significant improvement in math, coding, and multimodal benchmarks compared to our initial release in February.
Furthermore, the 1.5 Pro Model is now stronger than 1.0 Ultra.
The latest Gemini 1.5 Pro is now our most capable model for text and vision understanding tasks, surpassing 1.0 Ultra on 16 of 19 text benchmarks and 18 of 21 of the vision understanding benchmarks. The table below highlights the improvement in average benchmark performance for different categories in 1.5 Pro since Feb, and also shows the strength of the model relative to the 1.0 Pro and 1.0 Ultra models. The 1.5 Flash model also compares very well against the 1.0 Pro and 1.0 Ultra models.
One clear example of this can be seen on MMLU
On MMLU we find that 1.5 Pro surpasses 1.0 Ultra in the regular 5-shot setting scoring 85.9% versus 83.7%. However with additional inference compute, via majority voting on top of multiple language model samples, we can get a performance of 91.7% versus Ultra’s 90.0%, which extends the known performance ceiling of this task.
@OriolVinyalsML and I are very proud of the whole Gemini team, and it’s fantastic to see this progress and to share these highlights from our Gemini Model Family.
Read the updated report here: goo.gle/GeminiV1-5
If you want to learn about efficient inference in LLMs, stop by to our poster #733 on Wed (July 26) at 11am icml.cc/virtual/2023/poster/…, #ICML23
Check out our gradient-free structural pruning approach for large models. No need for retraining or labeled data! In just a few minutes on one GPU, reduces up to 40% of the original FLOPs at minimal loss of accuracy.
w/ @hanjundai, Dale Schuurmans
paper: arxiv.org/abs/2303.04185
ALT Kernelized Convex Masking (KCM): A gradient-free structured pruning framework with R2D2 as a core component. R2D2 combines the ranks of the Representative Ranking (R2) and Data-Driven (D2) rank.
Check out our gradient-free structural pruning approach for large models. No need for retraining or labeled data! In just a few minutes on one GPU, reduces up to 40% of the original FLOPs at minimal loss of accuracy.
w/ @hanjundai, Dale Schuurmans
paper: arxiv.org/abs/2303.04185
ALT Kernelized Convex Masking (KCM): A gradient-free structured pruning framework with R2D2 as a core component. R2D2 combines the ranks of the Representative Ranking (R2) and Data-Driven (D2) rank.
Join us tomorrow, Dec 3 at #NeurIPS2022 for the 6th edition of the ML for Systems workshop. We will kick off with a keynote presentation from Google SVP Jeff Dean (@JeffDean).
Tune in tomorrow at 8:30am Central Time to see what’s new at Google AI. (Room 396)
#mlforsystems
Organizers @martin_maas @Azade_na @BenoitSteiner Neel Kant @DZhang50
Steering @annadgoldie @Azaliamirh @jonathanrraiman @miladhash @kswersk
Azade Nova retweeted
Interested in learning more about ML for Systems research at Google? Stop by the booth today at 10:30 am to hear @martin_maas discuss the latest in ML-driven systems! And if you want to learn more about the area, join the ML for Systems workshop on Saturday, December 3rd.
Azade Nova retweeted
Join us on Saturday, December 3 at #NeurIPS2022 for the 6th edition of the ML for Systems workshop. We have an exciting program with 5 invited talks and 19 accepted papers, as well as plenty of opportunities to chat with researchers in the area. mlforsystems.org
Azade Nova retweeted
As a 4th time organizer, I must say we have an amazing program this year at ML for Systems at NeurIPS.
Before I forget —
Organizers @martin_maas @Azade_na @BenoitSteiner Neel Kant @DZhang50
Steering @annadgoldie @Azaliamirh @jonathanrraiman @miladhash @kswersk
+ PC.
🙏
Azade Nova retweeted
🔥 LLMs can do OOD reasoning:We show that we can teach algorithms to LLMs with only three examples and it generalizes to much longer input length as much as the context length allows! We can also teach multiple algorithms,compose them to teach complex ones & use them as tools! 🔥
“LLMs can’t even do addition”
📄🚨We show that they CAN add! To teach algos to LLMs, the trick is to describe the algo in enough detail so that there is no room for misinterpretation
w/ @Azade_na @hugo_larochelle @AaronCourville @bneyshabur @HanieSedghi
arxiv.org/abs/2211.09066
Azade Nova retweeted
Excited to share that our work has been published in Nature! Our RL agent generates chip layouts in just a few hours, whereas human experts can take months. These superhuman AI-generated layouts were used in Google's latest AI accelerator (TPU-v5)! nature.com/articles/s41586-0…
Azade Nova retweeted
Thrilled to announce that our work on RL for chip floorplanning was published in Nature & used in production to design next generation Google TPUs, with potential to save thousands of hours of engineering effort for each next generation ASIC:
nature.com/articles/s41586-0… (1/7)
ALT Image by Google internal agency Hook.
Azade Nova retweeted
Autoregressive graph generation is powerful but slow. Our recent work reduces its complexity from O(V^2) to O((E+V) log V), with sublinear mem cost and training parallelism.
Paper: arxiv.org/abs/2006.15502
Code: github.com/google-research/g…
w/ @Azade_na @liyuajia @daibond_alpha Dale
Azade Nova retweeted
Check out TraDE, our new transformer based density estimator arxiv.org/abs/2004.02441. It beats NAF, BNAF, TAN, FFJORD, NSF, AEM on reference datasets. Works well for #bumblebee, too (left: original, right: estimate). @rasoolfa @pratikac and Jonas Mueller are the real stars here.
Azade Nova retweeted
Today we are excited to release video recordings of lectures from "Advanced Deep Learning and Reinforcement Learning", a course on deep RL taught at @UCL earlier this year by DeepMind researchers:
youtube.com/playlist?list=PL…
Enjoy!
Azade Nova retweeted
We've seen @BostonDynamics' infamous robot dog open doors, and climb stairs. But now it can navigate complicated construction sites, and do detailed work inspections too 👀: wired.trib.al/0CcVEmR
Azade Nova retweeted
If you want to do ML research, consider applying for the 2019 Google AI Residency program! You'll have the opportunity to conduct cutting-edge research working in a wide variety of areas, and this year we're expanding to host residents in even more locations.
Applications for the 2019 Google AI Residency program are now open! Visit g.co/airesidency/apply for more information on how to apply. To learn more about the accomplishments of the recently graduated second class of residents, visit ↓ goo.gl/5QZbsF
Azade Nova retweeted
the paper behind BERT is now online: arxiv.org/abs/1810.04805
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova arxiv.org/abs/1810.04805
