Someone looking to change the world by using technology to improve society.
Canada
Joined February 2010
- Tweets854
- Following178
- Followers60
- Likes1.8K
David Marques retweeted
Looking forward to London in a week to visit the @cohere @CohereForAI team 🔥
We are also hosting invite-only event on July 10th w researchers, experts in technical governance interested in global models that are safe and performant.
Join us: bit.ly/4eN6i0m
David Marques retweeted
We've barely scratched the surface of the space of deep learning architectures.
It's a high dimensional space, so the volume is almost entirely contained in the surface.
But we've scratched a tiny subset of the surface.
"Simplifying Transformer Blocks" ranks easily among my favorite research papers that I've read this year.
Here, the authors look into how the standard transformer block, essential to LLMs, can be simplified without compromising convergence properties and downstream task performance.
Based on signal propagation theory and empirical evidence, they find that many parts can be removed to simplify GPT-like decoder architectures as well as encoder-style BERT models:
skip connections
normalization layers (LayerNorm)
projection and value parameters
sequential attention and MLP sub-blocks (in favor of a parallel layout)
The authors also did a great job referencing tons of related work motivating their experiments. I definitely recommend reading this paper just for the references alone: arxiv.org/abs/2311.01906
David Marques retweeted
To better enable the community to build on our work — and contribute to the responsible development of LLMs — we've published further details about the architecture, training compute, approach to fine-tuning & more for Llama 2 in a new paper.
Full paper➡️ bit.ly/44JAELQ
David Marques retweeted
Many generative audio models rely on auto-regressive decoding, which produces tokens one by one and can be slow. Read all about SoundStorm, a new method tailored for audio tokens that uses parallel decoding for efficient and high-quality audio generation ↓goo.gle/3CJrxi8
David Marques retweeted
Tuesday is going to be wild. If you care about AI, developer tools, or software engineering, there is a good chance it will 🤯 your mind! Join us 9am pacific at modular.com
Life update: I started a new job at @Modular_AI! I was just about to accept an offer from Apple after leaving Google but after talking to @clattner_llvm and realizing how they planned to disrupt AI I knew I had to join. The public unveiling is Tuesday! modular.com/
David Marques retweeted
Announced by Mark Zuckerberg this morning — today we're releasing DINOv2, the first method for training computer vision models that uses self-supervised learning to achieve results matching or exceeding industry standards.
More on this new work ➡️ bit.ly/3GQnIKf
David Marques retweeted
LlamaAcademy: a factory that teaches LLaMA's how write API code. ChatGPT Plugins, but no manifest file is needed -- it just reads the API docs like a human would!
For example, here is a finetuned model that learned how to talk to Notion by reading its API docs:
David Marques retweeted
Want to learn about meta-learning & few-shot learning?
All of the latest lecture videos for Stanford CS330 are now online!
youtube.com/playlist?list=PL…
New topics in Fall '22 include:
- self-supervised pre-training
- large scale meta-optimization
- domain adaptation & generalization
David Marques retweeted
Save the date! We are super excited to share what we have been up to, including some incredible product and tech and a big new step forward for AI, that I hope many will find interesting….
The world of AI changes, forever 🔥
Register now for our live stream event on May 2nd, 9AM PT 🍿🎥
Visit modular.com right now! It's going to be epic! 🚀
#ai #ml #modular_ai
David Marques retweeted
Spotlight, a vision-only model that achieves general user interface (UI) understanding from raw pixels, establishes state-of-the-art results on many UI tasks, including widget captioning, screen summarization, and tappability prediction. Learn more → goo.gle/3IBbK7E
David Marques retweeted
Looking to stay up to date with your ML subfield, or deep-dive into a new research area?
📚 Our community has learning groups for all of the following:
-ML + Audio
-CV
-ML + Healthcare
-NLP
-ML Theory
-RL
Join us? share.hsforms.com/10OrjljwpQ…
David Marques retweeted
ToolFormer: LLMs that can teach themselves to use tools, like calculators, database queries, search engines,....
From @MetaAI - FAIR.
Toolformer: Language Models Can Teach Themselves to Use Tools
introduce Toolformer, a model trained to decide which APIs to call, when to call them, what arguments to pass, and how to best incorporate the results into future token prediction
abs: arxiv.org/abs/2302.04761
David Marques retweeted
Looking back at the past 12 months, I can't wait for the next 12 @CohereAI! Last year we launched new models, new endpoints, our Discord co:mmunity, C4AI @forai_ml, a non-profit research lab, opened offices in London and San Francisco, and much more 🫡
txt.cohere.ai/reflecting-on-…
David Marques retweeted
There are very few places in the world that currently serve large language models efficiently at scale.
Learn from someone at the frontier of LLM system design. @commit_xact is a researcher + colleague I greatly admire, and I am very much looking to this talk.
See you there!🎉
Our first technical talk of the year features Sudip Roy, Engineering Manager of @CohereAI s Inference Team. 🎉
Join us on January 25th to learn more from Sudip about the challenges and opportunities of training and serving large language models!
events.zoom.us/ev/AtYKtizVBX…
David Marques retweeted
2 SOTA vision+language models have arrived @huggingface! 🔥
💥 GIT by @Microsoft, a Transformer decoder conditioned on CLIP patches besides text. The first model that can do video captioning and QA besides image captioning and QA in Transformers!
huggingface.co/docs/transfor…
(1/3)
David Marques retweeted
We're celebrating the new year by announcing our first speaker of 2023: @colinraffel 🎉
Join us for a lively chat about Colin's research journey AND check off your new year's resolution to learn more from inspiring people. ✨
Register here! events.zoom.us/e/view/tLdVdF…
David Marques retweeted
Announcing OPT-IML: a new language model from Meta AI with 175B parameters, fine-tuned on 2,000 language tasks — openly available soon under a noncommercial license for research use cases.
Research paper & more details on GitHub ⬇️
