Dan Busbridge (@danbusbridge) header
Dan Busbridge (@danbusbridge) profile picture

Machine Learning Research @ Apple (opinions are my own)

London, United Kingdom
Joined November 2014
Photo by Dan Busbridge (@danbusbridge)Photo by Dan Busbridge (@danbusbridge)Photo by Dan Busbridge (@danbusbridge)Photo by Dan Busbridge (@danbusbridge)Photo by Dan Busbridge (@danbusbridge)Photo by Dan Busbridge (@danbusbridge)Photo by Dan Busbridge (@danbusbridge)Photo by Dan Busbridge (@danbusbridge)Photo by Dan Busbridge (@danbusbridge)Photo by Dan Busbridge (@danbusbridge)
Stop by poster #596 at 10A-1230P tomorrow (Fri 25 April) at #ICLR2025 to hear more about Sigmoid Attention! We just pushed 8 trajectory checkpoints each for two 7B LLMs for Sigmoid Attention and a 1:1 Softmax Attention (trained with a deterministic dataloader for 1T tokens): - Sigmoid: gs://axlearn-public/experiments/gala-7B-sigmoid-hybridnorm-alibi-sprp-2024-12-03-1002/checkpoints/ - Softmax: gs://axlearn-public/experiments/gala-7B-hybridnorm-alibi-sprp-2024-12-02-1445/checkpoints/ Inference code at github.com/apple/ml-sigmoid-…
Small update on SigmoidAttn (arXiV incoming). - 1B and 7B LLM results added and stabilized. - Hybrid Norm [on embed dim, not seq dim], `x + norm(sigmoid(QK^T / sqrt(d_{qk}))V)`, stablizes longer sequence (n=4096) and larger models (7B). H-norm used with Grok-1 for example.
1
14
1
45
9,655
Parameterized Transforms 🚀 Here is a new tool that provides a modular and extendable implementation of torchvision-based image augmentations that provides access to their parameterization. [1/5]
1
9
2
15
2,144
Dan Busbridge retweeted
🚨 One question that has always intrigued me is the role of different ways to increase a model's capacity: parameters, parallelizable compute, or sequential compute? We explored this through the lens of MoEs:
4
65
8
282
48,574
Dan Busbridge retweeted
We attempted to make Normalizing Flows work really well, and we are happy to report our findings in paper arxiv.org/pdf/2412.06329, and code github.com/apple/ml-tarflow. [1/n]
4
47
10
231
53,891
Dan Busbridge retweeted
Thrilled to share the latest work from our team at @Apple where we achieve interpretable and fine-grained control of LLMs and Diffusion models via Activation Transport 🔥 📄 arxiv.org/abs/2410.23054 🛠️ github.com/apple/ml-act 1/9 🧵
4
29
4
126
11,411
📢Internships at Apple ML Research🍏 We’re looking for a PhD research intern with interests in uncertainty quantification, LLMs, probabilistic ML and/or decision making under uncertainty! See thread for more details 👇 [1/3]
4
67
4
486
71,928
Dan Busbridge retweeted
Check out ‘Dataset Decomposition’, a new paper from the Machine Learning Research team @Apple that improves the training speed of LLMs by up to 3x: arxiv.org/abs/2405.13226 Hadi Pouransari et al. #EfficientML #AI #ML #Apple
1
26
52
8,101
Have a research idea at the intersection of generative models and RL, IL or planning? Consider submitting a short (4 pages) or long (9 pages) paper to our GenAI+Decision Making workshop at ICLR 2024: sites.google.com/view/GenAI4… More information coming early January!
22
1
81
10,757
Interested in Bayes + Self-Supervised Learning? Come to the SSL workshop at #NeurIPS2023 Room 217 We show that coupling Bootstrap Your Own Latent (BYOL) with Bayes-by-Backprop (BBB) learns a distrbutionally aligned predictive variance with supervised learning without any labels.
1
18
81
8,467
Dan Busbridge retweeted
I'll present Manifold Diffusion Fields as an oral at the Diffusion Models workshop @NeurIPSConf, today 2:30 pm New Orleans time!! work with @YuyangW95 @jsusskin and @itsbautistam Don't forget to join and let's know your questions and feedback! openreview.net/pdf?id=0AMMdK…
2
15
1
149
75,609
Dan Busbridge retweeted
Stocked to present « Bilevel Optimization to Learn Training Distributions under Domain Shift » at the Neurips Distribution shift workshop! Joint work w. my amazing colleagues @GrangierDavid and @awnihannun ! Paper : arxiv.org/abs/2311.11973
11
45
5,174
MDF is an oral at the Diffusion Models workshop tomorrow at @NeurIPSConf! Catch @Ahmed_AI035’s talk (via zoom because well…visas) and also @YuyangW95 and I will be around in the poster session! Come say hi and lets chat about practical diffusion models on manifolds!
Introducing Manifold Diffusion Fields (MDF), our new work on learning generative models over fields defined on curved geometries. This is joint work with our intern @Ahmed_AI035 (who hasn’t even started his PhD yet!) and @jsusskin at @Apple MLR arxiv.org/abs/2305.15586 🧵
5
2
19
5,303
MLR at Apple just opened an office in Copenhagen, Denmark 🎉🎉💃🇩🇰 And we are hiring! jobs.apple.com/en-us/details… Please reach out if you're interested! This office opening is pretty neat just on its own, if I do say so myself. 😊 1/
12
78
4
368
82,403
I wanted to advertise that we have opportunities in my team at Apple MLR in scalable/distributed ML for multimodal foundational models, embodied AI, planning and reasoning. More info below but feel free to send me a note. tinyurl.com/ms77zkam
5
2
19
4,113
Dan Busbridge retweeted
Just in time for the holidays, we are releasing some new software today from Apple machine learning research. MLX is an efficient machine learning framework specifically designed for Apple silicon (i.e. your laptop!) Code: github.com/ml-explore/mlx Docs: ml-explore.github.io/mlx/bui…
95
672
135
4,377
962,052
Dan Busbridge retweeted
Apple ML Research in Cambridge, UK is looking for a PhD intern 🎓 Topics: Self supervised representation learning, generative modeling, diffusion models. Job Req: jobs.apple.com/en-gb/details…
3
33
4
114
24,981
Our Apple ML Research team in Barcelona is looking for a PhD intern! 🎓 Curiosity-driven research 🧠 with the goal to publish 📝 Topics: Confidence/uncertainty quantification and reliability of LLMs 🤖 Apple here: jobs.apple.com/en-gb/details…
4
47
5
271
47,174
Introducing Manifold Diffusion Fields (MDF), our new work on learning generative models over fields defined on curved geometries. This is joint work with our intern @Ahmed_AI035 (who hasn’t even started his PhD yet!) and @jsusskin at @Apple MLR arxiv.org/abs/2305.15586 🧵
Manifold Diffusion Fields present Manifold Diffusion Fields (MDF), an approach to learn generative models of continuous functions defined over Riemannian manifolds. Leveraging insights from spectral geometry analysis, we define an intrinsic coordinate system on the manifold via the eigen-functions of the Laplace-Beltrami Operator. MDF represents functions using an explicit parametrization formed by a set of multiple input-output pairs. Our approach allows to sample continuous functions on manifolds and is invariant with respect to rigid and isometric transformations of the manifold. Empirical results on several datasets and manifolds show that MDF can capture distributions of such functions with better diversity and fidelity than previous approaches paper page: huggingface.co/papers/2305.1…
1
28
2
111
37,990
Planning to attend @CVPR? Check out the workshop sessions on Sunday, I will be talking about generative modeling for fields and manifolds at the @_LXAI workshop!
👨🎙️ Miguel Bautista, Research Scientist at Apple MLR. Ph.D in Machine Learning from the University of Barcelona. 🚀🪩 He will be a Keynote Speaker at @_LXAI @CVPR! Introducing his current research focus: 🖌️ "Generative Modelling: from images to functions and manifolds."
1
9
34
10,432
We are hiring an ML Research Engineer to work on the forefront of AI in the health space. If you are passionate about building robust DL models, want to work with outstanding ML researchers + engineers, want to have real impact with your work: apply! jobs.apple.com/en-us/details…
6
52
6
241