Technical Fellow, Graphcore. Love beautiful code, and beautiful hardware to run it on.
Cambridge
Joined July 2009
- Tweets1.5K
- Following848
- Followers3.9K
- Likes1.6K
Andrew Fitzgibbon retweeted
A tiny implementation detail in low-precision arithmetic could be biasing your AI training 😲
This interactive deep dive from Graphcore Research's @Awfidius uncovers a subtle failure mode in stochastic rounding that only appears when randomness is limited, and how it can be addressed with one simple fix 🎲
Check it out in the link below! 👇
Andrew Fitzgibbon retweeted
🚨 Graphcore is hiring AI Research Interns! 🚨
Join us to work at the intersection of hardware and AI and help shape the future of AI systems. Whether you're excited about efficient inference, large-scale training, or advancing frontier-model capabilities, we’ve got cutting-edge projects for you to dive into.
Interested? Apply below 👇
Andrew Fitzgibbon retweeted
Our Papers of the Month for September is now live! We cover:
- LLM self-correction via RL
- Trillion-token FP8 training
- SOAP (Shampoo + Adam)
- Generative models for crystals
All framed in terms of "proper conditioning" (🧵)
graphcore-research.github.io…
Andrew Fitzgibbon retweeted
Graphcore Research internships are now open 🎉
We're looking for PhD students for next summer
We're interested in algorithms & tools for hardware-efficient ML, in areas like LLM training/inference, GNNs, knowledge graphs and frameworks
Spread the word! graphcore.ai/jobs
Andrew Fitzgibbon retweeted
Introducing `tandv` - a library for tracking and visualising the internal stats of your model. We hope this will help with low-precision, debugging and more.
(link in 🧵)
Andrew Fitzgibbon retweeted
Sure matplotlib is cool, but what if I want to load my loss curves into the 2006 hit Flash game LineRider?
Andrew Fitzgibbon retweeted
Our u-µP paper hit arXiv this morning! I'm so proud of this one — and grateful for a wonderful team who put so much into it 🥰
We add lots of good things to µP. Better sweeping, transfer, simple FP8. Already @cloneofsimo has a great thread on it, which I highly recommend
babe wake up, new muP paper dropped
arxiv.org/abs/2407.17465
And holy smokes does this look promising!
Andrew Fitzgibbon retweeted
Excited to present our work at @icmlconf WANT workshop: ⚖️ Scalify: scale propagation for efficient low-precision LLM training 🎉
github.com/graphcore-researc…
Andrew Fitzgibbon retweeted
babe wake up, new muP paper dropped
arxiv.org/abs/2407.17465
And holy smokes does this look promising!
Andrew Fitzgibbon retweeted
Our team's summaries & analysis of our favourite papers from the last month.
We give our take on: Mamba-2, sparse-µP, contextual position encoding & matmul-free models 🧵
graphcore-research.github.io…
Andrew Fitzgibbon retweeted
Our latest edition of *Papers of the Month* is now available 📚
These are summaries of our team's favourite papers from March, including a new low-rank training procedure GaLore, and the supposed "Era of 1-bit LLMs" (really 1.58 bits)
Mini-version in 🧵
graphcore-research.github.io…
Andrew Fitzgibbon retweeted
So pleased to share that our OGB-LSC winning model GPS++ has just been published in TMLR.
GPS++: Reviving the Art of Message Passing for Molecular Property Prediction
Dominic Masters, Josef Dean, Kerstin Klaeser et al..
Action editor: Ole Winther.
openreview.net/forum?id=moVE…
#molecular #pcqm4mv2 #graph
Andrew Fitzgibbon retweeted
I'll start with "Julia meets the Intelligence Processing Unit" (pretalx.com/juliacon2023/tal…), about running #JuliaLang on the @graphcoreai IPU, spectacular example of how @llvmorg enables adopting new cool hardware in Julia, and how Julia opens up new doors in scientific computing
Andrew Fitzgibbon retweeted
Neat work by colleagues showing a BF16 model ported to FP16!
This kind of knowledge is also really useful heading towards FP8. Being able to switch easily between number formats is useful stuff🤌
How Graphcore made T5 run on Float16 hardware. Now you can have your flan and eat it. #LLM #T5 #Float16
hubs.la/Q01RyX1Z0
Andrew Fitzgibbon retweeted
Very proud, and still a bit shocked, to share that I've been elected Fellow of the Royal Society. Thank you to my students and collaborators from Imperial College, Oxford, AIST, Dyson, Slamcore and beyond, and of course to my supportive family and friends!
royalsociety.org/people/andr…
We are very happy to announce the eighty exceptional scientists elected as Fellows of the Royal Society this year, selected for their outstanding contributions to the advancement of science. Meet the new Fellows and find out more about their research: royalsociety.org/news/2023/0…
Andrew Fitzgibbon retweeted
Generative AI is moving from a cool plaything to a serious commercial tool. We outline the three tests that users should apply when considering how they might deploy it professionally.
hubs.la/Q01Rk75z0
Andrew Fitzgibbon retweeted
[A 🧵 for floating-point enthusiasts]
It’s fairly easy to see that a FP format with 0 exponent bits is just a signed INT. But what if I told you that a FP format with 1 exponent bit is also just a signed INT? 🤔😵💫
Let’s dive in…
Andrew Fitzgibbon retweeted
Dolly 2.0 brings some of the instruction-following capability seen in ChatGPT to a truly open-source language model, useable for commerical applications. Try it now for free on Paperspace.
hubs.la/Q01MPg8n0
Andrew Fitzgibbon retweeted
I was privileged to attend Simon Peyton Jone's Order of the British Empire (OBE) investiture. The medal was presented by HM Lord-Lieutenant Mrs Julie Spence OBE QPM on behalf of The King for his services to computing and his leadership of the UK computing at schools initiative.
Andrew Fitzgibbon retweeted
If you're in London on April 13th, join us for an in-person meet-up and round-table to discuss the latest in graph learning in industry!
If you're in London, join us for an in-person evening of talks and panels from industry leaders in Graph Neural Networks, such as Jure Leskovec, to discuss the future of enterprise deep learning.
Register here:
info.kumo.ai/amplify-your-gn…