University of Calgary Machine Learning Lab
Calgary, AB
Joined June 2021
- Tweets16
- Following45
- Followers55
- Likes42
Calgary ML Lab retweeted
Olá, ICLR 2026 in Rio de Janeiro! 🇧🇷
Excited to present our paper, “REAP the Experts: Why Pruning Prevails for One-Shot MoE compression” this Friday from 10:30-13:00 at Pavilion 3 (P3-#701).
REAP is a one-shot MoE pruning algorithm that scales to 1T params. Details 🧵👇
Calgary ML Lab retweeted
No tricks, just treats from our ML team. 👻
(1/5) Compressing a 1T parameter MoE model by 50% with virtually no loss in quality is now possible.
Our new paper with @mikelasby @lzrvch @NishSinnadurai Sean Lie and @yanii introduces an expert pruning criterion for one-shot compression that aims to solve the memory bottleneck of these massive models, making them far more accessible for research and deployment.
Calgary ML Lab retweeted
@UCalgaryML will be at #ICML2025 in Vancouver🇨🇦 next week: our lab has 6 different works being presented by 5 students across both the main conference & 6 workshops!
We are also recruiting fully-funded ML PhD/MSc students🧑🎓If attending, reach out to me on Whova app to meet 📅 – at Vancouver, British Columbia
ALT The Calgary Machine Learning Lab, will have 6 different works being presented by 5 students across the International Conference on Machine Learning (ICML) 2025 workshops (HiLD, ES-FoMo III, DIG-BUGS, WiML, MusiML) and main conference! Please see our full conference schedule on the lab website for more details.
Calgary ML Lab retweeted
1/10 🧵
🔍Can weight symmetry provide insights into sparse training and the Lottery Ticket Hypothesis?
🧐We dive deep into this question in our latest paper, "Sparse Training from Random Initialization: Aligning Lottery Ticket Masks using Weight Symmetry", accepted at #ICML2025
Calgary ML Lab retweeted
I'm proud that the @UCalgaryML lab will have 6 different works being presented by 6 students across #NeurIPS2024, in workshops (@unireps, @WiMLworkshop, MusiML) and the main conference! 🎉
Hope to see you at our posters/talks 🤓, full schedule at calgaryml.com
🧵(1/4) – at Calgary, Alberta
Calgary ML Lab retweeted
For more details, check out our full paper … 👇🏼
🔗: neurips.cc/virtual/2024/1026…
Joint work w/ @adnan_ahmad1306, @EkanshSh, and @yanii from @VectorInst and @UCalgary.
Visit our poster session on Dec 14 at #NeurIPS2024! 👋🏻
Supported by: @UCalgaryML
(6/6)
Calgary ML Lab retweeted
Really excited to be presenting this work from my lab at the @NeurIPSConf 2024 @unireps workshop! 🎉
If you are at NeurIPS and interested in the Lottery Ticket Hypothesis/sparse training, don’t miss our poster on Sat Dec 14th, 3:45-5:00 PM 🖼️🤓
@UCalgaryML @VectorInst
✨Our new @unireps paper tries to answer why the Lottery Ticket Hypothesis (LTH) fails to work for different random inits through the lens of weight-space symmetry.
We improve the transferability of LTH masks to new random inits leveraging weight symmetries.
🧵(1/6)
Calgary ML Lab retweeted
Congratulations @adnan_ahmad1306 on receiving the @RBCBorealis Fellowship!
Adnan is a PhD student in the @UCalgaryML lab at @SchulichENGG/@UCalgary, and @VectorInst, co-supervised by @rahulgk.
🌟🎓 Meet @RBCBorealis's 2023-2024 Fellows! We're excited to introduce 10 exceptional #PhD students selected for our #Fellowships program, representing the brightest talent from Canada's top universities.
Learn more about the Fellows and their #research: rbcborealis.com/news/celebra…
Calgary ML Lab retweeted
Happy to share our work on Knowledge Distillation temperature🌡️ and fairness⚖️. Must-read for AI practitioners given the prevalence of distilled models! KD is crucial in deploying AI, e.g. limited hardware, real-time applications
🙏@sarahookr @mikelasby for feedback! @UCalgaryML
Knowledge #distillation is a widely used model compression method. We explore the nuanced impact of temperature on distilled models' #fairness ⚖️. Our findings show distilled students are less fair than their teachers at typical temps, but can be more fair in some instances…🧵👇
Calgary ML Lab retweeted
"Dynamic Sparse Training with Structured Sparsity" (openreview.net/forum?id=kOBk…) was accepted at ICLR 2024! DST methods learn state-of-the-art sparse masks, but accelerating DNNs with unstructured masks is difficult. SRigL learns structured masks, improving real-world CPU/GPU timings
ALT Real-world GPU batched inference timings for a linear layer extracted from ViT-B-16 trained with SRigL with a batch size of 256.
Calgary ML Lab retweeted
I’m excited to share that I’ve received an Amazon Research Award for my proposal "Addressing Catastrophic Forgetting with Dynamic Sparse Training" at @UCalgaryML/@UCalgary!
Learn more about the program on the @AmazonScience website: amzn.to/ara-fall-winter-2021 #AmazonResearchAwards
Calgary ML Lab retweeted
I was happy to recently be informed my 2022 DG proposal was also awarded a @NSERC_CRSNG and @NationalDefence DND Discovery Grant Supplement — one of only 20 awarded nationally — to further support and accelerate research at @UCalgaryML!
#DiscoveryGrants #NSERCsupport #CdnSci
Calgary ML Lab retweeted
Key takeaways:
- Initialization is important for older NNs: use sparsity aware init.
- For modern NNs, init doesn't matter: improving gradient flow during training helps (i.e. RigL)
- Lottery tickets work since they re-locate the pruning solutions.
Interested in understanding sparse training better? Join our oral presentation at #AAAI2022
"Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win 🎟️" tomorrow (Feb 24) @ 10:30-11:45am PST! Poster at 8:45-10:30am.
Work with @yanii, @ynd, @cem_keskin_.
Calgary ML Lab retweeted
Work recently published by our close collaborators Drs. @marianapbento and @richardfrayneca!
Deep Learning in Large and Multi-Site Structural Brain MR Imaging Datasets frontiersin.org/article/10.3…
Calgary ML Lab retweeted
Join us for our #AAAI2022 *Oral Presentation* of our paper on Sparse DNN training: "Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win" tomorrow (Feb 24) @ 10:30-11:45am PST!
Poster at 8:45-10:30am/Feb 27 4:45-6:30pm. Work with @utkuevci, @ynd, @cem_keskin_.
ALT AAAI Poster: Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win