Tarik Hammadou

I shared a new tutorial + experiments on finetuning LLMs for classification efficiently. In this video, I explain how to convert a decoder-style LLM into a classifier. Many business problems are text classification problems, and if classification is all we need for a given task, using "smaller" and cheaper LLMs makes a lot of sense! (But, of course, also always run a simple logistic regression or naive Bayes baseline to determine if you even need a small LLM.) 🧪 In addition, I also ran a series of 19 experiments to answer some "what if" questions around finetuning pretrained LLMs for classification. Here, I kept things simple and small (e.g., GPT-2 on a toy binary classification task): Here's a snapshot summary of some of the interesting ones: 1) As would be expected, training on the last token yields much better performance than the first 2) Training the last transformer block is way better than just the last layer 3) LoRA performs on par or better than full finetuning—while being faster and more memory-efficient 4) Padding to full context length hurts performance 5) No padding or smart position selection leads to consistently higher accuracy 6) Surprisingly, training from random weights isn't much worse than using pretrained 7) Averaging embeddings over all tokens can improve performance slightly with little cost The full video is available here: youtube.com/watch?v=5PFXJYme… PS: If you are wondering why GPT instead of BERT? Well, you can of course also use BERT. Based on experiments on the 50k Movie Review dataset It's interesting though that this 3x smaller LLM performs on par (actually slightly better) than BERT. (ModernBERT then again is 2% better.)