Roxana Noelia

An absolutely brilliant textbook "Foundations of Large Language Models" by Tong Xiao and Jingbo Zhu (2025) is now in @ChapterPal's collection. This book is a comprehensive, technically grounded treatment of modern generative language models for artificial intelligence researchers, engineers, and advanced students in natural language processing. Assuming a foundational background in deep learning, probability, and standard neural sequence modeling, the book explains the entire lifecycle of foundation models—from raw data pre-training and architectural design to alignment, inference acceleration, and system-level scaling. It's a great in-depth read specifically on Large LMs after my The Hundred-Page Language Models Book that covers neural network and deep learning basics, count-based LMs, RNNs, and transformers. Read the book with an AI tutor: chapterpal.com/ebook/14fa054… (All books on ChapterPal are free to read with a free account.) The book has 76 beautiful illustrations across 277 pages. Table of contents: Preface Notation 1. Pre-training - 1.1. Pre-training NLP Models * 1.1.1 Unsupervised, Supervised and Self-supervised Pre-training * 1.1.2 Adapting Pre-trained Models - 1.2. Self-supervised Pre-training Tasks * 1.2.1 Decoder-only Pre-training * 1.2.2 Encoder-only Pre-training * 1.2.3 Encoder-Decoder Pre-training * 1.2.4 Comparison of Pre-training Tasks - 1.3. Example: BERT * 1.3.1 The Standard Model * 1.3.2 More Training and Larger Models * 1.3.3 More Efficient Models * 1.3.4 Multi-lingual Models - 1.4. Applying BERT Models - 1.5. Summary - References 2. Generative Models - 2.1. A Brief Introduction to LLMs * 2.1.1 Decoder-only Transformers * 2.1.2 Training LLMs * 2.1.3 Fine-tuning LLMs * 2.1.4 Aligning LLMs with the World * 2.1.5 Prompting LLMs - 2.2. Training at Scale * 2.2.1 Data Preparation * 2.2.2 Model Modifications * 2.2.3 Distributed Training * 2.2.4 Scaling Laws - 2.3. Long Sequence Modeling * 2.3.1 Optimization from HPC Perspectives * 2.3.2 Efficient Architectures * 2.3.3 Cache and Memory * 2.3.4 Sharing across Heads and Layers * 2.3.5 Position Extrapolation and Interpolation * 2.3.6 Remarks - 2.4. Summary - References 3. Prompting - 3.1. General Prompt Design * 3.1.1 Basics * 3.1.2 In-context Learning * 3.1.3 Prompt Engineering Strategies * 3.1.4 More Examples - 3.2. Advanced Prompting Methods * 3.2.1 Chain of Thought * 3.2.2 Problem Decomposition * 3.2.3 Self-refinement * 3.2.4 Ensembling * 3.2.5 RAG and Tool Use - 3.3. Learning to Prompt * 3.3.1 Prompt Optimization * 3.3.2 Soft Prompts * 3.3.3 Prompt Length Reduction - 3.4. Summary - References 4. Alignment - 4.1. An Overview of LLM Alignment - 4.2. Instruction Alignment * 4.2.1 Supervised Fine-tuning * 4.2.2 Fine-tuning Data Acquisition * 4.2.3 Fine-tuning with Less Data * 4.2.4 Instruction Generalization * 4.2.5 Using Weak Models to Improve Strong Models - 4.3. Human Preference Alignment: RLHF * 4.3.1 Basics of Reinforcement Learning * 4.3.2 Training Reward Models * 4.3.3 Training LLMs - 4.4. Improved Human Preference Alignment * 4.4.1 Better Reward Modeling * 4.4.2 Direct Preference Optimization * 4.4.3 Automatic Preference Data Generation * 4.4.4 Step-by-step Alignment * 4.4.5 Inference-time Alignment - 4.5. Summary - References 5. Inference - 5.1. Prefilling and Decoding * 5.1.1 Preliminaries * 5.1.2 A Two-phase Framework * 5.1.3 Decoding Algorithms * 5.1.4 Evaluation Metrics for LLM Inference - 5.2. Efficient Inference Techniques * 5.2.1 More Caching * 5.2.2 Batching * 5.2.3 Parallelization * 5.2.4 Remarks - 5.3. Inference-time Scaling * 5.3.1 Context Scaling * 5.3.2 Search Scaling * 5.3.3 Output Ensembling * 5.3.4 Generating and Verifying Thinking Paths - 5.4. Summary - Appendix - References