Large Language Models: A Deep Dive

Foreword
Reviews
Preface
	Why This Book
	Who This Book Is For
	What This Book Covers
	How to Navigate This Book
Acknowledgments
Declarations
Notation
Contents
Selected Acronyms
Chapter 1 Large Language Models: An Introduction
	1.1 Introduction
	1.2 Natural Language
	1.3 NLP and Language Models Evolution
		1.3.1 Syntactic and Grammar-based methods: 1960s-1980s
		1.3.2 Expert Systems and Statistical Models: 1980s-2000s
		1.3.3 Neural Models and Dense Representations: 2000s-2010s
		1.3.4 The Deep Learning Revolution: 2010s-2020s
	1.4 The Era of Large Language Models
		1.4.1 A Brief History of LLM Evolution
		1.4.2 LLM Scale
		1.4.3 Emergent Abilities in LLMs
	1.5 Large Language Models in Practice
		1.5.1 Large Language Model Development
		1.5.2 Large Language Model Adaptation
		1.5.3 Large Language Model Utilization
	References
Chapter 2 Language Models Pre-training
	2.1 Encoder-Decoder Architecture
		2.1.1 Encoder
		2.1.2 Decoder
		2.1.3 Training and Optimization
		2.1.3 Issues with Encoder-Decoder Architectures
	2.2 Attention Mechanism
		2.2.1 Self-AttentionSelf-Attention
	2.3 Transformers
		2.3.1 EncoderEncoder
		2.3.2 EncoderDecoder
		2.3.3 EncoderTokenization and Representation
		2.3.4 EncoderPositional Encodings
		2.3.5 EncoderMulti-Head Attention
		2.3.6 EncoderPosition-Wise Feed-Forward Neural Networks
		2.3.7 EncoderLayer Normalization
		2.3.8 EncoderMasked Multi-Head Attention
		2.3.9 EncoderEncoder-Decoder Attention
		2.3.10 EncoderTransformer Variants
	2.4 Data
		2.4.1 Language Model Pre-Training Datasets
		2.4.2 Data Pre-Processing
		2.4.3 Effects of Data on LLMs
		2.4.4 Task-Specific Datasets
	2.5 Pre-trained LLM Design Choices
		2.5.1 Pre-Training Methods
		2.5.2 Pre-training Tasks
		2.5.3 Architectures
		2.5.4 LLM Pre-training Tips and Strategies
	2.6 Commonly Used Pre-trained LLMs
		2.6.1 BERT (Encoder)
		2.6.2 T5 (Encoder-Decoder)
		2.6.3 GPT (Decoder)
		2.6.4 Mixtral 8x7B (Mixture of Experts)
	2.7 Tutorial: Understanding LLMs and Pre-training
		2.7.1 OverviewOverview
		2.7.2 Experimental Design
		2.7.3 Results and Analysis
		2.7.4 Conclusion
	References
Chapter 3 Prompt-based Learning
	3.1 Introduction
		3.1.1 Fully Supervised Learning
		3.1.2 Pre-train and Fine-tune Learning
		3.1.3 Prompt-based Learning
	3.2 Basics of Prompt-based Learning
		3.2.1 Prompt-based Learning: Formal Description
		3.2.2 Prompt-based Learning Process
		3.2.3 Prompt-based Knowledge Extraction
		3.2.4 Prompt-based Learning Across NLP Tasks
	3.3 Prompt Engineering
		3.3.1 Prompt Shape
		3.3.2 Manual Template Design
		3.3.3 Automated Template Design: Discrete Search
		3.3.4 Automated Template Design: Continuous Search
		3.3.5 Prompt-based Fine-tuning
	3.4 Answer engineering
		3.4.1 Answer Shape
		3.4.2 Defining the Answer Space
		3.4.3 Manual Answer Mapping
		3.4.4 Automated Answer Mapping: Discrete Search
		3.4.5 Automated Answer Mapping: Continuous Search
	3.5 Multi-Prompt Inference
		3.5.1 Ensembling
		3.5.2 In-context Learning
		3.5.3 Prompt Decomposition
	3.6 First Tutorial: Prompt vs. Pre-train and Fine-tune Methods in Text Classification and NER
		3.6.1 Overview
		3.6.2 Experimental Design
		3.6.3 Results and Analysis
		3.6.4 Conclusion
	3.7 Second Tutorial: Approaches to Prompt Engineering
		3.7.1 Overview
		3.7.2 Experimental Design
		3.7.3 Results and Analysis
		3.7.4 Conclusion
	References
Chapter 4 LLM Adaptation and Utilization
	4.1 Introduction
	4.2 Instruction Tuning
		4.2.1 Instruction Tuning Procedure
		4.2.2 Instruction Tuning Data
		4.2.3 Instruction Tuning for Domain Adaptation
	4.3 Parameter-Efficient Fine-Tuning
		4.3.1 Adapters
		4.3.2 Reparameterization
	4.4 Compute-Efficient Fine-Tuning
		4.4.1 LLM Quantization
	4.5 End-User Prompting
		4.5.1 Zero-Shot Prompting
		4.5.2 Few-Shot Prompting
		4.5.3 Prompt Chaining
		4.5.4 Chain-of-Thought
		4.5.5 Self-Consistency
		4.5.6 Tree-of-Thoughts
	4.6 Tutorial: Fine-Tuning LLMs in a Resource-Constrained Setting
		4.6.1 Overview
		4.6.2 Experimental Design
		4.6.3 Results and Analysis
		4.6.4 Conclusion
	References
Chapter 5 Tuning for LLM Alignment
	5.1 Alignment Tuning
		5.1.1 Helpfulness
		5.1.2 Honesty
		5.1.3 Harmlessness
	5.2  Foundation: The Reinforcement Learning Framework
	5.3 Mapping the RL Framework to LLMs with Human Feedback
	5.4 Evolution of RLHF
		5.4.1 Safety, Quality, and Groundedness in LLMs
		5.4.2 Deep Reinforcement Learning from Human Preferences
		5.4.3 Learning Summarization from Human Feedback
		5.4.4 Aligning LLMs to be Helpful, Honest, and Harmless with Human Feedback
	5.5 Overcoming RLHF Challenges
		5.5.1 Instilling Harmlessness with AI Feedback
		5.5.2 Direct Preference Optimization
	5.6 Tutorial: Making a Language Model More Helpful with RLHF
		5.6.1 Overview
		5.6.2 Experimental Design
		5.6.3 Results and Analysis
		5.6.4 Conclusion
	References
Chapter 6 LLM Challenges and Solutions
	6.1 Hallucination
		6.1.1 Causes
		6.1.2 Evaluation Metrics
		6.1.3 Benchmarks
		6.1.4 Mitigation Strategies
	6.2 Bias and Fairness
		6.2.1 Representational Harms
		6.2.2 Allocational Harms
		6.2.3 Causes
		6.2.4 Evaluation Metrics
		6.2.5 Benchmarks
		6.2.6 Mitigation Strategies
	6.3 Toxicity
		6.3.1 Causes
		6.3.2 Evaluation Metrics
		6.3.3 Benchmarks
		6.3.4 Mitigation Strategies
	6.4 Privacy
		6.4.1 Causes
		6.4.2 Evaluation Metrics
		6.4.3 Benchmarks
		6.4.4 Mitigation Strategies
	6.5 Tutorial: Measuring and Mitigating Bias in LLMs
		6.5.1 Overview
		6.5.2 Experimental Design
		6.5.3 Results and Analysis
		6.5.4 Conclusion
	References
Chapter 7 Retrieval-Augmented Generation
	7.1 Introduction
	7.2 Basics of RAG
	7.3 Optimizing RAG
	7.4 Enhancing RAG
		7.4.1 Data Sources and Embeddings
		7.4.2 Querying
		7.4.3 Retrieval and Generation
		7.4.4 Summary
	7.5 Evaluating RAG Applications
		7.5.1 RAG Quality Metrics
		7.5.2 Evaluation of RAG System Capabilities
		7.5.3 Summarizing RAG Evaluation
	7.6 Tutorial: Building Your Own Retrieval-Augmented Generation System
		7.6.1 Overview
		7.6.2 Experimental Design
		7.6.3 Results and Analysis
		7.6.4 Conclusion
	References
Chapter 8 LLMs in Production
	8.1 Introduction
	8.2 LLM Applications
		8.2.1 Conversational AI, chatbots and AI assistants
		8.2.2 Content Creation
		8.2.3 Search, Information Retrieval, and Recommendation Systems
		8.2.4 Coding
		8.2.5 Categories of LLMs
	8.3 LLM Evaluation Metrics
		8.3.1 Perplexity
		8.3.2 BLEU
		8.3.3 ROUGE
		8.3.4 BERTScore
		8.3.5 MoverScore
		8.3.6 G-Eval
		8.3.7 Pass@k
	8.4 LLM Benchmark Datasets
	8.5 LLM Selection
		8.5.1 Open Source vs. Closed Source
		8.5.2 Analytic Quality
		8.5.3 Inference Latency
		8.5.4 Costs
		8.5.5 Adaptability and Maintenance
		8.5.6 Data Security and Licensing
	8.6 Tooling for Application Development
		8.6.1 LLM Application Frameworks
		8.6.2 LLM Customization
		8.6.3 Vector Databases
		8.6.4 Prompt Engineering
		8.6.5 Evaluation and Testing
	8.7 Inference
		8.7.1 Model Hosting
		8.7.2 Optimizing Performance
		8.7.3 Optimizing Cost
	8.8 LLMOps
		8.8.1 LLMOps Tools and Methods
		8.8.2 Accelerating the Iteration Cycle
		8.8.3 Risk Management
	8.9 Tutorial: Preparing Experimental Models for Production Deployment
		8.9.1 Overview
		8.9.2 Experimental Design
		8.9.3 Results and Analysis
		8.9.4 Conclusion
	References
Chapter 9 Multimodal LLMs
	9.1 Introduction
	9.2 Brief History
	9.3 Multimodal LLM Framework
		Modality Encoder
		Input Projector
		Pre-training: Core LLMs, Datasets and Task-Specific Objectives
		MMLLM Tuning and Enhancements
		Multimodal RLHF
		Output Projector
		Modality Generator
	9.4 Benchmarks
	9.5 State-of-the-Art MMLLMs
		Flamingo (Image-Video-Text)
		Video-LLaMA (Image-Video-Audio-Text)
		NExT-GPT (Any-to-Any)
	9.6 Tutorial: Fine-Tuning Multimodal Image-to-Text LLMs
		Overview
		Experimental Design
		Results and Analysis
		Conclusion
	References
Chapter 10 LLMs: Evolution and New Frontiers
	10.1 Introduction
	10.2 LLM Evolution
		10.2.1 Synthetic Data
		10.2.2 Larger Context Windows
		10.2.3 Training Speedups
		10.2.4 Multi-Token Generation
		10.2.5 Knowledge Distillation
		10.2.6 Post-Attention Architectures
	10.3 LLM Trends
		10.3.1 Small Language Models
		10.3.2 Democratization
		10.3.3 Domain-Specific Language Models
	10.4 New Frontiers
		10.4.1 LLM Agents
		10.4.2 LLM-Enhanced Search
	10.5 Closing Remarks
	References
Appendix A Deep Learning Basics
	A.1 Basic Structure of Neural Networks
	A.2 Perceptron
	A.3 Multilayer Perceptron
		A.3.1 Structure and Function of MLPs
		A.3.2 Training MLPs
	A.4 Deep Learning
		A.4.1 Key Components of Deep Neural Networks
		A.4.2 Activation Functions
		A.4.3 Loss Functions
		A.4.4 Optimization Techniques
		A.4.5 Model Training
		A.4.6 Regularization Techniques
Appendix B Reinforcement Learning Basics
	B.1 Markov Decision Process
		B.1.1 Tasks
		B.1.2 Rewards and Return
		B.1.3 Policies and Value Functions
		B.1.4 Optimality
	B.2 Exploration/Exploitation Trade-off
	B.3 Reinforcement Learning Algorithms
		B.3.1 Q-Learning
		B.3.2 Deep Q-Network (DQN)
		B.3.3 Policy Gradient-based Methods
Index