Mastering Large Language Models with Python

Cover Page
Title Page
Copyright Page
Dedication Page
About the Author
About the Technical Reviewers
Acknowledgements
Preface
Errata
Table of Contents
1. The Basics of Large Language Models and Their Applications
   Introduction
   Structure
   Introduction to Large Language Models
      Unfolding the Journey of Language Models
      Influence of Large Language Models
   Introducing Transformers and Their Importance
      Understanding Transformers
      Transformers in Large Language Models
      Attention Mechanisms
      Add and Norm (Residual Connection and Layer Normalization)
      Feed Forward
      Masked Multi-Head Attention
      Linear and Softmax Layers
      Transformers and Large Language Models
   Scaling Laws for Large Language Models
      KM Scaling Law
      Chinchilla Scaling Law
      Key Techniques for Large Language Models
      Scaling
      Training
      Ability Eliciting
      Alignment Tuning and Tools Manipulation
      Tools Manipulation
   Creating and Nurturing Large Language Models
      Publicly Available Model Checkpoints or APIs
      Publicly Available Corpora
         CommonCrawl
         Reddit Links
         Wikipedia
         Pile
      Collecting Data
      Data Source
      Formatting Existing Datasets
      Formatting Human Needs
      Publicly Accessible Libraries
      Configuring LLMs in Detail
      Normalization
      Activation Functions
      Position Embeddings
      Attention and Bias
      Optimizer for LLMs
      Stabilizing the Training
      Pre-Training
      Emergent Abilities of Large Language Models
      In-Context Learning
      Instruction Following
      Step-by-Step Reasoning
      Exploring the Inner Workings of LLMs
      Understanding Pre-training and ICL
      Performing ICL
   Chain-of-Thought Prompting in Large Language Models
      Confluence of ICL and CoT
      CoT Prompt Design
      Assessment Yardsticks for Large Language Models
      Massive Multitask Language Understanding (MMLU)
      BIG-bench
      HELM
      In-depth Analysis of the Capabilities of LLMs
      Generalist
      Specialist
   Conclusion
      References
2. Demystifying Open-Source Large Language Models
   Introduction
   Structure
   Overview
      Open-Source versus Proprietary Large Language Models
      Risks and Drawbacks of Open-Source LLMs
      Security Vulnerabilities in Open-Source LLMs
   Introducing the Models
      StableLM: Empowering Language Generation with Stability AI
      BERT: Advancing Language Representations with Bidirectional Encoder Representations from Transformers
      BLOOM: Empowering Open Science with the Largest Multilingual Language Model
      RedPajama: Advancing Open-Source Language Models
      Falcon-40B: Empowering Open-Source Language Models
      StarCoder: Empowering Developers with Code Generation
      Replit-Code: Empowering Developers with Intelligent Code Completion
      GPT-Neo: Empowering Open and Collaborative Research in Language Models
      Galactica: Revolutionizing Scientific Knowledge with Meta AI
      Segment Anything Model (SAM): Advancing Image Segmentation with Meta AI
      Dolly: Empowering Natural Language Processing with Databricks
   Conclusion
   References
3. Closed-Source Large Language Models
   Introduction
   Structure
   OpenAI
      GPT-4 Limited Beta
      GPT-3
      GPT-3.5
      DALL·E Beta
      Whisper Beta
      Embeddings
      Moderation
      Codex
      Accessing GPT Models via OpenAI API
      Experimenting with GPT Models via Playground
      Understanding Chat Completions API
      Function Calling with OpenAI
      Completions API
   DALL·E Image Generation Models
      Image Generation
      Image Editing
      Image Variation
      Content Moderation
      Language-specific tips
      Embedding Model - Understanding Embeddings
      Accessing Embeddings
      Embedding Models
      Limitations and Risks
      Whisper: OpenAI’s Speech-to-Text Model
      Transcriptions with Whisper
      Translations with Whisper
      Supported Languages
      Handling Longer Inputs
      Prompting
      Moderation Model: Ensuring Content Compliance
      Using the Moderation API
   Cohere
      Models
      Command
      Generation
      Representation
      Rerank (Beta)
      Summarize (Beta)
      Exploring Cohere Playground
      Selecting the Right Model Size
   Other AI-Powered Tools
   Conclusion
   References
4. LLM APIs for Various Large Language Model Tasks
   Introduction
   Structure
   A Tour of the Hugging Face Ecosystem
   Mastering the Inference API: Accessing Models in Hugging Face
   Advantages and Disadvantages of API Inferencing with Large Language Models
      Security Concerns when Using API Inferencing with Sensitive Data
   Use Cases across Natural Language Processing, Audio, and Computer Vision
      Natural Language Processing
      Audio
      Computer Vision
      Code Overview — Hugging Face APIs in Action
      Function Signature
      Setting Up
      Task Selection
      Sending the Request
      Example Usage
   Mastering the Inference API: Accessing Models in OpenAI
      Installation
      Authentication
      Models
      List Models
      Retrieve Model
      Chat
      Completion
      Edit
      Images
      Create Image API
      Create Image Edit API
      Create Image Variation API
      Embeddings
      Audio
      Creating transcription
      Create translation
      Moderation
      Create Moderation
      Mastering the Inference API: Accessing Models in Cohere
      Installation
      Authentication
      Co.Generate
      Co.Embed
      Co.Classify
      Co.Tokenize
      Co.Detokenize
      Co.Detect_language
      Co.Summarize
      Co.Rerank
   Conclusion
5. Integrating Cohere API in Google Sheets
   Introduction
   Structure
   Introduction to Google Sheets
   Introduction to Google Apps Script
   Understanding the Use Cases
      Text Classification
      Text Generation
      Text Summarization
   Prerequisites
      Required Knowledge and Tools
      Setting up Google Sheets and Google Apps Script
      Getting the Cohere API key
      Boilerplate code development and walkthrough
      Explanation of the Boilerplate code
      Walkthrough of the code and its structure
   Functionality-specific Development and Walkthrough
      Text Classification
      Code explanation and walkthrough
      Text Generation
      Code explanation and walkthrough
      Text Summarization
      Code explanation and walkthrough
      Testing the Use Cases
      Expected results and how to interpret them
   Putting it all Together
   Error Handling and Debugging
   Best Practices and Tips
   Conclusion
6. Dynamic Movie Recommendation Engine Using LLMs
   Introduction
   Structure
   Introduction to Vector Search-Based Recommendation Engine
      Understanding the Use Case: Movie Recommendations
   Introduction to Essential Components
   Background of Sentence Transformers/all-MiniLM-L12-v1
   Vector Databases: An Overview and Importance
   Redis as a Vector Database
   Installation and Pre-requisites
   Environment Preparation in Google Colab
   Upgrading Redis
   Understanding the Dataset
   The MovieRecommender Class: An Overview
      Data Preprocessing for Transformers
      Choosing the Right Transformer Model: MiniLM
      Defining Movie Data Loading and Vector Encoding
      Defining the Vector Database Indexing Process
      Defining the Search Function
      The Load and Index and Search Functions
      Wrapper Functions
      Summarizing the Use of Transformers and Vector Databases
   Discussion on Use Cases Beyond Movie Recommendations
      Future Improvements and Scalability Considerations
   Conclusion
7. Document-and Web-based QA Bots with Large Language Models
   Introduction
   Structure
   Understanding Large Language Models in the Context of a Chatbot
   Semantic Search and Its Role
   Building a Document- and Web-based QA Bot
   Applications of Document- and Web-based QA Bots
   Transitioning to Vector Databases and Their Role in Semantic Search
   Vector Databases and Pinecone: An In-Depth Look
      Benefits of Vector Databases over Traditional Databases
   Introduction to the Use Case: Document-based QA bot
      Tech-Stack Walkthrough and Explanation
      Pre-requisites
      OpenAI API Key
      Pinecone API Key
      Implementation Steps
      Detailed Code Walkthrough
   Introducing the Use Case: Web-based QA Bot
      Tech-Stack Walkthrough and Explanation
      Understanding FAISS and Pinecone
      Pre-requisites
      Cohere API Key
      Implementation Steps
      Detailed Code Walkthrough
   Conclusion
8. LLM Quantization Techniques and Implementation
   Introduction
   Structure
   Understanding Quantization
      Benefits and Importance of LLMs
      The Essence of Quantization
      Types of Quantization Techniques
      Specialized Quantization Strategies for LLMs
   Criteria for Quantization Strategy Selection
   Trade Offs and Challenges
   Implementing 4-bit Quantization using BitsandBytes and GPTQ Libraries
      Quantization Using BitsAndBytes
      Integration with Hugging Face Transformers
      Quantization Using GPTQ
      Comparative Analysis: GPTQ versus Other Algorithms
   The Role of Quantization in Real-World Applications
   Conclusion
9. Fine-Tuning and Evaluation of LLMs
   Introduction
   Structure
   Insight into LLM Variants
      Foundation LLM
      Pre-trained LLM
      Fine-Tuned LLM
   Benefits of Using Pre-trained Over Foundation LLMs
      Faster Training and Deployment
      Better Performance on Specific Domains
      Requires Less Data for Fine-Tuning
      Lower Risk
      Access to State-of-the-Art Models
   Impact of Training Corpus on Performance
      More Data, More Knowledge
      Model Scale and Architecture Matter
      Diminishing Returns
      Balancing Corpus Size with Compute Resources
      Corpus Relevance
      Multi-domain Versatility
   Best Practices for Fine-Tuning LLMs
      Understanding the Dataset
      Choosing the Right Pre-trained Model
      Targeted Parameter Fine-Tuning
      Customizing the Training Objective
      In-Context Learning and Other Advancements
      Tips for Creating an Instruction Dataset
   GPU Selection for LLM Training and Inference
      GPU Architecture: Core Components
      Specialized Components
      Programming GPUs
      GPUs in LLMs
      Selecting the Right GPU for LLM Training
      GPU for Model Inference
   Selecting the Right Model for Fine-Tuning
      Key Factors to Consider
      Task-Specific Recommendations
      General Guidelines
   Challenges in Fine-Tuning Pre-trained Models
   Sneak Peek into Efficiency in LLMs
      Token Economics
      Art of Prompt Optimization
      GPT Versions Cost Ratio
      Embedding and Fine-Tuning Costs
      Training and Fine-Tuning Costs
      GPU Memory Requirements
      Areas for Innovation
   Validating LLMs
      Evaluation Metrics
      Perplexity: The Speedometer of Language Models
      Language Fluency: The River’s Flow
      Coherence: The Structured Essay
      Contextual Understanding: The GPS System
      Factual Accuracy: The Journalist’s Fact-Check
      Evaluating General NLP Tasks
      Text Classification
      Text Generation
      Information Extraction
      Question Answering
      Text Similarity
      Dialogue Systems
      Challenges
      Metrics are Only Part of the Story
      Human Factor in Evaluation
      Ethical Dimension
   Future of Fine-Tuning LLMs
   Conclusion
   References
10. Recipes for Fine-Tuning and Evaluating LLMs
   Introduction
   Structure
   Fine-Tuning with OpenAI on Colab
      Implementation Walkthrough
   Fine-Tuning LLM on the Cohere Platform
   Fine-Tuning T5 Models on Kaggle GPUs for Text Classification on IMDB Reviews
      Implementation Walkthrough
      Environment Preparation for DeepSpeed
      Plugging the DeepSpeed Optimizer
   Evaluation of LLMs on Summarization Task
   Fine-Tuning on Colab Using Quantization Techniques
      Implementation Walkthrough
   Conclusion
   References
11. LLMOps - Operationalizing LLMs at Scale
   Introduction
   Structure
   Key Components of LLMOps
      Data Preparation
      Model Training
      Model Evaluation
      Model Deployment
      Model Monitoring
   Data Management in LLMOps
      Importance of Data Management
      Data Collection and Preprocessing
      Data Labeling and Annotation
      Data Storage, Organization, and Versioning
      Traditional Development Process
      Platform LLMOps Approach
   Difference Between LLMOps and MLOps
      Computational Resources
      Transfer Learning
      Human Feedback
      Hyperparameter Tuning
      Performance Metrics
      Prompt Engineering
      Building LLM Chains or Pipelines
   Best Practices for LLMOps
      Exploratory Data Analysis (EDA)
      Data Preparation and Prompt Engineering
      Model Fine-Tuning
      Model Review and Governance
      Model Inference and Serving
      Model Monitoring with Human Feedback
      General Best Practices
   Benefits of LLMOps
      Efficiency
      Scalability
      Risk Reduction
      Enhanced Customer Experience
   Challenges of LLMOps
      Large Model Size
      Complex Datasets
      Continuous Monitoring and Evaluation
      Scalability
      Model Optimization
      Infrastructure Optimization
      Security and Privacy
      Integration
      Automation
      Monitoring
      Validation
   Deployment Strategies for Large Language Models
      Latency Considerations
      Cost Management
      Resource Management
      Deployment Options: Cloud-based or On-premise
      Continuous Integration and Delivery (CI/CD)
      Monitoring and Maintenance
      Scalability and Performance Optimization
      Deployment Strategies
   Security and Privacy in LLM Deployment
      Data Privacy and Protection
      Data Encryption and Access Controls
      Model Security
      Regulatory Compliance
   Joint Recommendation for Language Model Deployment: Cohere, OpenAI, and AI21 Labs
      Prohibit Misuse
      Thoughtfully Collaboration with Stakeholders
   Confidence Checklist for Deploying Large Language Models in Production
      Output Validation
      Prepare for DDoS Attacks
      Building User Limits
      Care About Latency
      Avoid Retrofitting Logs and Monitoring Records for LLMs
      Implement Data Privacy
   Understanding the Economics of Large Language Models
      Costs
      Optimization
      Trade-offs
   LLMOps Adoption Blueprint
      Checklist for LLMOps Deployment
   Conclusion
12. Implementing LLMOps in Practice Using MLflow on Databricks
   Introduction
   Structure
   Technological Stacks in LLMOps Implementation
   High-Level Overview of the Implementation Pipeline
   Step-by-Step Overview of the Implementation
   Pre-requisite: Setup and Configuration
   Code Structure
   Detailed Code Walkthrough
      DataLoader
      Summarizer
      MLflowHandler
      Wrapping Up — The Pipeline
   Conclusion
13. Mastering the Art of Prompt Engineering
   Introduction
   Structure
   Prompt Engineering
      Prompt Shape
      Manual Template Engineering
   Automated Template Learning
      Discrete Prompts
      Continuous Prompts
   Answer Engineering
      Answer Shape
      Answer Space Design Methods
   Multi-prompt Learning
      Prompt Ensembling
      Prompt Augmentation
      Prompt Composition
      Prompt Decomposition
   Training Strategies for Prompting Methods
      Training Settings
      Parameter Update Methods
      Promptless Fine-Tuning
      Tuning-Free Prompting
      Fixed-LM Prompt Tuning
      Fixed-prompt LM Tuning
      Prompt+LM Tuning
   Prompt Engineering for Various Applications
      Knowledge Probing
      Classification-based Tasks
      Information Extraction
      “Reasoning” in NLP
      Question Answering
      Text Generation
   Unveiling the Intersections of Prompt Engineering with Existing Learning Techniques
      Ensemble Learning
      Few-Shot Learning
      Larger-Context Learning
      Query Reformulation
      QA-based Task Formulation
      Controlled Generation
      Supervised Attention
      Data Augmentation
   Challenges in Prompt Engineering
      Prompt Design
      Tasks Beyond Classification and Generation
      Prompting with Structured Information
      Answer Engineering
      Many-class and Long-answer Classification Tasks
      Multiple Answers for Generation Tasks
      Selection of Tuning Strategy
      Multiple Prompt Learning
      Prompt Ensembling
      Prompt Composition and Decomposition
      Prompt Augmentation
      Prompt Sharing
      Choosing Optimal Pre-trained Models
      Analyzing Prompting Theoretically and Empirically
      Exploring Prompts’ Transferability
      Calibration of Prompting Methods
      Combination of Different Paradigms
   Conclusion
14. Prompt Engineering Essentials and Design Patterns
   Introduction
   Structure
   Essence of Prompt Patterns
   Need for Prompt Design Patterns
   Anatomy of Prompts
      Three Pillars of Prompt Anatomy
      Stimulus
      Direction
      Constraints
      Significance of Understanding Prompt Anatomy
      Advanced Techniques
      Zero-Shot Prompting
      Few-Shot Prompting
      Chain-of-Thought Prompting
      Tree-of-Thoughts Framework
      Controlling Inconsistencies: Temperature and Self-Consistency
   Prompt Engineering Design Patterns
      Prompt Pattern Catalog
      Input Semantics
      Output Customization
      Error Identification
      Prompt Improvement
      Interaction
      Context Control
      Meta Language Creation Pattern
      Output Automater Pattern
      Understanding Flipped Interaction Pattern
      Persona Pattern
      Question Refinement Pattern
      Alternative Approaches Pattern
      Cognitive Verifier Pattern
      Fact Checklist Pattern
      Template Pattern
      Infinite Generation Pattern
      Visualization Generator Pattern
      Game Play Pattern
      Reflection Pattern
      Refusal Breaker Pattern
      Context Manager Pattern
      Recipe Pattern
   Prompt Engineering Best Practices
      Separate Instructions and Context
      Be Specific and Detailed
      Articulate Desired Output Format Through Examples
      Zero-Shot, Few-Shot, and Fine-Tuning
      Avoid Fluffy Descriptions
      Being Explicit About What to Do
      Code Generation Specifics
   Conclusion
   References
15. Ethical Considerations and Regulatory Frameworks for LLMs
   Introduction
   Structure
   Large Language Models: Categories, Life Cycle, and Key Techniques for Safety and Trustworthiness
   Dissecting the Applications of Large Language Models
      Text-based Conversational AI
      Text-based Image Synthesis
   Life Cycle of LLMs
   Critical Strategies for Improving Security and Reliability in Large Language Models
      The Power of Learning from Human Input (RLHF)
      Guardrails — Protective Measures
   Large Language Models Vulnerabilities
      Intrinsic Issues
      Performance Challenges
      Sustainability Challenges
      Other Inherent Issues Regarding Reliability and Responsibility
   Deliberate Attacks
      Unauthorized Information Disclosure and Privacy Issues
      Robustness Gap
      Backdoor Attack
      Poisoning and Disinformation
      Unintended Glitches
      Incidental Exposure of User Information
      Bias and Discrimination
   General Verification Framework for Large Language Models
      Evaluation Stage
      Runtime Monitoring
      Ethical Principles and AI Regulations
   Falsification and Evaluation
      Red Teaming
      Manipulating LLMs
      Prompt Injection in LLMs
      Assessing LLMs Against Human Expertise
      Benchmarks for LLMs
      Automating Testing and Evaluation
   Verification of Large Language Models
      Checking the Checkers: Verification of NLP Models
      Interval Bound Propagation: Establishing the Fence
      Navigating Uncertainty with Abstract Interpretation
      Bracing for Change with Randomized Smoothing
      Black-Box Verification: Cracking the Code
      Assessing the Resilience of LLMs
      A Case for Smaller Models
      Runtime Monitors: The Guardians of LLMs
      Detecting the Deviations: Monitoring Out-of-Distribution
      Guarding Against Output Failures
      Perspective
   Regulations and Ethical Use
      Regulate or Ban?
      Responsible AI Principles
      Transparency and Explainability
   Conclusion
   References
16. Towards Trustworthy Generative AI (A Novel Framework Inspired by Symbolic Reasoning)
   Introduction
   Structure
   Dichotomy of LLMs and Symbolic Systems
      Introduction to Symbolic Systems and Their Capabilities
      Introduction to Symbolic Systems and their Capabilities: A Deep Dive into Cyc
      History of Cyc
      Structure and Language
      Capabilities
      Real-World Applications
      Cyc Versus LLMs
      The Untapped Potential of Combining Both for Trustworthiness
   Revisiting the 16 Desiderata: A Critique
      Identifying Gaps and Proposing Extensions to the Desiderata
      Examination of the Desiderata
   The Concept of “Semantic Amplification”
   A Definitive Game-Changer: Enhancing the Trustworthiness of LLMs
      The Role of Semantic Amplification in the Trust-Enhanced Generative Framework (TEGF)
      Trust-Enhanced Generative Framework (TEGF)
      Statistical Language Model (SLM)
      Inner Workings
      Performance Metrics
      Symbolic Reasoning Engine
      Logic Rules and Ontology
      Validation and Refinement
      Performance Metrics
      Trustworthiness Layer
      Scoring Algorithm
      Real-time Scoring
      Performance Metrics
      Explainability Module
      Natural Language Generation (NLG)
      User Interface Design
      Performance Metrics
      Data Provenance Tracker
      Database Implementation
      Performance Metrics
      Contextual Understanding Module
      Adaptive Algorithms
      Performance Metrics
   Cohesion in the Trust-Enhanced Generative Framework (TEGF)
      Component Interactions and Trust Propagation
      Recommendations for Enhanced Cohesion
   Trustworthy Medical Diagnosis System: An End-to-End Walkthrough
   The Genesis of the Provenance Layer in TEGF
      The Mechanics of the Provenance Layer
      Real-World Implications: A Multi-Sector Focus
      Case Study: Healthcare - Complex Diagnoses
      User Experience
      Security Aspects
      Future Developments
   Introduction to the TIGAI Framework
      Components of TIGAI
      Adaptive Knowledge Base
      Contextual Reasoning Engine
      Trustworthiness Score
      Interpretability Layer
      TIGAI: Complementary or Contrasting Aspects with TEGF
      Case Studies
      Technical Depth
      Future Scope
      User Experience
      Security and Compliance
      Performance Metrics
   Ethical and Societal Implications: Navigating the Moral Labyrinth
      Data Privacy and Consent: The Double-Edged Sword
      Ethical Solutions
      Transparency and Accountability: The Pillars of Ethical AI
      Ethical Solutions
      Potential for Misuse: The Dark Side of Trustworthiness
      Ethical Solutions
   Setting a New Ethical Benchmark for Trustworthy AI
      Ethical Guidelines for TEGF in Healthcare
      Future Outlook and Public Policy
   Conclusion
   Reference
Index