Natural Language Processing on Oracle Cloud Infrastructure

Table of Contents
About the Author
About the Technical Reviewers
Acknowledgments
Introduction
Chapter 1: NLP Essentials
	Introduction to Natural Language Processing
		NLP Tasks
		NLP Key Concepts
		Common Challenges
	Transformers for NLP
		Transformer Architecture
		Transformer Taxonomy
		Transfer Learning
		Hugging Face Ecosystem
	Strategic Considerations for NLP Adoption
		Models
		Data
		Team
	Summary
	References
Chapter 2: Oracle Cloud for NLP
	Introduction to Oracle Cloud Infrastructure (OCI)
		History
		Core Concepts and Terminology
			Regions and Realms
			Tenancy/Compartment
			Core OCI Resources
				OCI Networking
				OCI Compute
				OCI Storage
				Identity and Access Management (IAM)
	Oracle’s AI Overview
		AI Strategy
		AI Stack
		OCI AI Services
		OCI ML Services
		AI Infrastructure
	OCI for NLP
		OCI Language
			Use Cases
		OCI Data Science
			AI Quick Actions
		OCI Data Labeling
		AI Samples
		High-Level Flow for Building NLP Models Using OCI
	Summary
	References
Chapter 3: Healthcare NLP Case Study
	MedTALN Inc. Case Study
		Company Background
		Healthcare NLP
			Business Drivers
		Healthcare NER Initiative
			What Is Named Entity Recognition (NER)
			Healthcare NER Benefits
				Use Cases
	Healthcare NER Inception
		Scope and Requirements
			Requirements
		Assembling the Team
			Engaging the NLP Consultant
	Healthcare NER Elaboration
		Architectural Design
			Methodology
			Preselection of Candidate Solution Options
				OCI Language-Based Models Option
				LLMs and OCI Data Science AI Quick Actions
				Fully Custom Healthcare NER Model
			Selection of the Optimal Approach
	Solution Blueprint
		High-Level Architecture
		High-Level Approach
		Project Preparation
			OCI Account
			Defining Roles and Responsibilities
	Summary
	Reference
Chapter 4: Tenancy Preparation
	Getting Started
		Cost-Saving Strategies
	OCI Tenancy Preparation
		Compartment Creation
		Network Configuration
		Storage
		Identity and Security
			IAM Setup for Data Scientists
				Users and Groups
				Dynamic Groups
				Policies
			IAM Setup for Data Labelers
	Data Science Environment Setup
		Project
		Notebook Sessions
			CPU-Based Notebook Session Setup
				Conda Installation
				Setup Check
			GPU-Based Notebook Session
	Summary
Chapter 5: Dataset Preparation
	Preliminaries
		Labeled Datasets
		Cost Saving
			Off-the-Shelf Datasets
			Cost Comparative Analysis
	Dataset Life Cycle
		Framing the Problem (Step 1)
		Dataset Selection (Step 2)
			Selecting Datasets on Hugging Face
			Candidate Healthcare NER Dataset
	Training Dataset Preparation
		Dataset Collection and Wrangling (Steps 3 and 4)
			Dataset Preparation Notebook
				Loading
				Wrangling Steps
		Dataset Labeling (Step 5)
			OCI Data Labeling Service (DLS)
			Dataset Import
				Dataset Import Notebook
					Initialization
					Dataset Import
			Dataset Labeling
			Quality Assurance (QA)
		Dataset Creation (Step 6)
		Additional Notes
			Dataset Import Using DLS UI
			Record Count Limit
	Summary
	References
Chapter 6: Model Fine-Tuning
	Preliminaries
		Language Models (LMs)
			Evolution of LMs
				Neural Language Models (2003)
				Word Embeddings: Word2Vec and GloVe (2013–2014)
				Transformers (2017)
				Pretrained Language Models (2018–2019)
				Large Language Models (LLMs) (2020s)
			Acronyms
			Taxonomy of Pretrained Language Models
		Healthcare-Specific Pretrained Language Models
			Why Domain-Specific Models for Healthcare
				Why Open Pretrained Models
		Cost-Saving Strategies for the Training Phase
		Transfer Learning–Based Fine-Tuning Workflow
	Pretrained Model Selection
		Framing the Problem (Step 1)
		MLM Model Selection from Hugging Face (Step 2)
			Pretrained Model Selection Notebook
				Identify a List of Candidate MLM Models from Hugging Face Hub
					Search MLM Models
					Check the Model Configuration
					Retrieve Mask Tokens
				Evaluate and Rank Models Based on Entity Prediction
	Healthcare NER Model Fine-Tuning
		Training Dataset Creation Notebook
			Declare Helper Functions
			Create HF Dataset from CoNLL File
			Create Splits for the HF Dataset
			Save Dataset
		Training Notebook
			Loading Training Dataset
			Training Initialization
				Set Pretrained Models for Fine-Tuning
				Declare Helper Functions
				Initialize the Training Objects
			Starting the Training
			Analyzing Training and Evaluation Losses
				Visual Analysis
				Automated Checkpoint Selection
	Healthcare NER Model Evaluation
		Evaluation Notebook
			Initialization
				Load the Training Dataset
				Define Helper Functions
			Evaluate
				Load Best Checkpoints
				Evaluate All the Models’ Best Checkpoints
				Select the Best Model
			Save the Best Model
			Test the Best Model
				Prepare the Test Examples
				Load and Use the Best Model
				Generate Predictions
	Summary
	References
Chapter 7: Model Deployment and Monitoring
	Model Inference Preliminaries
		Understanding Inference vs. Training
			Cost-Saving Strategies for the Inference Phase
		Preparing the Environment
			Setting Up Policies
			Setting Up Logging
			Publish Custom Conda Env.
	Deployment Process
		Oracle Data Science Model Catalog
		Oracle Data Science Model Deployment
		Oracle ADS HuggingFacePipelineModel
	Deployment Process Notebook
		Initializing the ADS Class “HuggingFacePipelineModel”
		Authenticate
			Initialize Hugging Face Pipeline
			Prepare Model Artifact
			Manually Correct score.py
			Run Introspection
			Call Model Summary
			Verify the Generated Model Artifacts
		Save the Model to the Model Catalog
			Create a Model Version Set
			Save the Model
		Deploy and Invoke
			Deploy and Generate Endpoint
			Run Prediction Against Endpoint
	Monitoring and Maintenance
		Logs
		Metrics
	Summary
	References
Chapter 8: MLOps and Conclusion
	MLOps with OCI Data Science
		OCI Data Science Pipelines
		Pipeline Example
		Pipeline Creation Step-by-Step
			Pipeline Creation Prerequisites
				Create Pipeline Step Artifacts
				Data Science Dynamic Group Rule
			Create Pipeline
	Journey Through NLP: From Theory to Practice
		Healthcare NER Model Life Cycle Summary
			Data Preparation
			Model Training and Evaluation
			Model Deployment and Monitoring
				Deploy
				Monitor
	Responsible AI
	Summary
	Reference