Conversational AI: Chatbots that work

Conversational AI
brief contents
contents
preface
acknowledgments
about this book
	Who should read this book
	How this book is organized: A roadmap
	About the code
	liveBook discussion forum
	Other online resources
about the author
about the cover illustration
Part 1: Foundations
	Chapter 1: Introduction to conversational AI
		1.1 Introduction to AI assistants and their platforms
			1.1.1 Types of AI assistants
			1.1.2 A snapshot of AI assistant platforms
		1.2 Primary use cases for AI assistant technology
			1.2.1 Self-service assistant
			1.2.2 Agent assist
			1.2.3 Classification and routing
		1.3 Follow along with this book
			1.3.1 What you need to create your assistant
			1.3.2 Useful spreadsheet software
			1.3.3 Recommended programming language and code repository
	Chapter 2: Building your first conversational AI
		2.1 Building a conversational AI for Fictitious Inc.
		2.2 What’s the user’s intent?
			2.2.1 What’s an utterance?
			2.2.2 What’s a response?
			2.2.3 How does the assistant understand what the user means?
			2.2.4 Why machine learning?
			2.2.5 What’s an intent?
			2.2.6 What’s an entity?
			2.2.7 Combining intents and entities
			2.2.8 Adding intents to the Fictitious Inc. assistant
		2.3 Responding to the user
			2.3.1 Simple question-and-answer responses
			2.3.2 Contextualizing a response by using entities
			2.3.3 An alternative way to provide contextual responses
			2.3.4 Responding with a process flow
		2.4 Other useful responses
			2.4.1 Detecting low confidence
			2.4.2 Counting misunderstandings
			2.4.3 Implementing confidence detection and the two-strikes rule
		2.5 Try to build this conversational AI yourself
Part 2: Designing for success
	Chapter 3: Designing effective processes
		3.1 What processes will the assistant handle?
			3.1.1 Designing for the most common user needs
			3.1.2 Assembling a plan and a dream team
			3.1.3 Managing the design process
			3.1.4 Cross-cutting design aspects
		3.2 Choosing the channel to implement first
			3.2.1 How users receive information in voice and web
			3.2.2 How the assistant receives information in voice and web
	Chapter 4: Designing effective dialogue
		4.1 Writing dialogue
			4.1.1 Take a conversational tone
			4.1.2 Don’t repeat yourself (much)
			4.1.3 Acknowledge the user
		4.2 Asking questions
		4.3 What if the assistant doesn’t understand?
			4.3.1 Reprompting
			4.3.2 Disambiguation
			4.3.3 Escalation
	Chapter 5: Building a successful AI assistant
		5.1 AI assistant use cases
		5.2 Conversational AI success metrics
			5.2.1 Containment
			5.2.2 Time to resolution
			5.2.3 Net promoter score
			5.2.4 Coverage
			5.2.5 Instrumenting your conversational AI
		5.3 Command interpreter success metrics
			5.3.1 Usage
			5.3.2 Stickiness
			5.3.3 Instrumenting your command interpreter
		5.4 Event classifier success metrics
			5.4.1 Time to resolution
			5.4.2 Number of hand-offs
			5.4.3 Other customer satisfaction metrics
			5.4.4 Instrumenting your classifier
Part 3: Training and testing
	Chapter 6: Training your assistant
		6.1 Training an AI assistant
		6.2 Finding training data
			6.2.1 Production logs
			6.2.2 A mock user interface
			6.2.3 Subject-matter experts
			6.2.4 Organizing training data into intents
		6.3 Assessing whether you have the right training data
			6.3.1 Training data variety
			6.3.2 Training data volume
			6.3.3 Training data veracity
	Chapter 7: How accurate is your assistant?
		7.1 Testing an AI assistant for accuracy
			7.1.1 Testing a single utterance
			7.1.2 Testing multiple utterances
			7.1.3 Selecting a test data set
		7.2 Comparing testing methodologies
			7.2.1 Blind testing
			7.2.2 k-folds cross-validation test
		7.3 Selecting the right accuracy metric for the job
	Chapter 8: Testing your dialogue flows
		8.1 Functionally testing a dialogue flow
			8.1.1 Manually testing a conversation flow
			8.1.2 Automating a conversation flow test
			8.1.3 Testing the dialogue flowchart
			8.1.4 Testing the unexpected error paths
		8.2 Nonfunctionally testing a dialogue flow
			8.2.1 User experience testing
			8.2.2 Load testing
Part 4: Maintenance
	Chapter 9: Deployment and management
		9.1 Where to store your code
			9.1.1 Taking the Wild West approach
			9.1.2 Using source control for code
		9.2 Where to run your code
			9.2.1 Development environment
			9.2.2 Test environment
			9.2.3 Production environment
			9.2.4 After the first production deployment
		9.3 Using source control for other assets
	Chapter 10: Improving your assistant
		10.1 Using a success metric to determine where to start improvements
			10.1.1 Improving the first flow to fix containment problems
			10.1.2 Inspecting other process flows for containment problems
		10.2 Analyzing the classifier to predict future containment problems
			10.2.1 Representative baseline
			10.2.2 Finding gaps in the training data
		10.3 When and why to improve your assistant
			10.3.1 You can’t fix everything at once
			10.3.2 You can’t always predict how users will react
			10.3.3 User needs will change
			10.3.4 Not every problem is technical
Part 5: Advanced/optional topics
	Chapter 11: Building your own classifier
		11.1 Why build your own classifier?
			11.1.1 Classification is a differentiator
			11.1.2 Classification is a core competency
			11.1.3 Traceability
			11.1.4 To learn
			11.1.5 Build or buy?
		11.2 Build a simple classifier from first principles
			11.2.1 The simplest text classifier
			11.2.2 The mathematics behind a simple classifier
		11.3 Expanding the simple classifier
			11.3.1 Predicting more than one class
			11.3.2 An all-in-one classifier
			11.3.3 Comparing binary classifiers to all-in-one classifiers
		11.4 Extending even further
			11.4.1 What happens when you add more training data?
			11.4.2 Exercise: Experiment on your own
	Chapter 12: Additional training for voice assistants
		12.1 Collecting data to test a speech-to-text model
			12.1.1 Call recordings as speech training data
			12.1.2 Generating synthetic speech data
		12.2 Testing the speech-to-text model
			12.2.1 Word error rate
			12.2.2 Intent error rate
			12.2.3 Sentence error rate
		12.3 Training a speech-to-text model
			12.3.1 Custom training with a language model
			12.3.2 Custom training with an acoustic model
			12.3.3 Custom training with grammars
appendix: Glossary of terms and abbreviations
index
	A
	B
	C
	D
	E
	F
	G
	H
	I
	J
	K
	L
	M
	N
	O
	P
	Q
	R
	S
	T
	U
	V
	W