Scaling Python with Ray: Adventures in Cloud and Serverless Patterns

Cover
Copyright
Table of Contents
Foreword
Preface
	What You Will Learn
	A Note on Responsibility
	Conventions Used in This Book
	License
	Using Code Examples
	O’Reilly Online Learning
	How to Contact Us
	Acknowledgments
		From Holden
		From Boris
Chapter 1. What Is Ray, and Where Does It Fit?
	Why Do You Need Ray?
	Where Can You Run Ray?
	Running Your Code with Ray
	Where Does It Fit in the Ecosystem?
		Big Data / Scalable DataFrames
		Machine Learning
		Workflow Scheduling
		Streaming
		Interactive
	What Ray Is Not
	Conclusion
Chapter 2. Getting Started with Ray (Locally)
	Installation
		Installing for x86 and M1 ARM
		Installing (from Source) for ARM
	Hello Worlds
		Ray Remote (Task/Futures) Hello World
		Data Hello World
		Actor Hello World
	Conclusion
Chapter 3. Remote Functions
	Essentials of Ray Remote Functions
	Composition of Remote Ray Functions
	Ray Remote Best Practices
	Bringing It Together with an Example
	Conclusion
Chapter 4. Remote Actors
	Understanding the Actor Model
	Creating a Basic Ray Remote Actor
	Implementing the Actor’s Persistence
	Scaling Ray Remote Actors
	Ray Remote Actors Best Practices
	Conclusion
Chapter 5. Ray Design Details
	Fault Tolerance
	Ray Objects
	Serialization/Pickling
		cloudpickle
		Apache Arrow
	Resources / Vertical Scaling
	Autoscaler
	Placement Groups: Organizing Your Tasks and Actors
	Namespaces
	Managing Dependencies with Runtime Environments
	Deploying Ray Applications with the Ray Job API
	Conclusion
Chapter 6. Implementing Streaming Applications
	Apache Kafka
		Basic Kafka Concepts
		Kafka APIs
	Using Kafka with Ray
	Scaling Our Implementation
	Building Stream-Processing Applications with Ray
		Key-Based Approach
		Key-Independent Approach
	Going Beyond Kafka
	Conclusion
Chapter 7. Implementing Microservices
	Understanding Microservice Architecture in Ray
		Deployment
		Additional Deployment Capabilities
		Deployment Composition
	Using Ray Serve for Model Serving
		Simple Model Service Example
		Considerations for Model-Serving Implementations
		Speculative Model Serving Using the Ray Microservice Framework
	Conclusion
Chapter 8. Ray Workflows
	What Is Ray Workflows?
	How Is It Different from Other Solutions?
	Ray Workflows Features
		What Are the Main Features?
		Workflow Primitives
	Working with Basic Workflow Concepts
		Workflows, Steps, and Objects
		Dynamic Workflows
		Virtual Actors
	Workflows in Real Life
		Building Workflows
		Managing Workflows
		Building a Dynamic Workflow
		Building Workflows with Conditional Steps
		Handling Exceptions
		Handling Durability Guarantees
		Extending Dynamic Workflows with Virtual Actors
		Integrating Workflows with Other Ray Primitives
		Triggering Workflows (Connecting to Events)
		Working with Workflow Metadata
	Conclusion
Chapter 9. Advanced Data with Ray
	Creating and Saving Ray Datasets
	Using Ray Datasets with Different Tools
	Using Tools on Ray Datasets
		pandas-like DataFrames with Dask
		Indexing
		Shuffles
		Embarrassingly Parallel Operations
		Working with Multiple DataFrames
		What Does Not Work
		What’s Slower
		Handling Recursive Algorithms
		What Other Functions Are Different
		pandas-like DataFrames with Modin
		Big Data with Spark
		Working with Local Tools
	Using Built-in Ray Dataset Operations
	Implementing Ray Datasets
	Conclusion
Chapter 10. How Ray Powers Machine Learning
	Using scikit-learn with Ray
	Using Boosting Algorithms with Ray
		Using XGBoost
		Using LightGBM
	Using PyTorch with Ray
	Reinforcement Learning with Ray
	Hyperparameter Tuning with Ray
	Conclusion
Chapter 11. Using GPUs and Accelerators with Ray
	What Are GPUs Good At?
	The Building Blocks
	Higher-Level Libraries
	Acquiring and Releasing GPU and Accelerator Resources
	Ray’s ML Libraries
	Autoscaler with GPUs and Accelerators
	CPU Fallback as a Design Pattern
	Other (Non-GPU) Accelerators
	Conclusion
Chapter 12. Ray in the Enterprise
	Ray Dependency Security Issues
	Interacting with the Existing Tools
	Using Ray with CI/CD Tools
	Authentication with Ray
	Multitenancy on Ray
	Credentials for Data Sources
	Permanent Versus Ephemeral Clusters
		Ephemeral Clusters
		Permanent Clusters
	Monitoring
	Instrumenting Your Code with Ray Metrics
	Wrapping Custom Programs with Ray
	Conclusion
Appendix A. Space Beaver Case Study: Actors, Kubernetes, and More
	High-Level Design
	Implementation
		Outbound Mail Client
		Shared Actor Patterns and Utilities
		Mail Server Actor
		Satellite Actor
		User Actor
		SMS Actor and Serve Implementation
	Testing
	Deployment
	Conclusion
Appendix B. Installing and Deploying Ray
	Installing Ray Locally
	Using Ray Docker Images
	Using Ray Clusters
		Installing Ray on AWS
		Installing Ray on IBM Cloud
		Installing Ray on Kubernetes
		Installing Ray on a kind Cluster
		Using ray up
		Using the Ray Kubernetes Operator
		Installing Ray on OpenShift
	Conclusion
Appendix C. Debugging with Ray
	General Debugging Tips with Ray
	Serialization Errors
	Local Debugging with Ray Local
	Remote Debugging
		Ray’s Integrated Debugger (via Pdb)
		Other Tools
	Ray and Container Exit Codes
	Ray Logs
	Container Errors
	Native Errors
	Conclusion
Index
About the Authors
Colophon