Data Processing with Optimus: Supercharge big data preparation tasks for analytics and machine learning with Optimus using Dask and PySpark

Cover
Title
Copyright and Credits
Table of Contents
Section 1: Getting Started with Optimus
Chapter 1: Hi Optimus!
	Technical requirements
	Introducing Optimus
		Exploring the DataFrame technologies
		Examining Optimus design principles
	Installing everything you need to run Optimus
		Installing Anaconda
		Installing Optimus
		Installing JupyterLab
		Installing RAPIDS
		Using Coiled
		Using a Docker container
	Using Optimus
		The Optimus instance
		The Optimus DataFrame
		Technical details
	Discovering Optimus internals
		Engines
		The DataFrame behind the DataFrame
		Meta
		Dummy functions
		Diagnostics
	Summary
Chapter 2: Data Loading, Saving, and File Formats
	Technical requirements
	How data moves internally
		File to RAM
		File to GPU memory
		Database to RAM
		Database to GPU memory
	Loading a file
		Loading a local CSV file
		Wildcards
		Loading large files
		Loading a file from a remote connection
		Loading data from a database
		Special dependencies for every technology
		Creating a dataframe from scratch
		Connecting to remote data sources
		Connection credentials
	Connecting to databases
	Saving a dataframe
		Saving to a local file
		Saving a file using a remote connection
		Saving a dataframe to a database table
		Loading and saving data in parallel
	Summary
Section 2: Optimus – Transform and Rollout
Chapter 3:  Data Wrangling
	Technical requirements
	Exploring Optimus data types
		Converting data types
	Operating columns
		Selecting columns
		Moving columns
		Renaming columns
		Removing columns
		Input and output columns
		Managing functions
		String functions
		Numeric functions
		Date and time functions
		URL functions
		Email functions
	Experimenting with user-defined functions
		Using apply
		Supporting multiple engines
	Summary
	Further reading
Chapter 4: Combining, Reshaping, and Aggregating Data
	Technical requirements
	Concatenating data
		Mapping
		Concatenating columns
	Joining data
	Reshaping and pivoting
		Pivoting
		Stacking
		Unstacking
		Melting
		Aggregations
	Aggregating and grouping
	Summary
Chapter 5: Data Visualization and Profiling
	Technical requirements
	Data quality
		Handling matches, mismatches, and nulls
	Exploratory data analysis
		Single variable non-graphical methods
		Single variable graphical methods
		Multi-variable non-graphical methods
		Multi-variable graphical methods
	Data profiling
	Cache flushing
	Summary
Chapter 6: String Clustering
	Technical requirements
	Exploring string clustering
	Key collision methods
		Fingerprinting
		N-gram fingerprinting
		Phonetic encoding
	Nearest-neighbor methods
		Levenshtein distance
	Applying suggestions
	Summary
Chapter 7: Feature Engineering
	Technical requirements
	Handling missing values
		Removing data
		Imputation
	Handling outliers
		Tukey
		Z-score
		Modified Z-score
	Binning
	Variable transformation
		Logarithmic transformation
		Square root transformation
		Reciprocal transformation
		Exponential or power transformation
		String to index
	One-hot encoding
	Feature splitting
	Scaling
		Normalization
		Standardization
		Max abs scaler
	Summary
Section 3: Advanced Features of Optimus
Chapter 8: Machine Learning
	Technical requirements
	Optimus as a cohesive API
		How Optimus can help
	Implementing a train-test split procedure
		When to use a train-test split procedure
		Test size
		Repeatable train-test splits
		Using k-fold cross-validation
	Training models in Optimus
		Linear regression
		Logistic regression
		Model performance
		K-means
		PCA
		Loading and saving models
	Summary
Chapter 9: Natural Language Processing
	Technical requirements
	Natural language processing
	Removing unwanted strings
		Stripping the HTML
		Removing stopwords
		Removing URLs
		Removing special characters
		Expanding contracted words
	Stemming and lemmatization
		Stemming
		Lemmatization
	word_tokenizer
		Part-of-speech tagging
		Applying the transformation
	Feature extraction from text
		Bag of words
	Summary
Chapter 10: Hacking Optimus
	Technical requirements
		Installing Git
	Adding a new engine
		Cloning the repository from GitHub
		How the project is organized
		The entry point
		Base class functions
		Applying functions
		I/O operations
		Plots
		Profiler data types
	Bumblebee
	Joining the community
	The future
	Limitations
	Summary
Chapter 11: Optimus as a Web Service
	Technical requirements
	Introducing Blurr
	Setting up the environment
		Pre-requisites for Blurr
		Installing the package
		Importing the package
		Creating a Blurr session
		Multiple engines in one session
		Quickest setup
	Making requests
		Loading a dataframe
		Saving a dataframe
		Getting information from the dataset
		Transforming a dataset
		Passing arguments
		Getting the content of the dataset
		Multiple operations in one request
		Using other types of data
	Summary
Index