Big Data Analytics: A Guide to Data Science Practitioners Making the Transition to Big Data

Cover
Half Title
Series Page
Title Page
Copyright Page
Dedication
Contents
Preface
I. Setting the Scene: Analyzing Big Data
	Introduction
	1. What is Big in “Big Data”?
	2. Approaches to Analyzing Big Data
	3. The Two Domains of Big Data Analytics
		3.1. A practical big P problem
			3.1.1. Simple logistic regression (naive approach)
			3.1.2. Regularization: the lasso estimator
		3.2. A practical big N problem
			3.2.1. OLS as a point of reference
			3.2.2. The Uluru algorithm as an alternative to OLS
II. Platform: Software and Computing Resources
	Introduction
	4. Software: Programming with (Big) Data
		4.1. Domains of programming with (big) data
		4.2. Measuring R performance
		4.3. Writing efficient R code
			4.3.1. Memory allocation and growing objects
			4.3.2. Vectorization in basic R functions
			4.3.3. apply-type functions and vectorization
			4.3.4. Avoiding unnecessary copying
			4.3.5. Releasing memory
			4.3.6. Beyond R
		4.4. SQL basics
			4.4.1. First steps in SQL(ite)
			4.4.2. Joins
		4.5. With a little help from my friends: GPT and R/SQL coding
		4.6. Wrapping up
	5. Hardware: Computing Resources
		5.1. Mass storage
			5.1.1. Avoiding redundancies
			5.1.2. Data compression
		5.2. Random access memory (RAM)
		5.3. Combining RAM and hard disk: Virtual memory
		5.4. CPU and parallelization
			5.4.1. Naive multi-session approach
			5.4.2. Multi-session approach with futures
			5.4.3. Multi-core and multi-node approach
		5.5. GPUs for scientific computing
			5.5.1. GPUs in R
		5.6. The road ahead: Hardware made for machine learning
		5.7. Wrapping up
		5.8. Still have insufficient computing resources?
	6. Distributed Systems
		6.1. MapReduce
		6.2. Apache Hadoop
			6.2.1. Hadoop word count example
		6.3. Apache Spark
		6.4. Spark with R
			6.4.1. Data import and summary statistics
		6.5. Spark with SQL
		6.6. Spark with R + SQL
		6.7. Wrapping up
	7. Cloud Computing
		7.1. Cloud computing basics and platforms
		7.2. Transitioning to the cloud
		7.3. Scaling up in the cloud: Virtual servers
			7.3.1. Parallelization with an EC2 instance
		7.4. Scaling up with GPUs
			7.4.1. GPUs on Google Colab
			7.4.2. RStudio and EC2 with GPUs on AWS
		7.5. Scaling out: MapReduce in the cloud
		7.6. Wrapping up
III. Components of Big Data Analytics
	Introduction
	8. Data Collection and Data Storage
		8.1. Gathering and compilation of raw data
		8.2. Stack/combine raw source files
		8.3. Efficient local data storage
			8.3.1. RDBMS basics
			8.3.2. Efficient data access: Indices and joins in SQLite
		8.4. Connecting R to an RDBMS
			8.4.1. Creating a new database with RSQLite
			8.4.2. Importing data
			8.4.3. Issuing queries
		8.5. Cloud solutions for (big) data storage
			8.5.1. Easy-to-use RDBMS in the cloud: AWS RDS
		8.6. Column-based analytics databases
			8.6.1. Installation and start up
			8.6.2. First steps via Druid’s GUI
			8.6.3. Query Druid from R
		8.7. Data warehouses
			8.7.1. Data warehouse for analytics: Google BigQuery example
		8.8. Data lakes and simple storage service
			8.8.1. AWS S3 with R: First steps
			8.8.2. Uploading data to S3
			8.8.3. More than just simple storage: S3 + Amazon Athena
		8.9. Wrapping up
	9. Big Data Cleaning and Transformation
		9.1. Out-of-memory strategies and lazy evaluation: Practical basics
			9.1.1. Chunking data with the ff package
			9.1.2. Memory mapping with bigmemory
			9.1.3. Connecting to Apache Arrow
		9.2. Big Data preparation tutorial with ff
			9.2.1. Set up
			9.2.2. Data import
			9.2.3. Inspect imported files
			9.2.4. Data cleaning and transformation
			9.2.5. Inspect difference in in-memory operation
			9.2.6. Subsetting
			9.2.7. Save/load/export ff files
		9.3. Big Data preparation tutorial with arrow
		9.4. Wrapping up
	10. Descriptive Statistics and Aggregation
		10.1. Data aggregation: The ‘split-apply-combine’ strategy
		10.2. Data aggregation with chunked data files
		10.3. High-speed in-memory data aggregation with arrow
		10.4. High-speed in-memory data aggregation with data.table
		10.5. Wrapping up
	11. (Big) Data Visualization
		11.1. Challenges of Big Data visualization
		11.2. Data exploration with ggplot2
		11.3. Visualizing time and space
			11.3.1. Preparations
			11.3.2. Pick-up and drop-off locations
		11.4. Wrapping up
IV. Application: Topics in Big Data Econometrics
	Introduction
	12. Bottlenecks in Everyday Data Analytics Tasks
		12.1. Case study: Efficient fixed effects estimation
		12.2. Case study: Loops, memory, and vectorization
			12.2.1. Naïve approach (ignorant of R)
			12.2.2. Improvement 1: Pre-allocation of memory
			12.2.3. Improvement 2: Exploit vectorization
		12.3. Case study: Bootstrapping and parallel processing
			12.3.1. Parallelization with an EC2 instance
	13. Econometrics with GPUs
		13.1. OLS on GPUs
		13.2. A word of caution
		13.3. Higher-level interfaces for basic econometrics with GPUs
		13.4. TensorFlow/Keras example: Predict housing prices
			13.4.1. Data preparation
			13.4.2. Model specification
			13.4.3. Training and prediction
		13.5. Wrapping up
	14. Regression Analysis and Categorization with Spark and R
		14.1. Simple linear regression analysis
		14.2. Machine learning for classification
		14.3. Building machine learning pipelines with R and Spark
			14.3.1. Set up and data import
			14.3.2. Building the pipeline
		14.4. Wrapping up
	15. Large-scale Text Analysis with sparklyr
		15.1. Getting started: Import, pre-processing, and word count
		15.2. Tutorial: political slant
			15.2.1. Data download and import
			15.2.2. Cleaning speeches data
			15.2.3. Create a bigrams count per party
			15.2.4. Find “partisan” phrases
			15.2.5. Results: Most partisan phrases by congress
		15.3. Natural Language Processing at Scale
			15.3.1. Preparatory steps
			15.3.2. Sentiment annotation
		15.4. Aggregation and visualization
		15.5. sparklyr and lazy evaluation
V. Appendices
	Appendix A: GitHub
	Appendix B: R Basics
	Appendix C: Install Hadoop
VI. Bibliography and Index
	Bibliography
	Index