Big Data Using Hadoop and Hive

Cover
Half-Title
Title
Copyright
Dedication
Contents
Preface
Chapter 1: Big Data
	Big Data Challenges for Organizations
	How We Are Using Big Data
	Big Data: An Opportunity
	Hadoop: A Big Data Solution
	Big Data in the Real World
Chapter 2: What is Apache Hadoop?
	Hadoop History
	Hadoop Benefits
	Hadoop's Ecosystem: Components
	Hadoop Core Component Architecture
	Summary
Chapter 3: The Hadoop Distribution Filesystem
	HDFS Core Components
	HDFS Architecture
	Data Replication
	Data Locality
	Data Storage
	Failure Handling on the HDFS
	Erasure Coding (EC)
	HDFS Disk Balancer
	HDFS Federation
	HDFS Architecture and Its Challenges
	Hadoop Federation: A Rescue
	Benefits of the HDFS Federation
	HDFS Processes: Read and Write
	Failure Handling During Read and Write
Chapter 4: Getting Started with Hadoop
	Hadoop Configuration
	Command-Line Interface
	Generic Filesystem CLI Command
	Distributed Copy (distcp)
	Hadoop's Other User Commands
	HDFS Permissions
	HDFS Quotas Guide
	HDFS Short-Circuit Local Reads
	Offline Edits Viewer Guide
	Offline Image Viewer Guide
Chapter 5: Interfaces to Access HDFS Files
	WebHDFS REST API
	FileSystem URIs
	Error Responses
	Authentication
	Java FileSystem API
	URI and Path
	FSDataInputStream
	FSDataOutputStream
	FileStatus
	Directories
	Delete Files
	C API libhdfs
Chapter 6: Yet Another Resource Negotiator
	YARN Architecture
	YARN Process Flow
	YARN Failures
	YARN High Availability
	YARN Schedulers
	The Fair Scheduler
	The Capacity Scheduler
	The YARN Timeline Server
	Application Timeline Server (ATS)
	ATS Data Model Structure
	ATS V2
	YARN Federation
Chapter 7: MapReduce
	MapReduce Process
	Key Features
	Different Phases in the MapReduce Process
	MapReduce Architecture
	MapReduce Sample Program
	MapReduce Composite Key Operation
	Mapper Program
	MapReduce Configuration
Chapter 8: Hive
	Hive History
	Hive Query
	Data Storage
	Data Model
	Complex Data Types
	Hive DDL (Data Definition Language)
	Tables
	View
	Partition
	Bucketing
	Hive Architecture
	Serialization/Deserialization (SerDe)
	Metastore
	Query Compiler
	HiveServer2
Chapter 9: Getting Started with Hive
	Hive Set-up
	Hive Configuration Settings
	Loading and Inserting Data into Tables
	Insert from a Select Query
	Load Table Data into File
	Create and Load Data into a Table
	Hive Transactions
	Enable Transactions
	Insert Values
	Update
	Delete
	Merge
	Locks
	Hive Select Query
	Select Basic Query
	Hive QL File
	Hive Select on Complex Datatypes
	Order By and Sort By
	Distribute By and Cluster By
	Group By and Having
	Built-in Aggregate Functions
	Enhanced Aggregation
	Table-Generating Functions
	Built-In Utility Functions
	Collection Functions
	Date Functions
	Conditional Functions
	String Functions
	Hive Query Language-Join
Chapter 10: File Format
	File Format Characteristics
	Columnar Format
	Schema Evolution
	Splittable
	Compression
	File Formats
	RC (Row-Columnar) File Input Format
	Optimized Row Columnar (ORC) File Format
	Parquet
	File Format Comparisons
	ORC vs. Parquet
Chapter 11: Data Compression
	Data Compression Benefits
	Data Compression in Hadoop
	Splitting
	Compression Codec
	Data Compressions
	References
Index