Smart Meter Data Analytics: Electricity Consumer Behavior Modeling, Aggregation, and Forecasting

Foreword
Preface
Acknowledgements
Contents
1 Overview of Smart Meter Data Analytics
	1.1 Introduction
	1.2 Load Analysis
		1.2.1 Bad Data Detection
		1.2.2 Energy Theft Detection
		1.2.3 Load Profiling
		1.2.4 Remarks
	1.3 Load Forecasting
		1.3.1 Forecasting Without Smart Meter Data
		1.3.2 Forecasting with Smart Meter Data
		1.3.3 Probabilistic Forecasting
		1.3.4 Remarks
	1.4 Load Management
		1.4.1 Consumer Characterization
		1.4.2 Demand Response Program Marketing
		1.4.3 Demand Response Implementation
		1.4.4 Remarks
	1.5 Miscellanies
		1.5.1 Connection Verification
		1.5.2 Outage Management
		1.5.3 Data Compression
		1.5.4 Data Privacy
	1.6 Conclusions
	References
2 Electricity Consumer Behavior Model
	2.1 Introduction
	2.2 Basic Concept of ECBM
		2.2.1 Definition
		2.2.2 Connotation
		2.2.3 Denotation
		2.2.4 Relationship with Other Models
	2.3 Basic Characteristics of Electricity Consumer Behavior
	2.4 Mathematical Expression of ECBM
	2.5 Research Paradigm of ECBM
	2.6 Research Framework of ECBM
	2.7 Conclusions
	References
3 Smart Meter Data Compression
	3.1 Introduction
	3.2 Household Load Profile Characteristics
		3.2.1 Small Consecutive Value Difference
		3.2.2 Generalized Extreme Value Distribution
		3.2.3 Effects on Load Data Compression
	3.3 Feature-Based Load Data Compression
		3.3.1 Distribution Fit
		3.3.2 Load State Identification
		3.3.3 Base State Discretization
		3.3.4 Event Detection
		3.3.5 Event Clustering
		3.3.6 Load Data Compression and Reconstruction
	3.4 Data Compression Performance Evaluation
		3.4.1 Related Data Formats
		3.4.2 Evaluation Index
		3.4.3 Dataset
		3.4.4 Compression Efficiency Evaluation Results
		3.4.5 Reconstruction Precision Evaluation Results
		3.4.6 Performance Map
	3.5 Conclusions
	References
4 Electricity Theft Detection
	4.1 Introduction
	4.2 Problem Statement
		4.2.1 Observer Meters
		4.2.2 False Data Injection
		4.2.3 A State-Based Method of Correlation
	4.3 Methodology and Detection Framework
		4.3.1 Maximum Information Coefficient
		4.3.2 CFSFDP-Based Unsupervised Detection
		4.3.3 Combined Detecting Framework
	4.4 Numerical Experiments
		4.4.1 Dataset
		4.4.2 Comparisons and Evaluation Criteria
		4.4.3 Numerical Results
		4.4.4 Sensitivity Analysis
	4.5 Conclusions
	References
5 Residential Load Data Generation
	5.1 Introduction
	5.2 Model
		5.2.1 Basic Framework
		5.2.2 General Network Architecture
		5.2.3 Unclassified Generative Models
		5.2.4 Classified Generative Models
	5.3 Methodology
		5.3.1 Data Preprocessing
		5.3.2 Model Training
		5.3.3 Metrics
	5.4 Case Studies
		5.4.1 Data Description
		5.4.2 Unclassified Generation
		5.4.3 Classified Generation
	5.5 Conclusion
	References
6 Partial Usage Pattern Extraction
	6.1 Introduction
	6.2 Non-negative K-SVD-Based Sparse Coding
		6.2.1 The Idea of Sparse Representation
		6.2.2 The Non-negative K-SVD Algorithm
	6.3 Load Profile Classification
		6.3.1 The Linear SVM
		6.3.2 Parameter Selection
	6.4 Evaluation Criteria and Comparisons
		6.4.1 Data Compression-Based Criteria
		6.4.2 Classification-Based Criteria
		6.4.3 Comparisons
	6.5 Numerical Experiments
		6.5.1 Description of the Dataset
		6.5.2 Experimental Results
		6.5.3 Comparative Analysis
	6.6 Further Multi-dimensional Analysis
		6.6.1 Characteristics of Residential & SME Users
		6.6.2 Seasonal and Weekly Behaviors Analysis
		6.6.3 Working Day and Off Day Patterns Analysis
		6.6.4 Entropy Analysis
		6.6.5 Distribution Analysis
	6.7 Conclusions
	References
7 Personalized Retail Price Design
	7.1 Introduction
	7.2 Problem Formulation
		7.2.1 Problem Statement
		7.2.2 Consumer Problem
		7.2.3 Compatible Incentive Design
		7.2.4 Retailer Problem
		7.2.5 Data-Driven Clustering and Preference Discovering
		7.2.6 Integrated Model
	7.3 Solution Methods
		7.3.1 Framework
		7.3.2 Piece-Wise Linear Approximation
		7.3.3 Eliminating Binary Variable Product
		7.3.4 CVaR
		7.3.5 Eliminating Absolute Values
	7.4 Case Study
		7.4.1 Data Description and Experiment Setup
		7.4.2 Basic Results
		7.4.3 Sensitivity Analysis
	7.5 Conclusions and Future Works
	References
8 Socio-demographic Information Identification
	8.1 Introduction
	8.2 Problem Definition
	8.3 Method
		8.3.1 Why Use a CNN?
		8.3.2 Proposed Network Structure
		8.3.3 Description of the Layers
		8.3.4 Reducing Overfitting
		8.3.5 Training Method
	8.4 Performance Evaluation and Comparisons
		8.4.1 Performance Evaluation
		8.4.2 Competing Methods
	8.5 Case Study
		8.5.1 Data Description
		8.5.2 Basic Results
		8.5.3 Comparative Analysis
	8.6 Conclusions
	References
9 Coding for Household Energy Behavior
	9.1 Introduction
	9.2 Basic Idea and Framework
	9.3 Load Profile Clustering
		9.3.1 GMM-Based Typical Load Profile Extraction
		9.3.2 X-Means-Based Load Profile Clustering
	9.4 Socioeconomic Genes Identification Method
		9.4.1 Socioeconomic Information Classification
		9.4.2 The Concept of Socioeconomic Genes
		9.4.3 Socioeconomic Genes Evaluation Indicators
		9.4.4 Socioeconomic Gene Search Method
	9.5 Load Profile Prediction
	9.6 Case Studies
		9.6.1 Consumer Load Profile Classification
		9.6.2 Socioeconomic Gene Search Result
		9.6.3 Consumer Load Profile Prediction
	9.7 Conclusions
	References
10 Clustering of Consumption Behavior Dynamics
	10.1 Introduction
	10.2 Basic Methodology
		10.2.1 Data Normalization
		10.2.2 SAX for Load Curves
		10.2.3 Time-Based Markov Model
		10.2.4 Distance Calculation
		10.2.5 CFSFDP Algorithm
	10.3 Distributed Algorithm for Large Data Sets
		10.3.1 Framework
		10.3.2 Local Modeling-Adaptive k-Means
		10.3.3 Global Modeling-Modified CFSFDP
	10.4 Case Studies
		10.4.1 Description of the Data Set
		10.4.2 Modeling Consumption Dynamics for Each Customer
		10.4.3 Clustering for Full Periods
		10.4.4 Clustering for Each Adjacent Periods
		10.4.5 Distributed Clustering
	10.5 Potential Applications
	10.6 Conclusions
	References
11 Probabilistic Residential Load Forecasting
	11.1 Introduction
	11.2 Pinball Loss Guided LSTM
		11.2.1 LSTM
		11.2.2 Pinball Loss
		11.2.3 Overall Networks
	11.3 Implementations
		11.3.1 Framework
		11.3.2 Data Preparation
		11.3.3 Model Training
		11.3.4 Probabilistic Forecasting
	11.4 Benchmarks
		11.4.1 QRNN
		11.4.2 QGBRT
		11.4.3 LSTM+E
	11.5 Case Studies
		11.5.1 Data Description
		11.5.2 Residential Load Forecasting Results
		11.5.3 SME Load Forecasting Results
	11.6 Conclusions
	References
12 Aggregated Load Forecasting with Sub-profiles
	12.1 Introduction
	12.2 Load Forecasting with Different Aggregation Levels
		12.2.1 Variance of Aggregated Load Profiles
		12.2.2 Scaling Law
	12.3 Clustering-Based Aggregated Load Forecasting
		12.3.1 Framework
		12.3.2 Numerical Experiments
	12.4 Ensemble Forecasting for the Aggregated Load
		12.4.1 Proposed Methodology
		12.4.2 Case Study
	12.5 Conclusions
	References
13 Prospects of Future Research Issues
	13.1 Big Data Issues
	13.2 New Machine Learning Technologies
	13.3 New Business Models in Retail Market
	13.4 Transition of Energy Systems
	13.5 Data Privacy and Security
	References