Statistical Learning Tools for Electricity Load Forecasting

Contents
1 Introduction
	1.1 Industrial Motivation
	1.2 Data Sets
		1.2.1 General Considerations
		1.2.2 Salient Features of Electricity Demand
		1.2.3 Irish Individual Electrical Demand Data
			1.2.3.1 Data Presentation
			1.2.3.2 Data Processing
			1.2.3.3 Getting the Data
		1.2.4 French National Demand Data
			1.2.4.1 Data Presentation
			1.2.4.2 Data Processing
			1.2.4.3 Getting the Data
		1.2.5 US Regional Demand Data from the GEFCOM 2014 Competition
			1.2.5.1 Data Presentation
			1.2.5.2 Data Processing
			1.2.5.3 Getting the Data
	1.3 Problems
		1.3.1 Short-Term Point Forecasting
		1.3.2 Probabilistic Forecasting
		1.3.3 Selection of Relevant Variables for Prediction
		1.3.4 Peak Demand Forecasting
		1.3.5 Adaptive Forecasting
		1.3.6 Bottom-up and Hierarchical Forecasting
	1.4 Assessment and Validation
		1.4.1 Assessment Scores
			1.4.1.1 Pointwise Forecasting
			1.4.1.2 Probabilistic Forecasting
		1.4.2 Validation Procedures
			1.4.2.1 Cross-Validation
			1.4.2.2 Bootstrapping
Part I A Toolbox of Models
	2 Additive Modelling of Electricity Demand with mgcv
		2.1 Introducing GAMs
			2.1.1 GAM Model Structure
			2.1.2 GAM Model Fitting in a Bayesian Framework
			2.1.3 Basic Smooth Effects and Penalties
				2.1.3.1 Thin Plate Splines and Derivative-Based Penalties
				2.1.3.2 Smooth Effects in mgcv
			2.1.4 Model Selection
				2.1.4.1 Model Selection via Smoothing Parameter Estimation
				2.1.4.2 Performing AIC-Based Model Selection Under Penalization
				2.1.4.3 Choosing the Type and the Basis Dimension of a Smooth Effect
			2.1.5 Example: Modelling Aggregated Irish Smart Meter Data
		2.2 More Smooth Effects and Big Data Methods
			2.2.1 Tensor-Product and By-variable Smooths
			2.2.2 GAM Methods for Large Data Sets
			2.2.3 Example: Modelling Aggregate Irish Smart Meter Data (Continued)
			2.2.4 Alternatives to mgcv for GAM Modelling
			2.2.5 Summary
	3 Probabilistic GAMs: Beyond Mean Modelling
		3.1 Introduction to GAMLSS Modelling in mgcv
			3.1.1 Probabilistic GAM Modelling of GEFCom 2014 Data
		3.2 Introducing QGAM Models
			3.2.1 QGAM Model Structure
			3.2.2 Fitting QGAM Models with qgam
			3.2.3 Distribution-Free QGAM Modelling of GEFCom 2014 Data
			3.2.4 Alternatives to mgcv and qgam for GAMLSS and QGAM Modelling
		3.3 Summary
	4 Functional Time Series
		4.1 Functional Data
		4.2 Wavelets
		4.3 KWF: A Nonparametric Regression for Stationary FTS
		4.4 Prediction Interval
			4.4.1 Bootstrap Generation
			4.4.2 Two Variants from the KWF Method
		4.5 Clustering Functional Data
			4.5.1 Clustering by Feature Extraction
			4.5.2 Clustering Using a Dissimilarity Measure
	5 Random Forests
		5.1 Random Forests: An Ensemble Based Method
			5.1.1 CART Trees
			5.1.2 Principle of Random Forests
			5.1.3 OOB Error
		5.2 Variable Importance Measures and Marginal Effects
			5.2.1 Permutation Variable Importance
			5.2.2 Group Variable Importance
			5.2.3 Marginal Effects
		5.3 Tuning Meta-Parameters
			5.3.1 Tuning for Prediction
			5.3.2 Tuning for Computing VI
		5.4 Theoretical Results
		5.5 A Variant Adapted to Time Series
		5.6 Electricity Data Modeling Using Random Forests
			5.6.1 CART Trees
				5.6.1.1 Nested Sequence of Pruned Subtrees
				5.6.1.2 Optimal and Suboptimal CART Trees
			5.6.2 Random Forests
	6 Aggregation of Experts
		6.1 Introduction
		6.2 Online Forecasting of Arbitrary Sequence with a Set of Experts
		6.3 The Notion of Regret
		6.4 Aggregation with Exponential Weights
		6.5 Gradient Trick
		6.6 Aggregation with Adaptive Learning Rates
		6.7 Specialized Experts
		6.8 Nonconvex Aggregation
			6.8.1 Ridge
			6.8.2 Tricks
				6.8.2.1 Constant Bias
				6.8.2.2 Random Walk
		6.9 Dealing with Breaks
			6.9.1 Shifting Oracle
			6.9.2 Fixed Share
	7 Mixed Effects Models for Electricity Load Forecasting
		7.1 Introduction
		7.2 The Standard Linear Mixed Effects Model
			7.2.1 Classical Linear Model
			7.2.2 Random Effects
			7.2.3 A Simple Example of a LME Model
		7.3 Stochastic Linear Mixed Models for Longitudinal Data
		7.4 Regression Trees for Mixed Effects Longitudinal Data
			7.4.1 The RE-EM Tree Algorithm
		7.5 Functional Mixed Effects Models
			7.5.1 A Penalized Spline Approach to Functional Mixed Effects Model Analysis
		7.6 Predicting Time Series of Electricity Consumption
Part II Case Studies: Models in Action on Specific Applications
	Case Studies Organization
	8 Disaggregated Forecasting of the Total Consumption of a Given Subset of Customers
		8.1 Data
			8.1.1 Original Data Set
			8.1.2 Other Data Sets
		8.2 Problems
		8.3 Modeling and Results
			8.3.1 From Individual Curves to a Hierarchy of Partitions for Forecasting
			8.3.2 Numerical Experiments
		8.4 Validation
		8.5 Interpretation
		8.6 Complements and Discussion
			8.6.1 Upscaling
			8.6.2 Discussion
	9 Aggregation of Multiscale Experts for Bottom-Up Load Forecasting
		9.1 Data
		9.2 Problem
		9.3 Methods
		9.4 Numerical Results
		9.5 Discussions
	10 Short-Term Electricity Load Forecasting for Fine-Grained Data with PLAM
		10.1 Data
			10.1.1 Data Transformation
			10.1.2 Generation of Aggregates
		10.2 Problem
		10.3 Modelling
			10.3.1 Estimation and Model Selection in PLAMs
		10.4 Analysis and Results
		10.5 Discussion and Conclusion
	11 Functional State Space Models
		11.1 Data
		11.2 Problems
		11.3 Modelling
		11.4 Model Construction
		11.5 Prediction Performances
		11.6 Supplements and Discussion
	12 Forecasting Daily Peak Demand Using GAMs
		12.1 Forecasting Problem
		12.2 Modelling
			12.2.1 A High-Resolution Approach
			12.2.2 A Multiresolution Approach
		12.3 Results on GEFCom 2014 Data
		12.4 Conclusion
	13 Forecasting During the Lockdown Period
		13.1 Data
		13.2 Problem
		13.3 Methods
			13.3.1 GAM and Adaptive GAM
			13.3.2 RF and Adaptive RF
			13.3.3 Stacking GAM and RF
			13.3.4 Aggregation Algorithms
		13.4 Numerical Results
		13.5 Discussions
References