Lifelong Machine Learning [2nd ed.]

Contents......Page 4
Preface......Page 9
Classic Machine Learning Paradigm......Page 11
Motivating Examples......Page 13
Brief History of Lifelong Learning......Page 16
Definition of Lifelong Learning......Page 19
Types of Knowledge & Key Challenges......Page 24
Evaluation Methodology a&nd Role of Big Data......Page 26
Outline of the Book......Page 28
Transfer Learning......Page 30
Naïve Bayes Transfer Classifier......Page 31
Deep Learning in Transfer Learning......Page 33
Multi-Task Learning......Page 34
Task Relatedness in Multi-Task Learning......Page 35
GO-MTL: Multi-Task Learning using Latent Basis......Page 36
Deep Learning in Multi-Task Learning......Page 38
Difference from Lifelong Learning......Page 39
Difference from Lifelong Learning......Page 40
Reinforcement Learning......Page 41
Meta Learning......Page 42
Summary......Page 43
Lifelong Supervised Learning......Page 44
Definition and Overview......Page 45
Learning a New Representation for Lifelong Learning......Page 46
MTL Net......Page 47
Lifelong EBNN......Page 48
ELLA: An Efficient Lifelong Learning Algorithm......Page 49
Objective Function......Page 50
Dealing with the First Inefficiency......Page 51
Dealing with the Second Inefficiency......Page 53
Active Task Selection......Page 54
Naïve Bayesian Text Classification......Page 55
Basic Ideas of LSC......Page 57
LSC Technique......Page 58
Discussions......Page 59
Domain Word Embedding via Meta-Learning......Page 60
Summary and Evaluation Datasets......Page 62
Catastrophic Forgetting......Page 64
Continual Learning in Neural Networks......Page 66
Learning without Forgetting......Page 68
Progressive Neural Networks......Page 70
Elastic Weight Consolidation......Page 71
Incremental Training......Page 73
Updating Representation......Page 74
Constructing Exemplar Sets for New Classes......Page 75
Expert Gate......Page 76
Autoencoder Gate......Page 77
Selecting the Most Relevant Expert for Testing......Page 78
Generative Adversarial Networks......Page 79
Generative Replay......Page 80
Evaluating Catastrophic Forgetting......Page 81
Summary and Evaluation Datasets......Page 82
Problem Definition and Applications......Page 85
Incrementally Updating a CBS Learning Model......Page 87
Testing a CBS Learning Model......Page 89
CBS Learning for Unseen Class Detection......Page 90
DOC: Deep Open Classification......Page 92
Feed-Forward Layers and the 1-vs.-Rest Layer......Page 93
Reducing Open-Space Risk......Page 94
DOC for Image Classification......Page 95
Unseen Class Discovery......Page 96
Summary and Evaluation Datasets......Page 97
Main Ideas of Lifelong Topic Modeling......Page 98
LTM: A Lifelong Topic Model......Page 101
LTM Model......Page 102
Topic Knowledge Mining......Page 103
Incorporating Past Knowledge......Page 104
Conditional Distribution of Gibbs Sampler......Page 106
Overall Algorithm of AMC......Page 107
Mining Must-link Knowledge......Page 108
Mining Cannot-link Knowledge......Page 110
Extended Pólya Urn Model......Page 111
Sampling Distributions in Gibbs Sampler......Page 113
Summary and Evaluation Datasets......Page 115
NELL: A Never-Ending Language Learner......Page 117
Extractors and Learning in NELL......Page 120
Lifelong Opinion Target Extraction......Page 123
Lifelong Learning through Recommendation......Page 124
AER Algorithm......Page 125
Knowledge Learning......Page 126
Recommendation using Past Knowledge......Page 127
Conditional Random Fields......Page 129
General Dependency Feature......Page 130
The L-CRF Algorithm......Page 131
Relaxation Labeling......Page 133
Lifelong Relaxation Labeling......Page 134
Summary and Evaluation Datasets......Page 135
Continuous Knowledge Learning in Chatbots......Page 136
LiLi: Lifelong Interactive Learning and Inference......Page 137
Basic Ideas of LiLi......Page 139
Components of LiLi......Page 141
A Running Example......Page 142
Summary and Evaluation Datasets......Page 143
Lifelong Reinforcement Learning......Page 144
Lifelong Reinforcement Learning through Multiple Environments......Page 145
Acquiring and Incorporating Bias......Page 146
Hierarchical Bayesian Approach......Page 147
MTRL Algorithm......Page 148
Updating Hierarchical Model Parameters......Page 149
PG-ELLA: Lifelong Policy Gradient Reinforcement Learning......Page 151
Policy Gradient Reinforcement Learning......Page 152
Policy Gradient Lifelong Learning Setting......Page 153
Objective Function and Optimization......Page 154
Safe Policy Search for Lifelong Learning......Page 155
Cross-domain Lifelong Reinforcement Learning......Page 156
Summary and Evaluation Datasets......Page 157
Conclusion & Future Directions......Page 158
Biblio......Page 163