AI Time Series Control System Modelling

243 Pages·2022·8.723 MB·English

by  Chuzo Ninagawa

Chuzo Ninagawa
AI Time
Series Control
System
Modelling
AI Time Series Control System Modelling
Chuzo Ninagawa
AI Time Series Control
System Modelling
Chuzo Ninagawa
Smart Grid Power Control Engineering
Joint Research Laboratory
Gifu University
Gifu, Japan
ISBN 978-981-19-4593-9 ISBN 978-981-19-4594-6 (eBook)
https://doi.org/10.1007/978-981-19-4594-6
© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature
Singapore Pte Ltd. 2023
This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether
the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse
of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and
transmission or information storage and retrieval, electronic adaptation, computer software, or by similar
or dissimilar methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors, and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or
the editors give a warranty, expressed or implied, with respect to the material contained herein or for any
errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.
This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd.
The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721,
Singapore
Preface
The Internet of Things (IoT) and artificial intelligence (AI) are without a doubt the
most important technology topics of the near future. As the world undergoes Digital
Transformation (DX), the Internet data collection through IoT is becoming the norm,
and an era of massive time series data accumulation is about to begin. Furthermore,
modeling technology will no longer be able to keep up with manually in extracting
relevant information from the large ocean of time series data. AI modeling will be
an inevitable core technology of the DX era.
Most image recognition, which is a representative of AI technology, can be said
to be static modeling that does not depend on past history, but the time series data
accumulated by DX can be said to be dynamic modeling in which the appearance of
values changes with past history. Long short-term memory (LSTM), a neural network
specialized for time series data, has been attracting attention as a neural network that
is good at prediction depending on the history of time series data, and its tools are
now readily available.
Since time series AI modeling is a sophisticated predictor that deals with history,
it is difficult for beginners to understand the learning theory and effective training
data collection techniques. For example, even if you study the theory and collect time
series data to test it in your own work, you will face problems such as obtaining only
biased training data or not being able to determine realistic convergence conditions.
However, there seem to be two extremes in the world: theoretical books with
only mathematical formulas, or how-to books on tools. For graduate students in
university laboratories and front-line engineers in the industrial world, there is a
need for specialized books that serve as a “bridge” by describing the development
of practical models based on theory.
In each chapter of this book, a structure is adopted that has never been seen before:
a section that presents the basic theory in mathematical form, followed by a section
that presents practical applications of the theory. In other words, the emphasis is on
showing concrete examples of the application of the basic algorithms in the field of
system control immediately after understanding them mathematically. By doing so,
the author aimed to take a different approach from mathematical books that develop
theories in an abstract manner by deriving pure mathematical formulas and from
v
vi Preface
how-to books that only describe how to input and output data to off-the-shelf tools
without describing theories.
In general, examples of AI machine learning may appear to be handled well,
but there may be cases where the reliability is not guaranteed. Therefore, all the
examples in this book were selected from peer-reviewed papers by IEEJ, IEEE, and
other experts to ensure reliability.
The structure of this book is as follows.
Chapter 1 begins with the definition of “time series” in system control and
describes the position of control design modeling from time series data of physical
quantities of interest.
Chapter 2 presents the basic theory of linear multiple regression modeling and
autoregressive (AR) modeling as the most fundamental methods in time series data
modeling and their practical applications.
Chapter 3 describes the basic theory of neural network modeling of dynamic
characteristics of control targets as a representative of AI machine learning modeling
with time series data and its application to modeling of step response characteristics.
Chapter 4 presents the theory of long short-term memory (LSTM) neural
networks, which have attracted attention in recent years as a method for modeling
control targets whose subsequent behavior differs depending on the time series
history, and an example of a prediction model for sudden events in control as an
application of the theory.
Chapter 5 describes the theory and examples of heuristic optimal search control
as optimal control using the above time series AI model.
Chapter 6 describes a practical method for collecting time series data in the field
of system control design, including a method for correcting collected data bias and
a method for estimating training data from normal operation data.
Chapter 7 describes a practical method for implementing the methods described
in the above chapters: a time series data collection platform, a method for extracting
zones of interest from field-collected data, and self-developed machine learning
software.
I would like to express my gratitude to many people for their help in compiling
this book. Morio Takahama, former professor at Nagoya University, as an expert in
control engineering, and Satoru Hayamizu, former professor at Gifu University, as
an expert in AI, provided valuable comments on the manuscript. Former Assistant
Professor Shun Matsukawa of Ninagawa Laboratory, Gifu University, and current
Lecturer at Hokkaido University of Science, helped to confirm the mathematical
expressions. Of course, I am grateful to Assistant Professor Yoshifumi Aoki, Ph.D.
student Asif Iqbal, and other members of the Ninagawa Laboratory at Gifu University
for their research. I would like to express my gratitude to all of them. Finally, I would
like to thank my wife for allowing me to write this book at home for a long time.
Gifu, Japan Chuzo Ninagawa
Contents
1 Introduction ................................................... 1
1.1 Time Series ................................................ 1
1.1.1 What is “Time Series” Dealt in This Book? .............. 1
1.1.2 Time Series for Statistical Control ...................... 2
1.1.3 Dissemination of Time Series Data for Control ........... 3
1.2 Time Series and Control Models .............................. 4
1.2.1 Control Modeling .................................... 4
1.2.2 Control Model Building Methods ...................... 5
1.3 Control Time Series and AI Methods .......................... 6
1.3.1 Control Model by Time Series Analysis ................. 6
1.3.2 Control and AI Methods .............................. 7
References ..................................................... 8
2 Linear Time Series Modeling .................................... 9
2.1 Linear Regression Models ................................... 9
2.1.1 One-Dimensional Linear Regression Model .............. 9
2.1.2 Multi-Dimensional Linear Regression Model ............ 12
2.2 Fundamentals of AR Models ................................. 14
2.2.1 Overview of the AR Model ............................ 14
2.2.2 Yule-Walker Method (One Variable) .................... 16
2.2.3 Yule-Walker Method (Multivariate) ..................... 18
2.3 Practical Example 1: Multiple Regression Model with Stable
Interval ................................................... 21
2.3.1 Air-Conditioning Stable Power Model .................. 21
2.3.2 Selection of Explanatory Variables ..................... 23
2.3.3 Linear Multiple Regression Analysis .................... 28
2.3.4 Model Evaluation and Validation ....................... 29
vii
viii Contents
2.4 Practical Example 2: Step Response AR Model ................. 31
2.4.1 Limited Control of Building Air-Conditioning Power ...... 31
2.4.2 Fitting the AR Mathematical Model .................... 33
2.4.3 Model Identification from Measured Data ............... 35
2.4.4 AR Model Identification Results ....................... 37
References ..................................................... 39
3 Deep Learning AI Modeling ..................................... 41
3.1 Fundamentals of Deep Learning .............................. 41
3.1.1 Fundamentals of Neural Networks ...................... 41
3.1.2 Principles of Deep Learning ........................... 43
3.1.3 Stacked Denoising Autoencoder Method ................ 45
3.2 Time Series Data Deep Learning .............................. 47
3.2.1 Time Series Parallel Input Neural Network .............. 47
3.2.2 Number of Layers for Time Series Deep Learning ........ 48
3.2.3 Hyperparameters for Time Series Deep Learning ......... 51
3.3 Practical Example 3: Step Response AR Neural Network ......... 54
3.3.1 Step Response AR Neural Network ..................... 54
3.3.2 Training a Step Response Time Series Model ............ 56
3.3.3 Evaluation of Step Response Time Series Models ......... 59
3.4 Practical Example 4: Deep Learning in Practice—Sudden
Event Prediction Model ..................................... 60
3.4.1 Examples of Sudden Events ........................... 60
3.4.2 Sudden Event Prediction Neural Network Model ......... 61
3.4.3 Training a Neural Network Model for Sudden Event
Prediction ........................................... 63
References ..................................................... 65
4 LSTM AI Modeling ............................................ 67
4.1 Fundamentals of LSTM Neural Networks ...................... 67
4.1.1 What is LSTM Neural Network? ....................... 67
4.1.2 LSTM Forward Propagation Calculation ................ 70
4.1.3 LSTM Back Propagation Calculation ................... 71
4.2 Performance Evaluation Methods for LSTM Time Series
Models ................................................... 73
4.2.1 LSTM Model of Rare and Unexpected Events ............ 73
4.2.2 Predictive Performance Evaluation Method .............. 76
4.2.3 Results of Predictive Performance Evaluation ............ 76
4.3 Practical Example 5: Electricity Wholesale Market LSTM
Model .................................................... 79
4.3.1 Prediction of Wholesale Electricity Prices ............... 79
4.3.2 Electricity Wholesale Price LSTM Forecasting Model ..... 81
4.3.3 Evaluation of Wholesale Electricity Price LSTM
Forecasting Model ................................... 82
Contents ix
4.4 Practical Example 6: LSTM Model for Prediction
of Disturbance Events ....................................... 84
4.4.1 Example of a Time Series Unexpected Event ............. 84
4.4.2 Facility Maintenance Operation as a Disturbance
for RTP Adaptive Control ............................. 86
4.4.3 Disturbance Prediction LSTM Model for RTP
Adaptive Control ..................................... 87
4.4.4 Evaluation of Disturbance Predictive LSTM Model
for RTP Adaptive Control ............................. 88
References ..................................................... 90
5 Optimal Control Using Time Series AI Models .................... 91
5.1 Fundamentals of Optimal Search and Control ................... 91
5.1.1 SA Optimal Search Method ........................... 91
5.1.2 Principle of Simulated Annealing (SA) Optimal
Search Algorithm .................................... 92
5.1.3 Example of Evaluation Function for SA Optimal
Search Control ...................................... 96
5.2 State Explosion and Parallel Search ........................... 97
5.2.1 Large-Scale Control Target State Space ................. 97
5.2.2 Parallel SA Search Algorithm .......................... 99
5.2.3 Trials of Large-Scale Parallel Search .................... 100
5.3 Practical Example 7: Electricity Price Optimal Search Control .... 105
5.3.1 Real-Time Electricity Pricing .......................... 105
5.3.2 Optimal Control of Air-Conditioning Power Rates ........ 107
5.3.3 Actual Equipment Tests of Optimal Search Control ....... 110
5.4 Practical Example 8: Practical Cessation of Large-Scale
Search .................................................... 114
5.4.1 Practicality of Optimal Search Control .................. 114
5.4.2 Censoring of Large-Scale Search ....................... 115
References ..................................................... 119
6 Reality of Time Series Learning Data Collection .................. 121
6.1 Practical Example 9: Generating Training Data
with Pseudo-step Response .................................. 121
6.1.1 Step Response Training Data .......................... 121
6.1.2 Break-Point Step Response Extraction Method ........... 123
6.1.3 Example of Break-Point Method Training Data
Collection ........................................... 127
6.2 Practical Example 10: Artificial Augmentation of Training
Data Collection ............................................ 130
6.2.1 Reality of Training Data Collection ..................... 130
6.2.2 Artificial Augmentation of Training Data ................ 131
6.2.3 Practice of Artificial Training Data Augmentation ........ 133
x Contents
6.3 Example 11: Generating Training Data with Emulators .......... 136
6.3.1 Baseline and Reproducibility .......................... 136
6.3.2 Baseline Emulator Training ............................ 141
6.3.3 Baseline Estimation Model Evaluation .................. 143
References ..................................................... 145
7 Practical Work on Time Series AI Modeling ...................... 147
7.1 IoT Time Series Data Collection Methods ...................... 147
7.1.1 IEEE1888 Standard for Time Series Data Collection ...... 147
7.1.2 IEEE1888 Time Series Data Transmission Method ........ 149
7.1.3 IEEE1888 Standard IoT Communication Software
Implementation ...................................... 151
7.2 Zone Selection for Time Series AI Training Data ................ 154
7.2.1 Reality of Time Series Training Data Collection .......... 154
7.2.2 Zone Selection of Time Series Training Data ............. 156
7.2.3 Practical Methods with Time Series Data Selection ....... 159
7.3 Self-developed Software for Time Series AI Modeling ........... 161
7.3.1 Off-the-Shelf Training Tools ........................... 161
7.3.2 Self-developing Machine Learning Software ............. 162
7.3.3 Visualization with Self-developed Machine Learning
Software ............................................ 163
References ..................................................... 166
8 Example Source Code of MLP Deep Learning Algorithm .......... 169
8.1 Execution Environment ..................................... 169
8.2 Example of MLP Code ...................................... 170
9 Example Source Code of LSTM Neural Network Learning
Algorithm ..................................................... 201
9.1 Execution Environment ..................................... 201
9.2 Baseline Estimation LSTM Time Series Learning Code
Example .................................................. 202