CSEE JOURNAL OF POWER AND ENERGY SYSTEMS, VOL. 7, NO.

2, MARCH 2021 209

Deep Learning in Power Systems Research:

A Review
Mahdi Khodayar, Student Member, IEEE, Guangyi Liu, Senior Member, IEEE, Fellow, CSEE, Jianhui Wang,
Fellow, IEEE, and Mohammad E. Khodayar, Senior Member, IEEE

Abstract—With the rapid growth of power systems measure- neighbor embedding (t-SNE) [4]. Such feature extraction tech-
ments in terms of size and complexity, discovering statistical niques dramatically increase the time and memory complexity
patterns for a large variety of real-world applications such as of data-driven algorithms and lead to insufficient accuracy
renewable energy prediction, demand response, energy disaggre-
gation, and state estimation is considered a crucial challenge. as they mainly cannot capture highly nonlinear and highly
In recent years, deep learning has emerged as a novel class of varying patterns inside the ambient space of the data [1].
machine learning algorithms that represents power systems data Recent machine learning studies on wind forecasting [5]–
via a large hypothesis space that leads to the state-of-the-art [8], photovoltaic (PV) power prediction [9]–[12], state es-
performance compared to most recent data-driven algorithms. timation [13], [14], power grid synthesis [15], and energy
This study explores the theoretical advantages of deep repre-
sentation learning in power systems research. We review deep disaggregation [16]–[18] show that developing data-driven
learning methodologies presented and applied in a wide range models with less dependencies on explicit preprocessing meth-
of supervised, unsupervised, and semi-supervised applications as ods (e.g., PCA) leads to dramatically better performance in
well as reinforcement learning tasks. We discuss various settings terms of classification and regression accuracy. Instead of
of problems solved by discriminative deep models including having an explicit preprocessing approach, the deep learning
stacked autoencoders and convolutional neural networks as well
as generative deep architectures such as deep belief networks studies form a composition of multiple nonlinear latent layers
and variational autoencoders. The theoretical and experimental in a multi-layer artificial neural network (ANN). The ANN
analysis of deep neural networks in this study motivates long- parameters (i.e., weights and biases) are generally trained in a
term research on optimizing this cutting-edge class of models greedy unsupervised layer-by-layer fashion [19], where each
to achieve significant improvements in the future power systems layer performs a nonlinear feature extraction on the features
research.
computed by its previous layer.
Index Terms—Autoencoder, convolution neural network, deep Based on the theoretical aspects, deep learning algorithms
learning, discriminative model, deep belief network, generative proposed in power engineering applications are generally
architecture, variational inference. categorized into three major classes:
1) Discriminative deep ANNs aim to directly learn a highly
nonlinear decision boundary between different classes and
regression regions of the power system data [20]–[22]. In
I. I NTRODUCTION this category, the rectified linear unit (ReLU) ANN is pre-
sented for real-time reliability management response [23],

T HE reliability and accuracy of data-driven models in

power systems operation and analysis closely rely on the
selection of data representation (i.e., features extracted from
online small signal stability assessment [24], and faulted line
localization [25]. Moreover, the stacked autoencoder (SAE) is
developed as a highly nonlinear version of the PCA for unsu-
the underlying data) [1]. As a result, most of the concerns re- pervised pattern recognition for wind energy prediction [7],
garding the application of classic data-driven models in power [26], PV power forecasting [27], and fault diagnosis [28].
systems is focused on the design of preprocessing techniques In addition, the long short-term memory (LSTM) ANN is
using unsupervised dimensionality reduction algorithms in- presented as a supervised temporal feature extractor with a
cluding the principal component analysis (PCA) [2], linear deep recurrent formulation to model the sequential behavior
discriminant analysis (LDA) [3], and t-distributed stochastic of the time-dependent power systems measurements [17], [29].
Convolutional neural network (CNN) is another major class
Manuscript received June 18, 2020; revised August 24, 2020; accepted
October 15, 2020. Date of online publication November 20, 2020; date of of discriminative models that captures coherent structures in
current version January 28, 2021. This work was supported by the Science and power system measurements using convolutional and pooling
Technology Project of State Grid Corporation of China (No. 5455HJ180018). operations [30]. The mixture of these operations incorporates
M. Khodayar is with Global Energy Interconnection Research Institute
North America (GEIRI North America), San Jose, CA, USA; and the the spatial charactristics of measurements into their temporal
Department of Computer Science at the University of Tulsa, Tulsa, OK, USA. features to solve spatiotemporal tasks in the area of renewable
G. Y. Liu (corresponding author, email: [email protected]) energy forecasting [9], transient stability analysis [31], and
is with Envision Digital, Redwood City, CA, USA.
J. H. Wang and M. E. Khodayar are with the Department of Electrical and fault detection [32].
Computer Engineering of Southern Methodist University, Dallas, TX, USA. 2) Probabilistic deep ANNs consider feature learning as a
DOI: 10.17775/CSEEJPES.2020.02700 procedure to find a parsimonious set of hidden variables that
2096-0042 © 2020 CSEE
210 CSEE JOURNAL OF POWER AND ENERGY SYSTEMS, VOL. 7, NO. 2, MARCH 2021

best describe the probability density function (PDF) of the Gaussian variation as well as the recently proposed GANs
data. The PDF is further mapped to the target class/value of the and VAEs. The applications and theoretical advantages of
problem. Deep belief network (DBN) is a well-known proba- these techniques are discussed in this section. Then, in Section
bilistic graphical model that learns the PDF of the data given IV, the paper reviews DRL algorithms and their vast area
its conditionally independent latent features. The features are of applications in power systems optimization and control.
learned by Gibbs sampling in order to provide an accurate Finally, the conclusions are provided in Section V.
estimation of the probabilistic behavior of the input data for
probabilistic applications that need to address large uncertainty II. D ISCRIMINATIVE D EEP L EARNING
factors in the data. DBN is mainly applied to wind and solar
power prediction [33], [34], transient stability assessment [35], Discriminative modeling is one of the major areas in
day-ahead and week-ahead load prediction [36], as well as machine learning that tends to estimate a function fθ pa-
probabilistic state estimation [37]. Moreover, the Generative rameterized by θ ∈ Rp that directly maps an input to
Adversarial Network (GAN) is presented that compares its the true output of the problem. Let us consider a training
generated data samples with the actual dataset to increase the dataset Dtr = {(x1 , y1 ), (x2 , y2 ), . . . , (xn , yn )} that contains
accuracy of its learned PDF. As this model efficiently learns n training samples (xi , yi ) with input xi corresponding to
the major characteristics of the PDF, it is recently introduced the true output/label yi , and a test dataset Dts = {(xn+1 ,
to important outlier and fault detection problems for small- yn+1 ), (xn+2 , yn+2 ), . . . , (xn+m , yn+m )} with m unobserved
sample wind turbines [38] and smart grid cyber attack de- test samples. The goal is to learn the optimal parameter θ∗
tection [39]. Furthermore, GANs are recently employed for where the average distance between fθ∗ (x) and y is the lowest
model-free renewable scenario generation [40]. The variational for all samples (x, y) ∈ Dtr . The test error is the average error
autoencoders (VAEs) are presented as a novel version of deep between the trained fθ∗ (x) and y for all (x, y) ∈ Dts .
generative ANNs that learn the PDF of the data by learning To obtain a nonlinear mapping between the inputs and out-
a high dimensional latent variable which is mapped to the puts, the classic multilayer perceptron (MLP) defines an input
original data samples. VAE is shown to create accurate data layer h0 ∈ Rd0 and L computational layers {h1 , h2 , . . . , hL }
samples used for power grid synthesis [15], unsupervised where each layer hi ∈ Rdi (i ∈ [1, L]) is a nonlinear function
anomaly detection in energy time series [41], [42], and electric of previous layer defined by hi = g i (W i hi−1 + bi ) where g i
vehicle load generation [43]. is a nonlinear transformation function usually computed by a
3) Deep Reinforcement Learning (DRL) algorithms are a sigmoid or hyperbolic tangent function, W i ∈ Rdi × Rdi−1 is
major class of machine learning approaches that seek to learn the weight matrix and bi ∈ Rdi is the bias of the activation
an optimal policy based on the feedback from the environment function in layer hi . Using the hidden layers, the MLP
computed by a reward function. This function reflects how provides a nonlinear transformation between the input h0 = x
much the problem’s objective is satisfied based on the current and output hL = y in the dataset.
state of the system. In contrast to the conventional deep To train each layer hi , the gradient descent (GD) method
learning that merely estimates a discrete target function for moves parameters W i and bi in the opposite direction of
classification and continuous target function for regression, the gradient of the training error with respect to W i and
DRL aims to decline a general error function defined by bi , respectively. As the gradients dramatically decline with
the experience in a fully observable or partially observable the increase in L, there is a trade-off between the number
environment. Hence, this method solves more general classes of computational layers L and the strength of GD to update
of problems compared to the classic deep learning. Due to its the model. As L becomes larger to address more complex
feedback-based nature, DRL is widely employed for control problems, GD becomes ineffective due to the vanishing gra-
problems including voltage control [44], adaptive emergency dients. Hence, the classic MLP does not provide sufficient
control [45], as well as self-learning control for energy effi- generalization capability to accurately solve complex real-
cient transportation [46]. Also, DRL is applied to optimiza- world problems. As a result, discriminative deep learning is
tion problems for learning the optimal bidding strategies in proposed to efficiently train deep ANNs with L > 1 in order to
electricity markets [47], [48], demand response strategies for have a high capacity mapping fθ while providing an effective
energy management [49]–[51], as well as finding the optimal training procedure to update the parameters.
wind and storage cooperative schedule to decrease the effect
of the uncertainty in renewable generation in smart grids [52]. A. Rectified Linear Unit ANN
Moreover, this class of methodologies are recently introduced ReLU ANN defines a rectified linear unit activation function
to cyber attack detection and recovery [53], dynamic power ReLU (x) = max(0, x) at the computational layers of MLP
allocation [54], and power system data integrity defense [55]. rather than using the classic nonlinear activation functions.
This paper reviews the three major categories of deep neural Since the gradient of ReLU (x) with respect to a positive input
networks in the domain of power systems research. First, x is always 1 regardless of x, this function solves the vanishing
the deep discriminative approach is introduced in Section II. gradient problem of the MLP. Hence, this model is applied
Various variations of this class of models are explained, and to power systems applications that require highly nonlinear
compared both mathematically and experimentally using sev- feature extraction.
eral real-world power system datasets. Section III introduces Table I summarizes the applications of discriminative mod-
probabilistic deep learning methods such as DBN and its eling in the power systems area. As shown in this table, a
KHODAYAR et al.: DEEP LEARNING IN POWER SYSTEMS RESEARCH: A REVIEW 211

TABLE I
D ISCRIMINATIVE D EEP L EARNING IN P OWER S YSTEMS A PPLICATIONS
Applications Dataset Model Performance Metric Result
Reliability Management ReLU Coefficient of determination 0.964
IEEE-RTS96
Response [23] SAE (R2 Score) 0.951
Stability Assessment ReLU 94.1%
[24], [56], [30], IEEE 39-bus SAE Classification Accuracy 92.6%
[31], [57] CNN 97.8%
Fault Detection ReLU Detection Accuracy, 93.20%, 91.12%
[24], [28], IEEE 39-bus SAE Location Accuracy 94.18%, 91.71%
[58], [32] CNN Rate 96.09%, 94.31%
ReLU 94.11%
PMU Event 16-machine 68-bus SAE Classification 95.07%
Classification [59] Test System LSTM Accuracy 96.34%
CNN 98.17%
Hourly Wind ReLU 1,38%, 1.74%
Power Prediction SAE 1.24%, 1.68%
Western Wind Dataset RMSE, MAPE
[7], [26], LSTM 1.13%, 1.53%
[29], [60] CNN 1.07%, 1.26%
ReLU 1.29%, 1.54%
Hourly PV Power
SAE 1.09%, 1.37%
Prediction [9], National Solar Radiation Database RMSE, MAPE
LSTM 0.97%, 1.10%
[27], [29]
CNN 0.85%, 0.92%
ReLU 0.0435, 0.0120
Load Modeling [61] 16-machine 68-bus Test System RMSE, MAPE
LSTM 0.008, 0.0071
ReLU 0.069
Hourly Load Forecasting [62] Industrial Power Demand Dataset SAE Normalized RMSE 0.051
LSTM 0.032
Power Fluctuation ReLU 0.042, 107.91%
Market Trading Reports MAE, MAPE
Identification [63] LSTM 0.038, 105.72%
Energy Disaggregation SAE Precision, Recall, F-score 84.63%, 61.04%, 70.62%
Reference Energy Disaggregation Dataset
[16], [17] LSTM 89.83%, 65.72%, 75.93%

ReLU ANN is implemented in [23] to estimate the cost of W i and bi of the layer i, respectively. Finally, the whole SAE
real-time resource allocations decisions in operation planning ANN is trained using GD on the training data Dtr .
of the modified IEEE-RTS96 single area network [64]. Also, Due to the unsupervised feature learning at each AE,
in [24], various ReLU ANN architectures are trained to learn the SAE model is suitable for situations where the training
the small signal stability assessment of the classic 16-machine data is limited or contains remarkable uncertainty and noise
68-bus test system [65]. As shown in [24], when the number factors. Hence, this method respectively outperforms the MLP,
of layers increases from 2 to 6, the assessment accuracy nonlinear autoregressive exogenous (NARX) ANN, and time
is significantly increased since the ReLU ANN’s hypothesis delay ANN (TDANN) by 23.66%, 21.54%, and 14.81% in
space becomes larger. In addition, the ReLU ANN is applied terms of the mean absolute percentage error (MAPE) for short-
to real-time faulted line localization in IEEE 39-bus and 68- term wind speed prediction [7], [26]. Moreover, as shown in
bus power systems which resulted in 98% and 93% location Table I, the SAE outperforms ReLU in both classification
accuracy rate for line-to-ground and double line-to-ground tasks (e.g., stability assessment [24] and PMU event clas-
faults, respectively. Furthermore, in [59], ReLU ANNs are sification [59]) as well as regression tasks with large data
shown to yield 98.17% accuracy for the classification of 6 variations (e.g., wind and PV power prediction [26], [27]
events including generation loss, load loss, as well as line-to- and load forecasting [62]). Furthermore, due to its powerful
ground faults in the IEEE 68-bus system. greedy layer-wise training process, the SAE yields an average
transformer fault diagnosis accuracy of 95.4% in the IEC
B. Stacked Autoencoder 60599 and IEC TC 10 databases [66]. In addition, SAE
To train a deep ANN with input h0 and L computational improves the transient stability analysis accuracy of extreme
layers hi (i = 1, 2, . . . , L), the SAE trains L AEs {AE i }L learning machines (ELMs) by 6.59% in the IEEE 39-bus
i=1 .
Each AE i is a MLP ANN with one hidden layer with an system [56].
encoding activation function fenc where a high-dimensional
input hi−1 ∈ Rdi−1 is encoded into a lower dimensional latent C. Long Short-term Memory Network
feature vector hi = fenc (hi−1 ) ∈ Rdi which is further mapped
back (decoded) to the original input hi−1 in the output layer LSTM is a widely used deep recurrent ANN that extracts
oi = fdec (hi ) using the decoding function fdec . Hence, the GD powerful temporal features from a time series x1 , x2 , . . . , xT .
error of AE i is computed by koi − hi−1 k22 to train the weight At each time step, 0 6 t 6 T , LSTM observes a sample xt
i
Wenc and bias bienc of its encoding layer as well as the weight and updates its temporal memory C t that describes the state
Wdec and bias bidec of its decoder. To update the parameters
i
of the time series at t, and produces a temporal feature vector
of the SAE, starting from i = 1, each AE i is trained and the ht that summarizes LSTM’s temporal information after the
i
trained encoder parameters Wenc and bienc are used to initialize observation xt . The recursive structure of LSTM features is
212 CSEE JOURNAL OF POWER AND ENERGY SYSTEMS, VOL. 7, NO. 2, MARCH 2021

defined by: l. The pooling layer sweeps an average or maximum function

 over small patches of the convolution output in (2) to further
it = σ(Wi .[ht−1 , xt ] + bi )


 reduce the dimension of the extracted features which enhances
ft = σ(Wf .[ht−1 , xt ] + bf )




 the sparsity of the kernel parameters and avoids overfitting
ot = σ(Wo .[ht−1 , xt ] + bo ) on the training set. Finally, the fully connected layer maps
(1)

C̃t = tanh(WC .[ht−1 , xt ] + bC ) the extracted features to the target label of the underlying

Ct = ft ∗ Ct−1 + it ∗ C̃t

 classification or regression task.

 As the convolution and pooling layers process their local
ht = ot ∗ tanh(Ct )

input patches simultaneously, the CNN yields the state-of-the-
where it is the input gate that decides the magnitude of art performance in tasks where the local spatial and temporal
information flow into the time-dependent memory Ct using correlations of the data play a crucial role. Therefore, this
the sigmoid activation σ with weight Wi and bias bi . ft is the model outperforms ReLU ANNs as well as SAE and LSTM
forget gate that determines how much information needs to in applications where the data has a strong spatiotemporal
be removed from Ct using weight Wf and bias bf . ot is the structure such as the wind and PV power prediction [26],
LSTM’s output at time t using weight Wo and bias bo while [27] as well as PMU event classification [59]. In [9], this
ht is the extracted tempoal feature at time t. At each time model is applied to 6-hr ahead spatiotemporal solar irradiance
step t, the memory is updated by C̃t as a nonlinear function prediction which obtains 21.62% and 16.78% better RMSE
parameterized by WC and bC . and MAPE, respectively, compared to the LSTM due to
In contrast to the classic recurrent MLPs, the LSTM does modeling the correlation between the radiation at neighboring
not encounter the vanishing gradient problem; hence, can solar sites by the convolution operation in (2). In addition,
be efficiently trained using GD. As a result, as shown in in [31], CNN is applied to the transient stability assessment
Table I, this method is applied to a large variety of time- of the IEEE 39-bus system. In a short period of time after a
dependent applications such as wind, PV, and load predic- disturbance, the bus voltage phasors sampled from PMUs from
tion [26], [27], [36] as well as load modeling [61] and power various points of the system are given to the CNN to judge if
fluctuation identification [63]. As Table I shows, the LSTM the system is stable, aperiodic unstable or oscillatory unstable.
generally outperforms both ReLU and SAE in the domain CNN’s classification accuracy is 98.7% while recent variations
of time-dependent applications due to its recurrent structure of support vector machines and decision trees lead to 95.2%
and powerful temporal memory. In [29], a novel attention and 92.1% accuracies. Furthermore, CNN is shown to yield
mechanism-based LSTM is developed to improve the hourly promising results in fault diagnosis [32], harmonic power grid
solar energy prediction of MLP by 6.17% and 0.27 in terms analysis [30], and voltage stability assessment [57].
of MAPE and root mean squared error (RMSE), respectively. Besides the kernel-based CNN in (2), recent studies pro-
Also, the LSTMs in [67] and [60] have shown the state-of-the- posed spectral graph convolutions to capture spatial patterns
art performance in wind prediction tasks. Moreover, in [61], of the graph-structured power system datasets [15]. Given
a LSTM is defined in a multimodal neural architecture to an N -node graph with D-dimensional features X ∈ RN ×D ,
simultaneously capture the temporal characteristics of dynamic adjacency matrix A, and degree matrix D, the convolution
load parameters as well as the voltage and power changes in operation of the graph CNN is computed by:
the IEEE 68-bus test system [65]. It is shown that the LSTM
f (X, A) = σ(D −1/2 AD−1/2 XW ) (3)
captures real-time dynamic behaviors of load parameters with
38.42% and 25.64% better RMSE and MAPE, respectively, where W is the trainable convolution weight matrix. The
compared to the TDNN method due to its larger hypoth- graph CNN is recently employed for short-term wind speed
esis space and overcoming the overfitting problem. Similar prediction [68] in Northern US. Due to capturing spatial
accuracy improvements are recently reported in other time- characteristics of the wind data, this model provides 11.61%
dependent applications including power fluctuation identifi- and 17.98% better RMSE and MAPE compared to the DBN,
cation [63], data-based line trip fault prediction [58], and respectively. Moreover, this model yields 20.04% better relia-
industrial load forecasting [62]. bility for probabilistic solar energy prediction compared to the
LSTM [9].
D. Convolutional Neural Network
CNNs contain a two dimensional input layer I, a set of E. Advantages and Restrictions of Deep Discriminative
hidden convolution and pooling layers, and a fully connected Modeling
output layer. Each neuron in the convolution layer is a non- Table II summarizes the advantages and restrictions of
linear kernel that divides the input into small slices called various deep discriminative models in power engineering
receptive fields. The output of convolution operation at the research. As shown in this table, the ReLU ANN has a simple
k-th kernel in the l-th convolution layer is computed by: implementation with low training time complexity and fast
XX feed-forward approach. However, this supervised model does
flk (p, q) = ic (x, y).ekl (u, v) (2) not explicitly model the time-dependent or spatial features
c x,y
in the datasets. Similar to the ReLU ANN, the SAE cannot
where ic (x, y) is the (x, y) element of the c-th channel of input directly capture the spatial and temporal patterns. However,
I, and ekl (u, v) is the (u, v) element of the k-th kernel of layer SAE is an unsupervised feature learner which is more suitable
KHODAYAR et al.: DEEP LEARNING IN POWER SYSTEMS RESEARCH: A REVIEW 213

TABLE II
A DVANTAGES AND R ESTRICTIONS OF D EEP D ISCRIMINATIVE M ODELS
Models Advantages Restrictions
1- Simple implementation 1- Lack of temporal and spatial feature extraction
ReLU 2- Fast feed-forward process 2- Lack of the feature coherence
3- Low training time complexity 3- Limited to supervised applications
1- Large estimation bias
1- Simple Implementation
2- Lack of temporal and spatial feature extraction
SAE 2- Fast feed-forward process
3- Lack of feature coherence
3- Unsupervised feature extraction
4- High chance of overfitting
1- High chance of overfitting
1- Extracting accurate temporal features 2- Lack of spatial data modeling
LSTM
2- Flexible input dimensions 3- High sensitivity to the initial state
4- Limited to supervised applications
1- Accurate spatial feature extraction 1- Lack of temporal data modeling
2- Sparse data representation 2- High training time complexity
CNN
3- Simple training process using gradient descent 3- High training memory complexity
4- Distributed implementation 4- Limited to supervised applications

for applications with limited number of training samples such To train W i , the Contrastive Divergence method [8] is em-
as cyber attack detection. In addition, SAE is suitable for ployed that adds the gradient of P (hi−1 ) with respect to W i
problems with large amounts of uncertain data points including to increase the likelihood of observing the visible vector hi−1
wind speed prediction, solar energy forecasting, and dynamic given the latent vector hi . Similar approach is used to train bi
load modeling. and bi−1 in an unsupervised fashion. When the unsupervised
Due to its recurrent structure, the LSTM can model temporal training is done for all layers, a dense layer o = hL+1 is added
dependencies between time-dependent observations and work on top of the last layer hL and the whole neural network is
with variable input lengths. However, since the number of trained by the supervised GD similar to the SAE.
LSTM parameters is generally larger than classic recurrent Table III shows the large variety of DBN’s applications
ANNs, this supervised model has a higher chance of overfitting in power systems area. As shown in this table, the DBN
and high sensitivity to the observation noise. leads to accurate wind and PV power prediction results due to
Using filtering and pooling layers, the CNN is able to capturing uncertainties in the energy time series [8]. Moreover,
provide powerful sparse representations from spatial datasets DBN shows a promising performance in transient stability
using simple gradient-based techniques. As the filters can be classification with 94.69% accuracy in the Central China
trained in a distributed manner, the CNN is a very efficient Regional Power Grid [35]. Furthermore, in [37], this method
method for pattern recognition in large-scale systems. How- is recently applied to the state estimation of the US PG&E69
ever, since this supervised model does not contain a recursive distribution network that led to a remarkably small MAPE of
structure, it cannot accurately capture time-dependent struc- 0.091% which shows the large hypothesis space and low bias
tures of the data. of this probabilistic model.

III. P ROBABILISTIC D EEP L EARNING B. Generative Adverserial Network

In contrast to discriminative deep learning where an explicit Assuming a training set Dtr , GAN is an unsupervised deep
function maps x to y where (x, y) ∈ Dtr , the objective of ANN that learns P (x) s.t. x ∈ Dtr using a generator ANN
probabilistic deep neural architectures [69] is to capture the G(z) that observes some input noise z ∼ P (z) and outputs a
PDF P (x) for all samples in the dataset Dtr . Then, an explicit sample x0 drawn from the generators PDF Pg . The produced
function is learned to map P (x) to P (y|x), hence learning the sample x0 as well as the training samples x ∈ Dtr are given
true output y for all samples (x, y) ∈ Dtr . to a discriminator ANN D, a binary classifier which decides
if the generated sample x0 comes from the true PDF P (x)
A. Deep Belief Network or the PDF of generated samples Pg . Training the generator
The DBN is a deep MLP with input h0 and L computational and discriminator simultaneously, we improve the generator
layers hi (i = 1, 2, . . . , L). Each layer hi is a Restricted to create realistic samples by decreasing the distance between
Boltzmann Machine (RBM) RBM i , a generative graphical the real PDF P (x) and the generated PDF Pg . To train
model that encodes the PDF of its input layer hi−1 into its the discriminator D, the following unsupervised objective is
latent feature vector hi . At each RBM i i = 1, 2, . . . , L, the applied:
conditional PDF of the j-th neurons in the visible layer hi−1
and hidden layer hi is computed by: max Ex∼P (x) [log D(x)] + Ex0 ∼Pg [log(1 − D(x0 ))] (5)
D
 !
i−1 Here, D(x) is trained to differentiate between the samples
P (hij = 1|hi−1 ) = σ i
+ bij
 P
k Wkj .hk

generated from G(z) and the true samples x ∼ P (x).



! (4) Using (5), to simultaneously optimize ANNs G(z) and D,

 i−1 i
P (hj = 1|h ) = σ
P i i i−1 the following min-max objective is optimized using the GD
k Wjk .hk + bj


method:
214 CSEE JOURNAL OF POWER AND ENERGY SYSTEMS, VOL. 7, NO. 2, MARCH 2021

TABLE III
P ROBABILISTIC D EEP L EARNING IN P OWER S YSTEMS A PPLICATIONS
Applications Dataset Model Performance Metric Result
Wind Speed DBN 0.5494, 6.39%
Shangchuan Island Wind Farm RMSE, MAPE
Prediction [33] VAE 0.4832, 4.81%
PV Power DBN 17.55 kW, 3.76%
North China Baoding Dataset RMSE, MAPE
Prediction [9], [34] VAE 15.48 kW, 3.63%
Transient Stability DBN 94.69%
Central China Regional Power Grid Classification Accuracy
Assessment [35] VAE 98.14%
Hourly Load DBN 0.4851, 5.81%
Texas Urbanized Area Dataset RMSE, MAPE
Forecasting [41], [42] VAE 0.4032, 5.02%
DBN 0.091, 0.073
State Estimation [37] US PG&E69 Distribution Network MAPE, Maximum Absolute Error
VAE 0.084, 0.069
Fault Detection DBN 79.11%
[38], [41], Northern China Wind Farm (SCADA) VAE Classification Accuracy 84.85%
[42] GAN 87.32%
Cyber Attack GAN 95.34%
5-bus Smart Grid Classification Accuracy
Detection [39] VAE 92.18%
Renewable Scenario GAN 0.61
Wind & Solar Integration Dataset Kullback–Leibler Divergence
Generation [40] VAE 0.52
Power Grid Columbia University Synthetic GAN Topological Distance, 0.678, 3.41 MW
Synthesis [15] Power Grid (CUSPG) VAE Power Flow Distance 0.0512, 3.06 MW

min max JD,G = Ex∼P (x) [log D(x)]+ an accurate estimation of the data PDF P (x) when marginal-
G D
ized over all valid z.
Ez∼P (z) [log(1 − D(G(z)))] (6)
As shown in Table III, the VAE is applied to learn the
To test the model on a testing set Dts , the Kullback-Leibler conditional PDF of future wind speed/power given its previous
(KL) divergence is used as a distance metric between the measurements for short-term wind prediction [33]. Moreover,
estimate PDF and the true PDF of samples x ∈ Dts . similar technique is applied in [15] and [34] to hourly and 6-
As shown in Table III, GAN leads to a promising perfor- hour ahead prediction of PV power with 2.07kW and 6.53kW
mance in a diverse set of complex classification problems in- better RMSE compared to the DBN, respectively. In addition
cluding fault detection [41] and cyber attack classification [39], to regression, VAE outperforms DBN in complex classifi-
as well as regression problems such as scenario generation cation tasks with 3.45% accuracy improvement in transient
for the wind and solar power [40]. Compared to the classic stability assessment [35] and 5.74% better fault detection
DBN, GAN has a larger hypothesis space which leads to accuracy [38]. Moreover, VAE is utilized to learn the PDF of
higher generalization capacity. Hence, as Table III shows, the physical and topological characteristics of power networks
GAN outperforms DBN in both fault detection and cyber for power network synthesis. As shown in Table III, VAE
attack classification. Moreover, since GAN explicitly models generates realistic power networks that accurately imitate not
the joint PDF of the data, it can be directly applied to realistic only the topological properties (e.g., diameter and density) but
data synthesis problems such as power grid synthesis [15], also the power flow statistics (maximum, minimum, and me-
[38] while DBN does not have such a capability. dian flow) of the large-scale transmission network in CUSPG
dataset [70].
C. Variational Autoencoder
D. Advantages and Restrictions of Deep Generative Modeling
Similar to GANs, the objective of VAE is to learn the PDF
Table IV presents the advantages and restrictions of deep
P (x) s.t. x ∈ Dtr in an unsupervised fashion. The VAE
generative modeling in power system research. As shown in
consists of an encoder ANN qθ (z|x) parametrized by θ and a
this table, DBN, GAN, and VAE can handle measurement
decoder ANN pφ (x|z) with parameters (weights and biases)
uncertainties while providing a powerful unsupervised data
φ. The encoder maps x into the latent representation z which
representation. Compared to GAN and VAE, the DBN has
has a Gaussian distribution estimated by qθ (z|x). Then, to find
smaller sample complexity which leads to less number of
the optimal z that is powerful enough to best reconstruct x,
training examples required for feature extraction. However,
the decoder maps z into the actual input x. Hence, training
since this model employs Gibbs sampling in its training
the VAE consists of maximizing the likelihood of x as well
process, it has a large training time complexity. Also, DBN
as minimizing the KL divergence KL of the distribution of
has a strong independence assumption on its latent variables
z (i.e. qθ (z|x)) and its actual distribution N (0, I) where I is
which makes it less suitable for pattern recognition in highly
the identity matrix. Therefore, the loss function of the VAE is
nonlinear datasets.
computed by:
In contrast to DBN, the GAN and VAE directly learn the
X  
data distributions with no prior assumptions. Thus, these mod-
JVAE = KL[qθ (z|x)kN (0, I)]−Eqθ (z|x) [log pφ (x|z)]
els can be effectively applied to power system data synthesis.
x∈Dtr
(7) Due to its larger architecture, GAN requires more number
of training examples compared to the DBN. Also, GAN
Training the VAE using GD, the decoder pφ (x|z) provides has limited feature diversity and lacks parameter convergence
KHODAYAR et al.: DEEP LEARNING IN POWER SYSTEMS RESEARCH: A REVIEW 215

TABLE IV
A DVANTAGES AND R ESTRICTIONS OF D EEP G ENERATIVE M ODELS
Models Advantages Restrictions
1- Modeling uncertainties 1- Large training time complexity
DBN 2- Unsupervised feature extraction 2- Strong prior knowledge (conditional independence)
3- Small sample complexity 3- Lack of parameter convergence guarantee
1- Large sample complexity
1- Modeling uncertainties 2- Lack of parameter convergence guarantee
GAN 2- Unsupervised feature extraction 3- Limited diversity
3- Data Synthesis 4- Diminished gradient
5- Sharp but unreliable estimations
1- Modeling uncertainties
2- Unsupervised feature extraction 1- Large sample complexity
VAE 3- Data Synthesis 2- Low sharpness of estimated distribution
4- Providing probabilistic classification and regression 3- Large testing time complexity
5- Reliable estimation of the actual probability distribution

guarantees. While VAE has similar sample complexity com- Moreover, this model shows a promising load shedding result
pared to GAN, it provides a more reliable distribution esti- of 26 MW for optimal emergency control of the IEEE 39-
mation. However, the smaller sharpness of VAE compared to bus system [45]. Furthermore, DQN is employed for power
GAN makes GAN a better choice for probabilistic applica- grid cost efficiency with transportation energy optimization,
tions. and showed 14.1% improvement compared to the classic
binary control method [46]. The high generalization power
IV. D EEP R EINFORCEMENT L EARNING of this method has encouraged the researcher to apply DQN
for various real-world applications ranging from electricity
Besides classification and regression, deep ANNs are em- marketing [47] and demand-response learning [49] to smart
ployed in reinforcement learning settings where the problem is grid scheduling [52] and cyber attck detection [53].
modeled as a Markov decision process (MDP) (S, A, Pa , Ra )
with the state set S, action domain A, and state transition B. Double DQN (DDQN)
probability Pa (s, s0 ) = P (st+1 = s0 |st = s, at = a) to
To reduce the overestimation effect of the state-action value
model the likelihood of going from state st at time t to
Q(s, a) in (9), the DDQN uses a target deep ANN parameter-
state st+1 at time t + 1. This transition leads to observing
ized by θ0 to compute the update value maxa Q(s0 , a) while
the immediate reward Ra (st = s, st+1 = s0 ) from the
the state-action Q(s, a) is computed by a deep ANN with
problem’s environment. The goal is find the optimal policy
the original DQN parameters θ. As shown in Table V, this
π ∗ (st ) that determines action at P
to maximize the expected
∞ t
 method improves the classic DQN with 2.2% improvement
discounted reward sum Ravg = E t=o γ Ra (st , st+1 ) . The in cost efficiency for transportation energy optimization [46]
discounting factor 0 6 γ 6 1 decides the contribution of the
and £43 ∗ 103 improvement in electricity market bidding
historical rewards to Ravg . The optimal policy π ∗ (s) for a state
profit [47].
s ∈ S is computed by:
π ∗ (s) = arg max Q(s, a) (8) C. Deep Deterministic Policy Gradient (DDPG)
a
DDPG is an actor-critic DRL algorithm. The actor µ(s)
where Q(s, a) is the optimal state-action value function that models the policy as a deep ANN that observes a states s and
estimates the reward of taking action a in state s. generates the corresponding continuous action a. The critic Q
is a deep ANN that estimates Q(s, a) for the state-action input
A. Deep Q-network (DQN) < s, a >. To compute the state’s value, the actor’s output is
DQN [44] directly learns Q(s, a) and employs (8) to find the given to the critic to calculate Q(s, a). Similar to DQN, The
optimal policy. To provide high generalization power and low critic’s TD-error function JQ is computed using the Bellman
estimation bias, the DQN implements Q(s, a) by a deep neural equation:
network QAN N that observes an input hs, ai and outputs
2
Q(s, a). To train QAN N , the Temporal Difference (TD) error JQ = Q(s, µ(s)) − (Ra (s, s0 ) + γQ0 (s0 , µ0 (s0 ))) (10)
δ is defined as the difference between the current Q(s, a) and where Q0 and µ0 are the target critic and actor deep ANNs,
the value function after the transition to s0 computed by: respectively. The target ANNs Q0 and µ0 are time delayed
δ = Q(s, a) − (Ra (st = s, st+1 = s0 ) + γ max Q(s0 , a)) (9) copies of Q and µ that slowly track the learned state-action
a values. The actor’s loss function Jµ is computed by Q(s, µ(s))
To train the DQN (i.e., minimize δ), the Huber loss is which is maximized to increase the DDPG’s return while JQ
computed by J(δ) = 21 δ 2 if |δ| 6 1 and J(δ) = |δ| − 21 is minimized. To learn Q and µ using GD, the gradients
otherwise. Applying GD, one can minimize J(δ) with respect of JQ and Jµ with respect to their weights and biases are
to the weights and biases of QAN N . computed, respectively. Moreover, the target networks Q0 and
Table V shows the applications of DLR in the power µ0 are updated by respectively adding a small fraction of
engineering domain. As shown in this table, DQN is recently their corresponding parameters in the original networks Q
applied for optimal voltage control of a 200-bus system [44]. and µ at each DRL episode. Table V shows the significant
216 CSEE JOURNAL OF POWER AND ENERGY SYSTEMS, VOL. 7, NO. 2, MARCH 2021

TABLE V
D EEP R EINFORCEMENT L EARNING A PPLICATIONS IN P OWER S YSTEMS

Applications Dataset Model Performance Metric Result

DQN 161.54
Voltage Control [44] Realistic 200-bus System (SCADA) Average Control Reward
DDPG 124.83
DQN 26 MW
Emergency Control [46] IEEE 39-bus Load Shedding
DDPG 23 MW
Transportation Energy California Freeway Performance DQN Cost Efficiency 14.1%
Optimization [46] Measurement System (PeMS) DDQN (compared to binary control) 16.3%
DQN 5.2 * 10ˆ5
Electricity Market [47], [48] Synthetic Market Dataset DDQN Profit(£) 5.63 * 10ˆ5
DDPG 5.86 * 10ˆ5
DQN
Demand-Response Strategy 161.93
Steel Powder Manufacturing Dataset TD-based Operation Cost($)
Learning [49]–[51] 134.85
Actor-Critic DRL
DQN $ 4268.17
Power Scheduling [52] Shaanix Wind Farm Dataset Average Income($)
Improved DQN $ 4730.21
DQN 0.120 p.u.
Cyber Attack Detection [53], [55] IEEE 9-bus System Transient Energy
DDPG 0.056 p.u.

experimental advantage of DDPG compared to DQN-based shown in this diagram, the combination of sparse coding and
methods. While DQN cannot handle high-dimensional action dictionary learning methods with discriminative models is one
spaces, the DDPG learns policies in these conditions. Thus, of the significant future domains that require further study.
DDPG is shown to generally provide better accuracy in both The sparse models can decrease the sample complexity of
regression problems such as autonomous voltage control [44], deep learning algorithms and help deep neural networks to
emergency control [45], strategic bidding [47] as well as better decompose, compress, and reconstruct the input data.
classification tasks including cyber attack detection [53] and Hence, this approach would lead to a remarkable accuracy
data integrity protection [55]. improvement in Behind-The-Meter net load disaggregation,
deep nonintrusive load monitoring, PMU data compression
D. Advantages and Restrictions of Deep Reinforcement and noise reduction, customer behavior estimation, and the
Learning Internet of Things data analytic. The graph knowledge repre-
Table VI provides a summary of DRL advantages and re- sentation in discriminative learning is another major area of
strictions in power system applications. As shown in this table, future research. In this class of approaches, the deep neural
both DQN and DDQN have stable and robust training proce- network captures highly nonlinear and highly varying charac-
dures. Therefore, they are very suitable for datasets with high teristics of graph-structured datasets. This category of models
uncertainty factors. However, these methods cannot guarantee is very effective to improve the accuracy of spatiotemporal
their parameter convergence. Also, DQN and DDQN mainly renewable energy and load forecasting techniques. Also, it
optimize deterministic policies in discrete action spaces. Thus, can be effectively employed for real-time state estimation,
compared to the DDPG, they are less suitable for real-world load and system parameter identification, as well as topology
applications with continuous actions. Although DDPG may detection. The last class of future works in the domain of
suffer from parameter instability during training, it provides discriminative models is the deep Bayesian learner. This model
a fast and guaranteed convergence to a promising stochastic incorporates the Bayes rule into ReLU ANNs to create robust
local policy. probabilistic deep learning solutions that can effectively handle
the uncertainties in the datasets. The future applications of
V. F UTURE R ESEARCH D IRECTIONS this method include probabilistic dynamic load modeling,
The future research on deep learning algorithms in the probabilistic pattern recognition of Behind-The-Meter sensor
area of power engineering is summarized in Fig. 1. As data, as well as power quality disturbance detection and

TABLE VI
A DVANTAGES AND R ESTRICTIONS OF D EEP R EINFORCEMENT L EARNING M ODELS
Models Advantages Restrictions
1- Lack of policy convergence guarantee
2- Slow policy convergence
1- Stability of the learning algorithm
DQN 3- Assuming deterministic policies
2- Robustness to measurement uncertainties
4- Lack of compatibility with continuous action spaces
5- Overestimating the state-action values
1- Lack of policy convergence guarantee
1- Stability of learning algorithm
2- Slow policy convergence
DDQN 2- Robustness to measurement uncertainties
3- Assuming deterministic policies
3- Realistic estimation of state-action values
4- Lack of compatibility with continuous action spaces
1- Fast convergence to the optimal policy
2- Guaranteed convergence in complicated state-action spaces 1- Instability of the learning algorithm
DDPG
3- Learning stochastic policies 2- High sensitivity of parameters to noise
4- Compatibility with continuous action spaces
KHODAYAR et al.: DEEP LEARNING IN POWER SYSTEMS RESEARCH: A REVIEW 217

Future Research Directions

Discriminative Modeling Generative Modeling Reingorcement Learning

Graph Bayesian Generative Conditional Network PDF Multi-agent Operational Multi-stage and
Sparse coding knowledge Discriminative graph PDF estimation DRL Control Cooperative
representation Modeling representation learning Games

1) Behind-The-Meter Net Load Disaggregation 1) Spatiotemporal probabilistic state estimation 1) Game modeling for dynamic demand response
2) Deep Nonintrusive load monitoring 2) Spatiotemporal probabilistic load forecasting 2) Decentralized control for optimal bidding
3) PMU data compression and noise reduction 3) Spatiotemporal probabilistic wind and PV 3) Optimal energy trade strategies
4) Customer Behavior Estimation power forecasting 4) Blockchain and smart contracts
5) Transaction energy

1) Spatiotemporal renewable energy forecasting 1) Anomaly Detection 1) Smart generation control schemes
2) Spatiotemporal load forecasting 2) Cyber threat and vulnerability analysis 2) Model-free load frequency control
3) State estimation 3) False data injection detection 3) Appliances and system control
4) Load and system parameter identification 4) Probabilistic system-wide load parameter
5) Topology detection identification

1) Behind-The-Meter Net Load Disaggregation 1) Power grid synthesis 1) Multi-stage game modeling for cybersecurity
2) Deep Nonintrusive load monitoring 2) Probabilistic state estimation 2) Cooperative DRL for distributed economic dispatch
3) PMU data compression and noise reduction 3) Topology detection 3) Electric power market modeling
4) Customer Behavior Estimation
5) Internet of Things

Fig. 1. Future research directions in the area of deep representation learning.

classification. can be combined with multi-stage games to provide reliable

solutions for modeling the interactions between cyber attackers
One of the major future works in the area of genera-
and cyber defenders. Also, cooperative DRL can be utilized
tive modeling is the generative graph representation model.
for distributed economic dispatch while satisfying the power
This approach combines graph CNNs with GANs and VAEs
balance and other operational constraints.
to provide accurate probabilistic representations of graph-
structured energy datasets and power grids. The future appli-
VI. C ONCLUSION
cations of this approach include spatiotemporal probabilistic
state estimation, spatiotemporal probabilistic load forecasting, With the growing time and memory complexity of power
as well as the spatiotemporal probabilistic wind and solar system applications, the need for advanced statistical pat-
energy forecasting. Another major class of future generative tern recognition tools has lead to the use of deep learning
models is the conditional PDF learners. In contrast to the methodologies. This novel class of methods can be mainly
classic GAN and VAE, this category of models can learn the categorized into discriminative, generative, and reinforcement
conditional PDF of an observation given another observation. learning approaches. This review studies the deep discrimi-
Thus, it is able to remarkably improve the accuracy of machine native models that provide an explicit method to map their
learning algorithms in various power system applications such complex input directly to the problem’s solution. Due to their
as anomaly detection, cyber threat detection and classification, high generalization capacity, these models are widely applied
false data injection detection, and probabilistic load parameter to stability assessment, fault detection, as well as renewable
identification. In addition to this class of models, the network generation prediction. Then, deep generative approaches are
PDF estimation models are another category of future works reviewed that provide a probabilistic approximation of data
that compute the joint PDF of node and edge features of large- PDFs; hence, learning complex probabilistic structures for
scale power networks. This class of algorithms improves future a wide range of power engineering applications including
generative solutions for power network synthesis, probabilistic state estimation, renewable scenario generation, and power
state estimation, and power grid topology detection. grid synthesis. Finally, deep reinforcement learning algorithms
are discussed that seek to optimize an objective using the
In the area of DRL, the multi-agent DRL can significantly observed rewards captured from the problem’s environment.
improve the game modeling solutions between power compa- The theoretical and experimental analysis of the employed
nies and customers for dynamic demand response. Also, this method motivates future research in the area of deep learning
technique can provide decentralized control strategies for opti- to further extend the applications of this powerful class of
mal bidding between the companies and customers. Moreover, models in new perspectives of power engineering.
the multi-agent DRL can improve energy market strategies and
establish smart contracts in peer-to-peer electricity trades with R EFERENCES
Blockchain. Future DRL works can also advance operational
[1] Y. Bengio, A. Courville, and P. Vincent, “Representation learning: A
control strategies. These models can be applied to smart gen- review and new perspectives,” IEEE Transactions on Pattern Analysis
eration control in multi-area interconnected grids, model-free and Machine Intelligence, vol. 35, no. 8, pp. 1798–1828, Aug. 2013.
load frequency control, as well as the appliances and system [2] L. F. Cai, N. F. Thornhill, S. Kuenzel, and B. C. Pal, “Wide-area
monitoring of power systems using principal component analysis and
control (e.g., smart grid emergency control and autonomous k-nearest neighbor analysis,” IEEE Transactions on Power Systems, vol.
grid operational control). Moreover, future DRL techniques 33, no. 5, pp. 4913–4923, Sep. 2018.
218 CSEE JOURNAL OF POWER AND ENERGY SYSTEMS, VOL. 7, NO. 2, MARCH 2021

[3] Y. Jimenez, J. Cortes, C. Duarte, J. Petit, and G. Carrillo, “Non- [24] J. Cao and Z. Fan, “Deep learning-based online small signal stability
intrusive discriminant analysis of loads based on power quality data,” in assessment of power systems with renewable generation,” in Proceedings
Proceedings of 2019 IEEE Workshop on Power Electronics and Power of 2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing,
Quality Applications (PEPQA), 2019, pp. 1–5. Advanced & Trusted Computing, Scalable Computing & Communica-
[4] S. Zhang, S. B. Zhang, S. F. Li, L. Du, and T. G. Habetler, “Visualization tions, Cloud & Big Data Computing, Internet of People and Smart
of multi-objective switched reluctance machine optimization at multiple City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI),
operating conditions with t-SNE,” in Proceedings of 2019 IEEE Energy IEEE, 2018, pp. 216–221.
Conversion Congress and Exposition (ECCE), 2019, pp. 3793–3798. [25] W. T. Li, D. Deka, M. Chertkov, and M. Wang, “Real-time faulted line
[5] Y. Q. Liu, H. Qin, Z. D. Zhang, S. Q. Pei, Z. Q. Jiang, Z. K. Feng, and localization and PMU placement in power systems through convolutional
J. Z. Zhou, “Probabilistic spatiotemporal wind speed forecasting based neural networks,” IEEE Transactions on Power Systems, vol. 34, no. 6,
on a variational Bayesian deep learning model,” Applied Energy, vol. pp. 4640–4651, Nov. 2019.
260, pp. 114259, Feb. 2020. [26] J. Yan, H. Zhang, Y. Q. Liu, S. Han, L. Li, and Z. X. Lu, “Forecasting
[6] K. P. Lin, P. F. Pai, and Y. J. Ting, “Deep belief networks with genetic the high penetration of wind power on multiple scales using multi-to-
algorithms in forecasting wind speed,” IEEE Access, vol. 7, pp. 99244– multi mapping,” IEEE Transactions on Power Systems, vol. 33, no. 3,
99253, Jul. 2019. pp. 3276–3284, May 2018.
[7] M. Khodayar, O. Kaynak, and M. E. Khodayar, “Rough deep neural [27] A. Gensler, J. Henze, B. Sick, and N. Raabe, “Deep learning for solar
architecture for short-term wind speed forecasting,” IEEE Transactions power forecasting—an approach using autoencoder and LSTM neural
on Industrial Informatics, vol. 13, no. 6, pp. 2770–2779, Dec. 2017. networks,” in Proceedings of 2016 IEEE international conference on
[8] M. Khodayar, J. H. Wang, and M. Manthouri, “Interval deep generative systems, man, and cybernetics (SMC), IEEE, 2016, pp. 002858–002865.
neural network for wind speed forecasting,” IEEE Transactions on Smart [28] Z. Xu, W. X. Mo, Y. Wang, S. M. Luo, and T. Liu, “Transformer fault
Grid, vol. 10, no. 4, pp. 3974–3989, Jul. 2019. diagnosis based on deep brief sparse autoencoder,” in Proceedings of
[9] M. Khodayar, S. Mohammadi, M. E. Khodayar, J. H. Wang, and G. 2019 Chinese Control Conference (CCC), IEEE, 2019, pp. 7432–7435.
Y. Liu, “Convolutional graph autoencoder: a generative deep neural [29] H. X. Zhou, Y. J. Zhang, L. F. Yang, Q. Liu, K. Yan, and Y. Du,
network for probabilistic spatio-temporal solar irradiance forecasting,” “Short-term photovoltaic power forecasting based on long short term
IEEE Transactions on Sustainable Energy, vol. 11, no. 2, pp. 571–583, memory neural network and attention mechanism,” IEEE Access, vol.
Apr. 2020. 7, pp. 78063–78074, Jun. 2019.
[10] H. X. Zang, L. L. Cheng, T. Ding, K. W. Cheung, Z. Liang, Z. [30] N. Severoğlu and O. Salor, “Harmonic analysis in power systems using
N. Wei, and G. Q. Sun, “Hybrid method for short-term photovoltaic convolutional neural networks,” in Proceedings of 2018 26th Signal
power forecasting based on deep convolutional neural network,” IET Processing and Communications Applications Conference (SIU), IEEE,
Generation, Transmission & Distribution, vol. 12, no. 20, pp. 4557– 2018, pp. 1–4.
4567, Nov. 2018. [31] Z. T. Shi, W. Yao, L. K. Zeng, J. F. Wen, J. K. Fang, X. M. Ai, and
[11] M. Y. Sun, T. Q. Zhang, Y. Wang, G. Strbac, and C. Q. Kang, “Using J. Y. Wen, “Convolutional neural network-based power system transient
Bayesian deep learning to capture uncertainty for residential net load stability assessment and instability mode prediction,” Applied Energy,
forecasting,” IEEE Transactions on Power Systems, vol. 35, no. 1, pp. vol. 263, pp. 114586, Apr. 2020.
188–201, Jan. 2020. [32] S. Chan, I. Oktavianti, V. Puspita, and P. Nopphawan, “Convolutional
[12] C. J. Huang and P. H. Kuo, “Multiple-input deep convolutional neural adversarial neural network (CANN) for fault diagnosis within a power
network model for short-term photovoltaic power forecasting,” IEEE system: addressing the challenge of event correlation for diagnosis by
Access, vol. 7, pp. 74822–74834, Jun. 2019. power disturbance monitoring equipment in a smart grid,” in Proceed-
[13] H. Z. Wang, J. Q. Ruan, G. B. Wang, B. Zhou, Y. T. Liu, X. Q. Fu, and ings of 2019 International Conference on Information and Communi-
J. C. Peng, “Deep learning-based interval state estimation of ac smart cations Technology (ICOIACT), 2019, pp. 596–601.
grids against sparse cyber attacks,” IEEE Transactions on Industrial [33] H. Z. Wang, G. B. Wang, G. Q. Li, J. C. Peng, and Y. T. Liu, “Deep belief
Informatics, vol. 14, no. 11, pp. 4766–4778, Nov. 2018. network based deterministic and probabilistic wind speed forecasting
[14] K. R. Mestav, J. Luengo-Rozas, and L. Tong, “Bayesian state estimation approach,” Applied Energy, vol. 182, pp. 80–93, Nov. 2016.
for unobservable distribution systems via deep learning,” IEEE Trans- [34] F. Xu, Y. Tian, Z. Wang, and J. L. Li, “One-day ahead forecast of
actions on Power Systems, vol. 34, no. 6, pp. 4910–4920, Nov. 2019. PV output based on deep belief network and weather classification,” in
[15] M. Khodayar, J. H. Wang, and Z. Y. Wang. Deep generative graph Proceedings of 2018 Chinese Automation Congress (CAC), 2018, pp.
distribution learning for synthetic power grids. arXiv preprint arXiv: 412–417.
1901.09674, 2019. [35] L. Zheng, W. Hu, Y. F. Zhou, Y. Min, X. L. Xu, C. M. Wang, and R.
[16] W. He and Y. Chai, “An empirical study on energy disaggregation via Yu, “Deep belief network based nonlinear representation learning for
deep learning,” in Proceedings of 2016 2nd International Conference on transient stability assessment,” in Proceedings of 2017 IEEE Power &
Artificial Intelligence and Industrial Engineering (AIIE 2016), 2016. Energy Society General Meeting, 2017, pp. 1–5.
[17] M. Khodayar, J. H. Wang, and Z. Y. Wang, “Energy disaggregation [36] Y. S. He, J. H. Deng, and H. J. Li, “Short-term power load forecasting
via deep temporal dictionary learning,” IEEE Transactions on Neural with deep belief network and copula models,” in Proceedings of 2017
Networks and Learning Systems, vol. 31, no. 5, pp. 1696–1709, May 9th International conference on intelligent human-machine systems and
2020. cybernetics (IHMSC), 2017, pp. 191–194.
[18] J. Jiang, Q. Q. Kong, M. Plumbley, and N. Gilbert. Deep learning based [37] Y. Huang, Q. S. Xu, C. Hu, Y. X. Sun, and G. Lin, “Probabilistic state
energy disaggregation and on/off detection of household appliances. estimation approach for AC/MTDC distribution system using deep belief
arXiv preprint arXiv: 1908.00941, 2019. network with non-Gaussian uncertainties,” IEEE Sensors Journal, vol.
[19] Y. Roy, H. Banville, I. Albuquerque, A. Gramfort, T. H. Falk, and 19, no. 20, pp. 9422–9430, Oct. 2019.
J. Faubert, “Deep learning-based electroencephalography analysis: a [38] J. H. Liu, F. M. Qu, X. W. Hong, and H. G. Zhang, “A small-sample wind
systematic review,” Journal of Neural Engineering, vol. 16, no. 5, pp. turbine fault detection method with synthetic fault data using generative
051001, Jan. 2019. adversarial nets,” IEEE Transactions on Industrial Informatics, vol. 15,
[20] M. Keshk, B. Turnbull, N. Moustafa, D. Vatsalan, and K. K. R. Choo, no. 7, pp. 3877–3888, Jul. 2019.
“A privacy-preserving-framework-based blockchain and deep learning [39] S. Ahmadian, H. Malki, and Z. Han, “Cyber attacks on smart energy
for protecting smart power networks,” IEEE Transactions on Industrial grids using generative adverserial networks,” in Proceedings of 2018
Informatics, vol. 16, no. 8, pp. 5110–5118, Aug. 2020. IEEE Global Conference on Signal and Information Processing (Glob-
[21] Z. Zhang, D. Zhang, and R. C. Qiu, “Deep reinforcement learning for alSIP), 2018, pp. 942–946.
power system applications: An overview,” CSEE Journal of Power and [40] Y. Z. Chen, Y. S. Wang, D. Kirschen, and B. S. Zhang, “Model-free
Energy Systems, vol. 6, no. 1, pp. 213–225, Mar. 2020. renewable scenario generation using generative adversarial networks,”
[22] S. Zhou, Z. Hu, W. Gu, M. Jiang, and X.-P. Zhang, “Artificial intel- IEEE Transactions on Power Systems, vol. 33, no. 3, pp. 3265–3275,
ligence based smart energy community management: A reinforcement May 2018.
learning approach,” CSEE Journal of Power and Energy Systems, vol. [41] J. Pereira and M. Silveira, “Unsupervised anomaly detection in energy
5, no. 1, pp. 1–10, Mar. 2019. time series data using variational recurrent autoencoders with attention,”
[23] L. Duchesne, E. Karangelos, and L. Wehenkel, “Machine learning of in Proceedings of 2018 17th IEEE International Conference on Machine
real-time power systems reliability management response,” in Proceed- Learning and Applications (ICMLA), 2018, pp. 1275–1282.
ings of 2017 IEEE Manchester PowerTech, IEEE, 2017, pp. 1–6. [42] R. C. Zheng and J. Gu, “Anomaly detection for power system forecasting
KHODAYAR et al.: DEEP LEARNING IN POWER SYSTEMS RESEARCH: A REVIEW 219

under data corruption based on variational auto-encoder,” in Proceedings [63] S. L. Wen, Y. Wang, Y. Tang, Y. Xu, P. F. Li, and T. Y. Zhao, “Real-
of 8th Renewable Power Generation Conference (RPG 2019), 2019. time identification of power fluctuations based on LSTM recurrent neural
[43] Z. X. Pan, J. M. Wang, W. L. Liao, H. W. Chen, D. Yuan, W. P. Zhu, network: a case study on Singapore power system,” IEEE Transactions
X. Fang, and Z. Zhu, “Data-driven EV load profiles generation using a on Industrial Informatics, vol. 15, no. 9, pp. 5266–5275, Sep. 2019.
variational auto-encoder,” Energies, vol. 12, no. 5, pp. 849, Mar. 2019. [64] C. Grigg, P. Wong, P. Albrecht, R. Allan, M. Bhavaraju, R. Billinton,
[44] J. J. Duan, D. Shi, R. S. Diao, H. F. Li, Z. W. Wang, B. Zhang, D. Q. Chen, C. Fong, S. Haddad, S. Kuruganty, W. Li, R. Mukerji, D.
S. Bian, and Z. H. Yi, “Deep-reinforcement-learning-based autonomous Patton, N. Rau, D. Reppen, A. Schneider, M. Shahidehpour, and C.
voltage control for power grid operations,” IEEE Transactions on Power Singh, “The IEEE reliability test system-1996. a report prepared by the
Systems, vol. 35, no. 1, pp. 814–817, Jan. 2020. reliability test system task force of the application of probability methods
[45] Q. H. Huang, R. K. Huang, W. T. Hao, J. Tan, R. Fan, and Z. Y. Huang, subcommittee,” IEEE Transactions on Power Systems, vol. 14, no. 3, pp.
“Adaptive power system emergency control using deep reinforcement 1010–1020, Aug. 1999.
learning,” IEEE Transactions on Smart Grid, vol. 11, no. 2, pp. 1171– [65] A. K. Singh and B. C. Pal, “Report on the 68-bus, 16-machine, 5-area
1182, Mar. 2020. system,” IEEE PES Task Force on Benchmark Systems for Stability
[46] X. W. Qi, Y. D. Luo, G. Y. Wu, K. Boriboonsomsin, and M. Barth, “Deep Controls, Version 3.3-3rd , 2013.
reinforcement learning enabled self-learning control for energy efficient [66] M. Duval and A. DePabla, “Interpretation of gas-in-oil analysis using
driving,” Transportation Research Part C: Emerging Technologies, vol. new IEC publication 60599 and IEC TC 10 databases,” IEEE Electrical
99, pp. 67–81, Feb. 2019. Insulation Magazine, vol. 17, no. 2, pp. 31–41, Mar. /Apr. 2001.
[47] Y. J. Ye, D. W. Qiu, M. Y. Sun, D. Papadaskalopoulos, and G. [67] B. W. Zhou, X. J. Ma, Y. H. Luo, and D. S. Yang, “Wind power
Strbac, “Deep reinforcement learning for strategic bidding in electricity prediction based on LSTM networks and nonparametric kernel density
markets,” IEEE Transactions on Smart Grid, vol. 11, no. 2, pp. 1343– estimation,” IEEE Access, vol. 7, pp. 165279–165292, Nov. 2019.
1355, Mar. 2020. [68] M. Khodayar and J. H. Wang, “Spatio-temporal graph deep neural
[48] H. C. Xu, H. B. Sun, D. Nikovski, S. Kitamura, K. Mori, and H. network for short-term wind speed forecasting,” IEEE Transactions on
Hashimoto, “Deep reinforcement learning for joint bidding and pricing Sustainable Energy, vol. 10, no. 2, pp. 670–681, Apr. 2019.
of load serving entity,” IEEE Transactions on Smart Grid, vol. 10, no. [69] M. Khodayar and J. Wang, “Probabilistic time-varying parameter iden-
6, pp. 6366–6375, Nov. 2019. tification for load modeling: A deep generative approach,” IEEE Trans-
[49] X. F. Huang, S. H. Hong, M. M. Yu, Y. M. Ding, and J. H. Jiang, actions on Industrial Informatics, 2020.
“Demand response management for industrial facilities: a deep rein- [70] S. Soltan, A. Soltan, and G. Zussman, “A learning-based method for
forcement learning approach,” IEEE Access, vol. 7, pp. 82194–82205, generating synthetic power grids,” IEEE Systems Journal, vol. 13, no.
Jun. 2019. 1, pp. 625–634, Mar. 2019.
[50] J. R. Va´zquez-Canteli and Z. Nagy, “Reinforcement learning for
demand response: a review of algorithms and modeling techniques,”
Applied Energy, vol. 235, pp. 1072–1089, Feb. 2019.
[51] Y. D. Yang, J. Y. Hao, Y. Zheng, X. T. Hao, and B. F. Fu, “Large-scale
home energy management using entropy-based collective multiagent Mahdi Khodayar received the B.Sc. degree in
reinforcement learning framework,” in Proceedings of the 18th Inter- Computer Engineering and the M.Sc. degree in Ar-
national Conference on Autonomous Agents and MultiAgent Systems, tificial Intelligence from Khajeh Nasir Toosi Univer-
2019, pp. 2285–2287. sity of Technology, Tehran, Iran, in 2013 and 2015,
[52] J. T. Qin, X. S. Han, G. J. Liu, S. Wang, W. B. Li, and Z. Jiang, “Wind respectively. In 2017, he was a research assistant
and storage cooperative scheduling strategy based on deep reinforcement at the College of Computer and Information Sci-
learning algorithm,” Journal of Physics: Conference Series, vol. 1213, ence, Northeastern University, Boston, MA, USA.
no. 3, pp. 032002, Jun. 2019. He received his Ph.D. in Electrical Engineering at
[53] F. R. Wei, Z. Q. Wan, and H. B. He, “Cyber-attack recovery strategy for Southern Methodist University, TX, USA, in 2020.
smart grid based on deep reinforcement learning,” IEEE Transactions He is currently an Assistant Professor in the De-
on Smart Grid, vol. 11, no. 3, pp. 2476–2486, May 2020. partment of Computer Science at the University of
[54] Y. S. Nasir and D. N. Guo, “Multi-agent deep reinforcement learning Tulsa, Tulsa, OK, USA. His main research interests include machine learning
for dynamic power allocation in wireless networks,” IEEE Journal on and statistical pattern recognition. He has served as a reviewer for reputable
Selected Areas in Communications, vol. 37, no. 10, pp. 2239–2250, Oct. journals including IEEE Transactions on Neural Networks and Learning
2019. Systems, IEEE Transactions on Industrial Informatics, IEEE Transactions
[55] D. An, Q. Y. Yang, W. M. Liu, and Y. Zhang, “Defending against on Fuzzy Systems, IEEE Transactions on Sustainable Energy, and IEEE
data integrity attacks in smart grid: a deep reinforcement learning-based Transactions on Power Systems. He is currently focused on deep learning,
approach,” IEEE Access, vol. 7, pp. 110835–110845, Aug. 2019. sparse modeling, and spatiotemporal pattern recognition with applications to
power networks, computer vision, and transportation.
[56] B. D. Tan, J. Yang, Y. F. Tang, S. B. Jiang, P. Y. Xie, and W. Yuan, “A
deep imbalanced learning framework for transient stability assessment
of power system,” IEEE Access, vol. 7, pp. 81759–81769, Jun. 2019.
[57] W. Q. Yang, Y. Z. Zhu, and Y. T. Liu, “Fast assessment of short-term
voltage stability of AC/DC power grid based on CNN,” in Proceedings of
Guangyi Liu (SM’12) received the bachelor, master
2019 IEEE PES Asia-Pacific Power and Energy Engineering Conference
and Ph.D degrees from Harbin Institute of Tech-
(APPEEC), 2019, pp. 1–4.
nology and China Electric Power Research Institute
[58] S. L. Zhang, Y. X. Wang, M. Q. Liu, and Z. J. Bao, “Data-based line in 1984, 1987 and 1990, respectively. He is Chief
trip fault prediction in power systems using LSTM networks and SVM,” Scientist, Smart Grid CoE with Envision Digital
IEEE Access, vol. 6, pp. 7675–7686, Dec. 2017. Corporation. His research interests include energy
[59] D. I. Kim, L. F. Wang, and Y. J. Shin, “Data driven method for event management system (EMS) and distribution man-
classification via regional segmentation of power systems,” IEEE Access, agement system (DMS), electricity market, smart
vol. 8, pp. 48195–48204, Mar. 2020. grid, autonomic power system, active distribution
[60] Z. X. Sun, S. S. Zhao, and J. X. Zhang, “Short-term wind power network and application of big data in power system.
forecasting on multiple scales using VMD decomposition, k-means
clustering and LSTM principal computing,” IEEE Access, vol. 7, pp.
166917–166929, Sep. 2019.
[61] M. J. Cui, M. Khodayar, C. Chen, X. N. Wang, Y. Zhang, and M. E.
Khodayar, “Deep learning-based time-varying parameter identification
for system-wide load modeling,” IEEE Transactions on Smart Grid, vol.
10, no. 6, pp. 6102–6114, Nov. 2019.
[62] M. Tan, S. P. Yuan, S. H. Li, Y. X. Su, H. Li, and F. He, “Ultra-short-
term industrial power demand forecasting using LSTM based hybrid
ensemble learning,” IEEE Transactions on Power Systems, vol. 35, no.
4, pp. 2937–2948, Jul. 2020.
220 CSEE JOURNAL OF POWER AND ENERGY SYSTEMS, VOL. 7, NO. 2, MARCH 2021

Jianhui Wang (M’07–SM’12–F’21) received the

Ph.D. degree in Electrical Engineering from Illinois
Institute of Technology, Chicago, Illinois, USA, in
2007. He is a Professor with the Department of
Electrical and Computer Engineering at Southern
Methodist University. Dr. Wang has authored and/or
co-authored more than 300 journal and conference
publications, which have been cited for more than
24,000 times by his peers with an H-index of 81.
He has been invited to give tutorials and keynote
speeches at major conferences including IEEE ISGT,
IEEE SmartGridComm, IEEE SEGE, IEEE HPSC and IGEC-XI. Dr. Wang is
the past Editor-in-Chief of the IEEE Transactions on Smart Grid and an IEEE
PES Distinguished Lecturer. He is also a guest editor of a Proceedings of the
IEEE special issue on power grid resilience. He is the recipient of the IEEE
PES Power System Operation Committee Prize Paper Award in 2015 and the
2018 Premium Award for Best Paper in IET Cyber-Physical Systems: Theory
& Applications. Dr. Wang is a Clarivate Analytics highly cited researcher for
production of multiple highly cited papers that rank in the top 1% by citations
for field and year in Web of Science (2018-2020).

Mohammad E. Khodayar received the B.S. and

M.S. degrees in Electrical Engineering from Amirk-
abir University of Technology (Tehran Polytechnic)
and Sharif University of Technology, respectively;
and the Ph.D. degree in Electrical Engineering from
Illinois Institute of Technology, Chicago, IL, USA,
in 2012. He was a Senior Research Associate in the
Robert W. Galvin Center for Electricity Innovation
at Illinois Institute of Technology. He is currently an
Associate Professor with the Department of Electri-
cal and Computer Engineering, Southern Methodist
University, Dallas, TX, USA. He is an associate editor of IEEE Transactions
on Sustainable Energy, IEEE Transactions on Vehicular Technology and IEEE
Access. His research interests include power system operation and planning,
transportation electrification and applications of machine learning to power
systems.