IEEE TRANSACTIONS ON ENERGY CONVERSION, VOL. 24, NO.

2, JUNE 2009 493

Coordinated Reactive Power Control of a Large Wind

Farm and a STATCOM Using Heuristic
Dynamic Programming
Wei Qiao, Member, IEEE, Ronald G. Harley, Fellow, IEEE,
and Ganesh Kumar Venayagamoorthy, Senior Member, IEEE

Abstract—A novel interface neurocontroller (INC) is proposed In order to meet power factor requirement (e.g., −0.95 to
for the coordinated reactive power control between a large wind 0.95) at the connection point, most wind farms are equipped
farm equipped with doubly fed induction generators (DFIGs) and with switched shunt capacitors for static reactive compensa-
a static synchronous compensator (STATCOM). The heuristic dy-
namic programming (HDP) technique and radial basis function tion [1], [2]. Moreover, because many wind farms are connected
neural networks (RBFNNs) are used to design this INC. It effec- to electrically weak power networks, characterized by low short
tively reduces the level of voltage sags as well as the over-currents circuit ratios and under-voltage conditions, dynamic power elec-
in the DFIG rotor circuit during grid faults, and therefore, signifi- tronic devices such as a static var compensator (SVC) and a static
cantly enhances the fault ride-through capability of the wind farm. synchronous compensator (STATCOM) [3] have been increas-
The INC also acts as a coordinated external damping controller
for the wind farm and the STATCOM, and therefore, improves ingly used in wind farms to provide rapid and smooth reactive
power oscillation damping of the system after grid faults. Simula- compensation and voltage control [4].
tion studies are carried out in PSCAD/EMTDC and the results are When connected to the grid and during a grid fault, the volt-
presented to verify the proposed INC. age sags at the connection point of the wind farm can cause
Index Terms—Heuristic dynamic programming (HDP), inter- a high current in the rotor circuit and the converter. Since the
face neurocontroller (INC), power oscillation damping, reactive power rating of the VFC converter is only 25%–30% of the
power control, static synchronous compensator (STATCOM), wind induction generator power rating, this over-current can lead to
farm. the destruction of the converter. Therefore, one of the key issues
I. INTRODUCTION related to the wind farms equipped with DFIGs is the grid fault
or low voltage ride-through capability. Much research effort has
ECAUSE of the concern about the environmental pollu-
B tion and a possible energy crisis, there has been a rapid
increase in renewable energy sources worldwide in the past
gone into this issue and several techniques have been proposed.
One technique is blocking the RSC and short circuiting the
rotor circuit by a crow-bar circuit to protect the converter from
decade. Among various renewable energy sources, wind power over current in the rotor circuit [1], [5], [6]. The wind turbine
is the most rapidly growing one. generators (WTGs) continue their operation to produce some
Nowadays, the majority of wind turbines are equipped with active power, and the GSCs can be set to control the reactive
doubly fed induction generators (DFIGs). In the DFIG concept, power and voltage. When the fault has been cleared and when
the wound-rotor induction generator is grid-connected at the the voltage and the frequency in the power network have been
stator terminals, as well as at the rotor mains via a partially rated reestablished, the RSC restarts and the WTG returns to normal
variable frequency ac/dc/ac converter (VFC), which only needs operation. In this uninterrupted operation feature, voltage sta-
to handle a fraction (25%–30%) of the total power to achieve bility is a crucial issue. In the case of a weak power network and
full control of the generator. The VFC consists of a rotor-side during a grid fault, the GSC cannot provide sufficient reactive
converter (RSC) and a grid-side converter (GSC) connected power and voltage support due to its small power capacity, and
back-to-back by a dc-link capacitor. there can be a risk of voltage collapse. As a result, the RSC
Manuscript received July 12, 2006; revised February 5, 2008. First published
will not restart and the WTG will be disconnected from the net-
January 13, 2009; current version published May 19, 2009. This work was work. This problem can be solved by using dynamic reactive
supported in part by the National Science Foundation, USA, under Grant ECS compensation. In [6], the authors investigated the application
0524183. Paper no. TEC-00323-2006.
W. Qiao is with the Department of Electrical Engineering, Univer-
of a STATCOM to help with the uninterrupted operation of a
sity of Nebraska-Lincoln, Lincoln, NE 68588-0511 USA (e-mail: wqiao@ wind farm equipped with DFIGs during grid faults. However,
engr.unl.edu). the focus of [6] was to investigate the DFIG behavior with the
R. G. Harley is with the Intelligent Power Infrastructure Consortium (IPIC),
School of Electrical and Computer Engineering, Georgia Institute of Technol-
STATCOM for voltage support during grid faults. In addition,
ogy, Atlanta, GA 30332-0250 USA (e-mail: [email protected]). the power network used in [6] is a simple single machine infinite
G. K. Venayagamoorthy is with the Real-Time Power and Intelligent Sys- bus system, and there is no coordination between the wind farm
tems Laboratory, Department of Electrical and Computer Engineering, Missouri
University of Science and Technology, Rolla, MO 65409-0249 USA (e-mail:
and the STATCOM for reactive power control.
[email protected]). The second solution to enhance the grid-fault ride-through
Color versions of one or more of the figures in this paper are available online capability of the DFIG wind turbines is to improve the con-
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TEC.2008.2001456
trol scheme of the RSC. A nonlinear controller and a fuzzy

0885-8969/$25.00 © 2009 IEEE

494 IEEE TRANSACTIONS ON ENERGY CONVERSION, VOL. 24, NO. 2, JUNE 2009

Fig. 2. Configuration of a DFIG wind turbine connected to a power grid.

Fig. 1. Single-line diagram of the multimachine benchmark power system that
includes a large wind farm and a STATCOM.
Areas 2 and 3 have switched shunt capacitors to support the volt-
age. The detailed description of the system is given in [13].
A STATCOM is placed at bus 6 to provide steady state as well
controller have been proposed in [7] and [8], respectively, for
as transient voltage support for the wind farm. This dynamic
controlling the RSC. Compared with the conventional linear
reactive compensator provides fast and smooth voltage control
control schemes, these control schemes reduce the over current
for the wind farm and enhances the capability of the wind farm
in the rotor circuit during grid faults.
to ride through grid disturbances.
Shunt flexible alternating current transmission system
G1 is modeled as a three-phase infinite source, while the
(FACTS) devices such as the SVC and the STATCOM pro-
other two conventional generators (G2 and G3) are modeled
vide rapid and smooth reactive compensation, and therefore,
in detail, with the exciter and turbine governor dynamics taken
can reduce the level of voltage sags during grid faults. The ap-
into account.
plication of a STATCOM to enhance the capability of a wind
farm (equipped with DFIGs) to ride through grid faults in a III. WIND FARM MODEL
multimachine power system has been reported in [9]. However,
The wind farm is represented by an aggregated model in
the reactive power control of the wind farm and the STATCOM
which over 100 individual wind turbines and DFIGs are mod-
in [9] are independent without coordination; during grid faults,
eled as one equivalent DFIG driven by a single equivalent wind
the voltage control is only achieved by the STATCOM.
turbine [1], [9]. Each individual DFIG wind turbine represents
This paper extends the work of [9] by proposing a novel co-
a 3.6-MW WTG system [6], [14]. The parameters of the equiv-
ordinated reactive power control scheme. It acts as an interface
alent wind turbine and DFIG are given in the Appendix.
controller between a wind farm and a STATCOM. The heuris-
The basic configuration of a DFIG wind turbine connected
tic dynamic programming (HDP) [10], [11] method and radial
to a power grid is shown in Fig. 2 [6], [9]. The wind turbine is
basis function neural networks (RBFNNs) [12] are employed to
connected to the induction generator through a mechanical shaft
design this nonlinear optimal adaptive interface neurocontroller
system, which consists of a low-speed shaft and a high-speed
(INC). Simulation studies are carried out in PSCAD/EMTDC
shaft and a gearbox in between. The wound-rotor induction ma-
to verify the proposed INC.
chine in this configuration is fed from both stator and rotor sides.
The stator is directly connected to the grid while the rotor is fed
II. POWER SYSTEM MODEL through a VFC. In order to produce electrical power at constant
The original four-machine 12-bus benchmark power system voltage and frequency to the utility grid over a wide operat-
in [13] is used as a platform system for studying FACTS device ing range from subsynchronous to supersynchronous speed, the
applications and integration of wind generation. Fig. 1 shows the power flow between the rotor circuit and the grid must be con-
single-line diagram of the extended four-machine 12-bus power trolled both in magnitude and in direction. Therefore, the VFC
system that now includes a large wind farm and a STATCOM. consists of two four-quadrant insulated-gate bipolar transistor
The system consists of six 230-kV busses, two 345-kV busses, (IGBT) pulse-width modulation (PWM) converters connected
and four 22-kV busses. It covers three geographical areas. Area 1 back-to-back by a dc-link capacitor. A crow-bar circuit is used
is predominantly a generation area with most of its generation to short circuit the RSC in order to protect the RSC from over
coming from hydro power (represented by G1 and G2). Area 2, current in the rotor circuit during transient disturbances.
located between the main generation area (area 1) and the main
load center (area 3), has a large 400 MW wind farm (represented A. Wind Power Model
by G4), but this is insufficient to meet local demand. Area 3, The mechanical power of the turbine extracting from the wind
situated about 500 km from area 1, is a load center with some is calculated by [1]:
thermal generation (represented by G3). Furthermore, since the
1
generation unit in area 2 has limited energy available, the system Pm = ρAr vw3 CP (λ, β) (1)
demand must often be satisfied through transmission. The trans- 2
mission system consists of 230-kV transmission lines except for where ρ is the air density in kg/m3 , Ar = πR2 is the area swept
one 345-kV link between areas 1 and 3 (between busses 7 and 8). by the rotor blades in m2 , vw is the wind speed in m/s, CP is the
QIAO et al.: COORDINATED REACTIVE POWER CONTROL OF A LARGE WIND FARM AND A STATCOM USING HDP 495

Fig. 3. Overall control scheme of the RSC: v d r 2 = −sω s σL r iq r , v q r 2 =

sω s (σL r id r + L 2m im s /L s ), σ = 1 − L 2m /L s L r . Fig. 4. Overall control scheme of the GSC.

power coefficient, which is a function of both tip speed ratio λ

and the blade pitch angle β. The CP –λ–β curves depend on the overall vector control scheme of the RSC. In order to achieve
blade design and are given by the wind turbine manufacturer. independent control of the stator active power Ps (by means of
speed control) and reactive power Qs (see Fig. 2) by means of
B. Modeling of the Shaft System rotor current regulation, the instantaneous three-phase rotor cur-
In transient stability studies, the WTG shaft system should be rents ir abc are sampled and transformed to dq components idr
represented by a two-mass model instead of a single lumped- and iq r in the stator-flux oriented reference frame. The reference
mass model [1]. In the two-mass model, separate masses are values for idr and iq r can be determined directly from Qs and
used to represent the low-speed turbine and the high-speed gen- ωr commands, respectively. The actual dq current signals (idr
erator, and the connecting resilient shaft is modeled as a spring and iq r ) are then compared with their reference signals (i∗dr and
and a damper. The motion equations are given by i∗q r ) to generate the error signals, which are passed through two
proportional–integral (PI) controllers to form the voltage signals
2Ht pωt = Tm − Dt ωt − Dtg (ωt − ωr ) − Ttg (2) vdr 1 and vq r 1 . The two voltage signals (vdr 1 and vq r 1 ) are com-
pensated by the corresponding cross coupling terms (vdr 2 and
2Hg pωr = Ttg + Dtg (ωt − ωr ) − Dg ωr − Te (3)
vq r 2 ) to form the dq voltage signals vdr and vq r . They are then
pTtg = Ktg (ωt − ωr ) (4) used by the PWM module to generate the IGBT gate control
signals to drive the rotor-side IGBT converter.
where p = d/dt; ωt and ωr are the turbine and generator rotor The objective of the GSC is to keep the dc-link voltage con-
speed, respectively; Tm and Te are the mechanical torque ap- stant regardless of the magnitude and direction of the rotor
plied to the turbine and the electrical torque of the generator, power. In this paper, the GSC control scheme is also designed
respectively; Ttg is an internal torque of the model; Ht and Hg to regulate the reactive power, Qg , exchanged between the GSC
are the inertia constants of the turbine and the generator, respec- and the grid. During normal operation, the GSC is considered
tively; Dt and Dg are the damping coefficients of the turbine to be reactive neutral by setting Q∗g = 0. This consideration is
and the generator, respectively; Dtg is the damping coefficient reasonable because the VFC rating is only 25%–30% of the
of the flexible coupling (shaft) between the two masses; and Ktg generator rating and the VFC is primarily used to supply the
is the shaft stiffness. active power from the rotor to the power grid. However, the
As discussed in [15], the WTG shaft system described by reactive power controllability of the GSC can be useful during
(2)–(4) has lightly damped low-frequency torsional oscillation the process of voltage reestablishment, after a grid fault has
modes. The natural frequencies of these modes depend on the been cleared and the RSC has been blocked. Fig. 4 shows the
mechanical parameters of the WTG systems, e.g., the inertia overall control scheme of the GSC. The actual signals of the
constants of the wind turbines and DFIGs, and are less than dc-link voltage and the reactive power (Vdc and Qg ) are com-
several hertz on most practical WTG systems. pared with their command values (Vdc ∗
and Q∗g ) to form the error
signals, which are passed through the PI controllers to generate
C. Control of the DFIG the reference signals for the d-axis and q-axis current com-
Control of the DFIG is achieved by control of the VFC, which ponents (i∗dg and i∗q g ), respectively. The instantaneous ac-side
includes control of the RSC and control of the GSC [1], [6], [15]. three-phase currents of the GSC are sampled and transformed
The objective of the RSC is to regulate both the stator active into dq current components idg and iq g by applying the syn-
and reactive powers, Ps and Qs , independently. The reactive chronously rotating reference frame transformation. The actual
power control using the RSC can be applied to keep the stator signals (idg and iq g ) are then compared with the corresponding
voltage Vs within the desired range, when the DFIG feeds into reference signals to form the error signals, which are passed
a weak power system with insufficient local reactive compen- through two PI controllers. The voltage signals (vdg 1 and vq g 1 )
sation. When the DFIG feeds into a strong power system, the are compensated by the corresponding cross coupling terms to
command of Qs can be simply set to zero. Fig. 3 shows the form the dq voltage signals vdg and vq g . They are then used by
496 IEEE TRANSACTIONS ON ENERGY CONVERSION, VOL. 24, NO. 2, JUNE 2009

Fig. 6. Schematic diagram of the INC.

In this section, an adaptive critic design (ACD) approach,

the HDP, and RBFNNs are used to design an external interface
controller for the coordinated reactive power control between
the wind farm and the STATCOM, as shown in Fig. 6. The
dashed line block denotes the plant to be controlled by the
INC. The voltage deviation, ∆V6 , at bus 6 and the active power
Fig. 5. Overall control scheme of the STATCOM. deviation, ∆Pg 4 , of the wind farm are fed into the INC to
produce two supplementary control signals, ∆Qs and ∆QC .
the PWM module to generate the IGBT gate-control signals to They are then added to the steady-state fixed set-point values,
drive the grid-side IGBT converter. Qs0 and QC 0 , respectively, to form the total commanded values
of the compensating reactive power, Q∗s and Q∗C , at the input of
the RSC and the STATCOM controllers. A basic principle is that
IV. STATCOM MODEL
by rapidly varying the amount of reactive power provided by
A STATCOM [3], [16], also known as an advanced SVC, is the DFIG and the STATCOM during grid faults, it is possible to
a shunt connected FACTS device. It generates a set of balanced reduce the level of voltage sags at the PCC, and therefore, control
three-phase sinusoidal voltages at the fundamental frequency, directly the transient imbalances between the electrical output
with rapidly controllable amplitude and phase angle. A typical power and the mechanical input power that are responsible for
application of a STATCOM is for voltage support. In this paper, over current in the rotor circuit. Because of the direct coupling
the STATCOM is modeled as a gate-turn-off thyristor (GTO) between voltage and reactive power, it is straightforward to
PWM converter with a dc-link capacitor. The overall control use the voltage deviation, ∆V6 , as an input signal of the INC.
scheme of the STATCOM is shown in Fig. 5. The objective However, the active power deviation, ∆Pg 4 , of the wind farm is
of the STATCOM is to provide the desired amount of reactive also used as an input of the INC because it provides the INC with
compensation [with the switch (SW) in position 1 in Fig. 5] or additional information of the plant dynamics. In addition, ∆Pg 4
to directly regulate the voltage at the point of common coupling contains the information of system oscillations and can therefore
(PCC) (bus 6) within the desired range (with SW in position 2 be used by the INC to damp postfault power oscillations of
in Fig. 5). This can enhance the capability of the wind farm to the system. The fixed set-point value Qs0 of the DFIG can
ride through transient disturbances in the grid. The block “grid” be determined based on the desired stator side or the net power
in Fig. 5 denotes the power network (see Fig. 1) to which the factor of the induction machine. The choice of Qs0 is also subject
wind farm and the STATCOM are connected. to the limit of the DFIG MVar rating. The value of QC 0 can be
determined by the results of a power flow calculation at a specific
V. DESIGN OF THE INTERFACE NEUROCONTROLLER operating point or to achieve some form of optimal power flow
operation of the network.
Grid faults, even far away from the location of the wind farm, The transfer functions from ∆V6 and ∆Pg 4 to ∆Qs and ∆QC
can cause voltage sags at the connection point of the wind farm. are complex, nonlinear, and depend on the network topology.
This voltage sag will result in an imbalance between the turbine A neural network can solve this problem and avoids having to
input power and the generator output power, and therefore, a derive such analytical functions.
high current in the stator windings of the DFIG. Because of
the magnetic coupling between stator and rotor, this current
will also flow in the rotor circuit and the converter. In addition,
A. Radial Basis Function Neural Network
the power imbalance during the fault will excite low-frequency
torsional oscillations on the WTG shaft system, which leads The neural networks used in this paper are three-layer
to oscillations of the shaft speed and the output active power. RBFNNs with the Gaussian density function as the activa-
These oscillations are lightly damped if there is no specifically tion function in the hidden layer (see Fig. 7) [12]. The over-
designed damping control for the WTG system. all input–output mapping for the RBFNN, fˆ : X ∈ Rn → Y ∈
QIAO et al.: COORDINATED REACTIVE POWER CONTROL OF A LARGE WIND FARM AND A STATCOM USING HDP 497

Fig. 8. Structure of the model network: TDL denotes time delay lock.

follows. If J(k) is optimal, then J(k + 1) and U (k) are both

optimal; if J(k + 1) is optimal, then J(k + 2) and U (k + 1) are
both optimal; and so on. In other words, if J(k) is optimal, then
U (n), for n = k, k + 1, . . . , ∞, are all optimal, and vice versa.
Therefore, if a control action optimizes the cost-to-go function
Fig. 7. Three-layer RBFNN. J(·) at time step k, then it optimizes the utility function U (·)
from time step k and onward.
Rm is The conventional dynamic programming approaches require
 


h
x − Cj 2
an accurate analytical model of the system dynamics, as well
ŷi = bi + vj i exp − (5) as knowledge of the system comprehensive dynamics known
βj2
j =1 a priori to develop an appropriate cost function J(·). These
where x is the input vector, Cj ∈ Rn is the center of the jth however are normally unavailable for many complex nonlin-
radial basis function (RBF) units in the hidden layer, h is the ear systems. Therefore, it is difficult to obtain an accurate so-
number of RBF units, bi and vj i are the bias term and the weight lution (i.e., an optimal control) for such systems in dynamic
between hidden and output layers, respectively, and ŷi is the ith programming. The ACD method offers an approach to find an
output. approximate solution to dynamic programming.
The locations of RBF centers are determined offline using a The HDP, belonging to the family of ACDs, requires three
k-means clustering algorithm [17]. Once the RBF centers are neural networks, one for the model, one for the critic, and one for
established, the width βi of the ith RBF unit in the hidden layer the action network for its implementation [10], [11]. The model
is calculated as follows network is used to provide a dynamical model of the plant for
 1/2 training the critic and action networks; the critic network esti-
1
h mates the cost function J in (7); the action network provides
βi =  Ci − Cj  
2
(6) the control action for the plant. Based on an accurate model net-
h j =1
work, the ACD method determines optimal control laws for a
where Ci and Cj are the center of the ith and jth RBF units, system by successively adapting the critic and action networks.
respectively. In (5) and (6),  · · ·  represents the Euclidean The adaptation process starts with a nonoptimal control by the
norm. action network; the critic network then guides the action net-
work toward the optimal solution at each successive adaptation.
B. Adaptive Critic Designs and Heuristic Dynamic This adaptation process uses the concept of reinforcement learn-
Programming ing. During the adaptations, neither of the networks needs any
information of the desired control trajectory, only the desired
ACDs, proposed by Werbos [10], is a neural-network-based
cost needs to be known.
optimization and control technique that solves the classical non-
linear optimal control problem by combining concepts of ap- C. Design of the Model Network
proximate dynamic programming and reinforcement learning.
Dynamic programming may provide the best approach to The model network is a three-layer RBFNN with 25 hidden
design the optimal control for highly constrained nonlinear sys- neurons. The plant inputs A = [∆Qs , ∆QC ] and outputs Y =
tems [18]. In dynamic programming, such an optimal control is [∆V6 , ∆Pg 4 ] at time k, k − 1, and k − 2 are fed into the model
obtained by solving the Bellman equation that optimizes some network to estimate the plant outputs Ŷ = [∆V̂6 , ∆P̂g 4 ] at time
cost-to-go function J of the system, defined as k + 1, as shown in Fig. 8. The sampling period for the RBFNN
∞
implementation is 1 ms.

The model network is pretrained offline using a suitably se-
J(k) = γ q U (k + q) = γJ(k + 1) + U (k) (7)
q =0
lected training data set collected from two sets of training. The
first set is called forced training in which the plant is perturbed
where U (·) is the utility function (user-defined function) that by injected small pseudorandom binary signals (PRBSs) (with
represents the one-stage cost or performance measure function S1 and S2 both in position 2 in Fig. 6), given by
of the system at each time step, and γ is a discount factor for
finite horizon problems (0 < γ < 1). Equation (7) describes the 0.1|Qs0 |[r0(k) + r1(k) + r2(k)]
PRBS Qs (k) = (8)
basic principle of dynamic programming: optimizing J(·) in the 3
short term is equivalent to optimizing U (·) in the long term, and 0.1|QC 0 |[r0(k) + r1(k) + r2(k)]
vice versa. This principle can be explained in more details as PRBS QC (k) = (9)
3
498 IEEE TRANSACTIONS ON ENERGY CONVERSION, VOL. 24, NO. 2, JUNE 2009

TABLE I
OPERATING CONDITIONS SELECTED FOR NATURAL TRAINING
OF THE MODEL NETWORK

Fig. 9. Structure of the critic network.

where r0, r1, and r2 are uniformly distributed random numbers

in [−1, 1] with frequencies 0.5, 1, and 2 Hz, respectively. The
second set is called natural training in which the PRBS is re-
moved (with S1 and S2 both in position 1 in Fig. 6), and the
system is exposed to natural disturbances and faults in the power Fig. 10. Adaptation of the critic network in HDP.
network. The forced training and natural training are carried out
at several different operating points to form the training data set, The critic network learns to minimize the following error
given by measure over time [11]:
  1
T
m m n  EC  = EC (k)EC (k) (11)
A = {X, Y} = AFi , ANij (10) 2
  k
i=1 i=1 j =1
where
where A is the entire training data set selected from m operat- EC (k) = J[Ŷ (k)] − γJ[Ŷ (k + 1)] − U (k). (12)
ing points; X and Y are the input and output data sets of the
The objective of the INC (see Fig. 6) is to provide an optimal
model network, respectively; AF i is the subset collected from
coordinating control that minimizes the voltage deviations at
the forced training at the operating point i; AN ij is the subset
bus 6, ∆V6 , as well as the active power oscillations, ∆Pg 4 , of
collected from the natural training caused by the jth natural
the wind farm. Therefore, the utility function is defined as
disturbance event at the operating point i. Table I shows the se-
lected five operating points for the natural training of the model 1 
U (k) = ∆V62 (k) + 0.5∆V62 (k − 1) + 0.1∆V62 (k − 2)
network in which vw , ω4 , Pg 4 , Qg , Qs , and QC are the wind 2
speed, DFIG rotor speed, output active power of the wind farm, 1 
reactive power of the GSC, reactive power of the DFIG stator, + ∆Pg24 (k)+0.5∆Pg24 (k − 1) + 0.1∆Pg24 (k−2) .
2
and the compensated reactive power from the STATCOM, re-
(13)
spectively. In this paper, three different natural disturbances are
applied at each operating point in Table I: 1) a 150-ms tempo- In (13), it is natural to use time-delayed values of ∆V6 and
rary three-phase short circuit at the bus 1 end of line 1–6; 2) a ∆Pg 4 because power systems are causal systems in which an
150-ms temporary phase A to ground short circuit at the bus 4 output depends on the present as well as past input values.
end of one of the parallel lines 3–4; and 3) wind speed variations Generally, two critic networks are required in HDP to estimate
around the mean values in Table I using the wind model in [19], the cost-to-go function J arising from the present state Ŷ (k)
which causes the variations of Pg 4 in the range of ±50 MW at and the future state Ŷ (k + 1), respectively. The critic’s output
each operating point. J(k + 1) is necessary to generate the target signal γJ(k +
The selected training data set ensures that the model network 1) + U (t), for training the critic network. In the case of mini-
can track the system dynamics over a wide operating range. mization in the LMS, the output weights of the critic network
After determining the training data set, the weights of the model are updated by
network are then calculated by a least mean squares (LMS)
∂J[Ŷ (k)]
method [20]. ∆WC (k) = −ηC {J[Ŷ (k)]−γJ[Ŷ (k+1)] − U (k)}
∂WC
(14)
D. Design of the Critic Network where ηC is a positive learning gain. The adaptation of the critic
network in HDP is shown in Fig. 10.
The critic network is a three-layer RBFNN with 15 hidden
neurons. The inputs to the critic network are the estimated plant
outputs, Ŷ = [∆V̂6 , ∆P̂g 4 ], from the model network and its E. Design of the Action Network
two time-delayed values. The output of the critic network is the The action network (see Fig. 11) is a three-layer RBFNN
estimate of the function J in (7) with respect to the estimated with 20 hidden neurons. The inputs to the action network are
plant output Ŷ , as shown in Fig. 9. the plant outputs Y = [∆V6 , ∆Pg 4 ], at time k − 1, k − 2, and
QIAO et al.: COORDINATED REACTIVE POWER CONTROL OF A LARGE WIND FARM AND A STATCOM USING HDP 499

Fig. 11. Structure of the action network.

Fig. 13. Comparison of the voltage magnitude at bus 6 with and without the
INC (STATCOM in reactive power control mode in the case of no INC).

the critic’s pretraining, the plant is perturbed by injecting small

Fig. 12. Adaptation of the action network in HDP.
PRBS given by (8) and (9) to Qs0 and QC 0 , respectively (with
S1 and S2 both in position 2 in Fig. 6).
k − 3. The outputs of the action network are the plant inputs, Once the critic’s pretraining is over, S1 and S2 switch to po-
A = [∆Qs , ∆QC ], at time k. sition 1 and the INC is used to provide an external control for
The objective of the action network adaptation is to find out the STATCOM and the RSC of the DFIG. Then, the critic’s
the optimal control trajectory, Aopt , in order to minimize the weights are fixed, the action network is trained by the proce-
cost-to-go function J over time: dure in Fig. 12 for NA cycles. Then, the action’s weights are
fixed, and the critic network is trained further for NC cycles.
Aopt (k) = arg min[J(k)] = arg min[U (k) + γJ(k + 1)]. This process of training the critic/action networks is repeated
A A
(15) one after the other until an accepted performance is achieved.
Such adaptation, as shown in Fig. 12, is achieved by Once the critic and action networks’ weights have converged,
training the action network with the error signal, EA (k) = the action network with the fixed weights is used to control the
∂J(k)/∂A(k), which is obtained by propagating the constant, plant during the real-time operation.
∂J/∂J = 1, back through the critic and model to the action net-
work [11]. The output weights of the action network are then VI. SIMULATION RESULTS
updated by
  Simulation studies are carried out in this section to examine
∂J(k) ∂ ∂J(k) the proposed INC. The wind farm initially operates at an oper-
∆WA (k) = −ηA . (16)
∂A(k) ∂WA ∂A(k) ating point with the wind speed vw = 11.0 m/s, generator rotor
speed ω4 = 1.2 pu, output active power Pg 4 = 300 MW, and
output reactive power Qg 4 = 0. The reactive power command
F. Overall Training Procedure of the GSC is set at Q∗g = 0. The steady-state fixed reactive
power commands of the RSC and the STATCOM are set at
The training procedure to implement the HDP algorithm con-
Qs0 = 0 and QC 0 = 165 MVar, respectively. The voltage at
sists of two training stages: one for the model network and the
bus 6 is regulated at V6 = 1.02 pu. A three-phase short circuit
other for the critic/action networks. The model network is first
is applied to the bus 1 end of line 1–6 at 1 s and is cleared after
pretrained offline to learn the plant dynamics before training
150 ms. This scenario has been used in the pretraining of the
the critic and action networks, as described in Section V-C.
model network, but has not been used for training the critic and
During the training of the critic and action networks, the wind
action networks. The dynamic performance of the wind farm,
speed is varied over a certain range (e.g., ±2 m/s around the
reinforced with the INC, is compared with the cases without the
mean wind speed) using the wind model in [19] to simulate the
INC.
real operation of the wind farm. Consequently, the output active
power of the wind farm varies significantly from time to time.
During this time, the model network can be trained further to A. STATCOM in Reactive Power Control Mode in the Case
of No INC
adapt to the operating conditions that are not covered by the
pretraining. Figs. 13–15 compare the system responses with and without
The training stage of the critic/action networks contains two the INC. In the case of no INC, the reactive power control is
separate training cycles: one for the critic and the other for the applied to the STATCOM (with SW in position 1 in Fig. 5).
action. The critic network is first pretrained by the procedure In this case, the reactive power commands of the RSC and the
in Fig. 10 to approximate the cost-to-go function J. During STATCOM are both constant. This control arrangement cannot
500 IEEE TRANSACTIONS ON ENERGY CONVERSION, VOL. 24, NO. 2, JUNE 2009

Fig. 16. Comparison of the voltage magnitude at bus 6 with and without the
Fig. 14. Comparison of the output active power of the wind farm with and
INC (STATCOM in voltage control mode in the case of no INC).
without the INC (STATCOM in reactive power control mode in the case of no
INC).

Fig. 17. Comparison of the output active power of the wind farm with and
without the INC (STATCOM in voltage control mode in the case of no INC).
Fig. 15. Comparison of the DFIG rotor current magnitude with and without
the INC (STATCOM in reactive power control mode in the case of no INC).

B. STATCOM in Voltage Control Model in the Case of No INC

contribute to improving the transient behavior of the wind farm Now the voltage control is applied to the STATCOM (with
or the damping of power oscillations in the system. On the SW in position 2 in Fig. 5). In this case, the reactive power com-
contrary, the INC provides the RSC and the STATCOM with mand of the RSC is still constant, but the STATCOM controller
supplementary control capability in response to voltage sags can contribute to improving the transient behavior of the wind
and power oscillations during a transient disturbance. As shown farm during voltage sags. As shown in Figs. 16–18, the voltage
in Fig. 13, the INC significantly reduces the magnitudes of volt- sag at bus 6 (see Fig. 16) and the maximum rotor current (see
age sag and voltage overshot at bus 6 during the three-phase Fig. 18) are almost the same for both cases with and without
short circuit. Fig. 14 shows the output active power of the wind the INC; however, the voltage overshoot (see Fig. 16) and the
farm. By using the active power deviation signal, ∆Pg 4 , as an magnitude of active power oscillations (see Fig. 17) in the case
input to the INC, the power oscillation damping with the INC of the INC are much smaller than for no INC. These results
is much better than that without the INC. Finally, the mag- are consistent with the design objectives, namely, the INC is
nitudes of the DFIG rotor current, Ir , are shown in Fig. 15. optimally designed to minimize the voltage deviation at bus 6
In this test, the reference values of the DFIG rotor currents, as well as the magnitudes of active power oscillations in terms
i∗dr and i∗q r in Fig. 3, are limited to 6.5 and 16 kA for both of the utility function U in (13).
cases with and without the INC. The peak value of the transient Another important result is shown in Fig. 19. It indicates
rotor current (from 1.05 s onward) without using the INC is that the amount of the compensated reactive power required by
about 18 kA, while this value reduces to 14 kA when using the the STATCOM when using the INC is less than half of that
INC. The INC significantly reduces the magnitude of the DFIG without the INC. Therefore, the size of the STATCOM can
rotor current transient during the 150 ms short circuit. There- be significantly reduced when using the INC to provide the
fore, it enhances the fault ride-through capability of the wind coordinated reactive power control for the wind farm and the
farm. STATCOM.
QIAO et al.: COORDINATED REACTIVE POWER CONTROL OF A LARGE WIND FARM AND A STATCOM USING HDP 501

Fig. 18. Comparison of the DFIG rotor current magnitude with and without Fig. 21. Comparison of the output active power of the wind farm with and
the INC (STATCOM in voltage control mode in the case of no INC). without the INC (STATCOM in voltage control mode with reactive power
limitation in the case of no INC).

Fig. 19. Comparison of the compensated reactive power by the STATCOM

with and without the INC (STATCOM in voltage control mode in the case of no Fig. 22. Comparison of the DFIG rotor current magnitude with and without
INC). the INC (STATCOM in voltage control mode with reactive power limitation in
the case of no INC).

rotor current (Ir ) transient is not a control objective of the INC,

but the rotor current is always controlled within its limit (e.g.,
16 kA in this application) during the 150 ms grid fault, as shown
in Fig. 18. Moreover, the postfault rotor currents (from 1.15 s)
decay rapidly for both cases with and without the INC.

C. STATCOM in Voltage Control Mode With Reactive Power

Limitation in the Case of No INC
In this test, the STATCOM is still operated in the voltage
control mode (with SW in position 2 in Fig. 5), but now the
compensated reactive power QC of the STATCOM is limited to
Fig. 20. Comparison of the compensated reactive power by the STATCOM 250 MVar (by putting suitable limits to the current references
with and without the INC (STATCOM in voltage control mode with reactive i∗dv and i∗q v in Fig. 5). Fig. 20 indicates that the maximum values
power limitation in the case of no INC). of QC are limited to 250 MVar for both cases with and without
the INC. However, the postfault power oscillations of QC (see
More reactive compensation from the STATCOM in the case Fig. 20) and Pg 4 (see Fig. 21) in the case of the INC are damped
of no INC (see Fig. 19) contributes to the rapid decay of the more rapidly than for no INC. In addition, the peak value of the
rotor current transient (see Fig. 18). On the other hand, in terms rotor current transient is reduced when using the INC, as shown
of the utility function (13), the INC actually optimally controls in Fig. 22. These results again confirm that the INC provides a
the reactive compensation from the RSC and the STATCOM smart coordinating control for the system. It improves the fault
to decay (or damp) the voltage (V6 ) and active power (Pg 4 ) ride-through capability of the wind farm and power oscillation
transients as fast as possible. In this design, fast decay of the damping of the system during this transient disturbance.
502 IEEE TRANSACTIONS ON ENERGY CONVERSION, VOL. 24, NO. 2, JUNE 2009

VII. CONCLUSION [11] D. V. Prokhorov and D. C. Wunsch, “Adaptive critic designs,” IEEE
Trans. Neural Netw., vol. 8, no. 5, pp. 997–1007, Sep. 1997.
A large wind farm equipped with DFIGs connected to a mul- [12] W. Qiao and R. G. Harley, “Indirect adaptive external neuro-control for a
timachine benchmark power system has been modeled in the series capacitive reactance compensator based on a voltage source PWM
converter in damping power oscillations,” IEEE Trans. Ind. Electron.,
PSCAD/EMTDC environment. A STATCOM has been placed vol. 54, no. 1, pp. 77–85, Feb. 2007.
at the bus where the wind farm is connected to the power network [13] S. Jiang, U. D. Annakkage, and A. M. Gole, “A platform for validation of
for steady-state and transient reactive power compensation. The FACTS models,” IEEE Trans. Power Del., vol. 21, no. 1, pp. 484–491,
Jan. 2006.
control schemes of the DFIG RSC, GSC, and the STATCOM [14] N. W. Miller, W. W. Price, and J. J. Sanchez-Gasca, “Dynamic modeling of
have been suitably designed. GE 1.5 and 3.6 wind turbine-generators,” in GE-Power Systems Energy
A novel INC, based on the HDP approach and RBFNNs, Consulting. Schenectady, NY: General Electric International, Oct. 27,
2003.
has been designed for the coordinated reactive power control [15] W. Qiao, W. Zhou, J. M. Aller, and R. G. Harley, “Wind speed estimation
between the wind farm and the STATCOM. Simulation studies based sensorless output maximization control for a wind turbine driving
have been carried out to examine the performance of the a DFIG,” IEEE Trans. Power Electron., vol. 23, no. 3, pp. 1156–1169,
May 2008.
proposed INC during grid faults. Results have shown that the [16] C. Schauder and H. Mehta, “Vector analysis and control of advanced static
INC effectively reduced the level of voltage sags as well as VAR compensators,” Inst. Elect. Eng. Proc.—Generation, Transmiss.
the over currents in the DFIG rotor circuit during grid faults, Distrib., vol. 140, no. 4, pp. 299–306, Jul. 1993.
[17] K. Alsabti, S. Ranka, and V. Singh, “An efficient k-means clustering
and therefore, significantly enhanced the fault ride-through algorithm,” presented at the First Workshop High Perform. Data Mining,
capability of the wind farm. Moreover, the INC acts as a Orlando, FL, Mar. 1998.
coordinated external damping controller for the wind farm and [18] D. A. White and M. I. Jordan, “Optimal control: A foundation for intelli-
gent control,” in Handbook of Intelligent Control, D. White and D. Sofge,
the STATCOM, and therefore, improves the postfault power Eds. New York: Van Nostrand Reinhold, 1992, pp. 185–214.
oscillation damping of system. [19] W. Qiao, R. G. Harley, and G. K. Venayagamoorthy, “Dynamic modeling
of wind farms with fixed-speed wind turbine generators,” in Proc. IEEE
PES 2007 General Meet., Tampa, FL.
APPENDIX [20] S. S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd ed.
Englewood Cliffs, NJ: Prentice-Hall, 1998.
Equivalent wind turbine: rated capacity = 400 MW, number
of blades = 3, rotor speed (variable) = 8.5–15.3 rpm. Wei Qiao (S’05–M’08) received the B.Eng. and
Mechanical shaft system (on 400 MW base): Ht = 4.29 s, M.Eng. degrees in electrical engineering from Zhe-
jiang University, Hangzhou, China, in 1997 and 2002,
Hg = 0.9 s, Dt = Dg = 0, Dtg = 1.5 pu, Ktg = 296.7 pu. respectively, the M.S. degree in high performance
Equivalent wound rotor induction generator: rated computation for engineered systems from Singapore–
power = 400 MW, rated stator voltage = 22 kV, power MIT Alliance (SMA), Singapore, in 2003, and the
Ph.D. degree in electrical engineering from Georgia
factor pf = −0.9 to + 0.9, rs = 0.0079 pu, rr = 0.025 Institute of Technology, Atlanta, in 2008.
pu, Lls = 0.07937 pu, Llr = 0.40 pu, Lm = 4.4 pu, base From 1997 to 1999, he was an Electrical Engi-
frequency f =60 Hz. neer in China Petroleum and Chemical Corporation
(SINOPEC). Currently, he is an Assistant Professor
of Electrical Engineering at the University of Nebraska-Lincoln. His research in-
REFERENCES terests include renewable energy systems and distributed generation, microgrids,
power system control, stability and performance optimization, power electron-
[1] V. Akhmatov, “Analysis of dynamic behavior of electric power sys- ics, electric machines, flexible ac transmission systems (FACTS) devices, and
tems with large amount of wind power” Ph.D. dissertation, Tech. Univ. the application of computation intelligence in electric energy systems. He is the
Denmark, Kgs. Lyngby, Denmark, Apr. 2003. author or coauthor of more than 30 papers published in refereed journals and
[2] J. Hochheimer, “Wind generation integration & operation—Technical international conferences and two book chapters.
challenges/issues,” in Proc. IEEE PES General Meet. 2006, Montreal, Dr. Qiao is the Technical Program Co-Chair of the 2009 IEEE Symposium
Canada. on Power Electronics and Machines in Wind Applications. He was the recipient
[3] N. G. Hingorani and L. Gyugyi, Understanding FACTS: Concepts and of the first prize in the Student Paper and Poster Competition of the IEEE Power
Technology of Flexible AC Transmission Systems. New York: IEEE, Engineering Society General Meeting 2006 in Montreal, Canada.
2000.
[4] S. W. Steven, “Wind parks as power plants,” in Proc. IEEE PES General
Meet. 2006, Montreal, Canada. Ronald G. Harley (M’77–SM’86–F’92) received the
[5] J. Morren and S. W. H. de Haan, “Ridethrough of wind turbines with M.Sc.Eng. degree (cum laude) in electrical engineer-
doubly-fed induction generator during voltage dip,” IEEE Trans. Energy ing from the University of Pretoria, Pretoria, South
Convers., vol. 20, no. 2, pp. 435–441, Jun. 2005. Africa, in 1965, and the Ph.D. degree in electrical
[6] W. Qiao, G. K. Venayagamoorthy, and R. G. Harley, “Real-time imple- engineering from London University, London, U.K.,
mentation of a STATCOM on a wind farm equipped with doubly fed in 1969.
induction generators,” in Proc. IEEE IAS 41st Annu. Meet., Tampa, FL, During 1971, he was the Chair of Electrical Ma-
Oct. 8–12, 2006, pp. 1073–1080. chines and Power Systems at the University of Natal,
[7] A. Mullane, G. Lightbody, and R. Yacamini, “Wind-turbine fault ride- Durban, South Africa. He is currently the Duke Power
through enhancement,” IEEE Trans. Power Syst., vol. 20, no. 4, pp. 1929– Company Distinguished Professor at the Georgia
1937, Nov. 2005. Institute of Technology, Atlanta. His current research
[8] R. G. de Almeida, J. A. P. Lopes, and J. A. L. Barreiros, “Improving interests include the dynamic behavior and condition monitoring of electric
power system dynamic behavior through doubly fed induction machines machines, motor drives, power systems and their components, and controlling
controlled by static converter using fuzzy control,” IEEE Trans. Power them by the use of power electronics and intelligent control algorithms. He is
Syst., vol. 19, no. 4, pp. 1942–1950, Nov. 2004. the author or coauthor of some 400 papers published in refereed journals and
[9] W. Qiao, R. G. Harley, and G. K. Venayagamoorthy, “Effects of FACTS international conferences and three patents.
devices on a power system which includes a large wind farm,” in Proc. Prof. Harley was one of the IEEE Industry Applications Society’s six Dis-
IEEE PES Power Syst. Conf. Expo. 2006, Atlanta, GA, pp. 2070–2076. tinguished Lecturers during 2000 and 2001. He was the recipient of the Cyril
[10] P. J. Werbos, “Approximate dynamic programming for real-time control Veinott Electromechanical Energy Conversion Award from the IEEE Power
and neural modeling,” in Handbook of Intelligent Control, D. White and Engineering Society for “Outstanding contributions to the field of electrome-
D. Sofge, Eds. New York: Van Nostrand Reinhold, 1992, pp. 493–526. chanical energy conversion” in 2005.
QIAO et al.: COORDINATED REACTIVE POWER CONTROL OF A LARGE WIND FARM AND A STATCOM USING HDP 503

Ganesh Kumar Venayagamoorthy (S’91–M’97– Dr. Venayagamoorthy is the Chair of the Working Group on Intelligent Con-
SM’02) received the Ph.D. degree in electrical en- trol Systems and the Secretary of the Intelligent Systems subcommittee of the
gineering from the University of KwaZulu Natal, IEEE Power Engineering Society. He is the General Chair of 2008 IEEE Swarm
Durban, South Africa, in February 2002. Intelligence Symposium and the Program Chair of the 2009 IEEE–International
He is currently an Associate Professor of Elec- Neural Network Society (INNS) International Joint Conference on Neural Net-
trical and Computer Engineering and the Director of works. He was the recipient of the 2007 U.S. Office of Naval Research Young
the Real-Time Power and Intelligent Systems Labo- Investigator Program Award, the 2004 National Science Foundation CAREER
ratory, Missouri University of Science and Technol- Award, the 2006 IEEE Power Engineering Society Walter Fee Outstanding
ogy (Missouri S&T), Rolla. During 2007, he was a Young Engineer Award, the 2005 IEEE Industry Applications Society (IAS)
Visiting Researcher at the ABB Corporate Research, Outstanding Young Member Award, the 2003 INNS Young Investigator Award,
Vasteras, Sweden. He has attracted in excess US$4 and the 2007/2005 Missouri S&T Faculty Excellence Award.
Million in competitive research funding from external funding agencies. His
current research interests include the development and applications of computa-
tional intelligence for power systems stability and control, alternative sources of
energy, and flexible alternating current transmission systems (FACTS) devices.
He is the author or coauthor of more than 57 refereed journals papers, 200
refereed international conference proceeding papers, two edited books, and five
book chapters.