DEPARTMENT OF INFORMATION TECHNOLOGY

IDLAB

DESIGN AND OPTIMIZATION OF

MODERN RF/MW SYSTEMS:
A BAYESIAN PERSPECTIVE
Tom Dhaene, Ivo Couckuyt, Domenico Spina
IDLab, Ghent University - imec, Belgium
 DEPARTMENT OF INFORMATION TECHNOLOGY
IDLAB

-1- GAUSSIAN PROCESS (GP)

-2- BAYESIAN OPTIMIZATION (BO)

-3- BO EXAMPLE
 DEPARTMENT OF INFORMATION TECHNOLOGY
IDLAB

-1- GAUSSIAN PROCESS (GP)

-2- BAYESIAN OPTIMIZATION (BO)

-3- BO EXAMPLE
GAUSSIAN PROCESS (GP)

• Definition
a Gaussian Process (GP) is a collection of random variables,
any finite number of which have a joint Gaussian distribution.

Gaussian Process Gaussian distribution

• distribution over functions • distribution over vectors

• fully specified by a • fully specified by a
mean function & mean &
covariance function covariance

6
GAUSSIAN PROCESS - HEARTBEAT

80
heartbeats/min
70
60

0 12 24
time

draw 1 sample from

Gaussian prior

60 70 80 @12:00
heartbeats/min 7
GAUSSIAN PROCESS - HEARTBEAT

80

80
heartbeats/min

heartbeats/min
70

70
60

60
0 12 24 0 6 12 18 24
time time

80
draw 2 samples from
Gaussian prior
heartbeats/min
@18:00

70

@1
8:0
60

0 :00
@06
60 70 80
heartbeats/min 8
@06:00
GAUSSIAN PROCESS - HEARTBEAT

80

80
80
heartbeats/min

heartbeats/min
heartbeats/min
70

70
70
60

60
60
0 12 24 0 12 24 0 12 24
time time time

draw N samples from

Gaussian prior

9
GAUSSIAN PROCESS - HEARTBEAT

3 observations
(evidence)

draw functions from draw functions from Gaussian Process

Gaussian prior posterior mean + variance
function
80

80
80
heartbeats/min

heartbeats/min
heartbeats/min
70

70
70
60

60
60

0 12 24 0 12 24 0 12 24
time time time
10
GAUSSIAN PROCESS - HEARTBEAT

3 observations
(evidence)

Gaussian Process
mean + variance
function

80
heartbeats/min
70
60
0 12 24
time
11
a Gaussian Process knows what it doesn’t know
GAUSSIAN PROCESS - HEARTBEAT

80

80
80
heartbeats/min

heartbeats/min
heartbeats/min
70

70
70
60

60
60
0 12 24 0 12 24 0 12 24
time time time
80

80
80
heartbeats/min

heartbeats/min
heartbeats/min
70

70
70
60

60
60

0 12 24 0 12 24 0 12 24
time time time
12
slide: Michael Pearce, University of Warwick & Juergen Branke, Warwick Business School
GAUSSIAN PROCESS

13
GAUSSIAN PROCESS

• Gaussian distributions over functions

14
GAUSSIAN PROCESS

• Gaussian distributions over functions

• specified by a mean function & covariance (kernel) function

15
GAUSSIAN PROCESS

• Gaussian distributions over functions

• specified by a mean function & covariance (kernel) function
• simple math

joint distribution

all Gaussians J :

closed-form conditional distribution

calculations possible

marginal distribution

16
GAUSSIAN PROCESS

• Gaussian distributions over functions

• specified by a mean function & covariance (kernel) function
• simple math
• simple code

17
GAUSSIAN PROCESS

• Gaussian distributions over functions

• specified by a mean function & covariance (kernel) function
• simple math
• simple code

• non-parametric
• data-driven

18
GAUSSIAN PROCESS

• Gaussian distributions over functions

• specified by a mean function & covariance (kernel) function
• simple math
• simple code

• non-parametric computational complexity

• data-driven
• training = O(N³)
• auto-hyperparametrization • prediction mean = O(N)
• learnt from data • prediction variance = O(N2)
• interpretable è practical limit : N ≅ 10 000

“ a GP lets the data speak for itself ”

... but all the data must speak
19
GAUSSIAN PROCESS

• Gaussian distributions over functions

• specified by a mean function & covariance (kernel) function GP = multilayer perceptron
• simple math with infinitely many hidden units
• simple code
& Gaussian priors on the weights
• non-parametric
• data-driven
deep GP :
• auto-hyperparametrization non-stationary behavior
• learnt from data
• Interpretable

• link: Gaussian Process (GP) <> Neural network (NN)

20
GAUSSIAN PROCESS

• Gaussian distributions over functions

• specified by a mean function & covariance (kernel) function
• simple math
• simple code

• non-parametric
• data-driven

• auto-hyperparametrization
• learnt from data
• Interpretable

• link: Gaussian Process (GP) <> Neural network (NN)

21
GAUSSIAN PROCESS

• Gaussian distributions over functions

• specified by a mean function & covariance (kernel) function
• simple math
• simple code

• non-parametric
• data-driven
• auto-hyperparametrization
• learnt from data
• Interpretable

• link: Gaussian Process (GP) <> Neural network (NN)

• probabilistic
• prior knowledge can be added
• well-calibrated uncertainty estimates
22
GAUSSIAN PROCESS

23
GAUSSIAN PROCESS

classification example

GP NN RF

24
GAUSSIAN PROCESS

example RF

GP NN

25
GAUSSIAN PROCESS

example NN RF

GP

26
GAUSSIAN PROCESS

example NN RF

GP

Achilles’ heel of ML
NN & RF : over-confidence far from training data

27
GAUSSIAN PROCESS

example NN RF

GP

Achilles’ heel of ML
NN & RF : over-confidence ➜ untrustworthy

Achilles’ heel of ML 28
GAUSSIAN PROCESS

example GP NN RF

Gaussian Processes know what they don’t know

Key benefit 29
 DEPARTMENT OF INFORMATION TECHNOLOGY
IDLAB

-1- GAUSSIAN PROCESS (GP)

-2- BAYESIAN OPTIMIZATION (BO)

-3- BO EXAMPLE
PROBLEM SETTING
• Modern engineering problems
• Large-scale problems
• Many parameters
• Multiple design requirements
• …
[Chinea]
• Difficult to design and optimize
• Expensive simulations / measurements
• no global analytical cost function

31
PROBLEM SETTING
• Global optimization problem
• Given !: Χ → ℝ where Χ ∈ ℝ'
() = argmin ! 1 3. 5. 6 1 <0
1∈2

• “Black box” system

• unknown & multimodal behaviour
• limited & expensive data samples
• (possibly) noisy data

32
PROBLEM SETTING
• Global optimization problem
• Given !: Χ → ℝ where Χ ∈ ℝ'
() = argmin ! 1 3. 5. 6 1 <0
1∈2

• data-efficient Bayesian Optimization approach

33
BAYESIAN OPTIMIZATION
• 2 key elements
• Surrogate-based approach
= Mathematical model
• Sequential sampling

34
BAYESIAN OPTIMIZATION
• Flow chart

Acquisition function

35
BAYESIAN OPTIMIZATION
Function
Initial samples evaluation
#
" # = "% %&' ! "#

36
BAYESIAN OPTIMIZATION
Function
Initial samples evaluation
#
" # = "% %&' ! "#

Expensive

37
BAYESIAN OPTIMIZATION
Function
Initial samples evaluation
#
" # = "% %&' ! "#

Mathematical
model

38
BAYESIAN OPTIMIZATION
Function
Initial samples evaluation
#
" # = "$ $)& ! "#

New sample Mathematical

"$%& → " #%& model

39
BAYESIAN OPTIMIZATION
Function
Initial samples evaluation
#
" # = "& &)% ! " #$%

New sample Mathematical

"&$% → " #$% model

40
BAYESIAN OPTIMIZATION
Function Model update
Initial samples evaluation
#
" # = "& &)% ! " #$%

New sample Mathematical

"&$% → " #$% model

41
BAYESIAN OPTIMIZATION
Function Model update
Initial samples evaluation
#
" # = "& &)% ! " #$%

New sample Mathematical

"&$% → " #$% model

42
BAYESIAN OPTIMIZATION
Function Model update
Initial samples evaluation
#
Final Results
" # = "& &)% ! " #$%

New sample Mathematical

"&$% → " #$% model

43
BAYESIAN OPTIMIZATION
Function Model update
Initial samples evaluation
#
Final Results
" # = "& &)% ! " #$%

New sample Mathematical

"&$% → " #$% model

Key 2 Key 1

44
 BAYESIAN OPTIMIZATION (BO) Key 1

• 2 key elements
Key 1 • Surrogate-based approach
= Mathematical model
Key 2 • Sequential sampling

45
 BAYESIAN OPTIMIZATION (BO) Key 1

• 2 key elements
Key 1 • Surrogate-based approach
• Probabilistic surrogate model: Gaussian Process (GP)
• Sequential sampling

46
BO: MODEL BUILDING Key 1

• Probabilistic surrogate model

1. Prior measure over the objective
• Initial belief

2. Posterior measure over the objective

• Updated belief, given the data

47
BO: MODEL BUILDING Key 1

• Probabilistic surrogate model

1. Prior measure over the objective
• Initial belief

2. Posterior measure over the objective

• Updated belief, given the data

48
BO: MODEL BUILDING Key 1

• Probabilistic surrogate model

1. Prior measure over the objective
• Initial belief

2. Posterior measure over the objective

• Updated belief, given the data

49
BO: MODEL BUILDING Key 1

• Probabilistic surrogate model

1. Prior measure over the objective
• Initial belief

2. Posterior measure over the objective

• Updated belief, given the data

50
BO: MODEL BUILDING Key 1

• Prior: distribution over functions

• Gaussian Process (GP)
= Generalization of Gaussian distribution

51
BO: MODEL BUILDING Key 1

• Prior: distribution over functions

• Gaussian Process (GP)
= Generalization of Gaussian distribution

Mean

/
1 -
.-"
!",$ % = , +$/
2)* +

Variance

52
BO: MODEL BUILDING Key 1

• Prior: distribution over functions

• Gaussian Process (GP)
= Generalization of Gaussian distribution
x GP ( m ( x ) , k ( x, x ' ) ) µ ( x ) ,s ( x )

53
BO: MODEL BUILDING Key 1

• Prior: distribution over functions

• Gaussian Process (GP)
= Generalization of Gaussian distribution
x GP ( m ( x ) , k ( x, x ' ) ) µ ( x ) ,s ( x )

Fully characterized by:

Mean function & Covariance function

(often 0) (kernel)

54
BO: MODEL BUILDING Key 1

• Prior: distribution over functions

• Gaussian Process (GP)
= Generalization of Gaussian distribution
x GP ( m ( x ) , k ( x, x ' ) ) µ ( x ) ,s ( x )

Fully characterized by:

Mean function & Covariance function

(often 0) (kernel)

55
 BO: MODEL BUILDING - KERNELS Key 1

• Example Covariance functions

Squared-Exponential Matérn Periodic
æ 2sin 2 ( 0.5 ( x - x¢ ) ) ö
n
æ 1 D æ x - x¢ ö 2 ö 21-n æ 2n x - x ' ö æ 2n x - x ' ö
k ( x, x¢ ) = b exp ç - å ç d d ÷ ÷ k ( x, x ) =
¢ ç ÷ Kn ç ÷ k ( x, x¢ ) = exp ç - ÷
ç 2 d =1 è ld ø ÷ G (n ) çè l ÷
ø
ç
è l ÷
ø
ç
è l 2 ÷
ø
è ø

Samples from the corresponding prior

[Adams]

66
 BO: MODEL BUILDING - KERNELS Key 1

• Example Covariance functions

Squared-Exponential Matérn Periodic
æ 2sin 2 ( 0.5 ( x - x¢ ) ) ö
n
æ 1 D æ x - x¢ ö 2 ö 21-n æ 2n x - x ' ö æ 2n x - x ' ö
k ( x, x¢ ) = b exp ç - å ç d d ÷ ÷ k ( x, x ) =
¢ ç ÷ Kn ç ÷ k ( x, x¢ ) = exp ç - ÷
ç 2 d =1 è ld ø ÷ G (n ) çè l ÷
ø
ç
è l ÷
ø
ç
è l 2 ÷
ø
è ø

Choosing the ‘optimal’ covariance function

• user choice

Selecting the ‘optimal’ hyperparameters

69
 BO: MODEL BUILDING - KERNELS Key 1

• Example Covariance functions

Squared-Exponential Matérn Periodic
æ 2sin 2 ( 0.5 ( x - x¢ ) ) ö
n
æ 1 D æ x - x¢ ö 2 ö 21-n æ 2n x - x ' ö æ 2n x - x ' ö
k ( x, x¢ ) = b exp ç - å ç d d ÷ ÷ k ( x, x ) =
¢ ç ÷ Kn ç ÷ k ( x, x¢ ) = exp ç - ÷
ç 2 d =1 è ld ø ÷ G (n ) çè l ÷
ø
ç
è l ÷
ø
ç
è l 2 ÷
ø
è ø

Choosing the ‘optimal’ covariance function

• user choice

Selecting the ‘optimal’ hyperparameters

70
 BO: MODEL BUILDING - KERNELS Key 1

• Example Covariance functions

Squared-Exponential Matérn Periodic
æ 2sin 2 ( 0.5 ( x - x¢ ) ) ö
n
æ 1 D æ x - x¢ ö 2 ö 21-n æ 2n x - x ' ö æ 2n x - x ' ö
k ( x, x¢ ) = b exp ç - å ç d d ÷ ÷ k ( x, x ) =
¢ ç ÷ Kn ç ÷ k ( x, x¢ ) = exp ç - ÷
ç 2 d =1 è ld ø ÷ G (n ) çè l ÷
ø
ç
è l ÷
ø
ç
è l 2 ÷
ø
è ø

Choosing the ‘optimal’ covariance function

• user choice

Selecting the ‘optimal’ hyperparameters

• data-driven approach, during training phase (MLE)

“ a GP lets the data speak for itself ”

71 71
BO: MODEL BUILDING Key 1

• Probabilistic surrogate model

1. Prior measure over the objective kernel
• Initial belief choice

2. Posterior measure over the objective

• Updated belief, given the data

74
BO: MODEL BUILDING Key 1

• Probabilistic surrogate model

1. Prior measure over the objective
• Initial belief

2. Posterior measure over the objective

• Updated belief, given the data

75
BO: MODEL BUILDING Key 1

• Bayes’s rule:
the posterior of a model M given data D
is proportional to the likelihood of D given M
multiplied by the prior probability of M
marginal
posterior prior
likelihood

P(D | M ) P(M )
P (M | D) =
P ( D)

76
BO: MODEL BUILDING - INFERENCE Key 1

• Posterior mean: model prediction

µ ( x ) = k ( x , x1:i ) ( K + s I ) f1:i
T 2 -1
n

x : chosen point in the design space

x1:i , f1:i : chosen points up to i - th iteration
and corresponding function evaluation
k: covariance function
K : covariance matrix, k evaluated on x1:i
s n2 : variance of the predicted noise associated with the data

78
BO: MODEL BUILDING - INFERENCE Key 1

• Posterior mean: model prediction

µ ( x ) = k ( x , x1:i ) ( K + s I ) f1:i
T 2 -1
n

• Posterior variance: model uncertainty

(K +s I ) k ( x, x )
-1
( x ) = k ( x , x ) - k ( x , x1:i )
T
s 2 2
n 1:i

79
BO: MODEL BUILDING - INFERENCE Key 1

• Posterior mean: model prediction

µ ( x ) = k ( x , x1:i ) ( K + s I ) f1:i
T 2 -1
n

• Posterior variance: model uncertainty

(K +s I ) k ( x, x )
-1
( x ) = k ( x , x ) - k ( x , x1:i )
T
s 2 2
n 1:i

computational complexity
• training = O(N³)
• prediction mean = O(N)
• prediction variance = O(N2)
80 80
BO: MODEL BUILDING - EXAMPLE Key 1

• 4 data samples = anchor points

81
BO: MODEL BUILDING - EXAMPLE Key 1

• Find minimum of 1D function

[Gonzàlez]

82
BO: MODEL BUILDING - EXAMPLE Key 1

• Find minimum of 1D function

1 curve

83
BO: MODEL BUILDING - EXAMPLE Key 1

• Find minimum of 1D function

3 curves

[Gonzàlez]

84
BO: MODEL BUILDING - EXAMPLE Key 1

• Find minimum of 1D function

100 curves

[Gonzàlez]

85
BO: MODEL BUILDING - EXAMPLE Key 1

• Find minimum of 1D function

many curves

[Gonzàlez]

86
BO: MODEL BUILDING - EXAMPLE Key 1

• Find minimum of 1D function

infinite curves

’ensemble’

[Gonzàlez]

87
BAYESIAN OPTIMIZATION
Function Model update
Initial samples evaluation
#
Final Results
" # = "& &)% ! " #$%

New sample Mathematical

"&$% → " #$% model

Key 2 Key 1

88
BAYESIAN OPTIMIZATION
Function Model update
Initial samples evaluation
#
Final Results
" # = "& &)% ! " #$%

New sample Mathematical

"&$% → " #$% model

Key 2 Key 1

Gaussian Process
89
 BAYESIAN OPTIMIZATION (BO) Key 2

• 2 key elements
• Surrogate-based approach
• Gaussian Process (GP)
Key 2 • Sequential sampling
• Acquisition Function

90
BO: ACQUISITION FUNCTION Key 2

• Sequential sampling
• Find a balance between:
• Exploitation
• Sample where the model prediction is high (low)
• Exploration
• Sample where the model uncertainty is high

• Solution: use a mathematical function

• Acquisition function

91
BO: ACQUISITION FUNCTION - EXAMPLE Key 2

• Example: Find minimum of 1D function

Exploitation
Exploration

92
BO: ACQUISITION FUNCTION - EXAMPLE Key 2

• Example: Find minimum of 1D function

Exploitation
Exploration

93
BO: ACQUISITION FUNCTION - EXAMPLE Key 2

• Example: Find minimum of 1D function

Exploitation

Exploration

94
BO: ACQUISITION FUNCTION Key 2

• Sequential sampling
• Find a balance between:
• Exploitation
• Sample where the model prediction is high (low)
• Exploration
• Sample where the model uncertainty is high

• Solution: use a mathematical function

• Acquisition function ! "

95
BO: ACQUISITION FUNCTION - POI Key 2

• Probability of Improvement (PoI) [Kushner 1964]

æ µ ( x ) - f ( xbest ) ö
a ( x ) = P ( x > xbest ) = F çç ÷÷
è s ( x) ø

96
BO: ACQUISITION FUNCTION - POI Key 2

• Probability of Improvement (PoI) [Kushner 1964]

æ µ ( x ) - f ( xbest ) ö
a ( x ) = P ( x > xbest ) = F çç ÷÷
è s ( x) ø

97
BO: ACQUISITION FUNCTION - POI Key 2

• Probability of Improvement (PoI) [Kushner 1964]

Posterior
mean

æ µ ( x ) - f ( xbest ) ö
a ( x ) = P ( x > xbest ) = F çç ÷÷
è s ( x) ø

Posterior
Standard normal
standard deviation
cumulative distribution function
(CDF)

98
BO: ACQUISITION FUNCTION - POI Key 2

• Probability of Improvement (PoI) [Kushner 1964]

PoI

99
 BO: ACQUISITION FUNCTION - EI Key 2

• Expected Improvement (EI) [Mockus 1978]

$ " − & "'()* $ " − & "'()*

! " = $ " − & "'()* Φ +, " .
, " , "
Bend

Standard normal
probability density function
(PDF)

100
 BO: ACQUISITION FUNCTION - EI Key 2

• Expected Improvement (EI) [Mockus 1978]

$ " − & "'()* $ " − & "'()*

! " = $ " − & "'()* Φ +, " .
, " , "
Bend

Exploitation Exploration

101
BO: ACQUISITION FUNCTION Key 2

• Many acquisition functions (AQ) exist

• Probability of Improvement (PoI) [Kushner 1964]
• Expected Improvement (EI) [Mockus 1978]
• GP Upper Confidence Bound (UCB) [Srinivas et al. 2010]

• For a given problem,

impossible to define the “best” AQ a-priori

102
BO: ACQUISITION FUNCTION Key 2

• How to use AQ?

xi +1 = arg max a ( x )
xÎC

• Solving an optimization problem at every iteration

• AQ is fast to evaluate,
• its gradients are (typically) available

104
BAYESIAN OPTIMIZATION IN A NUTSHELL
• Bayesian Optimization = strategy to transform
xM = arg max f ( x ) unsolvable
xÎC

into a series of problems

xi +1 = arg max a ( x ) solvable

xÎC

105
WRAP-UP: BAYESIAN OPTIMIZATION
Function Model update
Initial samples evaluation
#
Final Results
" # = "& &)% ! " #$%

New sample Mathematical

"&$% → " #$% model

Key 2 Key 1
Acquisition function Gaussian Process
106
BAYESIAN OPTIMIZATION - EXAMPLE
• Problem: Find minimum of 1D function

2 samples

107
BAYESIAN OPTIMIZATION - EXAMPLE
• Problem: Find minimum of 1D function

2 samples

108
BAYESIAN OPTIMIZATION - EXAMPLE
• Problem: Find minimum of 1D function

3 samples

109
BAYESIAN OPTIMIZATION - EXAMPLE
• Problem: Find minimum of 1D function

4 samples

110
BAYESIAN OPTIMIZATION - EXAMPLE
• Problem: Find minimum of 1D function

Continue
until ….

111
BAYESIAN OPTIMIZATION - EXAMPLE
• Problem: Find minimum of 1D function

7 samples

112
104
BAYESIAN OPTIMIZATION - EXAMPLE
• Problem: Find minimum of 1D function

Continue
until ….

113
BAYESIAN OPTIMIZATION - EXAMPLE
• Problem: Find minimum of 1D function

11 samples

114
 DEPARTMENT OF INFORMATION TECHNOLOGY
IDLAB

-1- GAUSSIAN PROCESS (GP)

-2- BAYESIAN OPTIMIZATION (BO)

-3- BO EXAMPLE
 BENDED INTERCONNECTION
• Described in [Gazda, 2010]
• Simulated in ADS Momentum
• Objective function:
Bend

118
 BENDED INTERCONNECTION – INIT
Function Model update
Initial samples evaluation
#
Final Results
" # = "& &)% ! " #$%
Bend

New sample Mathematical

"&$% → " #$% model
 BENDED INTERCONNECTION – INIT
• Design parameters
• Width ϵ [0.5 - 2.1] mm
• Length ϵ [ 45 - 55] mm
• Goal: Minimize differential-to-common-mode conversion
Bend

120
 BENDED INTERCONNECTION – INIT
• Design parameters
• Width ϵ [0.5 - 2.1] mm
• Length ϵ [ 45 - 55] mm
• Goal: Minimize differential-to-common-mode conversion
//1
Bend

+()*
1 1 //1
TDMCM = & ,-.// 0 + ,-.1/ 0 40
'()*

121
 BENDED INTERCONNECTION – INIT
• Design parameters
• Width ϵ [0.5 - 2.1] mm
• Length ϵ [ 45 - 55] mm
• Goal: Minimize differential-to-common-mode conversion
//1
Bend

+()*
1 1 //1
TDMCM = & ,-.// 0 + ,-.1/ 0 40
'()*

elements of the modal S-parameters matrix

122
 BENDED INTERCONNECTION – INIT
• Design parameters
• Width ϵ [0.5 - 2.1] mm
• Length ϵ [ 45 - 55] mm
• Goal: Minimize differential-to-common-mode conversion
//1
Bend

+()*
1 1 //1
TDMCM = & ,-.// 0 + ,-.1/ 0 40
'()*

• Evaluation: ADS Momentum

nodal modal TDMCM
layout
S-parameters S-parameters
205 freq’s
[0-6] GHz

expensive
123
 BENDED INTERCONNECTION – INIT
• Design parameters
• Width ϵ [0.5 - 2.1] mm
• Length ϵ [ 45 - 55] mm
• Goal: Minimize differential-to-common-mode conversion
//1
Bend

+()*
1 1 //1
TDMCM = & ,-.// 0 + ,-.1/ 0 40
'()*

• Evaluation: ADS Momentum

nodal modal TDMCM
layout
S-parameters S-parameters
input : 2D output : 1D
design/feature cost/goal
expensive
124
 BENDED INTERCONNECTION – GP
Function Model update
Initial samples evaluation
#
Final Results
" # = "& &)% ! " #$%
Bend

New sample Mathematical

"&$% → " #$% model

Key 1

Gaussian Process
 BENDED INTERCONNECTION – GP Key 1

• GP Model Building
• Covariance Function: Matérn 3/2
= able to model a wide class of functions (non-differentiable ones)
3 " − "′ 3 " − "′
! ", "′ = & ' 1 + exp −
ℓ ℓ
Bend

126
 BENDED INTERCONNECTION – GP Key 1

• GP Model Building
• Covariance Function: Matérn 3/2
= able to model a wide class of functions (non-differentiable ones)
3 " − "′ 3 " − "′
! ", "′ = & ' 1 + exp −
ℓ ℓ
Bend

• Hyperparameters tuning
• 3 initial samples chosen over a LHD
• tuning via Maximum Likelihood Estimation (MLE) [Williams, 2006]

127
 BENDED INTERCONNECTION – AQ
Function Model update
Initial samples evaluation
#
Final Results
" # = "' '(% ! " #$%
Bend

New sample Mathematical

"'$% → " #$% model

Key 2
Acquisition function
 BENDED INTERCONNECTION – AQ Key 2

• Acquisition function
• Expected Improvement
$ " − & "'()* $ " − & "'()*
! " = $ " − & "'()* Φ +, " .
, " , "
Bend

129
 BENDED INTERCONNECTION – BO
Function Model update
Initial samples evaluation
#
Final Results
" # = "& &)% ! " #$%
Bend

New sample Mathematical

"&$% → " #$% model

Key 2 Key 1
Acquisition function Gaussian Process
130
 BENDED INTERCONNECTION – BO
• Bayesian Optimization software
• Several BO software packages
• GPflowOpt, GPyOpt, Spearmint, BayesOpt, RoBO
Bend

131
 BENDED INTERCONNECTION – BO
• Bayesian Optimization software
• Several BO software packages
• GPflowOpt, GPyOpt, Spearmint, BayesOpt, RoBO
• open Source Python package, based on TensorFlow
Bend

132
 BENDED INTERCONNECTION
• Optimization Results
Design Min Objective Number of
parameters function iterations

w=2.094 mm
0.6739 50
L=54.398 mm
Bend

133
 BENDED INTERCONNECTION
• Optimization Results
Design Min Objective Number of
parameters function iterations

w=2.094 mm
0.6739 50
L=54.398 mm
Bend

Computational Time
S-parameters 86 min 35.82 s
simulations (ADS) (103.92 s per sample)
Bayesian Optimization 58.58 s
Total 87 min 34.4 s

134
 DEPARTMENT OF INFORMATION TECHNOLOGY
IDLAB

-1- GAUSSIAN PROCESSES (GP)

-2- BAYESIAN OPTIMIZATION (BO)

-3- BO EXAMPLE
-4- WRAP-UP
WRAP-UP: BAYESIAN OPTIMIZATION
Function Model update
Initial samples evaluation
#
Final Results
" # = "& &)% ! " #$%

New sample Mathematical

"&$% → " #$% model

Key 2 Key 1
Acquisition function Gaussian Process
183
Questions?

184
REFERENCES
• [Gonzàlez]: J. Gonzàlez, “Introduction to Bayesian Optimization”, Masterclass, Lancaster University, Lancaster, UK, 2107
• [Adams]: R. P. Adams, “A Tutorial on Bayesian Optimization for Machine Learning”, Harvard University, Harvard, UK
• [Gazda]: C. Gazda, D. Vande Ginste, H. Rogier, R. Wu and D. De Zutter, “A Wideband Common-Mode Suppression Filter for
Bend Discontinuities in Differential Signaling Using Tightly Coupled Microstrips”, IEEE Transactions on Advanced Packaging,
vol. 33, no. 4, pp. 969-978, Nov. 2010
• [Williams]: C. E. Rasmussen and C. K. Williams, “Gaussian processes for machine learning”, vol. 1, MIT press Cambridge,
2006
• [BO-GU]: Available at https://github.com/GPflow/GPflowOpt . Related publication: N. Knudde, J. van der Herten, T. Dhaene,
and I. Couckuyt, “GPflowOpt: A Bayesian Optimization Library using TensorFlow,” arXiv preprint – arXiv:1711.03845, 2017.
Available: https://arxiv.org/abs/1711.03845
• [Knudde]: N. Knudde, I. Couckuyt, D. Spina, K. Łukasik, P. Barmuta, D. Schreurs and T. Dhaene, “Data-Efficient Bayesian
Optimization with Constraints for Power Amplifier Design”, Proceedings of IEEE MTT-S International Conference on Numerical
Electromagnetic and Multiphysics Modeling and Optimization (NEMO), Reykjavik, Iceland, 2018
• [Torun]: H. M. Torun, M. Larbi, M. Swaminathan, “A Bayesian Framework for Optimizing Interconnects in High-Speed
Channels”, Proceedings of IEEE MTT-S International Conference on Numerical Electromagnetic and Multiphysics Modeling
and Optimization (NEMO), Reykjavik, Iceland, 2018

185
GENERAL REFERENCES ON BO
• C. E. Rasmussen and C. K. Williams, “Gaussian processes for machine learning”, vol. 1, MIT press Cambridge,
2006
• B. Shahriari, K. Swersky, Z. Wang, R. P. Adams and N. de Freitas, “Taking the Human Out of the Loop: A Review of
Bayesian Optimization”, Proceedings of the IEEE, vol. 104, no. 1, pp. 148-175, Jan. 2016.
• E. Brochu, V. M. Cora, N. de Freitas, “A Tutorial on Bayesian Optimization of Expensive Cost Functions, with
Application to Active User Modeling and Hierarchical Reinforcement Learning”, arXiv preprint – arXiv: 1012.2599,
2010. Available: http://arxiv.org/abs/1012.2599
• P. I. Frazier, “A Tutorial on Bayesian Optimization ”, arXiv preprint – arXiv: 1807.02811, 2018. Available:
https://arxiv.org/abs/1807.02811

186
Dr. Ivo Couckuyt
Dr. Domenico Spina
Prof. Tom Dhaene

IDLAB, DEPARTMENT OF INFORMATION TECHNOLOGY

Email: [email protected]

www.ugent.be
www.ugent.be/ea/idlab/en
www.imec.be