Microelectronics Journal 37 (2006) 510–518

www.elsevier.com/locate/mejo

Design of a Hamming neural network based

on single-electron tunneling devices
J.G. Guimarães1, L.M. Nóbrega, J.C. da Costa*
Department of Electrical Engineering, Universidade de Brası́lia, C.P. 4386, 70904-970 Brası́lia, DF, Brazil

Received 13 May 2005; received in revised form 23 July 2005; accepted 27 July 2005
Available online 10 October 2005

Abstract
In this paper, the first complete implementation of a Hamming neural network based on single-electron devices is presented. A large-scale
network for character recognition simulation based on building block approach was successfully carried out. Simulations were done using
SIMON and MATLAB softwares. Effects such as offset charges and dynamic behavior are taken into account. Moreover, room temperature
operation is considered.
q 2005 Elsevier Ltd. All rights reserved.

PACS: 73.23HK; 84.35.Ci; 85.35Gv; 85.40.Ke

Keywords: Single-electron; Neural network; Room temperature; Hamming network; Block; Recognition

1. Introduction and co-tunneling [9]. Such effects can degrade their

electrical performance [1,10]. To overcome these limi-
Nanoelectronic devices [1,2] are an extremely attractive tations, parallel processing architectures, like neural net-
option for developing GSI (Giga-Scale Integration) or even works, should be considered [3,11]. Artificial neural
TSI (Tera-Scale Integration) circuits [3] with dimensions networks seem advantageous because of their high
and performance limits [4] beyond the last Semiconductor parallelism and redundancy. Consequently, these networks
Industry Association’s (SIA) roadmap projections [2]. present robustness against local fluctuations [12].
Among nanoelectronic devices, single-electron tunnel- Competitive neural networks, like winner-take-all
ing (SET) devices based on tunnel junctions [1,5–7] (WTA), provide easiness of operation due their unsuper-
present the following features: low power consumption, vised training [13,14]. In addition, these neural networks
reduced dimensions and current control. These features have a reduced number of control signals, self-organization
should allow building chips with a number of devices and local memory [13,15]. Usually, WTA networks are used
orders of magnitude greater than the indicated by the together with another neural network layer to implement
roadmap [2] still respecting area and power consumption tasks such as: decision making, pattern recognition, feature
restrictions. In this sense, a TSI processor may be feasible extraction, image processing, video compression, Hamming
in the future. network and others [16].
However, nanoscaled SET devices present instabilities In the literature, some SET neurons have already been
resulting from local phenomena, like offset charges [8] proposed. Goossens et al. [17] provided some examples of
single-electron circuits for synapses and neurons. Kirihara
and Taguchi [18] showed a more complex neuron circuit
* Corresponding author. Tel.: C55 61 3307 2308; fax: C55 61 3274
with n inputs and 6nC2 SET transistors. However, both
6651.
E-mail address: [email protected] (J.C. da Costa). worked with supervised networks.
1
Janaina Guimarães acknowledges CAPES-Brazil and CNPq-Brazil for Yamada and Ameniya developed a SET Hopfield
support. network [19] and a SET Boltzmann machine [20]. Never-
0026-2692/$ - see front matter q 2005 Elsevier Ltd. All rights reserved. theless, the operation of these circuits had not considered
doi:10.1016/j.mejo.2005.07.007 offset charges and room temperature operation.
 J.G. Guimarães et al. / Microelectronics Journal 37 (2006) 510–518 511

A SET winner-take-all network has already been the matching score (MS) is obtained from [13]:
proposed [21–23]. This network was simulated implement- ðn
ing a Hamming network taking into account effects such as MSi Z nKHDi MSi Z nK jxj Keji j (1)
offset charges and temperature variations. Nevertheless, the
jZ1
input layer was simulated mathematically, i.e. it was not
implemented with nanoscaled devices. where HDi is the Hamming distance [13]. The Hamming
In this paper, a Hamming neural network completely distance is the number of elements in the input vector that do
designed using SET devices is presented for the first time. A not match the corresponding exemplar vector.
character recognition task is simulated at room temperature These matching scores values are inputs for the WTA
using building blocks [22]. Robustness against offset layer that determines which exemplar is closest to the given
charges, as well as dynamic behavior are evaluated. input vector x/ [13].

3. SET Hamming network implementation

2. Hamming network
The first complete implementation at circuit level using
The Hamming network [24] is a maximum likelihood
tunnel junctions of a Hamming network is illustrated in
classifier for disturbed bipolar binary inputs. So, for a set of
Fig. 2.
m exemplar vectors e1/; e2/; .; em/ it finds the exemplar
The input layer in Fig. 2 is presented for the first time in
which is most similar to a given input vector x/.
this work. In this circuit, the input vector x/Zðx1 ; .; xn Þ is
A Hamming network has two neural layers, as shown in
represented using input voltages. The synaptic weights are
Fig. 1:
represented using capacitors C11, C21, C31,.,C1m, C2m,
(1) input layer with n neurons; Cnm. In this way, input voltages injected in weight
(2) WTA output layer with m neurons. capacitances will result in pondered charges q1, q2,.,qm
expressed in Eq. (2).
The input layer consists of n neurons which provides q1 Z x1 C11 C x2 C21 C/C xn Cn1
matching scores MS1, MS2,.,MSm from the input vector
x/ to each one of the exemplar vectors e1/; e2/; .; em/ q2 Z x1 C12 C x2 C22 C/C xn Cn2
(2)
which are stored in the weights of this layer [24]. The «
matching score is the number of matching elements between
qm Z x1 C1m C x2 C2m C/C xn Cnm
the input vector and the corresponding exemplar vector.
Considering a input vector x/Zðx1 ; x2 ; .; xn Þ and an These charges will result in voltages VS1, VS2,.,VSm
exemplar vector ei/Zðe1i ; e2i ; .; eni Þ, where iZ1,.,m, respectively at nodes S1, S2,.,Sm shown in Fig. 2.

Fig. 1. Hamming network diagram.

512 J.G. Guimarães et al. / Microelectronics Journal 37 (2006) 510–518

Fig. 2. Hamming network SET circuit with n inputs and m outputs.

q1 current passing through nodes 1,2,.,m when the input

VS1 Z
CS1 voltages VS1, VS2,.,VSm are larger than the blockage
q voltage (VB), as shown in Fig. 3.
VS2 Z 2
CS2
(3)
«
4. Building blocks
q
VSm Z m
CSm Considering that the goal of this work is the simulation of
large-scale SET–WTA circuits, the idea of creating a
where CS1, CS2,.,CSm are the equivalent capacitances seen building block comes from two basic issues. The first one is
at nodes S1, S2,.,Sm. The SET transistors T1, T2,.,Tm are the difficulty of editing large circuits on SET simulators.
implementing activation functions for each neuron in the The schematic capture of a large SET circuit with SIMON
input layer. These transistors will convert voltages VS1, [25], for example, is a long and tedious task due to its ‘pick
VS2,.,VSm in output currents to the SET–WTA layer. Due and place’ editing system. A circuit with 400 tunnel
to the Coulomb blockage characteristic, there will only be junctions would take weeks to be totally edited with that
 J.G. Guimarães et al. / Microelectronics Journal 37 (2006) 510–518 513

means 400 tunnel junctions and 300 capacitors requires the

observation of all individual neuron outputs, i. e. 100 nodes.
In a Pentium 4, 1.5 GHz, 528 MB RAM only 45 nodes can
be observed with SIMON [22].
Taking these issues into consideration, a building block
approaching would be useful to overcome those limitations.
The whole network can be described by blocks. Those
blocks are small enough to be simulated by SIMON and are
completely built using SET devices [22]. The outputs
provided by SIMON will be processed with MATLAB to
determine which one is the winner output.
Considering a SET–Hamming network with N neurons.
This network is broken into blocks of n neurons (n!N),
in such a way that N/n building blocks will be created.
Each block is simulated with SIMON separately. After all
simulations, MATLAB is used to read all output voltages
Fig. 3. Activation function characteristic for the input neurons. of each block and find the winner neuron. For example, a
system. A netlist description of SET circuits can be made in SET–Hamming network with NZ1000 neurons could be
SIMON, but it is also time consuming, tedious and prone to broken into 40 blocks of nZ25 neurons for simulation. In
mistakes. The second issue is the processing capability. For the next section, an application example will be
example, a SET–WTA circuit with 100 neurons, which presented.

Fig. 4. Character recognition system.

514 J.G. Guimarães et al. / Microelectronics Journal 37 (2006) 510–518

In each building block, 4 exemplars were stored,

representing the weights. In block 1, for example, P1 was
stored in capacitances C11, C21,.,C36,1, P2 was stored in
C12, C22,.,C36,2 and so on.
Taking into account that tunneling is the major transport
desired in SET circuits, even at room temperature (T300 K),
the electrostatic energy (EC) should be greater than the
thermic energy (ET) [1,26,27]. This inequality ranges all
SET circuit capacitance values, as shown in Eq. (4).

e2 e2
EC OOET OOkB T300 K C!!
2C 2kB T300 K (4)
C!!3:1aF

In this SET–Hamming circuit, the matching score is

reinforced each time the input vector black spaces are in the
Fig. 5. 25 examples of exemplar characters.
same position as in the exemplar characters. In this sense,
the black space logic level (1) was represented with a
5. Character recognition task capacitance value of 0.1 aF, which is higher than 0.01 aF
that was the value assigned to the white space logic level
In order to check if the circuit proposed in Fig. 2 can be (K1). All circuit values chosen consider physically
used as a building block for implementing large-scale implemented devices [26].
neural networks, a character recognition task was simu- Moreover, 20 disturbed patterns were created from 20
lated using it. The system developed in this work is shown randomly chosen exemplar patterns. These disturbed
in Fig. 4. patterns were generated randomly choosing 10% of the
As can be seen in Fig. 4, the whole network with NZ100 points of each exemplar and changing them. Fig. 7 shows
neurons was broken into 25 blocks (nZ4) for simulation. four examples of disturbed patterns.
These disturbed patterns are the system’s input voltage
5.1. Design signals. The recognition system should indicate, to each
disturbed character presented in the input, the correspondent
For this task, 100 exemplar characters P1, P2,.,P100 [23] original exemplar pattern. Considering that one electron e is
were generated. In Fig. 5, 25 examples of these characters charged when an input black space is in the same position as
are shown. in an exemplar character stored, if the weight capacitance is
The next step was transforming each exemplar character 0.1 aF, then,
into a matrix. This representation is suitable for neural Q e
networks [23]. The dimension of each matrix was 6!6, i.e. VZ VZ V Z 1:6V (5)
C 0:1a
36 elements. In these matrices, each character’s black space
was converted into a logic level 1. The white spaces were Disturbed patterns are also transformed into matrices.
converted into logic levelsK1. Fig. 6 shows how an Considering the same logic level representation showed in
exemplar character is turned into matrix. Fig. 6, for each positive value was assigned the voltage

Fig. 6. Character to matrix conversion.

 J.G. Guimarães et al. / Microelectronics Journal 37 (2006) 510–518 515

1.6 V and for negative values were assigned the voltage

K1.6 V.
The activation function transistors, T1, T2,.,Tm were
designed for a Coulomb blockage voltage of 0.25 V. This
value was chosen according to the voltages VS1, VS2,.,VSm
provided by the input layer. From the Coulomb voltage
expression [1] assigning 0.25 V to VB led to
e e 1:6 !10K9
VB Z CST Z CST Z
2CST 2VB 2 !0:25 (6)
CST z0:3aF
where CST is the total capacitance seen from the SET
transistor island. Considering that CSTZ2CjTCCgT, assign-
ing CjTZ0.1 aF result in:
CST Z 2CjT C CgT CgT Z CST K2CjT
Fig. 7. Disturbed patterns.
(7)
CgT Z 0:3aFK2ð0:1aFÞ CgT Z 0:1aF
Table 1 The WTA layer was designed using the same values
SET–WTA circuit values
presented in [28]. These values are presented in Table 1.
CN CjN CgN VbiasN R
100 aF 0.01 aF 0.09 aF K 0.5 V 1MU
5.2. Simulations

Table 2 Building block simulations were executed with SIMON

Simulation parameters values using the simulation parameters showed in Table 2.
Parameter Value The disturbed inputs X1, X2,.,X20 were presented in
sequence to the network to simulate a dynamic behavior, i.e.
Temperature 1 and 300 K
Start time 0 ns in the first 1 ns time interval X1 was presented to the
End time 20 ns network, in the second interval X2 was presented and so on,
Time step 0.1 ns totalizing 20 ns. This time interval was chosen to fit the
Event number 1000 roadmap projections [2,3].
Tunnel order 1
The 25 building blocks were simulated using SIMON.
After that, MATLAB was used to read all voltage outputs

Fig. 8. Output voltages final values for input X3 and TZ1 K.

516 J.G. Guimarães et al. / Microelectronics Journal 37 (2006) 510–518

Fig. 9. Output voltages final values for input X3, TZ1 K and random offset charges (ROC) in the rangeK0.12e%ROC%0.12e.

and find the block with the winner neuron in each time In Fig. 9 all output voltages final values for input X3
interval. considering random offset charges (ROC) in the range
K0.12e%ROC%0.12e are shown. These simulations were
also done at 1 K.
5.3. Results
In this case, the recognition rate still was 100%.
For TZ1 K all disturbed patterns were recognized as Considering TZ300 K the output values for input X3 are
their original exemplar patterns. It means that the system in Fig. 10.
provided a recognition rate of 100%. Fig. 8 presents the Also in these 300 K simulations the recognition rate
output final voltage values v1(t), v2(t),.,v100(t) when X3 obtained was 100%. It is important to notice in Fig. 10 that
was presented to the system. The disturbed pattern X3 was there was a reduction in the difference between the first and
generated from exemplar pattern P15. the second largest output voltage values.

Fig. 10. Output voltages final values for input X3 and TZ300 K.
 J.G. Guimarães et al. / Microelectronics Journal 37 (2006) 510–518 517

Fig. 11. Output voltages final values for input X3, TZ300 K and random offset charges (ROC) in the rangeK0.08e%ROC%0.08e.

Simulations shown in Fig. 11 still obtaining a recognition and a significant improvement was obtained using a
rate of 100%. building block strategy. Building blocks simulations were
The building block was edited taking around 40 min. done using SIMON and MATLAB was used to obtain the
Each one of the 25 blocks was simulated in approximately final results. An application task, aiming at graphical
14 s, using a Pentium 4, 1.5 GHz, 528 MB RAM. MATLAB characters recognition using Hamming networks was
simulation, using the same computer, was done in successfully implemented. This application was validated
approximately 3 s. It took around 46 min to edit and for different temperatures and taking into account random
simulate the whole recognition system. offset charges. Further on, simulations considering co-
tunneling effects will be done, as well as a more complex
application will be implemented.
5.4. Discussion

From recognition rates provided by simulations shown in

Figs. 8–11, it can be seen that the designed SET–Hamming Acknowledgements
circuit worked properly in the range from 1 K to room
temperature. The authors gratefully acknowledge PADCT-Brazil for
The SET circuit architecture presented in this work support.
presents robustness against considerable levels of random
offset charges [10] without prejudice to the recognition rate.
Moreover, the proposed building block approach for
simulation, using SIMON and MATLAB, reduced the total References
time consumed to edit the SET–Hamming circuit (Fig. 2)
and simulate the recognition system (Fig. 4) and made it [1] H. Grabert, M.H. Devoret, Single Charge Tunneling—Coulomb
Blockade Phenomena in Nanostructures NATO ASI Series, Series B:
easier to implement larger networks. Physics, 1991.
Finally, the SET–Hamming circuit supported dynamic [2] Semiconductor Industry Association, The National Roadmap for
inputs varying in nanoseconds range, which is a suitable Semiconductors 2003.
value for a state of art nanoelectronic circuit [2]. [3] J.C. da Costa, J. Hoekstra, M.J. Goossens, C.J.M. Verhoeven,
H.M. Van Roermund, A.H.M. Van Roermund, Considerations about
nanoelectronic GSI processors, Analog Integrated Circuits and Signal
Processing 24 (2000) 59–71.
6. Conclusions [4] J. Carruthers, D. Hammerstrom, B. Colwell, G. Bourianoff, V.
Zhirnov. Computer architectures for nanoscale devices, Technical
report, Semiconductor Industry Association, 2003.
In this work a single-electron Hamming network [5] D.V. Averin, K. Likharev, Coulomb blockade of tunneling and
was completely implemented using tunnel junctions coherent oscillations in small tunnel junctions, Journal of Low
for the first time. A design methodology was developed Temperature Physics 62 (1986) 345–372.
518 J.G. Guimarães et al. / Microelectronics Journal 37 (2006) 510–518

[6] K.K. Likharev, N.S. Bakhvalov, G.S. Kazacha, I. Serdyukova, Single- [18] M. Kirihara, K. Taniguchi, A single-electron neuron device, Japanese
electron device, IEEE Transactions on Magnetics 25 (1989) 1436– Journal of Applied Physics 36 (1997) 4172–4175.
1439. [19] T. Yamada, Y. Amemiya, A multi-valued Hopfield network device
[7] T.A. Fulton, D. Dolan, Observation of single-electron charging effects using single-electron circuits, IEICE Transactions on Electronics E82-
in small tunnel junctions, Physics Review Letters 59 (1987) 109–112. C (1999) 1615–1622.
[8] H. Wolf, F.J. Ahlers, J. Niemeyer, H. Scherer, T. Weimann, A.B. Worin, [20] T. Yamada, M. Akazawa, T. Asai, Y. Amemiya, Boltzmann machine
V.A. Krupenin, S.V. Lotkohov, E. Presnov, Investigation of offset neural network devices using single-electron tunneling, Nanotechnol-
charge noise in single-electron tunneling devices, IEEE Transactions ogy 12 (2001) 60–67.
on Instrumentation and Measurements 46 (1997) 303–306. [21] J.G. Guimaraes, J.C. Costa, Performance analysis of single-electron
[9] D.K. Ferry, et al., Transport in Nanostructures, Cambridge University winner-take-all networks Proceedings of the 18th Symposium on
Press, USA, 1997. Technology and Devices SBMICRO2003 2003 pp. 120–129.
[10] K.K. Likharev, Single-electron devices and their applications, [22] J.G. Guimaraes, L.M. Nobrega, J.C. Costa, Single-electron winner-
Proceedings of the IEEE 87 (1999) 606–632. take-all macro block for large-scale integrated neural networks
[11] Ö Türel, K. Likharev, Crossnets: possible neuromorphic networks Proceedings of the 19th Symposium on Technology and Devices
based on nanoscale components, International Journal of Circuit SBMICRO2004 2004 pp. 145–150.
Theory and Applications 31 (2003) 37–54. [23] J.G. Guimaraes, H.C. do Carmo, J.C. Costa, Single-electron winner-
[12] J. Hoekstra, J.C. Costa, M.J. Goossens, C.J.M. Verhoeven, take-all network, Microelectronics Journal 35/2 (2004) 173–178.
A.H.M. Van Roermund, The application of neural networks for [24] R. Lippmann, An introdution to computing with neural nets, IEEE
nanoelectronic circuits Proceedings of the International Conference ASSP Magazine 4 (1987) 4–22.
on Neural Networks and Their Applications 1998 pp. 27–30. [25] C. Wasshuber, H. Kosina, S. Selberherr, SIMON—a simulator for
[13] L. Fausett, Fundamentals of Neural Networks, Prentice-Hall, New single-electron tunneling devices and circuits, IEEE Transactions on
Jersey, USA, 1994. Computer-Aided Design of Integrated Circuits and Systems 16 (1997)
[14] S. Haykin, S. Haykin, Neural Networks — A Comprehensive 937–944.
Foundation, Prentice-Hall, New Jersey, USA, 1994. [26] K. Yamamura, Y. Suda, Improvement of operation reliability at room
[15] K.F. Goser, C. Pacha, A. Kanstein, M.L. Rossman, Aspects of systems temperature for a single-electron pump, IEICE Transactions of
and circuits for nanoelectronics, Proceedings of the IEEE 85 (1997) Electron E81-C (1998) 16–20.
558–573. [27] Y. Takahashi, M. Nagase, H. Namatsu, K. Kurihara, K. Iwdate,
[16] Y. Fang, M.A. Cohen, T.G Kincaid, Dynamics of a winner-take-all Y. Nakajima, S. Horiguchi, K. Murase, M. Tabe, Fabrication
neural network, Neural Networks 9 (1997) 1141–1154. technique for Si single-electron transistor operating at room
[17] M.J. Goossens, C.J.M. Verhoeven, A.H.M. Van Roermund, Single- temperature, Electronics Letters 31 (1995) 136–137.
electron tunneling technology for neural networks Proceedings of the [28] J.G. Guimaraes, L.M. Nobrega, J.C. Costa, Single-electron hamming
Fifth International Conference on Microelectronics for Neural network Proceedings of the 20th Symposium on Technology and
Networks and Fuzzy Systems 1996 pp. 125–130. Devices SBMICRO2005 2005.