TURING AWARD LECTURE

Ref l ect i ons on Trusti ng Trust

To what extent should one trust a statement that a program is free of Trojan
horses? Perhaps it is more important to trust the people who wrote the
software.
KEN THOMPSON
I NTRODUCTI ON
I t ha nk t he ACM for t hi s awar d. I can' t hel p but feel
t hat I am r ecei vi ng t hi s honor for t i mi ng a nd s er endi p-
i t y as muc h as t echni cal mer i t . UNIX 1 s wept i nt o popu-
l ar i t y wi t h an i ndus t r y- wi de change f r om cent r al mai n-
f r ames to a ut onomous mi ni s. I s us pect t hat Dani el Bob-
r ow [1] woul d be her e i ns t ead of me i f he coul d not
afford a PDP-10 and ha d had t o "set t l e" for a PDP-11.
Mor eover , t he c ur r e nt st at e of UNIX is t he r es ul t of t he
l abor s of a l arge n u mb e r of peopl e.
Ther e is an ol d adage, "Dance wi t h t he one t hat
br ought you, " whi c h me a ns t hat I s houl d t al k about
UNIX. I have not wor ke d on ma i ns t r e a m UNIX i n ma n y
year s, yet I cont i nue to get unde s e r ve d cr edi t for t he
wor k of ot hers. Ther ef or e, I am not goi ng to t al k about
UNIX, but I wa nt to t ha nk e ve r yone who has cont r i b-
ut ed.
That br i ngs me to Denni s Ri t chi e. Our col l abor at i on
has been a t hi ng of beaut y. In t he t en year s t hat we
have wor ke d t oget her , I can r ecal l onl y one case of
mi s coor di nat i on of wor k. On t hat occasi on, I di s cover ed
t hat we bot h had wr i t t e n t he s ame 20- l i ne as s embl y
l anguage pr ogr am. I c ompa r e d t he sour ces and was as-
t ounde d to f i nd t hat t he y ma t c h e d char act er - f or - char -
act er. The r esul t of our wor k t oget her has be e n far
gr eat er t han t he wor k t hat we each cont r i but ed.
I am a pr ogr ammer . On my 1040 form, t hat is wha t I
put down as my occupat i on. As a pr ogr ammer , I wr i t e
1 UNIX is a t rademark of AT&T Bell Laboratories.
1 984 0001-0782/84/0800-0761 75
pr ogr ams. I woul d l i ke to pr es ent to you t he cut est
pr ogr am I ever wr ot e. I wi l l do t hi s i n t hr e e st ages a nd
t r y to br i ng i t t oget her at t he end.
STAGE I
In col l ege, bef or e vi deo games, we woul d a mus e our -
sel ves by posi ng pr ogr a mmi ng exer ci ses. One of t he
f avor i t es was to wr i t e t he s hor t es t s el f - r epr oduci ng pr o-
gram. Si nce t hi s is an exer ci s e di vor ced f r om r eal i t y,
t he usual vehi cl e was FORTRAN. Act ual l y, FORTRAN
was t he l anguage of choi ce for t he s ame r eas on t hat
t hr ee- l egged r aces ar e popul ar .
Mor e pr eci s el y st at ed, t he pr obl e m is to wr i t e a
sour ce pr ogr am t hat , wh e n c ompi l e d a nd execut ed, wi l l
pr oduce as out put an exact copy of i t s sour ce. If you
have ne ve r done t hi s, I ur ge you to t r y i t on your own.
The di s cover y of how to do i t is a r e ve l a t i on t hat far
sur passes any benef i t obt a i ne d by bei ng t ol d how to do
it. The par t about "shor t est " was j ust an i nc e nt i ve to
de mons t r a t e ski l l a nd de t e r mi ne a wi nner .
Fi gur e 1 shows a s el f - r epr oduci ng pr ogr am i n t he C 3
pr ogr ammi ng l anguage. (The pur i s t wi l l not e t hat t he
pr ogr am is not pr eci s el y a s el f - r epr oduci ng pr ogr am,
but wi l l pr oduc e a s el f - r epr oduci ng pr ogr am. ) Thi s en-
t r y is muc h t oo l ar ge to wi n a pr i ze, but i t de mons t r a t e s
t he t e c hni que a nd has t wo i mpor t a nt pr oper t i es t hat I
need to compl et e my st or y: 1) Thi s pr ogr am can be
easi l y wr i t t e n by a not he r pr ogr am. 2) Thi s pr ogr am can
cont ai n an a r bi t r a r y a mount of excess baggage t hat wi l l
be r e pr oduc e d al ong wi t h t he ma i n al gor i t hm. In t he
exampl e, even t he c omme nt is r epr oduced.
August 1984 Volume 27 Number 8 Communications of the ACM 781
Turing Award Lecture
c h a r s [ ] = I
t 0 1 1
i ; i
" ~ l l l
I ' V , ] ' ~
I ' ~ t ' ] l p
( 2 1 3 l i n e s d e l e t e d )
0
1;
/ ,
T h e s t r i n g s i s a
r e p r e s e n t a t i o n o f t h e b o d y
o f t h i s p r o g r a m f r o m ' 0'
t o t h e e n d .
, /
m a i n ( )
{
i n t i ;
p r i n t f ( " c h a r \ t s [ ] = { k n " ) ;
f o r ( i = 0 ; s [ i ] ; i + + )
p r i n t f ( " ~ t % d , \ n " , s [ i ] ) ;
p r i n t f ( " % s " , s ) ;
I
H e r e a r e s o m e s i m p l e t r a n s l i t e r a t i o n s t o a l l o w
a n o n - C p r o g r a m m e r t o r e a d t h i s c o d e .
= a s s i g n m e n t
= = e q u a l t o . E Q .
! = n o t e q u a l t o . N E .
+ + i n c r e m e n t
' x ' s i n g l e c h a r a c t e r c o n s t a n t
" xxx" m u l t i p l e c h a r a c t e r s t r i n g
% d f o r m a t t o c o n v e r t t o d e c i m a l
% s f o r m a t t o c o n v e r t t o s t r i n g
k t t a b c h a r a c t e r
k n n e w l i n e c h a r a c t e r
F I G U R E 1 .
STAGE I I
The C c ompi l e r is wr i t t e n i n C. What I am about to
des cr i be is one of ma n y " chi cken a nd egg" pr obl e ms
t hat ar i se whe n compi l er s ar e wr i t t e n i n t he i r own l an-
guage. In t hi s case, I wi l l use a speci f i c e xa mpl e f r om
t he C compi l er .
C al l ows a st r i ng cons t r uct to speci f y an i ni t i al i zed
c ha r a c t e r ar r ay. The i ndi vi dua l c ha r a c t e r s i n t he st r i ng
can be es caped to r e pr e s e nt unpr i nt a bl e char act er s For
exampl e,
"Hel l o wo r l d \ n "
r epr es ent s a st r i ng wi t h t he c ha r a c t e r "\ n, " r e pr e s e nt i ng
t he new l i ne char act er .
Fi gur e 2.1 is an i deal i zat i on of t he code i n t he C
compi l er t hat i nt er pr et s t he c ha r a c t e r escape s equence.
Thi s is an a ma z i ng pi ece of code. It "knows" i n a com-
pl et el y por t abl e wa y wha t c ha r a c t e r code is c ompi l e d
for a ne w l i ne i n any c ha r a c t e r set. The act of knowi ng
t hen al l ows i t to r ecompi l e itself, t hus pe r pe t ua t i ng t he
knowl edge.
Suppose we wi sh to al t er t he C c ompi l e r to i ncl ude
t he s equence " \ v" to r epr es ent t he ver t i cal t ab char ac-
ter. The ext ens i on to Fi gur e 2.1 is obvi ous and is pr e-
s ent ed in Fi gur e 2.2. We t hen r ecompi l e t he C com-
pi l er , but we get a di agnost i c. Obvi ous l y, si nce t he bi -
nar y ver si on of t he compi l er does not know about "\ v, "
t he sour ce is not l egal C. We mus t "t r ai n" t he compi l er .
Af t er it "knows" wha t " \ v" means, t hen our ne w
change wi l l become l egal C. We l ook up on an ASCII
char t t hat a ver t i cal t ab is de c i ma l 11. We al t er our
sour ce to l ook l i ke Fi gur e 2.3. Now t he ol d c ompi l e r
accept s t he ne w sour ce. We i nst al l t he r es ul t i ng bi na r y
as t he ne w offi ci al C c ompi l e r and now we can wr i t e
t he por t abl e ver s i on t he wa y we ha d i t i n Fi gur e 2.2.
Thi s is a deep concept . It is as cl ose to a " l ear ni ng"
pr ogr am as I have seen. You s i mpl y t el l it once, t he n
you can use t hi s s el f - r ef er enci ng def i ni t i on.
STAGE I I I
Agai n, i n t he C compi l er , Fi gur e 3.1 r e pr e s e nt s t he hi gh
l evel cont r ol of t he C c ompi l e r whe r e t he r out i ne "com-
c = n e x t ( );
i f ( c ! = ' \ V )
r e t u r n ( c ) ;
c = n e x t ( );
i f ( c = = ' \ V )
r e t u r n ( ' \ \ ' ) ;
i f ( c = = ' n ' )
r e t u r n ( ' k n ' ) ;
F I G U R E 2 . 2 .
c = n e x t ( );
i f ( c ~= ' \ v )
r e t u r n ( c ) ;
c = n e x t ( ) ;
i f ( c = = ' \ V )
r e t u r n ( ' k V ) ;
if(c = = ' n ' )
r e t u m ( ' k n ' ) ;
i f ( c = = ' v ' )
r e t u r n ( ' \ v ' ) ;
F I G U R E 2 . 1 .
c = n e x t ( );
i f ( c ! = ' \ V )
r e t u r n ( c ) ;
c = n e x t ( );
i f ( c = = ' \ v )
r e t u r n ( ' \ \ ' ) ;
i f ( c = = ' n ' )
r e t u r n ( ' \ n ' ) ;
i f ( c = = ' v ' )
r e t u r n ( 1 1 );
F I G U R E 2 . 3 .
762 Communications of the ACM August 1984 Volume 27 Number 8
Turing Award Lecture
pi l e" is cal l ed to compi l e t he next l i ne of source. Fi gur e
3.2 s hows a s i mpl e modi f i cat i on to t he c ompi l e r t hat
wi l l de l i be r a t e l y mi s c ompi l e s our ce wh e n e v e r a par t i c-
ul ar pa t t e r n is mat ched. If t hi s wer e not del i ber at e, i t
woul d be cal l ed a c ompi l e r "bug. " Si nce i t is del i ber at e,
it s houl d be cal l ed a "Tr oj an hor se. "
The act ual bug I pl a nt e d i n t he c ompi l e r woul d
ma t c h code i n t he UNIX "l ogi n" c omma nd. The re-
pl a c e me nt code woul d mi s compi l e t he l ogi n c o mma n d
so t hat it woul d accept ei t her t he i nt e nde d e nc r ypt e d
pas s wor d or a pa r t i c ul a r known passwor d. Thus i f t hi s
code wer e i ns t al l ed i n bi na r y and t he bi na r y wer e used
to compi l e t he l ogi n c omma nd, I coul d l og i nt o t hat
syst em as any user.
Such bl at ant code woul d not go unde t e c t e d for long.
Even t he most casual per us al of t he sour ce of t he C
compi l er woul d r ai se suspi ci ons.
The fi nal st ep is r e pr e s e nt e d i n Fi gur e 3.3. Thi s si m-
pl y adds a s econd Tr oj an hor se to t he one t hat a l r e a dy
exi st s. The s econd pa t t e r n is a i me d at t he C compi l er .
The r e pl a c e me nt code is a St age I s el f - r epr oduci ng pr o-
gr am t hat i nser t s bot h Tr oj an hor ses i nt o t he compi l er .
Thi s r equi r es a l ear ni ng phas e as i n t he St age II e xa m-
ple. Fi r st we compi l e t he modi f i ed sour ce wi t h t he nor -
mal C compi l er to pr oduc e a bugged bi nar y. We i nst al l
t hi s bi na r y as t he offi ci al C. We can now r emove t he
bugs f r om t he sour ce of t he c ompi l e r and t he ne w bi -
nar y wi l l r ei ns er t t he bugs wh e n e v e r i t is compi l ed. Of
cour se, t he l ogi n c o mma n d wi l l r e ma i n bugged wi t h no
t r ace i n sour ce a nywhe r e .
c o mp i l e ( s )
c h a r , s ;
I
F I G U R E 3 . 1 .
c o mp i l e ( s )
c h a r , s ;
I
if( m a tch ( s, " p a ttern" ) ) {
c o mp U e ( " b u g " ) ;
r e t u r n ;
J
F I G U R E 3 . 2 .
c o m p i l e ( s )
ch a r , s ;
if( m a tch ( s, " p a ttern1 " )) {
com p i l e ( ' b u g 1 ");
r e t u r n ;
I
if( m a tch ( s, =p a ttern 2 " ) ) I
com p i l e ( ' b u g 2 " ) ;
r e t u r n ;
J
F I G U R E 3 . 3 .
MORAL
The mor al is obvi ous. You can' t t r ust code t hat you di d
not t ot al l y cr eat e your sel f . ( Especi al l y code f r om com-
pani es t hat empl oy peopl e l i ke me. ) No a mo u n t of
s our ce- l evel ver i f i cat i on or s c r ut i ny wi l l pr ot ect you
f r om usi ng unt r us t e d code. In de mons t r a t i ng t he possi -
bi l i t y of t hi s ki nd of at t ack, I pi c ke d on t he C compi l er .
I coul d have pi cked on a ny pr ogr a m- ha ndl i ng pr ogr a m
such as an assembl er , a l oader , or even ha r dwa r e mi -
crocode. As t he l evel of pr ogr am get s l ower , t hes e bugs
wi l l be ha r de r and ha r de r to det ect . A we l l - i ns t a l l e d
mi cr ocode bug wi l l be al mos t i mpos s i bl e to det ect .
Af t er t r yi ng to convi nce you t hat I cannot be t r ust ed,
I wi sh to mor al i ze. I woul d l i ke to cr i t i ci ze t he pr ess i n
i t s ha ndl i ng of t he "hacker s, " t he 414 gang, t he Dal t on
gang, etc. The act s per f or med by t hes e ki ds ar e vandal -
i sm at best and pr obabl y t r espass and t hef t at wor st . It
is onl y t he i na de qua c y of t he c r i mi na l code t hat saves
t he hacker s from ver y s er i ous pr os ecut i on. The compa-
ni es t hat ar e vul ne r a bl e to t hi s act i vi t y, ( and most l arge
compani es ar e ver y vul ner abl e) ar e pr es s i ng ha r d to
updat e t he c r i mi na l code. Una ut hor i z e d access to com-
put e r syst ems is a l r e a dy a ser i ous cr i me i n a few st at es
and is c ur r e nt l y bei ng addr es s ed i n ma n y mor e st at e
l egi sl at ur es as wel l as Congress.
Ther e is an expl os i ve s i t uat i on br ewi ng. On t he one
hand, t he press, t el evi si on, a nd movi es ma ke her os of
vandal s by cal l i ng t he m whi z ki ds. On t he ot he r hand,
t he act s per f or med by t hes e ki ds wi l l soon be puni s ha -
bl e by year s i n pri son.
I have wa t c he d ki ds t est i f yi ng bef or e Congress. It is
cl ear t hat t hey ar e c ompl e t e l y u n a wa r e of t he ser i ous-
ness of t hef t acts. The r e is obvi ous l y a c ul t ur a l gap. The
act of br e a ki ng i nt o a c omput e r s ys t em has to have t h e
s ame soci al st i gma as br e a ki ng i nt o a nei ghbor ' s house.
It s houl d not ma t t e r t hat t he nei ghbor ' s door is un-
l ocked. The pr ess mus t l e a r n t hat mi s gui de d use of a
c omput e r is no mor e amazi ng t han dr unk dr i vi ng of an
aut omobi l e.
Acknowl edgment . I fi rst r ead of t he pos s i bi l i t y of s uch
a Tr oj an hor se i n an Ai r For ce cr i t i que [4] of t he secu-
r i t y of an ear l y i mpl e me nt a t i on of Mul t i cs. I cannot f i nd
a mor e speci f i c r ef er ence to t hi s document . I woul d
appr eci at e i t i f a nyone who can s uppl y t hi s r ef er ence
woul d l et me know.
REFERENCES
1, Bobrow, D.G., Burchfiel, J.D., Murphy, D.L., and Tomlinson, R.S.
TENEX, a paged time-sharing syst em for t he PDP-10. Commun. ACM
15, 3 {Mar. 1972}, 135-143.
2. Kernighan, B.W., and Ritchie, D.M. The C Programming Language.
Prentice-Hall, Englewood Cliffs, N.J., 1978.
3. Ritchie, D.M., and Thompson, K. The UNIX time-sharing system.
Commun. ACM 17, 0uly 1974), 365-375.
4. Unknown Air Force Document.
Author's Present Address: Ken Thompson, AT&T Bell Laboratories,
Room 2C-519, 600 Mountain Ave., Murray Hill, NJ 07974.
Permission to copy without fee all or part of this material is granted
provided that the copies are not made or distributed for direct commer-
cial advantage, the ACM copyright notice and the title of the publication
and its date appear, and notice is given that copying is by permission of
the Association for Computing Machinery. To copy otherwise, or to
republish, requires a fee and/or specific permission.
August 1984 Volume 27 Number 8 Communications of the ACM 763