1

Chapter 2: Computer Memory and Storage,

Representing Numbers, Random Numbers

Memory and Computer Representation of Data

The computer memory contains millions of transistors which at any time can be in one of two physical
states, usually labelled 0 and 1.
Each transistor carries 1 bit of information.
8 bits = 1 byte
210 bytes = 1 K byte ≡ 1024 bytes
220 bytes = 1 Mega byte
230 bytes = 1 Giga byte
240 bytes = 1 Tera byte
How many bits are needed?
(A) Characters
English: 26 lower case: a,b,c,....
26 upper case: A,B,C,....
10 numerals: 1,2,3,....
≈ 15 ‘extras’ +–.,;:* etc.
Total ≈ 77
Given N bits we can create 2N different combinations
Example: N=2, {0, 0}, {0, 1}, {1, 0}, {1, 1}
N=6 gives 26 = 64 combinations, too low for CHARACTERS.
In fact use 1 byte = 8 bits for each character.

(B) Book
say 40 lines × 80 characters per page,
1 page = 3200 bytes ≈ 3 K bytes.
300 pages ≈ 1 M byte.

(C) Music
To detect a frequency of 20KHz you need 40,000 valves of pressure per second. If each valve is given by 16
bits (CD quality) this amounts to
80 K Bytes / second
10 M Bytes / minute
650 M Bytes / 65 minute CD...about the capacity of a single disc.

(D) Pictures
Each dot on a computer image is called a PIXEL.
A good screen might have 1080 × 780 ≈ 1 M PIXELS.
Each pixel has colour and brightness specified by (about) 3 bytes.
Therefore a single image requires 3 Mb.
(A chemical photograph contains approximately 30-40 Mb of information.)
 2

Representation of Numbers

Integers: Two’s Complement Arithmetic

Integers are usually stored using an integer number of bytes, hence one usually refers to 8-bit (see below),
16-bit, 32-bit (default value on many computers) or 64-bit integers. The number of bits controls the range
of integers that can be stored, e.g., 8-bits allows 28 = 256 combinations, and so allows only 256 integers to
be stored.
One method for storing (8-bit) integers on the computer, that leads to a convenient binary ‘arithmetic’, is
known as the two’s complement method. According to this method, the left-most bit represent −2 7 = −128,
(i.e. minus the expected value), so that, for example

11010101 = −1 × 27 + 1 × 26 + 1 × 24 + 1 × 22 + 1 × 20 = −43

i.e. the left-most bit represents −27 then 26 , 25 ,...., 20 . This allows the range [−128, 127] to be stored.
How are these numbers added and subtracted?
Addition
Addition proceeds just like ordinary decimal addition
Example

1001 1101 − 99
+ 0001 0100 + 20
1011 0001 − 79

Subtraction
For subtraction we exploit the following theorem and then use addition.
Theorem
Suppose f (n) is a function that flips all the bits in n (e.g. f (1001 0111) = 0110 1000) then

−n = f (n) + 1

Proof

f (n) + n = 1111 1111 = −1 × 27 + 1 × 26 + 1 × 25 + ... + 1 × 20

= −128 + (64 + 32 + 16 + 8 + 4 + 2 + 1)
Xm
= −1 as 2i = 2m+1 − 1
i=0

Therefore a calculation
a=b−c
can be rewritten as
a = b + (−c) = b + f (c) + 1
This uses only addition and bit flipping, both of which are fast operations. Note, however, that in two’s
complement arithmetic we have the unusual results

2 × 64 = −128 and 1 + 127 = −128

 3

both because the left bit = -128. Also

2 × −128 = 0,
as multiplying by 2 shifts bits to the left, i.e.

2 × 0000 1101 = 0001 1010

(compare multiplying by 10 in decimal!)

Reals and Round-Off Error

Reals are stored as floating point numbers
eeeeeeee
±1. f f f f f f f f f f f × 2| {z }
| {z }
mantissa exponent

In the case of 32-bit (standard) floating point numbers, the mantissa is usually 23 bits long, and the exponent
is an 8-bit (two’s complement) integer in the range [−128, 127].
Representing the reals in this way has several consequences:

Rational Fractions
In base 2, just like base 10, many rational fractions have a repeating pattern after the decimal point.
Example
Store 1/7 (in base 10) as a decimal in binary.

1
7 = 2−3 (1 + 71 )
= 2−3 + 2−6 (1 + 71 ) ! substituting for 1/7 from above
= (0.001)2 ! in binary
Example
Store 1/10 (in base 10) as a decimal in binary.

1 16
10 = 2−4 ( 10 )
6
= 2 (1 + 10
−4
)
1
= 2 + 2 + 2−4 . 10
−4 −5
1
= 2 + 2 + 2 + 2−9 + 2−8 . 10
−4 −5 −8

= (0.00011)2
In binary, then, both 1/7 and 1/10 are repeating fractions! Since the reals are stored with a finite mantissa
(typically 23 bits) even 1/10 will only be approximately stored.
To improve precision we can use reals with more bits, e.g. Salford allows 64-bit reals, which can be called
with the declaration
integer, parameter::long=selected real kind(p=12)
real(kind=long)::x ! makes x a 64 bit real
Ordinary reals have 23 digits past the decimal point in binary ≈ 7 digits past the decimal point in base 10.
p=12 in the expression above asks for AT LEAST 12 digits in base 10 (in fact you get 16 for 64 bit reals).
 4

Examples of Round-off Error

Consider the sum 1+x for some small number x.
For 32-bit reals, x can be as small as 10−38 (1.0000000 × 211111111 = 2−128 = 10−38 ).
BUT 1+x cannot have an exponent of -128. Because 20 = 1, it must have exponent = 0. Therefore

1 + x = 1.f1 f2 f3 .......f22 f23 × 20

The smallest possible value of x is therefore 2−23 ≈ 10−7 (giving f1 = f2 = ... = f22 = 0, f23 = 1). Then

1 + x = 1.00000000000000000000001 × 20

If |x| < 2−23 then 1+x=1.

Summing Convergent Series

Suppose we want to calculate the summation
5000
X 1
n=1
n2

The final 1500 terms in this series are all small < 10−7 so if we add the summation FORWARDS (starting)
from the first term, they will not contribute to the final answer as there is a 1+x =1 round-off error for
every term. However, taken together, the final 1500 terms add to give ≈ 1.36 × 10 −4 , a significant error!
Solution One (not entirely foolproof) way to get around this is to add the terms in the summation BACK-
WARDS, i.e. from the smallest term first. This will minimise the accumulated round-off error.

Quadratic Equations
Consider a quadratic equation
ax2 + bx + c = 0.
and assume that b > 0 for what follows (although the argument is easily modified). The roots of this
equation are √ √
−b − b2 − 4ac −b + b2 − 4ac
x1 = , and x2 =
2a 2a
2
√ 2
2
Supposing we have b  4ac. Then b − 4ac ≈ b(1 − 2ac/b ) and

b c
x1 ≈ − , and x2 = −
a b
Note that |x1 |  |x2 | since x1 /x2 = b2 /ac  1.
A problem with round-off error may arise if 4ac < 10−7 b2 . Then the computer will calculate b2 − 4ac = b2
and will then calculate x2 = 0!!!!
Solution A robust quadratic solver proceeds as follows. First calculate
√
−b − b2 − 4ac
x1 =
2a
 5

as normal. Then note that

√ √
−b − b2 − 4ac −b + b2 − 4ac 4ac c
x1 x2 = × = 2 =
2a 2a 4a a
Now recover the ‘problem’ root x2 from
c
x2 = .
ax1
Clearly if x1 ≈ −b/a then x2 ≈ −c/b as it should! We have avoided round-off error.

Random Numbers
A computer is an entirely deterministic device, i.e. it does not have access to any genuinely random process.
‘Random’ numbers must therefore be generated from a deterministic sequence - ideally one which ‘appears’
to be random to the casual observer (although of course is not really). ‘Random’ numbers generated in this
fashion are adequate for most pratical purposes.

The Linear Congruence Algorithm

One popular and relatively simple algorithm is as follows:

1. Choose 3 numbers a, c and m where a < c < m.

2. Choose a number I0 < m

3. Generate the sequence I0 , I1 , I2 ,.... by

Ij+1 = (aIj + c) mod m

Example: m = 7, a = 2, c = 3

then I0 = 5, I1 = (2 × 5 + 3) mod 7 = 6, I2 = 1, I3 = 5, I4 = 6, etc.

4. This generates a sequence of integers between 0 and m − 1. Dividing by m gives a sequence of reals
between 0 and 1 − 1/m.

5. Some choices of a, c, m are better than others

GOOD CHOICE : m = 233280, a = 9301, c = 49297 (Numerical Recipes)

BAD CHOICE : m = 8, a = 2, c = 4 I0 = 2, I1 = 0, I2 = 4, I3 = 4, I4 = 4, etc.

The Instrinsic Subroutine

In FORTRAN random numbers can be called using the intrinsic subroutine
Call random number(x)
This uses an algorithm chosen by the compiler company. Like the linear congruence algorithm, it will be
deterministic, i.e. every time it runs it will give the same random numbers.
To change from run to run, can use
Call random seed()
 6

or alternatively add
print *,’enter number < 1000
read *,nran
do i=1,nran
call random number(x)
end do !returns last value of x

Generating Other Random Variables using the Intrinsic Subroutine

We can use the random variable X from the intrinsic subroutine to generate random variables with the
probability distribution of our choice.
Recall: X is uniformly distributed on [0, 1].

Continuous Random Variables

Suppose we want to generate a random variable Y with probability density P (Y ). Recall that for a reali-
sation Yi of Y Z Ymax
Prob(Y ≤ Yi < Y + δY )
P (Y ) = lim , P (Y ) dY = 1.
δY →0 δY Y
min
To obtain Y in terms of X,
Set P (Y ) dY = P (X) dX = dX
Then rearrange and solve
dX
= P (Y ) with x(Ymin ) = 0,
dY
and invert to get Y (X). Then the random variable Y = Y (X) will have the correct distribution.
Example
Find Y(X) so that
P (Y ) dY = exp {−Y } dY, 0<Y <∞
Following the above procedure
dX
= exp {−Y }, X(0) = 0.
dY
Solving get
X = 1 − exp {−Y },
and rearranging
1
Y (X) = log
1−X
gives the correct distribution.

Discrete Random Variables

We can also generate discrete random variables using the intrinsic subroutine
This can be done using integer variables. Suppose we want a discrete random variable Z to take integer
values i1 to i2 inclusive with equal probability. Then the following FORTRAN lines can be used
Integer::Z,i1,i2
 7

call random number(x)

Z=(i2-i1+1)*x+i1
Example: Coin Toss
Want a discrete random variable a that takes the values 0 or 1 with equal probability. Then i1=0, i2-i1+1=2,
so
Integer::a
call random number(x)
a=2*x