9/10/2024

CHEN 101
Chemical Engineering Principles I
Dr. Ajaz Rashid

Introduction to Engineering
Calculations
Chapter 2

1
 9/10/2024

2.1 Units and Dimensions

• A dimension is a property that can be

• measured, such as length, time, mass, or temperature, or
• calculated by multiplying or dividing other dimensions, such as length/time (velocity),
length3 (volume), or mass/length3 (density).

• A unit is a way to assign a numerical value to the measurement of that dimension

• mass is a dimension
• kilogram (kg) is a unit
• kilogram helps us to assign a numerical value to mass (e.g. mass of a bottle of HCl is
measured to be 1.2 kg)
• Using a different unit will result in a different numerical value, generally

2.1 Units and Dimensions

• Units can be treated like algebraic variables when quantities are added,
subtracted, multiplied, or divided
• Two quantities may be added or subtracted only if their units are the same

4 cm – 1 cm = 3 cm 3 cm + 2 mm = ??
4𝑥 – 𝑥 = 3𝑥 3𝑥 + 2𝑦 = ??

2
 9/10/2024

2.1 Units and Dimensions

• Numerical values and their corresponding units may always be combined by

multiplication or division.

3 N × 4 m = 12 N.m 2 m × 2 m = 4 m2
3𝑥 × 4𝑦 = 12 𝑥.𝑦 2𝑦 × 2𝑦 = 4 𝑦2
5 m / 1 s = 5 m/s
6 m / 3 m/s = 2 s
6 g / 3 g = 2 (dimensionless) 5𝑦 ÷ 5𝑦 = 1

2.2 Conversion of Units

• A measured quantity can be expressed in terms of any units having the

appropriate dimension.
• Velocity maybe expressed in m/s, ft/s, miles/h, cm/min, km/yr or any other
ratio of a length unit to a time unit.
• The numerical value of the velocity will depend on the units chosen.

10 m/s = 32.8 ft/s = 22.4 mph = 60000 cm/min = 315360 km/yr

3
 9/10/2024

2.2 Conversion of Units

• The equivalence between two expressions of the same quantity may be defined in
terms of a ratio:
1 cm 10 mm 1 hr 1 kg 2.20462 lb𝑚
10 mm 1 cm 60 min 2.20462 lb𝑚 1 kg
• These ratios are known as conversion factors.
• To convert a quantity expressed in terms of one unit to its equivalent in terms of
another unit, multiply the given quantity by the conversion factor (new unit/old
unit). For example, to convert 1.2 m to its equivalent in cm, write

100 cm
1.2 m × = 120 cm
1m

2.2 Conversion of Units

• TEST YOURSELF (page 7)

1. What is a conversion factor?

2. What is the conversion factor for s/min?
3. What is the conversion factor for min2/s2?
4. What is the conversion factor for m3/cm3?

4
 9/10/2024

Quantity Equivalent values

1 kg = 1000 g = 0.001 metric ton = 2.20462 lbm = 35.27392 oz
2.2 Conversion of Units
Mass
1 lbm = 16 oz = 5 x 10-4 ton = 453.593 g = 0.453593 kg
Conversion Factors Table

Length 1m = 100 cm = 1000 mm = 106 microns = 1010 angstroms

= 39.37 in = 3.2808 ft = 1.0936 yd = 0.0006214 mile
1 ft = 12 in. = 1/3 yd = 0.3048 m = 30.48 cm
Volume 1m3 = 1000 L = 106 cm3 = 106 mL
= 35.3145 ft3 = 220.83 imperial gallons = 264.17 gal
= 1056.68 qt
1 ft3 = 1728 in.3 = 7.4805 gal = 0.028317 m3 = 28.317 L
= 28317 cm3
Force 1N = 1 kgm/s2 = 105 dynes = 105 gcm/s2 = 0.22481 lbf
1 lbf = 32.174 lbmft/s2 = 4.4482 N = 4.4482 x 105 dynes
1 atm = 1.01325 x 105 N/m2 (Pa) = 101.325 kPa = 1.01325 bar
Pressure = 1.01325 x 106 dynes/cm2 = 760 mm Hg at 0oC (torr)
= 10.333 m H2O at 4oC = 14.696 lbf/in.2 (psi)

Energy 1J = 1 Nm = 107 ergs = 107 dyne.cm = 2.778 x 10-7 kWh

= 0.23901 cal = 0.7376 ft-lbf = 9.486 x 10-4 Btu
Power 1W = 1 J/s = 0.23901 cal/s = 0.7376 ft.lbf/s = 9.486 x 10-4 Btu/s
1.341 x 10-3 hp

2.2 Conversion of Units

• Example 2.2-1

• Convert an acceleration of 1 cm/s2 to km/yr2.

1 cm 36002 s 2 242 h2 3652 day 2 1 m 1 km

= 9.95 × 109 km/yr 2
s2 12 h2 12 day 2 12 yr 2 100 cm 1000 m

10

5
 9/10/2024

2.3 Systems of Units

• Common Systems of Units

SI Units CGS system English units

Centimeter Gram US Customary

International
Second system system of units
system of units
of units (ft, s, lb)
(m, s, kg)
(cm, s, g) also called FPS

11

2.3 Systems of Units

A system of Mass, length, time, temperature, electrical current, light

Base units
units has the intensity, quantity of matter
following
components
Multiple units Multiples or fractions of base units

mm cm m dm km

Derived units Obtained in one of the two ways:

Compound unit: by multiplying and Defined equivalent of compound unit:

dividing base or multiple e.g. 1 erg = 1 g.cm/s2,
units; e.g. cm2, ft/min, m/s2 1 lbf = 32.174 lbm.ft/s2

12

6
 9/10/2024

2.3 Systems of Units

Base Units
Quantity SI CGS English

Length meter (m) centimeter (cm) foot (ft)

Mass kilogram (kg) gram (g) pound (lb)

Moles gram-mole (gmol) gram-mole (gmol) pound-mole (lb-mol)

Time second (s) second (s) second (s)

Temperature Kelvin (K) Kelvin (K) Rankine (oR)

Electric current Ampere (A) Ampere (A) Ampere (A)

Light intensity Candela (cd) Candela (cd) Candela (cd)

13

2.3 Systems of Units

Multiple Unit Preferences
Factor Name Symbol Factor Name Symbol
1024 yotta Y 10-1 deci d
1021 zetta Z 10-2 centi c
1018 exa E 10-3 milli m
1015 peta P 10-6 micro 
1012 tera T 10-9 nano n
109 giga G 10-12 pico p
106 mega M 10-15 femto f
103 kilo k 10-18 atto a
102 hecto h 10-21 zepto z
101 deca da 10-24 yocto y
Examples: 1 GJ = 109 J 1 mg = 10-3 g

14

7
 9/10/2024

2.3 Systems of Units

• TEST YOURSELF (page 9)

1. What are the factors (numerical values and units) needed to convert
a) meters to millimeters?
b) nanoseconds to seconds?
c) square centimeters to square meters?
d) cubic feet to cubic meters (use the conversion factor table on the inside front cover)?
e) horsepower to British thermal units per second?
2. What is the derived SI unit for velocity? The velocity unit in the CGS
system? In U.S. customary units?

15

2.3 Systems of Units

• Example 2.2-1

• Convert 23 lbm.ft/min2 to its equivalent in kg.cm/s2.

Answer: 0.088 kg.cm/s2

16

8
 9/10/2024

2.4 Force and Weight

• According to Newton’s Second Law of Motion

𝐹=𝑚.𝑎

• Natural force units are therefore:

• SI units: kg.m/s2
• cgs units: g.cm/s2
• American units: lbm.ft/s2
• Since these units are too long, Derived Force Units have been defined

17

2.4 Force and Weight

• SI Units: 1 Newton (1 N) = 1 kg.m/s2

• 1 N is the force required to accelerate a mass of 1 kg at the rate of 1 m/s2
• cgs Units: 1 dyne = 1 g.cm/s2
• 1 dyne is the force required to accelerate a mass of 1 g at the rate of 1 cm/s2
• American Engineering Units: pound force (lbf)
• 1 pound (lbf) is the force exerted by earth on an object that has a mass of 1
lbm and falling at acceleration of gravity at sea level and 45o latitude (which
is 32.174 ft/s2).
• 1 pound force (lbf) = 32.174 lbm.ft/s2

18

9
 9/10/2024

2.4 Force and Weight

Example:
• What is the force required to accelerate a mass of 4.0 lbm at a rate of 80.435 ft/s2.
Express the force in the unit of lbf.

𝐹 = 𝑚. 𝑎
4.0 𝑙𝑏𝑚 80.435 𝑓𝑡
=
𝑠2
321.74 𝑙𝑏𝑚 . 𝑓𝑡
=
𝑠2
321.74 𝑙𝑏𝑚 . 𝑓𝑡 1𝑙𝑏𝑓
=
𝑠 2 32.174 𝑙𝑏𝑚 𝑓𝑡Τ𝑠 2
= 10𝑙𝑏𝑓

19

2.4 Force and Weight

Mass vs Weight
• The weight of an object is the force exerted on the object by gravitational
attraction.
Example: If an object has a mass of 25 kg then what is its weight?

𝑭=𝒎×𝒂
or
𝑾=𝒎×𝒈

25 kg 9.81 m 245.25 kg. m

𝑊= ቤ = = 245.25 N
s2 s2

20

10
 9/10/2024

2.4 Force and Weight

• The value of 𝑔 at sea level and 45o latitude in different units is:
𝑔 = 9.8066 m/s2
= 980.66 cm/s2
= 32.174 ft/s2

• If the value of 𝑔 is not specified, then you may assume the above values

21

2.4 Force and Weight

• Example 2.4-1
Water has a density of 62.4 lbm/ft3. How much does 2.000 ft3 of water weigh
• (1) at sea level and 45o latitude and
• (2) in Denver, Colorado, where the altitude is 5374 ft and the gravitational
acceleration is 32.139 ft/s2?
Answer: (1) 124.8 lbf
(2) 124.7 lbf

22

11
 9/10/2024

2.4 Force and Weight

• TEST YOURSELF (page 11)

1. What is a force of 2 kg m/s2 equivalent to in newtons? What is a force of

2 lbm ft/s2 equivalent to in lbf?
2. If the acceleration of gravity at a point is 𝑔= 9.8 m/s2 and an object is resting on
the ground at this point, is this object accelerating at a rate of 9.8 m/s2?
3. Suppose an object weighs 9.8 N at sea level. What is its mass? Would its mass be
greater, less, or the same on the moon? How about its weight?
4. Suppose an object weighs 2 lbf at sea level. What is its mass? Would its mass be
greater, less, or the same at the center of the earth? How about its weight?
(Careful!)

23

2.6 Dimensional Homogeneity and Dimensionless

Quantities

• All quantities can be added and subtracted ONLY if their units are the same.

• Dimensional Homogeneity in Equations:

Every valid equation must be dimensionally homogeneous; that is all additive
terms on both sides of the equation must have the same dimensions.

𝑆 𝑚 = 𝑢 𝑚Τ𝑠 × 𝑡 𝑠 + ½ 𝑎 𝑚Τ𝑠 2 × 𝑡 2 (𝑠 2 )

24

12
 9/10/2024

2.6 Dimensional Homogeneity and Dimensionless

Quantities

• In this equation,

𝑆 𝑚 = 𝑢 𝑚Τ𝑠 × 𝑡 𝑠 + ½ 𝑎 𝑚Τ𝑠 2 × 𝑡 2 𝑠 2

• Each additive term has dimension of Length

• Hence, we say it is dimensionally homogenous
• Also, each additive term has a unit of meter (m)
• Hence, we say it is consistent in units

25

2.6 Dimensional Homogeneity and Dimensionless

Quantities

• Consider the same equation but in different units,

𝑆 𝑓𝑡 = 𝑢 𝑚Τ𝑠 × 𝑡 𝑚𝑖𝑛 + ½ 𝑎 𝑐𝑚Τ𝑠 2 × 𝑡 2 𝑠 2

• Each additive term has dimension of Length

• Hence, we say it is dimensionally homogenous
• But the additive terms have different units
• Hence, we say it is inconsistent in units
• We can fix such an equation by converting the units

26

13
 9/10/2024

2.6 Dimensional Homogeneity and Dimensionless

Quantities

• Consider the same equation but in different units,

𝑆 𝑚 = 𝑢 𝑚Τ𝑠 + ½ 𝑎 𝑐𝑚Τ𝑠 2 × 𝑡 2 𝑠 2

• Additive terms have different dimensions

• Hence, we say it is NOT dimensionally homogenous
• Such an equation cannot be valid
• For an equation to be valid it must be dimensionally homogenous

27

2.6 Dimensional Homogeneity and Dimensionless

Quantities

• A dimensionless quantity can be a pure number

(2, 1.5, 3/2 )
• or a multiplicative combination of variables with no net dimensions:
• Relative Values: e.g., relative densities: (the ratio of density of a substance to a
density of a standard)
• Dimensionless Group: e.g., Reynolds Number:
𝜌 𝑔Τ𝑐𝑚3 𝐷(𝑐𝑚)𝑣(𝑐𝑚Τ𝑠)
𝑅𝑒 =
𝜇 𝑔Τ𝑐𝑚. 𝑠

28

14
 9/10/2024

2.6 Dimensional Homogeneity and Dimensionless

Quantities

• Exponents: e.g. 2 in 𝑦2
• Transcendental Functions: e.g. log, ln, exp or e, sin
• Arguments of Transcendental Functions: e.g. 𝑥 in log(𝑥) or sin (𝑥)

RIGHT WRONG

102 ft 102ft
log (10) log (10 kg)
sin(0.5) sin(0.5 N)

29

2.6 Dimensional Homogeneity and Dimensionless

Quantities

• Example 2.6-1
• Consider the equation

𝐷 𝑓𝑡 = 3𝑡 𝑠 + 4

• If the equation is valid, what are the dimensions of the constants 3 and 4?
• If the equation is consistent in its units, what are the units of 3 and 4?
• Derive an equation for distance in meters and time in minutes.

30

15
 9/10/2024

2.6 Dimensional Homogeneity and Dimensionless

Quantities

• For the equation to be valid, it must be dimensionally homogeneous, so that

each term must have the dimension of length. The constant 3 must
therefore have the dimension length/time , and 4 must have the dimension
length.
• For consistency, the constants must be 3 ft/s and 4 ft .

31

2.6 Dimensional Homogeneity and Dimensionless

Quantities

• Define new variables 𝑫’ (m) and 𝒕’ (min). The equivalence relations between
the old and new variables are:

𝐷′ m 3.2808 ft
𝐷 ft = อ = 3.2808 𝐷′
1m

𝑡 ′ min 60 𝑠
𝑡 s = ቤ = 60 𝑡′
1 min

32

16
 9/10/2024

2.6 Dimensional Homogeneity and Dimensionless

Quantities

• Substitute these expressions in the given equation

3.2808𝐷′ = 3 60𝑡 ′ + 4

• Or we can simplify it by dividing both sides by 3.2808:

𝐷′ 𝑚 = 54.86𝑡 ′ 𝑚𝑖𝑛 + 1.22

33

2.6 Dimensional Homogeneity and Dimensionless

Quantities

• Example 2.6-2
• A quantity 𝑘 depends on the temperature 𝑇 in the following manner:
𝑚𝑜𝑙 5
20000
𝑘 = 1.2 × 10 𝑒𝑥𝑝 −
𝑐𝑚3 . 𝑠 1.987𝑇

• The units of the quantity 20000 are cal/mol and 𝑇 is in K (kelvin). What are
the units of 1.2x105 and 1.987?

34

17
 9/10/2024

2.6 Dimensional Homogeneity and Dimensionless

Quantities

• Since the equation must be consistent in its units and exp is dimensionless,
1.2×105 should have the same units as 𝑘, mol/(cm3.s). Moreover, since the
argument of exp must be dimensionless, we can write

20000 1 mol. K
=1
𝑇 (K) 1.987 cal

1 1
20000 = = = cal/mol
1 mol. K mol/cal
ฬ
K cal

35

2.6 Dimensional Homogeneity and Dimensionless

Quantities

• Since the equation must be consistent in its units and exp is dimensionless,
1.2×105 should have the same units as 𝑘, mol/(cm3.s). Moreover, since the
argument of exp must be dimensionless, we can write

20000 1 mol. K
=1
𝑇 (K) 1.987 cal

1 1
20000 = = = cal/mol
1 mol. K mol/cal
Kฬ cal

36

18
 9/10/2024

2.6 Dimensional Homogeneity and Dimensionless

Quantities

• TEST YOURSELF (page 21)

1. What is a dimensionally homogeneous equation? If an equation is

dimensionally homogeneous, is it necessarily valid? If an equation is valid,
is it necessarily dimensionally homogeneous?
2. If 𝑦 𝑚/𝑠 2 = 𝑎 𝑧 𝑚3 , what are the units of 𝑎?
3. What is a dimensionless group? What multiplicative combination of 𝑟(𝑚),
𝑠(𝑚/𝑠 2 ) and 𝑡(𝑠) would constitute a dimensionless group?
4. If 𝑧 𝑙𝑏𝑓 = 𝑎 sin(𝑄), what are the units of 𝑎 and 𝑄?

37

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

• In process calculations, we come across very large numbers as well as very

small numbers
• A convenient way to express these numbers is using the scientific notation
• In scientific notation, a number is expressed as the product of another
number (usually between 0.1 and 10) and a power of 10.
• Examples:
123,000,000 = 1.23 × 108

0.000028 = 2.8 × 10−5

38

19
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

• Significant Figures of a number are the digits from the first nonzero digit on
the left of the number to either:
• the last digit (zero or nonzero) on the right if there is a decimal point
• (e.g., 86.030 or 8.6030 X 101 has 5 significant figures)
or
• the last nonzero digit of the number if there is no decimal point
• (e.g., 12,000 has 2 significant figures)

39

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

Number Scientific Notation Significant Figures

6.3528
34.020
0.005708
0.00007835
850950
2500
250.0
40

20
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

Number Scientific Notation Significant Figures

6.3528 6.3528×100 5
34.020 3.4020×101 5
0.005708 5.708×10-3 4
0.00007835 7.835×10-5 4
850950 8.5095×105 5
2500 2.5×103 2
250.0 2.500×102 4
41

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

• The rule for significant figures applies only to measured

quantities or numbers calculated from measured quantities
• If a quantity is known precisely — like a pure integer (2) or a
counted rather than measured quantity (16 oranges)—its value
implicitly contains an infinite number of significant figures

42

21
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

• Accuracy is how close a measurement is to the correct value for that

measurement.
• For example, let us say that you are measuring the length of a paper
that is 11.0 inches long.
• You measure the length of the paper three times and obtain the
following measurements: 11.1 in., 11.2 in., and 10.9 in.
• These measurements are quite accurate because they are very close to
the correct value of 11.0 inches.
• In contrast, if you had obtained a measurement of 12 inches, your
measurement would not be very accurate.

43

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

• The precision of a measurement system refers to how close the

agreement is between repeated measurements (which are
repeated under the same conditions).
• Consider the example of the paper measurements. The precision
of the measurements refers to the spread of the measured
values.

44

22
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

45

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

• The number of significant figures in a given value is an indication

of the precision with which the quantity is known.
• The more significant figures, the more precise is the value

46

23
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision
28.3 mL

28 mL

30 mL

47

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

• Mathematical Operation and Significant Figures

• 4.30 x 2.444 = 10.5, 10.51, 10.509, or 10.5092 ?

• 4.30 + 2.444 = 6.7, 6.74, 6.744, or 6.7440 ?

Always Remember!!

The precision of the final answer

cannot be greater than the least precise measurement

48

24
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

• Mathematical Operation and Significant Figures

• Multiplication and Division:

• 4.30 x 2.444 = 10.5

10.5092
3 s.f 4 s.f 3 s.f

49

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

• Mathematical Operation and Significant Figures

• Multiplication and Division:

• Example:

• 10.523 x10-3 x 11 x103 ÷ 2.50 = 46

50

25
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

• Mathematical Operation and Significant Figures

• Addition and Subtraction:

• When two or more numbers are added or subtracted, the

positions of the last significant figures of each number relative to
the decimal point should be compared. Of these positions, the
one farthest to the left is the position of the last permissible
significant figure of the sum or difference.

51

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

• Mathematical Operation and Significant Figures

• Addition and Subtraction:

position of last
s.f.

• Example 1530
– 2.56
• 1530 – 2.56 = 1527.44 1530

position of last
s.f.

52

26
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

• Mathematical Operation and Significant Figures

• Addition and Subtraction:

position of last
s.f.

• Example 4.30
+ 2.444
• 4.30 + 2.444 = 6.744 6.74

position of last
s.f.

53

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

• Mathematical Operation and Significant Figures

• Addition and Subtraction:

• Example

• 1.2 + 2.03 = 3.2

• 1230 + 2.03 = 1230

54

27
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

• A small area in a chemical plant which is 20.5 meters by 8.5

meters needs to be covered by waterproofing material. A tarp is
available, which is 5.50 meters by 10.00 meters. If the tarp is laid
over part of this area, how much of the area remains uncovered?

• Keep in mind the significant figures

55

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

• TEST YOURSELF (page 13)

1. Express the following quantities in scientific notation and indicate how many
significant figures each has.
a) 12,200
b) 12,200.0
c) 0.003040
2. Express the following quantities in standard decimal form and indicate how many
significant figures each has.
a) 1.34×105
b) 1.340×10-2
c) 0.00420×106

56

28
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

• TEST YOURSELF (page 13)

3. How many significant figures would the solution of each of the following
problems have? What are the solutions of (c) and (d)?
a) (5.74)(38.27) / (0.001250)
b) 1.000 + 10.2
c) (1.76×104)(0.12×10-6)
d) 18.76 – 7
4. Round off each of the following numbers to three significant figures.
a) 1465
b) 13.35
c) 1.765×10-7

57

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

• TEST YOURSELF (page 13)

5. When the value of a number is given, the significant figures provide an indication
of the uncertainty in the value; for example, a value of 2.7 indicates that the
number lies between 2.65 and 2.75. Give ranges within which each of the
following values lie.
a) 4.3
b) 4.30
c) 2.778×10-3
d) 2500
e) 2.500×103

58

29
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5b Validating Results

• The two most important questions to ask when solving problems

1. How do I get a solution?

• Asked when solving problems that arise in the design and analysis of
chemical processes
2. When I get a solution, how do I know it’s right?
• This is also very important, and serious problems can arise when it is not
asked by an engineer.

59

2.5 Numerical Calculation and Estimation

2.5b Validating Results

• You can validate your answers in the following ways:

• Back Substitution
• Order-of-Magnitude Estimation
• Test of Reasonableness

60

30
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5b Validating Results
• Back substitution
• After you solve a set of equations, substitute your solution back into the equations
and make sure it works:
3𝑥 2 – 12 = 0
• I solve the equation and find the solution as ±2.
• I substitute this value in the equation and check:
3(±2)2 – 12
= 3 × 4 – 12
= 12 – 12
= 0
Hence my solution is right

61

2.5 Numerical Calculation and Estimation

2.5b Validating Results
• Order of magnitude estimation
• The procedure for checking an arithmetic calculation by order-of-magnitude
estimation is as follows:

1. Substitute simple integers for all numerical quantities, using powers of 10

(scientific notation) for very small and very large numbers.
27.36 → 20 or 30 (whatever makes calculation easier)
63472 → 6x104

62

31
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5b Validating Results
2. Do the resulting arithmetic calculations by hand, continuing to round off
intermediate answers.

(36720)(0.0624) (4 × 104 )(5 × 10−2 )

= = 4 × 10(4−2+4) = 4 × 106
0.000478 5 × 10−4
3. If a number is added to a second, much smaller, number, drop the second
number in the approximation.

1 1 1
≈ ≈ = 0.25
4.13 + 0.04762 4.13 4

63

2.5 Numerical Calculation and Estimation

2.5b Validating Results

• Test of Reasonableness
• You calculate the diameter of a distillation column to be 2.3x105 meters
Does this look right?

• You calculate the amount of chlorine required for treatment of drinking

water for a city to be 0.32 mL
Does this make sense?

• You calculate the temperature of superheated steam to be -10 oC

Is this reasonable?

64

32
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5b Validating Results
• Example 2.5-1
• The calculation of a process stream volumetric flow rate has led to the
following formula:

254 13 1
𝑉ሶ = + ×
(0.879)(62.4) (0.866)(62.4) (31.3145)(60)

65

2.5 Numerical Calculation and Estimation

2.5b Validating Results
• Example 2.5-1
• The calculation of a process stream volumetric flow rate has led to the
following formula:
254 13 1
𝑉ሶ = + ×
(0.879)(62.4) (0.866)(62.4) (31.3145)(60)

Estimate the answer without using a calculator.

250 10 1 5
𝑉ሶ ≈ + × 1 1
≈ ≈ 0.2 × 10−2 = 0.002
50 60 4 × 10 6 × 10 25 × 102

66

33
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean
• Suppose we carry out a chemical reaction of the form A → Products, starting
with pure A in the reactor and keeping the reactor temperature constant at
45 oC.

• After two minutes we draw a sample from the reactor and analyze it to
determine X, the percentage of the A fed that has reacted.

67

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean
• In theory should have a unique value;
• However, in a real reactor X is a changing in an unpredictable manner from
one run to another at the same experimental conditions.
• The values of X obtained after 10 successive runs might be as follows:

Run 1 2 3 4 5 6 7 8 9 10
X (%) 67.1 73.1 69.6 67.4 71.0 68.2 69.4 68.2 68.7 70.2

68

34
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean
• Why don’t we get the same value X of in each run? There are several
reasons.
• It is impossible to replicate experimental conditions exactly in successive
experiments. If the temperature in the reactor varies by as little as 0.1
degree from one run to another, it could be enough to change the measured
value of X
• Even if conditions were identical in two runs, we could not possibly draw our
sample at exactly 2.000 minutes both times, and a difference of a second
could make a measurable difference in X
• Variations in sampling and chemical analysis procedures invariably introduce
scatter in measured values.

69

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean
• What is the true value of X?
• Theoretically there exists a True Value for X.
• Practically, there is no means to determine it.
• The best we can do is estimate the true value.
• How can we estimate of the true value of X?
• The most common estimate is the “Sample Mean” (or the “arithmetic
mean”)

70

35
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean
• Sample Mean:
𝑁
1 1
𝑋ത = 𝑋1 + 𝑋2 + 𝑋3 + ⋯ + 𝑋𝑁 = ෍ 𝑋𝑗
𝑁 𝑁
𝑗=1
• For the given data:
Run 1 2 3 4 5 6 7 8 9 10
X (%) 67.1 73.1 69.6 67.4 71.0 68.2 69.4 68.2 68.7 70.2

1
𝑋ത = 67.1 + 73.1 + 69.6 + ⋯ + 70.2 = 69.3%
10

71

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean
• Graphically we can represent this as:
80

75

70
ഥ
𝑿

65

60
1 3 5 7 9
Run

72

36
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean
• The more measurements of a random variable, the better the estimated
value based on the sample mean.
• However, even with a huge number of measurements the sample mean is at
best an approximation of the true value.
• In some cases, the mean could in fact be way off (e.g., if there is something
wrong with the instruments or procedures used to measure X).

73

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean

• TEST YOURSELF (page 16)

• The weekly production rates of a pharmaceutical product over the past six
weeks have been 37, 17, 39, 40, 40, and 40 batches per week.
1. Think of several possible explanations for the observed variation in the
weekly production rate.
2. If you used the sample mean of the given data as a basis, what would you
predict the next weekly production rate to be?
3. Come up with a better prediction, and explain your reasoning.

74

37
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean
• How scattered is the data?

75

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean
• Range:
𝑅 = 𝑋𝑚𝑎𝑥 − 𝑋𝑚𝑖𝑛

• For Data set (a), 𝑅 = 72 – 68 = 4

• For Data set (b), 𝑅 = 95 – 52 = 43
• The range is the crudest measure of scatter: it involves only two of the
measured values and gives no indication of whether or not most of the
values cluster close to the mean or scatter widely around it.

76

38
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean
• Sample Variance:
• The sample variance is a much better measure of the scatter of data:

1
𝑆𝑋2 = 𝑋1 − 𝑋ത 2
+ 𝑋2 − 𝑋ത 2
+ ⋯ + 𝑋𝑁 − 𝑋ത 2
𝑁−1

• Sample Standard Deviation:

𝑆𝑋 = 𝑆𝑋2

77

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean
• The smaller the Standard Deviation for a given set of data the less scattered
the data
• The smaller the Standard Deviation, the more reliable is the set of data
• For typical random variables, roughly two-thirds of all measured values fall
within one standard deviation of the mean;
• about 95% fall within two standard deviations;
• and about 99% fall within three standard deviations.

78

39
 9/10/2024

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean

𝑋ത − 3𝑆𝑋 𝑋ത − 𝑆𝑋 𝑋ത 𝑋ത + 𝑆𝑋 𝑋ത + 3𝑆𝑋
𝑋ത − 2𝑆𝑋 𝑋ത + 2𝑆𝑋

79

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean

• TEST YOURSELF (page 18)

• The volumetric flow rate of a process fluid, 𝑉ሶ (cm3/s), is measured five times, with the
following results:
Measurement 1 2 3 4 5
𝑽ሶ (cm3/s) 232 248 227 241 239

1. Calculate the sample mean (𝑉),ത range, sample variance (𝑆𝑉2 ), and sample standard
deviation (𝑆𝑉 ).
2. There is a high probability (above 90%) that a measured value of 𝑉ሶ will fall within two
ሶ the form 𝑉ሶ = 𝑎 ± 𝑏, choosing
standard deviations of the mean. Report the value of 𝑉in
the values of 𝑎 and 𝑏 to define this range.

80

40
 9/10/2024

2.7 Process Data Representation and Analysis

• The operation of any chemical process is ultimately based on the

measurement of process variables like temperatures, pressures, flow rates,
concentrations, and so on.
• It is sometimes possible to measure these variables directly, but more
commonly indirect techniques are used.

81

2.7 Process Data Representation and Analysis

• Suppose, for example, that you wish to measure the concentration, 𝐶, of a

solute in a solution.
• To do so, you normally measure a quantity 𝑋, such as a thermal or electrical
conductivity, a light absorbance, or the volume of a titer, that varies in a
known manner with 𝐶, and then calculate 𝐶 from the measured value of 𝑋.
• The relationship between 𝐶 and 𝑋 is determined in a separate calibration
experiment in which solutions of known concentration are prepared and is
measured for each solution.

82

41
 9/10/2024

2.7 Process Data Representation and Analysis

• Concentration of a solution is measured by using a light absorbance

technique
• Calibration experiment gives the following data:
Calibration Experiment Data
30
Concentration
Absorption
mol/L 25

Concentration (mol/L)
3 0.106 20

9 0.295 15

15 0.512 10

5
20 0.720
0
28 0.880 0 0.2 0.4 0.6 0.8 1
Absorption

83

2.7 Process Data Representation and Analysis

• What if we need concentration for an absorption of 0.20?

• What if we need concentration for an absorption of 0.95?

• Interpolation: Estimation of values at points between the tabulated data

• Extrapolation: Estimation of values at points that lie outside the range of the
tabulated data

84

42
 9/10/2024

2.7 Process Data Representation and Analysis

85

2.7 Process Data Representation and Analysis

• A number of interpolation and extrapolation methods are commonly used,

including two-point linear interpolation, graphical interpolation, and curve
fitting.
• Which one is most appropriate?
• This depends on the nature of the relationship
between the variables under consideration.

86

43
 9/10/2024

2.7 Process Data Representation and Analysis

• Two Point Linear Interpolation:

• Slope of straight line is same between any two points
• Slope = 𝑦 / 𝑥

𝑦2 − 𝑦1 𝑦 − 𝑦1
=
𝑥2 − 𝑥1 𝑥 − 𝑥1

𝑥 − 𝑥1
𝑦 = 𝑦1 + (𝑦 − 𝑦1 )
𝑥2 − 𝑥1 2

87

2.7 Process Data Representation and Analysis

• What if we need concentration for an absorption of 0.20?

𝒚 𝒙
𝑥 − 𝑥1
Concentration
Absorption 𝑦 = 𝑦1 + (𝑦 − 𝑦1 )
mol/L 𝑥2 − 𝑥1 2
𝒚𝟏 3 𝒙𝟏 0.106
𝒚 =? ? ← ← 𝒙 = 𝟎. 𝟐𝟎
𝒚𝟐 9 𝒙𝟐 0.295 0.2 − 0.106
15 0.512 𝑦=3+ 9−3
0.295 − 0.106
20 0.720
28 0.880
= 5.984 𝑚𝑜𝑙/𝐿
Dependent Independent
Variable Variable

88

44
 9/10/2024

2.7 Process Data Representation and Analysis

• TEST YOURSELF (page 22)

1. Values of a variable (𝑓) are measured at several times (𝑡):

𝑓 1 4 8
𝑡 1 2 3
Show that if two-point linear interpolation is used:
(a)𝑓(𝑡 = 1.3) ≈ 1.9
(b) 𝑡(𝑓 = 5) ≈ 2.25

89

2.7 Process Data Representation and Analysis

• TEST YOURSELF (page 22)

2. If a function 𝑦(𝑥) appears as shown in each of the diagrams shown below, would two-
point linear interpolation yield estimates of 𝑦 that are too high, too low, or correct? If the
two-point linear interpolation formula were used to estimate 𝑦 𝑥3 from the tabulated
values of (𝑥1 , 𝑦1 ) and (𝑥2 , 𝑦2 ) in plot (b), would the estimated value be too high or too
low?

90

45
 9/10/2024

2.7 Process Data Representation and Analysis

• Fitting a straight line:

• The equation for a straight line is:
𝑦 = 𝑎𝑥 + 𝑏
𝑎 is the slope of the line
𝑏 is the intercept of the line 𝑏 𝑎

91

2.7 Process Data Representation and Analysis

• Fitting a straight line:

• The equation for a straight line is: Concentration
Absorption
𝑦 = 𝑎𝑥 + 𝑏 (mol/L)
Fit the given data to a straight line 3 0.106
9 0.295
∆𝑦 𝑦2 − 𝑦1 28 − 3 15 0.512
𝑎= = = = 32.3 20 0.720
∆𝑥 𝑥2 − 𝑥1 0.88 − 0.106
28 0.880

𝑏 = 𝑦 − 𝑎𝑥 = 3 − 32.3 0.106 = −0.4238 𝑪 = 𝟑𝟐. 𝟑𝑨 − 𝟎. 𝟒𝟐𝟑𝟖

𝑏 = 𝑦 − 𝑎𝑥 = 28 − 32.3 0.880 = −0.4238

92

46
 9/10/2024

2.7 Process Data Representation and Analysis

• Fitting a straight line:

• We can now use this equation
𝐶 = 32.3 𝐴 − 0.4238
to determine values of 𝐶 for any given Absorption value.
e.g. Find 𝑪 when 𝑨 = 0.6
𝐶 = 32.3 (0.6) – 0.4238
= 18.96 mol/L

93

2.7 Process Data Representation and Analysis

• Fitting non-linear data:

• If the data looks like this:

• I want to find a value of 𝑦 using a value of 𝑥 that is not in the Data Table
Can I use a fitted linear equation?
Can I use linear interpolation or extrapolation?

94

47
 9/10/2024

2.7 Process Data Representation and Analysis

• Fitting non-linear data:

• Fitting a nonlinear equation to data is usually much harder than fitting a line
• However, with some nonlinear equations you can still use straight-line fitting
if you plot the data in a suitable manner.
• Suppose that 𝑥 and 𝑦 are related by the equation 𝑦 2 = 𝑎𝑥 3 + 𝑏.
• A plot of measured 𝑦 versus 𝑥 data would clearly be curved
• However, a plot of 𝑦2 versus 𝑥 3 would be a straight line with a slope 𝑎 and
intercept 𝑏.

95

2.7 Process Data Representation and Analysis

• Here are some examples of plots that yield straight lines:

𝒚 = 𝒂𝒙𝟐 + 𝒃 → plot 𝒚 𝐯𝐬 𝒙𝟐
𝒂 𝟏
𝒚𝟐 = + 𝒃 → plot 𝒚𝟐 𝐯𝐬
𝒙 𝒙
𝟏 𝟏
=𝒂 𝒙+𝟑 +𝒃 → plot 𝐯𝐬 (𝒙 + 𝟑)
𝒚 𝒚
𝐬𝐢𝐧 𝒚 = 𝒂(𝒙𝟐 − 𝟒) → plot 𝐬𝐢𝐧 𝒚 𝐯𝐬 (𝒙𝟐 − 𝟒)

𝟏 𝟏
𝒚= → plot 𝐯𝐬 𝒙
𝑪𝟏 𝒙 − 𝑪𝟐 𝒚
𝒚−𝟏 𝟐
𝟏Τ𝟐
𝒚 = 𝟏 + 𝒙 𝒎𝒙𝟐 + 𝒏 → plot 𝐯𝐬 𝒙𝟐
𝒙𝟐

96

48
 9/10/2024

2.7 Process Data Representation and Analysis

• Example 2.7-2
• A mass flow rate 𝑚ሶ (g/s) is measured as a function of temperature T (oC)
T 10 20 40 80
𝑚ሶ 14.76 20.14 27.73 38.47

• There is reason to believe that 𝑚ሶ varies linearly with the square root of T :
𝑚ሶ = 𝑎𝑇 0.5 + 𝑏
• Use a straight-line plot to verify this formula and determine 𝒂 and 𝒃

97

2.7 Process Data Representation and Analysis

• We find the values of T 0.5 from the given data

T 10 20 40 80
T 0.5 3.162 4.472 6.325 8.944
𝑚ሶ 14.76 20.14 27.73 38.47
40

30

• We now plot 𝑚ሶ vs T 0.5 which should be a straight line,

with a slope = 𝒂 and an intercept = 𝒃 20
𝑚ሶ

10

0
0 2 4 6 8 10

𝑇 0.5
98

49
 9/10/2024

2.7 Process Data Representation and Analysis

T 10 20 40 80
T 0.5 3.162 4.472 6.325 8.944
𝑚ሶ 14.76 20.14 27.73 38.47

• To find 𝑎 and 𝑏, we can use the first and last points

38.47 − 14.76 23.71
𝑎= = = 4.1
8.944 − 3.162 5.782

𝑏 = 𝑚ሶ − 𝑎𝑇 0.5 = 14.76 − 4.1 3.162 = 1.8

• Verification: At the last point, 𝑚ሶ = 4.1 800.5 + 1.8 = 38.47

99

2.7 Process Data Representation and Analysis

• TEST YOURSELF (page 26)

1. The following plot is generated from experimental (𝑥, 𝑦) data:

What equation would you use to relate 𝑥 and 𝑦?

100

50
 9/10/2024

2.7 Process Data Representation and Analysis

• TEST YOURSELF (page 26)

2. How would you plot (𝑥, 𝑦) data to get a straight line, and how would you
determine 𝑎 and 𝑏 for each of the following functions?
a) y = 𝑎 𝑥 + 𝑏
b) 1/y = 𝑎 𝑥 − 3 3 + 𝑏
c) y = 𝑎𝑥 2 − 𝑏 1/3
d) sin (y) = 𝑥 𝑎𝑥 + 𝑏 −2
e) y = 𝑎𝑒 𝑏𝑥
f) y = 𝑎𝑥 𝑏

101

2.7 Process Data Representation and Analysis

• Suppose you wish to fit an exponential function 𝑦 = 𝑎𝑒 𝑏𝑥 to measured

(𝑥, 𝑦) data.
• The determination of 𝑎 and 𝑏 is simplified by plotting ln 𝑦 as a function of 𝑥.
• There are two plotting procedures you might use:
1. Calculate ln 𝑦 for each value of 𝑦, and plot ln 𝑦 versus 𝑥 on rectangular
axes,
2. Plot 𝑦 directly on a logarithmic scale versus 𝑥 on a rectangular scale.

102

51
 9/10/2024

2.7 Process Data Representation and Analysis

• Logarithmic Coordinates:
Parallel axis

No need to
calculate ln(y)
Plot directly
the y values

103

2.7 Process Data Representation and Analysis

• Semi-log plot:
• For a function of the form: 𝑦 = 𝑎𝑒 𝑏𝑥
1200 10000

1000
1000
800

600 100

400
10
200

0 1
0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8

Rectangular Plot Semi-log Plot

104

52
 9/10/2024

2.7 Process Data Representation and Analysis

• Example

• A plot of 𝑦 versus 𝑥 yields a straight line on a semilog plot (the vertical axis is
a logarithmic scale while the horizontal axis is a rectangular scale). The line
passes through (8.68,0.01) and (2,1). Sketch the plot and calculate the
equation 𝑦(𝑥).

105

2.7 Process Data Representation and Analysis

10

0.1

0.01
2 4 6 8 10

106

53
 9/10/2024

2.7 Process Data Representation and Analysis

ln 𝑦 = ln 𝑎 + 𝑏𝑥

ln 1 − ln 0.01 4.608
𝑏= = = −0.69
2 − 8.68 −6.68

ln 𝑎 = ln 𝑦 − 𝑏𝑥 = ln 1 + 0.69 2 = 1.38
𝑎 = 𝑒 1.38 = 3.97

𝑦 = 3.97𝑒 −0.69𝑥

107

2.7 Process Data Representation and Analysis

• If one axis is logarithmic and the other is rectangular then it is called a

semi-log plot
• If both axis are logarithmic, then it is called a log-log or simply a log plot
• If 𝑦 versus 𝑥 data appear linear on a log plot, then ln 𝑦 versus ln 𝑥 would be
linear on a rectangular plot, and the data can therefore be correlated by a
power law:
𝑦 = 𝑎𝑥 𝑏

108

54
 9/10/2024

2.7 Process Data Representation and Analysis

• Log paper

109

2.7 Process Data Representation and Analysis

• TEST YOURSELF (page 29)

1. The following plots yield straight lines. What are the equations that relate
the variables?

a) 𝑃 versus 𝑡 on rectangular coordinates.

b) 𝑃 (logarithmic axis) versus 𝑡 on a semi-log paper.
c) 𝑃 versus 𝑡 on a log plot
d) 𝑦 2 − 3 (logarithmic axis) versus 1/𝑥 2 on a semi-log paper.
e) 1/𝐹 versus 𝑡 2 − 4 on a log paper.

110

55
 9/10/2024

2.7 Process Data Representation and Analysis

• TEST YOURSELF (page 29)

2. What would you plot against what on what kind of axes to get a straight
line for the following relationships (𝑎 and 𝑏 are constants)?

a) 𝑃 = 𝑎𝑒 𝑏𝑡
b) 𝑃 = 𝑎𝑡 𝑏
c) 𝑃2 = exp 𝑎𝑡 3 + 𝑏
d) 1/𝑃 = 𝑎 𝑡 − 4 −𝑏

111

2.7 Process Data Representation and Analysis

• Fitting a Line to Scattered Data

Easy to fit a straight line to

such data

most common technique = least squares method

More realistic data

Many lines can be drawn, so

which one to use?

112

56