Counting

|𝛀| = 𝑁,
|𝑨| = 𝑚
𝑃(𝑨) = 𝑚/𝑁

1
Counting

2
Counting
• While counting is in principle straightforward, it is frequently
challenging.
• The art of counting constitutes a large portion of the field of
combinatorics.
• In this section, we present the basic principle of counting and apply it to
a number of situations that are often encountered in probabilistic
models.
• Most common approach to the counting process is based on a divide-
and-conquer approach, whereby the counting is broken down into
stages through the use of a tree.
• This approach describes the possible outcomes of a multi-stage
experiment through a sequential process.

3
Counting

4
Counting

5
Counting
• In what follows, we will focus primarily on two types of counting
arguments that involve the selection of k objects out of a
collection of n objects.
• If the order of selection matters, the selection is called a
permutation,
• If the order of selection does not matter, it is called a
combination.
• We will then discuss a more general type of counting, involving a
partition of a collection of n objects into multiple subsets.

6
k-permutations
• We start with n distinct objects, and let k be some positive integer, with
k n.
• We wish to count the number of different ways that we can pick k out of
these n objects and arrange them in a sequence, i.e. , the number of
distinct k-object sequences.
• We can choose any of the n objects to be the first one.
• Having chosen the first, there are only n - 1 possible choices for the
second; given the choice of the first two, there only remain n - 2
available objects for the third stage, etc.
• When we are ready to select the last (the kth) object, we have already
chosen k - 1 objects, which leaves us with n - (k - 1) choices for the last
one.

7
k-permutations
• By the Counting Principle, the number of possible sequences, called k-
permutations is,

8
k-permutations

9
k-permutations

10
Combinations
• There are n people and we are interested in forming a committee of k.
• How many different committees are possible?
• More abstractly, this is the same as the problem of counting the number of k-
element subsets of a given n-element set.
• Notice that forming a combination is different than forming a k-permutation,
because in a combination there is no ordering of the selected elements.
• For example, while the 2-permutations of the letters A, B, C, and D are
AB, BA, AC, CA, AD, DA, BC, CB, BD, DB, CD, DC,
the combinations of two out of these four letters are
AB, AC, AD, BC, BD, CD.

11
Combinations
• In the preceding example, the combinations are obtained from the
permutations by grouping together «duplicates».
• For example, AB and BA are not viewed as distinct, and are both
associated with the combination AB.
• This reasoning can be generalized: each combination is associated with
"duplicate" k-permutations,
• So the number of k-permutations is equal to the number
of combinations times k!.
• Hence, the number of possible combinations, is equal to

12
Combinations vs Permutations

k
n

k!

. k! =

13
Combinations

14
Combinations

15
Partitions
• Recall that a combination is a choice of k elements out of an n-element
set without regard to order.
• Thus, a combination can be viewed as a partition of the set in two: one
part contains elements and the other contains the remaining .
• We now generalize by considering partitions into more than two subsets.
• We are given an n-element set and nonnegative integers 1 2 𝑟.
whose sum is equal to .
• We consider partitions of the set into disjoint subsets, with the th
subset containing exactly 𝑖 elements. Let us count in how many ways
this can be done.

16
Partitions

17
Partitions: Example 1

18
Partitions: Example 2

19
Partitions: Example 2

20
Partitions: Example 2

21
Summary

22
Random Variables
• Given an experiment and the coresponding set of possible
outcomes (sample space), a random variable is a function which
associates a particular number with each outcome.
• We refer to this number as the numerical value or simply the
value of the random variable.
• Mathematically, a random variable is a real-valued function of
the experimental outcome.

23
Discrete Random Variables

24
Random Variables

25
Random Variables

26
Discrete Random Variables
• A random variable is called discrete if its range (the set of values that it can take)
is either finite or countably infinite.
• For example, the random variables mentioned in (a) (5 tosses of a coin) and (b) (2
rolls of a die) above can take at most a finite number of numerical values, and
are therefore discrete.
• A random variable that can take an uncountably infinite number of values is not
discrete.
• For an example, consider the experiment of choosing a point a from the interval
[- 1. 1] .
• The random variable that associates the numerical value to the outcome is
not discrete.
• On the other hand, the random variable that associates with a the following
numerical value is discrete:

27
Discrete Random Variables

28
Probability Mass Functions

• The most important way to characterize a random variable is

through the probabilities of the values that it can take.
• For a discrete random variable , these are captured by the
probability mass function (PMF for short) of , denoted .
• In particular, if is any possible value of the probability mass of
denoted is the probability of the event
consisting of all outcomes that give rise to a value of equal to :

29
Probability Mass Functions

• For example. let the experiment consist of two independent tosses

of a fair coin, and let be the number of heads obtained. Then the
PMF of is

30
Probability Mass Functions

31
Probability Mass Functions: Some Notation

• We will often omit the braces from the event/set notation when no
ambiguity can arise. In particular, we will usually write in
place of the more correct notation ,
• We will write for the probability that takes a value
within a set .
• We will also adhere to the following convention throughout: we will
use upper case characters to denote random variables,
• … and lower case characters to denote real numbers such as the
numerical values of a random variable.

32
Probability Mass Functions: Notation

an event

33
The Bernaulli Random Variable
• Consider the toss of a coin, which comes up a head with probability ,
and a tail with probability .
• The Bernoulli random variable takes the two values 1 and 0
depending on whether the outcome is head or tail:

34
The Bernaulli Random Variable

35
The Binomial Random Variable

36
The Binomial Random Variable

37
The Geometric Random Variable

38
The Geometric Random Variable

39
The Geometric Random Variable

40
The Poisson Random Variable

41
The Poisson Random Variable

42
The Poisson Random Variable
• To get a feel for the Poisson random variable, think of a binomial random
variable with very small p and very large n.
For example, consider the number of typos in a book with a total of n words,
when the probability p that any one word is misspelled is very small
(associate a word with a coin toss which comes a head when the word is
misspelled),
or the number of cars involved in accidents in a city on a given day
(associate a car with a coin toss which comes a head when the car has an
accident).
Such a random variable can be well-modeled as a Poisson random variable.

43
The Poisson Random Variable

44
 Example: Binomial vs Poisson Approximation

Solution with Binomial Dist (Exact)

Solution with Poisson Approximation

Source: Helm (2008), Workbook 37.3 45

 Example 2: Binomial vs Poisson Approximation

Source: Helm (2008), Work Book 37.3

46
 Example 3

Source: Helm (2008), Workbook 37.3 47

Functions of Random Variables

48
Exercise

49
Exercise

50
Exercise

51
Homework

• Solve the previous exercise for .

Answer:

52