CAROL ASH »~ 4 REVISED PRINTING AN INTUITIVE COURSE FOR ENGINEERS AND SCIENTISTS (and everyone else!)The Probability Tutoring Book AN INTUITIVE COURSE FOR ENGINEERS AND SCIENTISTS. (AND EVERYONE ELSE!) Revised Printing Carol Ash University of Minois at Urbana-Champaign IEEE * PRESS The Institute of Electrical and Electronics Engineers, Inc., New YorkIEEE PRESS PO Box 1331, 445 Hoes Lane Piscataway, New Jersey 08855-1331 1992 PRESS Editorial Board William Perkins, Editor in Chief K.K Agarwal K. Hess A.C Schell R.S, Blicq J.D. Irwin L Shaw R.C. Dorf A. Michell M. Simaan D.M. Etter E.K. Miller Y. Sunahara J.J. Farrell III J.M.F Moura J. W. Woods J.G. Nagle Dudley R. Kay, Executive Editor Carrie Briggs, Administrative Assistant Denise Gannon, Production Supervisor Anne Reifsnyder, Associate Editor This book may be purchased at a discount from the publisher when ordered in bulk quantities. For more information contact: IEEE PRESS Marketing Attn: Special Sales PO Box 1331 445 Hoes Lane Piscataway, NJ 08855-1331 Fax: (732) 981-9334 ©1993 by the Institute of Electrical and Electronics Engineers, Inc. 345 East 47th Street, New York, NY 10017-2394 All rights reserved. No part of this book may be reproduced in any form, nor may it be stored in a retrieval system or transmitted in ‘any form, without written permission from the publisher. Printed in the United States of America 109876 ISBN 0-7803-1051-9 (pbk) IEEE Order Number: PP0288-1 [pbk] ISBN 0-87942.293.0 IEEE Order Number: PC0288-1 Library of Congress Cataloging-in-Publication Data Ash, Carol (date) ‘The probability tutoring book : an intuitive course for engineers and scientists (and everyone else!) / Carol Ash. pcm, “IEEE order number: PC0288-1"-Tp. verso. Includes index. ISBN 0-87942-293.9 1. Engineering mathematics. 2. Probabilities. I. Title. TASO.A7S 1993 620.001'S1—de20 92.53183, crContents Preface Introduction DISCRETE PROBABILITY CHAPTER 1 Basic Probability 1-1 12 13 14 15 Probability Spaces Counting Techniques Counting Techniques Continued ORs and AT LEASTS Continuous Uniform Distributions Review for the Next Chapter CHAPTER 2 _ Independent Trials and 2-Stage Experiments 2-1 22 2-3 24 2-5 Conditional Probability and Independent Events The Binomial and Multinomial Distributions Simulating an Experiment The Theorem of Total Probability and Bayes’ Theorem The Poisson Distribution Review Problems for Chapters 1 and 2 vii 37 37 a 56 63iv___Contents CHAPTER 3 — Expectation 3-1 Expected Value of a Random Variable 3-2. The Method of Indicators 3-3. Conditional Expectation and the Theorem of Total Expectation 3-4 Variance Review Problems for Chapter 3 Review for the Next Chapter CONTINUOUS PROBABILITY CHAPTER 4 = Continuous Random Variables 4-1 Density Functions 4.2. Distribution Functions 4-3 The Exponential Distribution 4-4 The Normal (Gaussian) Distribution 4-5 The Election Problem 4-6 Functions of a Random Variable 4-7. Simulating a Random Variable Review for the Next Chapter CHAPTERS _ Jointly Distributed Random Variables 5-1 Joint Densities 5-2. Marginal Densities 5-3 Functions of Several Random Variables Review Problems for Chapters 4 and 5 CHAPTER6 Jointly Distributed Random Variables Continued 6-1 Sums of Independent Random Variables 6-2 Order Statistics CHAPTER 7 Expectation Again 7-1 Expectation of a Random Variable 7-2. Variance 7-3 Correlation CHAPTER 8 Conditional Probability 8-1 Conditional Densities 8-2 2-Stage Experiments 74 SRIF 92 95 201 201 207 213 213 235 242 242 25083 Mixed 2-Stage Experiments Review Problems for Chapters 6, 7, CHAPTERS Limit Theorems 9-1 The Central Limit Theorem 9-2. The Weak Law of Large Numbers Review Problems for Chapters 4-9 SOLUTIONS INDEX and 8 Contents 257 263 266 266 269 274 275 466Preface This is the second in a series of tutoring books,! a text in probability, written for students in mathematics and applied areas such as engineering, physics, chemistry, economics, computer science, and statistics. The style is unlike that of the usual mathematics text, and I'd like to describe the approach and explain the rationale behind it. Mathematicians and consumers of mathematics (such as engineers) seem to disagree as to what mathematics actually is. To a mathematician, it’s im- portant to distinguish between rigor and informal thinking. To an engineer, intuitive thinking, geometric reasoning, and physical argument are all valid if they illuminate a problem, and a formal proof is often unnecessary or coun- terproductive. The typical mathematics text includes applications and examples, but its dominant feature is formalism. Theorems and definitions are stated precisely, and many results are proved at a level of rigor that is acceptable to a working mathematician. This is bad. After teaching many undergraduates, most quite competent and some, in fact, blindingly bright, it seems entirely clear to me that most are not ready for an abstract presentation. At best, they will have a classroom teacher who can translate the formalism into ordinary English (“what this really means is ...”). At worst, they will give up. Most will simply learn to read around the abstractions so that the textbook at least becomes useful as a source of examples. This text uses informal language and thinking whenever possible. This is the appropriate approach even for mathematics majors: Rigorous probability 'The first is The Calculus Tutoring Book by Carol Ash and Robert Ash (New York: IEEE Press, 1986). vilvill__ Preface isn’t even possible until you've had a graduate-level course in measure theory, and it isn’t meaningful until you’ve had this informal version first. In any textbook, problems are as important to the learning process as the text material itself. I chose the problems in this book carefully, and much consideration was given to the number of problems, so that if you do most of them you will get a good workout. To be of maximum benefit to students, the text includes detailed solutions (prepared by the author) to all problems. T’dlike to thank the staff at the IEEE PRESS, Dudley Kay, Executive Ed- itor; Denise Gannon, Production Supervisor; and Anne Reifsnyder, Associate Editor. I appreciate the time and energy spent by the reviewers, Dr. Robert E. Lover and members of the IEEE Press Board. Most of all I owe my husband, Robert B. Ash, for patiently and critically reading every word and for being better than anyone else at teaching mathe- matics in general, and probability in particular, to students and wives. Carol AshIntroduction Probability is a hard subject. Students find it hard, teachers find it hard, text- book writers find it hard. And you can’t put the blame on the “theory” because this text will not be theoretical. It emphasizes informal ideas and problem solv- ing, and the solutions will seem simple in retrospect, but you may find them hard to figure out on your own. This book will help you all it can, but in the end the only way to learn is to do many problems. On the plus side, probability is useful and intensely interesting. Many courses and some entire disciplines have a probability prerequisite, for ex- ample, stochastic processes, information theory, signal detection, signal pro- cessing, control systems, quantum mechanics, system theory, and of course statistics. There are two types of probability problems, discrete and continuous. Experiments with a “limited” number of outcomes are called discrete. There are only finitely many poker hands, so finding the probability of a royal flush in poker is a discrete problem. The discrete category includes experiments with an infinite number of outcomes as long as it is a “countably” infinite number. If the experiment is to toss a coin until a head turns up, the possible outcomes are outcome 1 never get a head outcome 2 Hon Ist toss outcome 3 = TH outcome 4 TTH outcomeS TITHx Introduction The list of outcomes is countably infinite, so finding the probability that it takes at most 10 tosses to get the first head is a discrete problem. Here’s an example of a continuous problem: John will be passing the corner of Main and First at some time between 1:00 and 2:00. Mary will be passing the same corner between 1:30 and 2:00. Each agrees to wait 5 minutes for the other. Find the probability that they meet. There is an infinite number of possible arrival times for John (all the numbers between 1 and 2), and it is an “uncountably” infinite number since there is no way to list all the times and label then 1st, 2nd, 3rd, .... Similarly for Mary. So the problem is continuous. Discrete probability has more charm, but if you are in engineering, con- tinuous probability will most likely be more useful for you. Discrete probabil- ity has no special prerequisite—high school algebra is enough. For continuous probability, you'll need integral calculus. Techniques of integration are not important—a computer can always do the hard integrals—but you will have to remember how to set up a double integral; a review of double integrals is included before they’re used in Chapter 5. The text begins with discrete probability in Chapters 1-3. The rest of the book, Chapters 4-9, covers continuous probability with occasional flashbacks to the discrete case. Discrete and continuous probability have certain basic ideas in common but in practice they will seem quite different. I hope you enjoy them both.CHAPTER 1 Basic Probability SECTION 1-1 PROBABILITY SPACES We want to answer the questions “What are probabilities?” and “How does an event get a probability?” Sample Space of an Experiment A sample space corresponding to an experiment is a set of outcomes such that exactly one of the outcomes occurs when the experiment is performed. The sample space is often called the universe, and the outcomes are called points in the sample space. There is more than one way to view an experiment, so an experiment can have more than one associated sample space. For example, suppose you draw one card from a deck. Here are some sample spaces. sample space I (the most popular) The space consists of 52 outcomes, 1 for each card in the deck. sample space 2 This space consists of just 2 outcomes, black and red. sample space 3 This space consists of 13 outcomes, namely, 2, 3, 4, 10,J,G, K, A. sample space 4 This space consists of 2 outcomes, picture and non- picture. Any outcome or collection of outcomes in a sample space is called an event, including the null (empty) set of outcomes and the set of all outcomes. In the first sample space, “black” is an event (consisting of 26 points). It is also an event in sample space 2 (consisting of 1 point). It is not an event in 12 Sec.1-1 Probability Spaces sample spaces 3 and 4, so these spaces are not useful if you are interested in the outcome black. Similarly, “king” is an event in sample spaces 1 and 3 but not in 2 and 4. Probability Spaces Consider a sample space with n points. Probabilities are numbers assigned to events satisfying the following rules. (1) Each outcome is assigned a non-negative probability such that the sum of the n probabilities is 1. This axiom corresponds to our intuitive understanding of probabilities in real life. The weather reporter never predicts a negative chance of snow, and the chance of snow plus the chance of rain plus the chance of dry should be 100%, that is, 1. (2) If Ais an event and P(A) denotes the probability of A, then P(A) = sum of the probabilities of the outcomes in the event A A sample space together with an assignment of probabilities to events is called a probability space. Note that probabilities are always between 0 and 1. Figure 1 shows a probability space with six outcomes a, b, c, d, e, f and their respective probabilities. The indicated event B contains the three out- comes d, e, f and P(B) =.14+.2+.3=6 Figure 1 Probabilities may be initially assigned to outcomes any way you like, as long as (1) is satisfied. Then probabilities of events are determined by (2). To make our probabilities useful, we try to assign initial probabilities to make a “good” model for the experiment.Chap.1 Basic Probability 3 Suppose you put slips of paper labeled a, b,c, d, e, f in a bag, shake it thoroughly, put on a blindfold, and draw one out; that is, you pick one of a, 8, ¢, d,e, f at random. The appropriate model should have the six outcomes equally likely, so instead of the probabilities in Fig. 1, you should assign each outcome the probability 1/6. Then number of outcomes in the event total number of outcomes i = This suggests an important special case. Suppose a sample space has n points and we make the special assignment (1’) P(each outcome) = + (Note that axiom (1) is satisfied as required.) Then the outcomes are equally likely. In this case it follows that number of outcomes in the event total number of outcomes __ favorable outcomes “total outcomes P(event) = @) Use (1') and (2!) if an experiment is “fair,” in particular if an outcome is picked at random. For the problems in this book, you may assume that dice and coins and decks of cards are fair unless specifically stated otherwise. If an experiment is not fair (e.g,, toss a biased coin), you will be given the initial probabilities. In real life you might select the initial probabilities by playing with the coin: If you toss it many times and 63% of the tosses are heads, then it is reasonable to make the initial assignment P(heads) = .63. How do you decide if your mathematical model (the probability space) is a good one? Suppose you assign initial probabilities so that P(event B) turns out to be .37. If many people each perform the experiment many times and “most” of the people find that B happens “close to” 37% of the time, then you have a good model. Example 1 (a fair deck) Draw one card from a deck. Consider sample space | (containing 52 points). In the absence of any special information to the contrary we always choose to assign probabilities using (1') and (2’). Then4 Sec.1-1 Probability Spaces P(ace of spades) = = favorable 4 _ 1 P(ace) = fvorable _ 4 _ 1 (ace) = Total 52 18 P(card) > 10) = ES In this text, the ace is considered to be a picture, along with jack, queen, king. Unless otherwise stated, the ace is high, so that ace > king > queen > jack > 10 Example 2 (a biased deck) Suppose that the 16 pictures in sample space 1 are assigned probability 1/32 and the 36 non-pictures are assigned prob 1/72. (Note that the sum of the probs is 1, as required.) This corresponds to a deck in which the pictures are more likely to be drawn (maybe the pictures are thicker than the non-pictures). Then P(ace of spades) = z P(ace) = sum of probs of the aces o 1 a a 4 = 32 + 307 got 32 — 32 P(card > 10) = probs of the 10’s + probs of the pictures 1 1 =4 +1635 P(card > 2) = sum ofall the probs = 1 P(red spade) = 0 since the event red spade contains no outcomes Impossible Events and Sure Events Probabilities are always between 0 and 1. Assure event is one that contains all points in the sample space (e.g., card > 2). The probability of a sure event is 1.Chap.1 Basic Probability 5 An impossible event is an event that contains no points in the sample space (e.g,, red spade). The probability of an impossible event is 0. The converses are not necessarily true: There are possible events that have probability 0 and non-sure events with probability 1. (See the footnote in Section 2.1 and see (8) in Section 4.1.) Complementary (Opposite) Events If Ais an event, then its complement A is the set of outcomes not in A. For example, if A is red card, then A is black card; if A is king, then A is non-king. It follows from (1) and (2) that in any probability space @) P(A) = 1— P(A) Example 3 (tossing two fair dice) Let’s find the probability of getting an 8 (as a sum) when you toss a pair of fair dice. The most useful sample space consists of the following 36 points; think of one die as red and the other as blue—each point indicates the face value of the red die and the face value of the blue die. 1,1 21 3,1 4,1 5,1 6,1 iva 210 912 Ara 5126.2 ay 1,3 2,3 3,3 4,3 5,3 6,3 1,4 2,4 3,4 4,4 5,4 6,4 1,5 2,5 3,5 4,5 5,5 6,5 1,6 2,6 3,6 4,6 5,6 6,6 There are five outcomes favorable to 8, namely, (2,6), (6,2), (5,3), (3,5), (4,4). So P@)= 5 Example 4 ‘Toss two dice. Find the probability that they show different values, for example, (4,6) and (2, 3) but not (2,2). You can count the favorable outcomes directly, or better still, by (3), (non-matching dice) = 1 — P(matching dice) =6 _Sec.1-2 Counting Techniques Problems for Section 1-1 ‘Toss two dice. Find the probability of each of the following events. 1, sumis7 2. Tori 3. second die > first die 4. at least one of the dice is a6 5. both dice are > 5 6. at least one die is > 5 7. neither die is over 4 8. both dice are even 9. at least one die is odd SECTION 1-2 COUNTING TECHNIQUES Ina probability space where the outcomes are equally likely, P(event) = number of favorable outcomes ~~ total number of outcomes In order to take advantage of this rule you must be able to find the total number of outcomes in an experiment and the number that are favorable to your event. For the dice problems in the last section, they were easy to find by inspection. But it usually isn’t that simple, so we'll first derive some counting procedures before continuing with probability. The Multiplication Principle Suppose you have 3 shirts (blue, red, green) and 2 pairs of pants (checked, striped). The problem is to count the total number of outfits. The tree diagram (Fig. 1) shows all possibilities: there are 2 x 3 = 6 outfits. Instead of drawing the tree, which takes a lot of space, think of filling a pants slot and a shirt slot (Fig. 2). The pants slot can be filled in 2 ways and the shirt slot in 3 ways, and the total number of outfits is the product 2 x 3. ‘You'll get the same answer if the tree is drawn with 3 shirt branches first, each followed by 2 pants branches. Equivalently, it doesn’t matter if you name the first slot pants and the second shirts as in Fig. 2, or vice versa. If an event takes place in successive stages (slots), decide in how (1) | many ways each slot can be filled, and then multiply to get the total number of outcomesChap. 1 Basic Probability 7 Pants Shins Outfits oe ee des < cnt checked esti ee ue sped pan, be shi < eee een siped puns, geen shit Figure 1 a0 roid a aun pants slot shi slot Figure 2 Number of outfits = 3 x 2 Example 1 The total number of 4etter words is 26 - 26 - 26 - 26 (Each spot in the word can be filled in 26 ways.) Example 2 The total number of 4-letter words that can be formed from 26 different scrab- ble chips is 26-25 24-23 (The first spot can be filled in 26 ways, the second in only 25 ways since you have only 25 chips left, etc.) 7x7 x 7versus7 x 6x5 The answer 7 x 7 x 7 is the number of ways of filling 3 slots from a pool of 7 objects where each object can be used over and over again; this is called sampling with replacement. The answer 7 x 6x5 is the number of ways of filling 3 slots from a pool of 7 objects where an object chosen for one slot cannot be used again for another slot; this is called sampling without replacement. Review Let’s review the product n! (n factorial), which turns up frequently in applications of the multiplication principle. By definition,8 Sec.1-2 Counting Techniques nl =n(n—1)(n—2)...1 Another definition sets o!=1 (This turns out to be a convenient choice for 0!, as you'll soon see.) For example, 61=6-5-4-3-2-1=720 10! HOE SS Se (eG se Ansan ga ro 7 7-6-5-4-3-2-1 5Ix6=6! Permutations (Lineups) Consider 5 objects A1,...,As. To count all possible lineups (permutations) such as Ay A5A4AgAz, AsAgA1.4Aq, and so on, think of filling 5 slots, one for each position in the line, and note that once an object has been picked, it can’t be picked again. The total number of lineups is 5-4-3-2- 1, or 5!. In general, n objects can be permuted in n! ways. Suppose you want to find the number of permutations of size 5 chosen from the 7 items Ay, ..., Az, for example, A1A44246 43, A1AgA7A2Ag. There are 5 places to fill, so the answer is eee (Some authors will write the answer in the more compact notation 7!/21.) Combinations (Committees) Now that we've counted permutations, let’s try counting committees. I'll illus- trate the idea by finding the number of committees of size 4 that can be chosen from the 17 objects Aj,...,A17. Note how committees differ from permutations: Ay, Ai7, Az, Aia is the same committee as Ay2, Ai, Aiz, Az, but AyAy7Ag Ata and Ay2Ai Ai7Ag are different permutations. Order doesn’t count in a committee; it does count ina permutation. It isn’t correct to fill four committee slots and get “answer” 17- 16-15-14 because a committee doesn’t have a first member, a second member, and soChap 1 Basic Probability 9 on—the order of the members doesn’t matter. The number 17 - 16 - 15 - 14 counts permutations, not committees. But you can get the committee answer by comparing the list of permutations with the list of committees. List of Committees List of Lineups (a) Py PPPs (a1) PPrPsPp (a2) PrP. PsPs There are 4! ; of these. (024) PoPrPiPs (b) Ps, Pa, Pic, Ps (bi) PaPaPi2Ps | (62) PsP2PsPs | There are 4! F of these. (bea) Pi2P3PaPo etc. Each committee gives rise to 4! lineups, so number of committees x 4! = number of lineups number of lineups _ 17-16 - 15-14 number of committees = Zi ti The symbol ('f) stands for the number of committees of size 4 from a population of size 17 (it’s pronounced 17 on 4 or 17 choose 4). It is also written as C(17,4). We've just shown that (ind The symbol ("), called a binomial coefficient, stands for the number of committees of size r that can be chosen from a population of size n of, equivalently, the number of combinations of n things taken rata time. Its value is given by > Oem Here’s the general result:10 Sec. 1-2 Counting Techniques For example, the number of 4-person committees that can be formed from a group of 10is 10\ _ Ale _10-9-8-7 ~4-3-261 =10-3-7=210 (Cancel as much as you can before doing any arithmetic.) ‘Some Properties of (?) ; O-() C3 This holds because picking a committee of size 4 automatically leaves a com- mittee of 13 leftovers and vice versa so the list of committees of size 4 and the list of committees of size 13 are the same length. Alternatively, (‘7) and (11) are equal because by the formula in (2) each is 17!/4! 13!. ° (Ga) This follows because there are clearly n committees of size 1 that can be chosen from a population of size n. It also follows from the formula in (2) since n\_(n\__ nt 1) > \n-1) “=i which cancels down to n. ° 0-0)"Chap.1 Basic Probability 11 We have (") = 1 because there is just one way to form a committee of size n from a population of size n. If you try to use the formula in (2), you get (:)- ata and it will be 1, provided you define 0! = 1. In other words, you can define 0! any way you like, but it is convenient to call it 1 because then the formula in (2) continues to hold even when r is n or 0. Once you have (”) = 1, it follows from (3) that () = 1 also. Ifyou want to interpret (5) as the number of committees with no members, look at it like this: There is 1 committee that can be formed with no members, namely, the null (empty) committee. Example 4 Pil find the probability of getting the queen of spades (denoted Qs) in a poker hand. A poker hand is a committee of 5 cards drawn from 52. The total number of poker hands is (°2). Finding favorable hands amounts to selecting a committee of size 4 (the rest of the hand) from 51 (the rest of the deck). So there are (°!) favorable hands and _&) _ 5! i) sl47!_ 5 (3) 47 52l 52 P(Qs) Example 5 Plfind the probability of nor getting the queen of spades in poker. Method 1 (directly) Again, the total number of poker hands is (52). Each favorable hand contains 5 cards chosen from the 51 non-Qs’s. So there are (3) favorable hands, and 51 P(not getting the queen of spades) = {3 5 Method 2 (indirectly) P(Qs) = 1— P(Qg) = 1— answer to example 4 = 1 — 5 The two answers agree: ! 5f47t pues te method 2 answer 4 method 1 answer = Brae Bat = 5212 Sec.1-2 Counting Techniques 9 x 8 x 7 versus (2) Both count the number of ways in which 3 things can be chosen from a pool of 9. But 9x 8x7 corresponds to choosing the 3 things to fill labeled slots (such as president, vice president, secretary), while (3) corresponds to the case where the 3 things are not given different labels or distinguished from one another in any way (such as 3 co-chairs). From a committee/lineup point of view, 9 x 8 x 7 counts lineups of 3 from a pool of 9, while (8) counts committees. Poker Hands versus Poker Lineups Consider the probability of getting a poker hand with all hearts. A poker hand is a committee of cards, and 3) (6) P(all hearts) = o 5 (For the numerator, pick 5 cards from the 13 hearts.) If we consider a poker lineup, that is, a lineup of 5 cards drawn from 52, then 13-12-11-10-9 P(all hearts) = 5515-49-48 The answers in (6) and (7) agree since (3) _ 13! 5L47! 13-12-11. 10-9 (2) SIs! 52! 52-51-50- 49-48 In (6) the underlying sample space is the set of unordered samples of size 5 chosen without replacement from a population of 52. In (7), the sample space is the set of ordered samples so that AgKyJp2s3c and KyAsJp2s3c are dif- ferent outcomes. The probability of all hearts is the same in both spaces so it’s OK to use poker lineups instead of poker hands. (But (6) seems more natural to most people.) Warning In some instances, such as the probability of all hearts in a poker hand, you can use ordered samples or unordered samples as long as you are consistent in the numerator and denominator. Jf you use order in one place, you must use it in the other place as well.Chap.1 Basic Probability 13 Problems for Section 1-2 1 ° 5. 9. Acommittee of size 5 is chosen from A1,..., Ag. Find the probability that (a) the committee contains Ag (b) the committee contains neither Az nor Ag Find the probability of a bridge hand (13 cards) with the AKQJ of spades and no other spades. If2 people are picked from 5 men, 6 women, 7 children, find the prob- ability that they are not both children. Find the probability of a poker hand with (a) no hearts (b) the ace of spades and the ace of diamonds (other aces allowed also) (c) the ace of spades and the ace of diamonds and no other aces Four women check their coats and the coats are later returned at ran- dom. Find the probability that (a) each woman gets her own coat (b) Mary gets her own coat Compute (a) 7/5! © ®/¢ (©) (345) ) (isa ©) @ @ (7%) © G38 . In a certain computer system you must identify yourself with a pass- word consisting of a single letter or a letter followed by as many as 6 symbols which may be letters or digits, for example, Z, ZZZZZZ6, RUNNER, JIMBO, R2D2. Assuming that any password is as likely to be chosen as any other, what is the probability that John and Mary choose the same password? Consider samples of size 3 chosen from Aj,...,A7. (a) Suppose the samples are drawn with replacement so that after an item is selected, it is replaced before the next draw (e.g., Ai can be drawn more than once) and order counts (e.g., Ar A Ai is different from A, A241). How many samples are there? (b) Suppose the samples are drawn without replacement and order counts. How many are there? (©) How many samplesare there if the sampling is without replacement and order doesn’t count? What is the fourth type of sampling? (Counting in this case is tricky and won't be needed in this course.) Find the probability of a royal flush in poker (a hand with AKQJ 10 all in the same suit). @14 Sec.1-3 Counting Techniques Continued 10. There are 7 churches in a town. Three visitors pick churches at random to attend. Find the probability that (a) they all choose the same church (b) they do not all choose the same church (©) they choose 3 different churches (d) at least 2 of them choose the same church 11. In the state lottery the winning ticket is decided by drawing 6 different numbers from 1 to 54. The order in which the numbers are drawn is irrelevant so that the draws 2, 54, 46, 37, 1, 6 and 54, 2, 37, 46, 1, 6 are the same. For $1 you get two tickets, each with your choice of 6 numbers on it. What is the probability that you win the lottery when you spend $1? SECTION 1-3 COUNTING TECHNIQUES CONTINUED Here are some examples that are more intricate but can still be done using the multiplication principle and permutation and combination rules from the last section. Example 1 A committee of size 7 is chosen from 6 men, 7 women, 8 children. Let’s find the probability that the committee contains (a) 2men, 4 women, 1 child (b) 2men (a) The total number of committees is (7). For the favorable outcomes, the 2 men can be picked in (8) ways, the 4 women in (7) ways, and the child in 8 ways. (Think of filling three slots: a 2-man subcommittee, a 4-woman subcommittee, and a L-child subcommittee.) So P(2M, 4W,1C) = fe 8 7 (b) For the favorable outcomes, the 2 men can be picked in (8) ways, the 5 others in ('5) ways. So P(2men) = @(3) @)Chap.1 Basic Probability 15 Example 2 Find the probability that a poker hand contains only one suit. To count the favorable hands, pick the suit in 4 ways. Then pick the 5 cards from that suit in (7) ways. So P(only one suit) = ie 5 Example 3 A box contains 40 white, 50 red, 60 black balls. Pick 20 without replacement. Find the prob of getting (@) 10 white, 4 red, 6 black (b) 10white (a) The total number of outcomes is (15). For the favorable, pick 10 white out of 40, pick 4 red out of 50, pick 6 black out of 60: (a) (3) e) (b) For the favorable, pick 10 white out of 40, pick 10 others from the 110 non-white: P(10W, 4R, 6B) Poow) = (Ge) (20) Indistinguishable versus Distinguishable Balls ‘You may not have realized it, but as we picked committees and used fav/total in example 3, we assumed that balls of the same color could be distinguished from ‘one another; for example, we assumed that balls were named Wy,..., Wao; Rj,..-,Rso; and By,...,Bgo. The probability of getting 10W, 4R, 6B is the same whether or not the balls have names painted on them, so our assumption is not only convenient but legal. If an experiment involves n white balls, to find any probabilities we assume the balls are named W; Example 4 Consider strings of digits and letters of length 7 without repetition. Find the probability that a string contains 2 digits, 4 consonants, and 1 vowel.16 Sec.1-3 Counting Techniques Continued Method 1 For the favorable outcomes, pick 2 digits, 4 consonants, and 1 vowel and then line them up. (2G) +57! Prob = 35735 -34-35-82- 31-30 Method 2 For the favorable outcomes, pick 2 positions in the string for the digits, 4 places for the consonants, leaving 1 for the vowel. Then fill the spots. G)G)-10-9-21-20-19-18-5 prob = "36-35-34: 33-82-3130 Double Counting T'll find the probability of a poker hand with two pairs. . If I regard a poker hand as a committee, then the total number of hands is (°2). " Now I need the number of favorable hands. ‘To get started, it often helps to write out some favorable outcomes—the steps involved in constructing an outcome may suggest the slots to be filled in the counting process. Here are some poker hands with two pairs: Hand 1: Qu, Qp, Js, Ju, An Hand 2: 4c, 44, Kp, Ke, 24 First, let’s analyze an incorrect method. Each outcome involves a first face (e.g., queen), two suits in that face (eg. heart and diamond), a second face (e.g., jack), two suits in that face (e.g., spade and heart), and a fifth card not of either face (to avoid a full house). So use these slots: Step 1. Pick a face value. Step 2. Pick 2 of the 4 cards in that face. Step 3. Pick another face value. Step 4, Pick 2 out of the 4 cards in that face. Step 5. Pick a fifth card from the 44 of neither face. se Filling these 5 slots we get “answer” (*) ce (3) _ (:) Ad This is wrong because it counts the following as different outcomes when they are really the same hand:Chap.1 Basic Probability 17 Outcome 1 Outcome 2 Pick queen, hearts and spades. Pick jack, hearts and clubs. Pick jack, hearts and clubs. Pick queen, hearts and spades. Pick ace of clubs. Pick ace of clubs. Step 1 implicitly fills a slot named first face value and step 3 fills a slot named second face value. But in a poker hand, the two faces for the two pairs can’t be distinguished as first and second so the slots are illegal. The “answer” in (+) counts every outcome twice. (Once you notice this, you can divide by 2 to get the right answer.) Here’s a correct version (from scratch): Pick a committee of 2 face values for the pairs. Pick 2 cards from each face value. Pick a fifth card from the 44 not of either face. The number of favorable hands is (1) (4) (4) - 44. And the probability of two pairs is G)QG) -44 3) In general, an “answer” that counts some outcomes more than once is re- ferred to as a double count (the preceding double count happens to count every outcome exactly twice). Double counts can be hard to resist. Warning A common mistake is to use, say, 7 - 6 instead of (3), that is, to fill slots implicitly named first thingo and second thingo when you should pick a committee of two things. Symmetries Here are some typical symmetries. Draw cards either with or without replacement. Place on Ist draw) = P(ace 2nd) = P(ace 3rd), etc. 1 ()) Peace 3rd, king 10th) = P(A Ist, K 2nd) = P(A 2nd, K Ist.) ete.18 Sec.1-3 Counting Techniques Continued Intuitively, each position in the deck of cards has the same chance of harboring an ace; each pair of positions has the same chance of containing an ace and king, and so on. Similarly, draw with or without replacement from a box containing red, white, and black balls. So P(RWBW drawn in that order) = P(WWRB) = P(RBWW), etc, Whatever the distribution of colors you draw, you're just as likely to get them in one order as another. It isn’t safe to rely on intuition, so Pll do one proof as a justification. I'll show that P(A on 3rd draw, K on Sth draw) = P(K on ist, Aon 2nd) This is immediate if the drawings are with replacement: Each probability is Suppose the drawing is without replacement. Then P(Kon Ist, A on 2nd) = $Y = a total 5: fave ee fav total ~ 52-51-50-49- 48 P(A on 3rd, K on Sth) = For the fav, there are 5 slots to fill. The 3rd slot can be filled in 4 ways, the Sth slot in 4 ways, and then the other 3 slots in 50 - 49 - 48 ways. So P(A on 3rd, K on Sth) = ——e = — The “other 3 slots” canceled out, leaving the same answer as P(K Ist, A 2nd), QED. Example 5 Draw without replacement from a box with 10 white and 5 black balls. To find the prob of W on the 1st and 4th draws (no information about 2nd and 3rd), take advantage of symmetry and switch to an easier problem: 10-9 P(W on Ist and 4th) = P(W on 1st and 2nd) = ===Chap.1 Basic Probability 19 Problems for Section 1-3 1. Find the prob that a poker hand contains (a) 3 diamonds and 2 hearts (b) 2 spades, one of which is the ace (©) 4 black and 1 red (a) 2aces (©) the ace of spades but not the king of spades 2. Three Americans Ay, Ag, Ag, 7 Russians Ry,...,Rr and 8 Germans Gi,...,Gs try to buy concert tickets. Only 5 tickets are left. If the tickets are given out at random, find the prob that (a) Rg gets a ticket and so do 2 of the 3 Americans (b) only 1 of the Germans gets a ticket 3. If3 people are picked from a group of 4 married couples, what is the prob of not including a pair of spouses? 4. Ifa 12-symbol string is formed from the 10 digits and 26 letters, repe- tition not allowed, what is the prob that it contains 3 even digits? 5. Find the prob of getting 3 whites and 2 reds if you draw 11 balls from a box containing 25 white, 30 red, 40 blue, and 50 black. 6. If four people are assigned seats at random in a 7-seat row, what is the prob that they are seated together? 7. Find the prob that a 3-card hand contains 3 of a kind (i.¢., 3 of the same value). 8. (a) Find the prob that a 4-card hand contains 2 pairs. (b) Find the prob that a 5-card hand contains a full house (3 of a kind and a pair). 9, Find the prob that a poker hand contains (a) a flush (5 cards of the same suit) (b) 4 aces (©) 4ofakind (eg., 4 jacks) (@) a pair (and nothing better than one pair) 10. Suppose b boys and g girls are lined up at random. Find the prob that there is a girl in the ith spot. Here are some counting problems with proposed answers that dou- ble count. Explain how they double count (produce specific outcomes which are counted as if they are distinct but are really the same) and then get correct answers. (a) To count the number of poker hands with 3 of a kind: Pick a face value and 3 cards from that value. Pick one of the remaining 48 cards not of that value (to avoid 4 of a kind). 11.20 Sec.1-4 _Ors and At Leasts Pick one of the remaining 44 not of the first or second value (to avoid 4 of a kind and a full house). Answer is 13 (§) 48-44. WRONG (b) To count 7-letter words with 3 A's: Pick a spot for the first A. Pick a spot for the second A. Pick a spot for the third A. Pick each of the remaining 4 places with any of the non-A’s. Answer is 7-6-5 - 254, WRONG (©) To count 2-card hands not containing a pair: Pick any first card. Pick a second card from the 48 not of the first face value. Answer is 52-48, WRONG SECTION 1-4 ORS AND AT LEASTS OR versus XOR The word “or” has two different meanings in English. If you've seen Boolean algebra you know that engineers have two different words, OR and XOR, to distinguish the two meanings: AOR B means A or B or both, called an inclusive or. Similarly, AOR B OR C means one or more of A, B, C (exactly one of A, B, C or any two of A, B,C or all three of A, B, C). On the other hand, AXOR B means A or B but not both, an exclusive or. In this book, “or” will always mean the inclusive OR unless specified otherwise. Inthe real world you'll have to decide for yourself which kind is intended. Ifa lottery announces that any number containing a 6 or a7 wins, then you win with a 6 or 7 or both; that is, 6 OR 7 wins (inclusive or). On the other hand, if you order a Coke or a 7-Up, you really mean a Coke or a 7-Up but not both; that is, Coke XOR 7-Up. OR Rule (Principle of Inclusion and Exclusion) For 2 events, oy P(A or B) = P(A) + P(B) — P(A and B)Chap.1 Basic Probability 21 For 3 events, P(Aor Bor C) = P(A) + P(B) + P(C) (2) —|P(A&B) + P(A&C) + P(B&C)} +P(AKB&C) Here’s the general pattern for n events: ® +3-at-a-time terms —4-at-a-time terms ete, For example, P(Aor Bor C or D) = P(A) + P(B) + P(C) + P(D) -[P(A& B) + P(A&C) + P(A& D) + P(B&C) + P(B& D) + P(C&D)) + [P(A& B&C) + P(A& B&D) + P(BECKD) + P(ALCED)} -P(A&B&CKD) Proof of (1) Suppose event A contains the 4 outcomes indicated in Fig. 1 with respective probs pi, ...,ps. And suppose B contains the indicated 3 outcomes. eve A event B Figure 1 The event “A or B” contains all 5 outcomes in Fig. 1, so P(Aor B) = pi t+ po +3 + pats On the other hand,22 Sec.1-4 Ors and AtLeasts P(A)+P(B)=pit+pe+pst+p, + pa+patPs This is nor the same as P(A or B) because it counts the probs ps and p4 twice. We do want to count them since this is an inclusive or, but we don’t want to count them twice. So to get P(A or B), start with P(A) + P(B) and then subtract the probs in the intersection of A and B, that is, subtract P(A and B) as in (1). Warning The “or” in rule (1) is inclusive; it means A or B or both. We subtract away P(A&B) not because we want to throw away the both’s but because we don’t want to count them twice. In other words, P(Aor B) = P(A or Bor both) = P(A) + P(B) — P(A&B) Proof of (2) Suppose A contains the 5 outcomes in Fig. 2 with indicated probs, B contains the indicated 6 outcomes, and C contains the indicated 4 outcomes. Then P(Aor BorC) =p +: +9 8 Figure 2 Look at P(A) + P(B) + P(C). It does include py, ... ,po, but it counts po, ps, Ps, pg twice each and counts p, three times. If we subtract away P(A&B), P(A&C), P(B&C) ae ass patpstpe | PAtPS pate then pa, ps, Ps, Ps Will be counted once each, not twice, but now p, isn’t counted at all. So formula (2) adds P(A&B&C) back in to include ps again.Chap.1 Basic Probability 23 Example 1 Pilfind the probability that a bridge hand (13 cards) contains 4 aces or 4 kings. By the OR rule, P(4aces or 4 kings) = P(4 aces) + P(4 kings) — P(4 aces and 4 kings) The total number of hands is ($3 When you count the number of ways of getting 4 aces, don’t think about kings at all (the hand may or may not include 4 kings—you don’t care). The other 9 cards can be picked from the 48 non-aces so there are G) ') hands with 4 aces. Similarly, for outcomes favorable to 4 kings, pick the other 9 cards from the 48 non-kings. And for the outcomes favorable to the event 4 kings and 4 aces, pick the other 5 cards from the 44 remaining cards. So 48) (48) (44) Ped aces or kings) = 3+ fh { -9 13 Unions and Intersections Many books use the union symbol instead of “or” and the intersection symbol instead of “and” so that the OR rule for two events looks like this: P(AUB) = P(A) + P(B) — P(ANB) I use “or” and “and” because they seem more natural. Most people would refer to 3 aces or 3 kings rather than to the union of 3-ace hands and 3-king hands. Mutually Exclusive (Disjoint) Events Suppose events A, B,C, D are mutually exclusive, meaning that no two can happen simultaneously (A, B, C, D have no outcomes in common). Then all the “and” terms in (1), (2), and (3) drop out, and we have a P(Aor B) = P(A) + P(B) 2’) P(A or B or C) = P(A) + P(B) + P(C) G') P(Aj or ... or An) = P(A1) + +++ + P(An) Example 2 Consider poker hands containing all spades or all hearts. The events “all spades” and “all hearts” are mutually exclusive since they can’t happen simul- taneously. So by (1),24 Sec. 1-4 _Ors and At Leasts P(all spades or all hearts) = P(all spades) + P(all hearts) _ Warning P(Aor B) is not P(A) + P(B) unless A and B are mutually exclusive. If they are not, don’t forget to subtract P(A&B). At Least One ‘Tpillustrate the general idea I'll find the probability that a poker hand contains at least one ace. Method 1 (the best for this particular example) 485 P(at least one ace) = 1— P(no aces) = 1 — 3 5 Method 2 P(at least one ace) = P(1A or 2A or 3A or 4A) The events one ace (meaning exactly one ace), two aces, three aces, four aces are mutually exclusive, so we can use the abbreviated OR rule. P(at least one ace) = P(1A) + P(2A) + P(3A) + P(4A) herr 7 @) Method 3 P(at least one ace) = P(Ag) + P(Au) + P(Ac) + P(Ap) — [P(As & Ax) + other 2-at-a-time terms] +[P(As & Au & Ac) + other 3-at-a-time terms] — P(As & Ay & Ac & Ap) This long expansion isn’t as bad as it looks. The first bracket contains 4 terms all having the same value, namely, (2)/@)- The second bracket contains (3) terms; all have the value (°?) /(°2).Chap.1___Basic Probability 25 The third bracket contains (3) terms, each with the value (4) /(*?). So @) In this example, method 1 was best, but you'll see examples favoring each of the other methods. P(at least one ace) = Warning Here is a wrong way to find the probability of at least one ace in a poker hand. The denominator is (°?) (right so far). For the numerator: © Pick 1 ace to be sure of getting at least one. Pick 4 more cards from the remaining 51. & so the numerator is 4(*!). The numerator is wrong because it counts the following as different outcomes when they are the same: Outcome 1 Outcome 2 Pick the Ag as the sure ace. Pick the Ay as the sure ace. Then pick Qu, Au, 2p, 6c. Then pick Qu, As, 2p, 6c- So the numerator double counts. Don’t try to count at least n thingos by presetting n thingos to be sure and then going on from there. It just won’t work. Example 3 Here’s how to find the prob that a bridge hand contains at most 2 spades. Method 1 P(at most 2 spades) = 1 — P(3S or 4S or ... or 138) = 1—[P(3S) + P(4S) + --- + P(138)] OK, but too slow!26 Sec. 1-4 __Ors and At Leasts Method 2 P(at most 2 spades) = P(no S) + P(1S) + P(28) = G) +13) + G)GP 7 Ga) Exactlies Combined with At Leasts I'll find the probability of a poker hand with 2 spades and at least 1 heart. Method 1 Use a variation of the rule P(at least 1 heart) = 1 — P(no hearts) to get P(2 spades and at least 1 heart) = P(2S) — P(2S and no H) _ MQ- OB (3) Method 2 P(2 spades and at least 1 heart) = P(2S and H) + P(2S and 2H) + P(2S and 3H) — (2)-13-(@) + (2) G) -26 + (2) (8) 7 ° At Least One of Each Form committees of 6 from a population of 10 Americans, 7 Russians, and 5 Germans. P(at least 1 of each European nationality) = P(at least one R and at least one G) 1— P(no R or no G) = 1 —[P(no R) + P(no G) — P(no R and no G)} 1 c+ 09) @)Chap.1 Basic Probability 27 Warning The complement of at least 1 of each European nationality is not no Europeans; that is, the complement is not noR and noG Rather, the complement is noR or noG Some Basic Pairs of Complementary Events Here is a brief list of some complementary events. (You should understand the logic of the list rather than force yourself to memorize it.) Event Complement AorB not A and not B Aand B not A or not B at least 1 king no kings at least 1 king and at least 1 queen _no kings or no queens at least 1 king or atleast 1 queen _no kings and no queens at least 1 of each suit no Sor no Horno CornoD Problems for Section 1-4 1. Find the prob that a poker hands contains (@) 2 aces or 2 kings (b) 3 aces or 3 kings (©) the ace or king of spades 2, A box contains 10 white balls, 20 reds, and 30 greens. Draw 5 without replacement. Find the prob that (a) the sample contains 3 white or 2 red or 5 green (b) all 5 are the same color 3. Consider computing P(Ai or Az or... or Ag). (a) Eventually, you have to subtract away the 2-at-a-time terms such as P(As & Ag). How many of these terms are there? (b) Eventually, you will have to add in the 3-at-a-time terms such as P(A; & Aq & As). How many are there?28 4, 5. ce 10. il. Problems for Section 1-4 A jury pool consists of 25 women and 17 men. Among the men, 2 are Hispanic, and among the women, 3 are Hispanic, If a jury of 12 people is picked at random what is the prob that it (a) contains no women or no Hispanics (b) contains no women and no Hispanics Find the prob that a poker hand contains the jack of spades XOR the queen of spades (the jack or queen but not both). Find the prob that a poker hand contains (a) at least 1 spade (b) at least 3 spades (©) at most 2 aces (a) 4 pictures including at least 2 aces Find the prob that a bridge hand contains (a) at least one royal flush (AKQJ 10 in the same suit) (b) at least one 4-of-a-kind (e.g., 4 kings) Ahundred people including the Smith family (John, Mary, Bill, Henry) buy a lottery ticket apiece. Three winning tickets will be drawn with- out replacement. Find the prob that the Smith family ends up happy. (Find three methods if you can, for practice.) ). Find the prob that a 4-person committee chosen from 6 men, 7women, and 5 children contains (a) 1 woman (b) at least 1 woman (c) at most 1 woman (d) at least 1 of each category (e) no women and at least 1 man There are 50 states and 2 senators from each state. Find the prob that acommittee of 15 senators contains (a) at least 1 from each of Hawaii, Massachusetts, and Pennsylvania (b) at least 1 from the three-state region composed of Hawaii, Mas- sachusetts, and Pennsylvania (c) 1 from Hawaii and at least 1 from Massachusetts (the game of rencontre—the matching game) Seven husbands Hh, ... H; and their wives Wi,...,Wz are matched up at random to form 7 new coed couples. (a) Find the prob that Hs is matched with his own wife. (b) Find the prob that H, Hs, and H are all matched with their own wives.Chap. 1 Basic Probability 29 (©) Find the prob that at least one husband is matched with his wife and simplify to get a pretty answer. Suggestion: The only feasible method is to useChap.1 Basic Probability 31 because there are an uncountably infinite number of each (that’s why the prob- lem is continuous rather than discrete), But in the same spirit we'll use P(wait 5 minutes or less) = #vOrable length _ 545 _ 1 total length 30°=«3 Figure 1 Suppose a point is chosen at random in an interval—the outcome of the experiment is said to be uniformly distributed on (or in) the interval. We choose as the mathematical model a sample space (universe) consisting of all points in the interval, where favorable length Pevent) = al Tength ‘Two-Dimensional Continuous Uniform Distributions John arrives at random between time 0 and time ¢;, and Mary arrives at ran- dom between time 0 and ta, where t; < ta. Find the probability that John arrives before Mary. Let x be John’s arrival time and y be Mary’s arrival time. Each outcome of the experiment is a pair of arrival times. The universe is the set of points in the rectangle in Fig. 2. The experiment amounts to picking a point at random from the rectangle. line x=y Figure The favorable points are those in the rectangle where « < y (see the review section if you don’t remember how to graph inequalities).92 Sec. 1-5 Continuous Uniform Distributions For the 2-dimensional continuous analog of fav/total, we'll use (John before Mary) = {2v0table area total area Perhaps the easiest way to compute the fav area in Fig. is indirectly, by first finding the unfavorable area: entire rectangle — unfavorable triangle P(John before Mary) = ee _ tits - #3 tte Suppose a point is chosen at random in a region in the plane—the outcome of the experiment is said to be uniformly distributed on (or in) the region. As a special case, if a number z is chosen at random from the interval [a, 6] and a second number y is chosen (independently from the « choice) at random from the interval c, d], then the point (x,y) is uniformly distributed in a 2-dimensional rectangle. ‘We choose as the mathematical model a sample space (uni- verse) consisting of all points in the region, where favorable area Pi it) = (event) total area Warning If one number is picked at random in an interval {a,6], then probabilities in- volving that number are found using fav length/total length in the 1-dimension- al universe (a,b). Suppose two numbers (e.g,, his arrival time and her arrival time) are picked at random in a, bj and [c, d], respectively. The probability of an event involving both numbers is found using fav area/total area in the 2-dimensional rectanglea <2 y34 ___ Problems for Section 1-5 axis y= Leos Figure 5 Figure 6 So the favorable region is the graph of y < L-cos6, the region under the graph of y = Los8. Figure 6 shows the favorable region inside the universe. Then favorable area total area {g/? Leos 6 0 (#/2)D LoL (7/2)D~ 7D P(needle hits a crack) = (As expected, the probability goes down if L gets smaller or if D gets larger.) Problems for Section 1-5 1. A number z is chosen at random between —1 and 1 so that < is uni- formly distributed in [—1, 1]. Find the prob thatChap.1 Basic Probability 35 (a) -}<2<0 © |z- | <1 b) (d) 32? >a 2. A point is chosen at random in a circle of radius 9. Find the prob that it’s within distance 2 of the center. 3. Consider the quadratic equation 4x” + 4Qz + Q +2 = 0, where Q is uniformly distributed on (0, 5]. Find the prob that the roots are real. 4. Suppose 0 is uniformly distributed on [—7/2, x/2]. Find the prob that sind > }. 5. Consider a circle with radius R. Here are two ways to choose a chord in the circle. In each case find the prob that the chord is longer than the side of an inscribed equilateral triangle (a) One end of the chord is point Q. The other end is determined by rotating the needle in the diagram by @ degrees where 0 is uni- formly distributed on (0, 180}. (b) Pick a point at random on the radius AB in the diagram (so that the indicated distance d is uniformly distributed on [0, R]). Draw a chord through the point perpendicular to AB. Ae needle Figure P5(a) Figure P5(b) 6. Choose a number at random between 0 and 1 and choose a second number at random between 1 and 3. Find the prob that (a) their sum is < 3 (b) their product is > 1 7, John and Mary agree to meet and each arrives at random between 10:00 and 11:00. Find the prob that (a) the first to arrive has to wait more than 10 minutes for the other (b) Mary arrives at least 20 minutes before John (©) they arrive at the same time36___ Review for the Next Chapter REVIEW FOR THE NEXT CHAPTER Series for e* 2 2 of atata += forall x l+a+ Series for e~* (replace z by —z in the e* series) l-2+ =e forall « Geometric Series l+zte +3 ¢aty-.- for -1l 5 10. (pokerdice) Toss 5 dice. Find the prob of getting a pair (and nothing better than a pair). 11. Ten pieces of candy are given out at random in a group of 5 boys, 7 girls, and 9 adults. Find the prob that 4 pieces of candy go to the girls if (a) people are allowed to get more than one piece (b) no one is allowed to get more than one piece 12. If 40% of marriages end in divorce and we assume that divorces are independent of one another, find the prob that of 8 couples (a) only the Smiths and Joneses will stay married (b) exactly 2 of the 8 couples will stay married 5 Ata particular intersection with a stop sign you observe that 1 out of every 20 cars fails to stop. Find the prob that among the next 100 cars at least 3 don’t stop. 14, Toss 6 balls at random into 10 boxes. Find the prob that (a) they split 4-2 (4 go into one box and 2 into a second box) (b) they split 3-3 (©) they all go into different boxes 15. A drawer contains 10 left gloves and 12 right gloves. If you pull out a handful of 4 gloves, what’s the prob of getting 2 pairs (2L and 2R)? 16. Fifteen percent of the population is left-handed. If you stop people on the street what's the prob that (a) it takes at least 20 tries to get a lefty (b) it takes exactly 20 tries to get a lefty (c) it takes exactly 20 tries to get 3 lefties (d) it takes at least 20 tries to get 3 lefties (the more compact your answer, the better) (e) the number of tries to get a lefty is a multiple of 5 Acoin has P(head) = p. Find the prob that it takes 10 tosses to get a head and a tail (i.c., at least one of each). 17. SECTION 2-3 SIMULATING AN EXPERIMENT A box contains 12 red balls and 8 black balls. Draw 10 times without replace- ment. Then54 Sec.2.3 Simulating an Experiment 12) (8) P(6 reds) = Co 35 (io) The physical interpretation of the mathematical model is that if you do this 10-draw experiment many times, it is likely (but not guaranteed) that the per- centage of times you'll get 6 reds will be close to (but not necessarily equal to) 35%. (Notice how much hedging there has to be in the last paragraph. Within a mathematical model, theorems can be stated precisely and proved to hold. But when you try to apply your model to the real world, you are stuck with imprecise words like many, very likely, and close to.) The random number generator in a computer can be used to simulate drawing balls from a box so that you can actually do the 10-draw experiment many times. The program that follows was done with Mathematica. The subprogram enclosed in the box is a single 10-draw experiment: It draws 10 balls without replacement from an urn containing 12 reds and 8 blacks. When it’s over, the counter named Red tells you the total number of reds in the sample. Here’s how this part works. Start the Red counter at 0. Draw the first ball by picking an integer z at random between 1 and 20. © The integers from 1 to 12 are the red balls; the integers from 13 to 20 are the blacks. © Step up Red by 1 if zis a red ball, that is, if z < 12 (in Mathematica this has to be written as z <= 12). ¢ On the second draw pick an integer z at random between 1 and 19. © If the first draw was red then the integers from 1 to 11 are the red balls, the integers from 12 to 19 are the blacks. e If the first draw was black then the integers from 1 to 12 are the red balls, the integers from 13 to 19 are the blacks. « Again, the counter Red is stepped up by 1 if zis red. e And so on through the 10 draws. The program as a whole repeats the 10-draw experiment n times. The counter named SixReds keeps track of how often you get 6 reds in the 10 draws. The final output divides SixReds by n to get the fraction of the time that this happens, that is, the relative frequency of 6 red results. Here’s what happened when I entered the program and ran it twice. In[1) Percent6RedwWO[n_] := (SixReds = 0;Chap.2 __ Independent Trials and 2-Stage Experiments 55 DoL (Red = 0; total = 20; Do (z = Random[Integer,{1,total}] ; If [z <= 12 - Red, Red = Red + 1]; total = total - 1, {10} ] 5 If[Red == 6, SixReds = SixReds + 1], {a} 13 SixReds/n//N (#Here is the output; The N makes it a decimal rather than a fraction*) > In(2] Percent6RedW0[600] (*Repeat the ten-draw experiment 500 times+) Out [2] 0.308 For that run, 31% of the time I got 6 reds in 10 draws. In[3] Percent6RedW0[1000] (+Repeat the ten-draw experiment 1000 times+) Out [3] 0.335 For the second run, 34% of the time I got 6 reds in 10 draws. If the drawing is done with replacement, then56 Sec.24 The Theorem of Total Probability and Bayes’ Theorem 12) 6 4 P(6 reds in 10 draws) = 9) (§) >) 2 25 (binomial distribution) Here’s the program adjusted so that it simulates n 10-draw experiments where the drawing is with replacement. Only the boxed subprogram is changed: now it just picks 10 integers at random between 1 and 20 where on each draw the integers 1 to 12 are red. In(4] Percent6RedWITH{n_] := (SixReds = 0; Dol (Red = 0; total = 20; Do [z = Random[(Integer ,{1,total}] ; If [z <= 12, Red = Red + 1); {10} 1 % If[Red == 6, SixReds = SixReds + 1], in} q3 SixReds/n//N ) Inf5] Percent6RedWITH(1000] Out (5) 0.265 SECTION 2-4 THE THEOREM OF TOTAL PROBABILITY AND BAYES’ THEOREM This section is about 2-stage (or multi-stage) experiments where the second stage depends on the first.Chap.2 _ Independent Trials and 2-Stage Experiments 57 The Theorem of Total Probability Here’s a typical 2-stage experiment. A box contains 2 green and 3 white balls. Draw 1. (yy If the ball is green, draw a card from a fair deck. If the ball is white, draw a card from a deck consisting of just the 16 pictures. ‘We'll find the probability of drawing a king. The tree diagram in Fig. 1 describes the situation. The labels on the first set of branches are P(green) = Z and P(wite) = 2 The labels on the second set of branches are fs 4 ‘i 48 P(king|green) = 5? P(non-king|green) = a + e 4 . . 12 P(king|white) = ie P(non-king|white) = ie Note that at each vertex of the tree, the sum of the probabilities is 1. Stage 1 Stage 2 4)52__— king “ green Us 48/52 —~non-king #2 4n16_— king ” aS white “ Tae Poking Figure1 Since one of green and white has to occur at the first stage, P(king) = P(green and king or white and king) = P(green and king) + P(white and king) (by the OR rule for mutually exclusive events) = P(green) P(king|green) + P(white) P(king|white) (by the AND rule) , ont Sl> ontos ale If we use the notation58 Sec.2-4 The Theorem of Total Probability and Bayes’ Theorem xin Bla branch #1 = product of the probabilities along the branch = then the answer can be written as P(king) = #1 + #3 = sum of favorable branches Here’s the general rule, called the theorem of total probability. If at the first stage the result is exactly one of A, B,C, then the probability of Z at the second stage is (2) P(Z) = P(A)P(Z|A) + P(B)P(Z|B) + P(C)P(ZIC) And here’s a restatement that makes it easy to use. (3) P(Z) = sum of favorable branches in Fig. 2 = #1 + #3 + #5 PEI zw Zz Pay Pelz ows rey z ao PZIO-z 45 << z Figure 2 The theorem of total probability as stated in (2) can be thought of in a more general context, without reference to a 2-stage experiment: If a prob- ability space can be divided into, say, 3 mutually exclusive exhaustive events A, B,C, then (2) holds for any event Z. Bayes’ Theorem Let’s use the same experiment again:Chap.2 Independent Trials and 2-Stage Experiments 59 A box contains 2 green and 3 white balls. Draw 1 ball. If the ball is green, draw a card from a fair deck. If the ball is white, draw a card from a deck consisting of just the 16 pic- tures, Suppose you draw a king on the second stage (Fig. 1 again). We'll go backward and find the probability that it was a green ball on the first stage. In other words, we'll find P(green on first stage|king on second) stage 1 Sage 2 sz king green <_ ws gsr nowking #2 se king 83 8 white << ine non-king #4 Figure 1 By the rule for conditional probability, . P(green and king) P(green|king) = —t8°e0 SNe Nine (green|king) Pking) The numerator is P(green) P(king|green) = branch #1 By the theorem of total probability, the denominator is #1 + #3. So #1 __ fav king branches #1+ #3 _ total king branches P(green|king) =60__Sec.2-4 The Theorem of Total Probability and Bayes’ Theorem The a posteriori (backward conditional) probability of A at the first stage, given Z on the second stage (Fig. 3), is P(Aand Z) P(Z) (4) _ Z-branches that are favorable to A total Z-branches = #1 - #14884 P(A|Z) = PIA Zz om A Zz Pay PIB) Zz #3 a < Zz PO PUI Zz as z i Example 1 Suppose }% of the population has a disease D. There is a test to detect the disease. A positive test result is supposed to mean that you have the disease, but the test is not perfect. For people with D, the test misses the diagnosis 2% of the time; that is, it reports a false negative. And for people without D, the test incorrectly tells 3% of them that they have D; that is, it reports a false positive. (a) Find the probability that a person picked at random will test positive. (b) Suppose your test comes back positive. What is the (conditional) proba- bility that you have D? (a) Figure 4 shows the tree diagram. By the theorem of total probability, P(positive) = #1 + #3 = (.005)(.98) + (.995)(.03)Chap.2 __ Independent Trials and 2-Stage Experiments 61 (b) By Bayes’ theorem, #1 total pos branches #1+ #3 (.005)(.98) (-005)(.98) + (.995)(.03) ~~ favorable pos branches P(Dpos) = SVorarle pos Dranches 14 HEALTH ‘Test SPs D << ns ” on Nes OB Pos 995 = D “ 7 Neg. Figure 4 So even though the test seems fairly accurate (with success rates of 98% and 97%), if you test positive, the probability is only .14 that you actually have the disease. (The probability came out low because so few people have the disease to begin with.) What we would really like to know in this situation is a first stage result: Do you have the disease? But we can’t get this information without an autopsy. The first stage is hidden. But the second stage (the result of the test) is not hidden. The best we can do is make a prediction about the first stage by looking at the second stage. This illustrates why backward conditionals are so useful. Problems for Section 2-4 1, The prob of color blindness is .02 for a man and .001 for a woman. Find the prob that a person picked at random is color blind if the population is 53% men. 2. Draw a card. If it’s a spade, put it back in the deck and draw a sec- ond card. If the first card isn’t a spade, draw a second card without replacing the first one. Find the prob that the second card is the ace of spades. A multiple-choice exam gives 5 choices per question. On 75% of the questions, you think you know the answer; on the other 25% of the questions, you just guess at random. Unfortunately when you think you know the answer, you are right only 80% of the time (you dummy). 3.4 Problems for Section 2-4 (a) Find the prob of getting an arbitrary question right. (b) Ifyou do get a question right, what’s the prob that it was a lucky guess? Box A has 10 whites and 20 reds, box B has 7 whites and 8 reds, and box C has 4 whites and 5 reds. Pick a box at random and draw one ball. If the ball is white, what’s the prob that it was from box B? . Toss a biased coin where P(H) = 2/3. If it comes up heads, toss it again 5 times. If it comes up tails, toss it again 6 times. Find the prob of getting at least 4 heads overall. Of 10 egg cartons, 9 contain 10 good eggs and 2 bad while a tenth carton contains 2 good and 10 bad. Pick a carton at random and pull out 2 eggs (without replacement). If both are bad, find the prob that you picked the tenth carton. A box of balls contains 3 whites and 2 blacks. round 1 Draw a ball. Don’t replace it. round 2 _If the ball is white, toss a fair coin. If the ball is black, toss a biased coin where P(H) = 8. round3 If heads, draw 2 balls from the (depleted) box. If tails, draw 1 ball. Find the prob of getting at least 1 white at round 3. . An insurance company unofficially believes that 30% of drivers are careless and the prob that a driver will have an accident in any one year is .4 for a careless driver and .2 for a careful driver. Find the prob that a driver will have an accident next year given that she has had an accident this year. .. Look at the accompanying tree diagram. What is each of the following the probability of? @) 2 @) 3 © (.2)(3)Chap.2 Independent Trials and 2-Stage Experiments 63 10. Tossa die 3 times. Find the probability that the result of the third toss is larger than each of the first two. Suggestion: Condition on the third toss and use the theorem of total probability. SECTION 2-5 THE POISSON DISTRIBUTION In Section 2.2 we found a formula, called the binomial distribution, for the probability of k successes in n Bernoulli trials. Now we'll look at a related formula called the Poisson distribution. Once you see how it’s connected to the binomial, you can apply it in appropriate situations. The Poisson Probability Function Let ) be fixed. Consider an experiment whose outcome can be 0, 1, 2, 3, ... . If dF — hr a) P(outcome is k) = © zi then we say that the outcome has a Poisson distribution with parameter A. The formula in (1) is the Poisson probability function. The Poisson prob function is legitimate because the sum of the probs is 1, as it should be: oo g-AQk k= The Poisson Approximation to the Binomial Consider n Bernoulli trials where P(success) = p. Let \ = np, interpreted as the average number of successes to be ex- pected in the n trials.64 Sec. 2-5 _The Poisson Distribution (To see the physical interpretation of A, consider tossing a coin 200 times where P(heads) = .03, so that n =200, p = .03. If many people toss 200 times each, one person might get no heads and another might get 199 heads, but it is very likely that the average number of heads per person is near .03 x 200 = 6.) Suppose n is large, p is small, their product is moderate, and k is much smaller than n. We’ll show that Ay P(k successes in n trials) = (7) = prt e = binomial diet So the Poisson can be used to approximate the binomial. The advantage of the Poisson, as you'll soon see, is that it has only the one parameter A, while the binomial distribution has two parameters, n and p. Here’s why the approximation holds. (")a = pyr MOLD BED gy _pyet Substitute and rearrange to get = —k+1) M* (1-A/n)P a) (yogrs teateseteng gas For large n, small p, moderate 4, and k much smaller than n, (- *) ~e™> (remember that lim (+3) =e’) a aba Ne ke (:-3) = (1=small)* v1 n(n —1)(n-1)---(n—k +1) _ n* + lower degree terms 2 nk nk So (2) becomes n\ Kapka Me eM (i) a-p) Mad EtChap.2 Independent Trials and 2-Stage Experiments 65 Application of the Poisson to the Number of Successes In Bernoulli Trials Your record as a typist shows that you make an average of 3 mistakes per page. Pll find the probability that you make 10 mistakes on page 437. Each symbol typed is an experiment where the outcome is either error or OK. The symbols on a page are typed independently, so they are Bernoulli trials. The number of mistakes on a page has a binomial distribution, where nis the number of symbols on a page and p is the probability of an error in a symbol. But we don’t know n or p, so we can’t use the binomial distribution. On the other hand, we do know that average number of mistakes per page = 3 so the next best choice is the Poisson approximation to the binomial, with A\=3: en3 310 10! Suppose you want the probability of fewer than 4 mistakes in the 10-page introduction. If the typist averages 3 mistakes per page, then on the average there are 30 mistakes in the introduction, so use the Poisson with \ = 30: P(10 mistakes on page 437) = P(fewer than 4 mistakes in Intro) = P(0 mistakes in Intro) + P(1) + P(2) + P(3) 2 398 =e" ( +9045 4 +) (Remember that 1° = 1 and 0! = 1.) Here’s the general rule: Suppose you have a bunch of Bernoulli trials. ‘You don’t know n, the number of trials in the bunch, or the probability p of success on any one trial (if you did, you could use the binomial distribution). But you do know that the average number of successes in a bunch is A. Then use the Poisson distribution to get Aye ki @) P(k successes in a bunch) = (provided that it’s a large bunch and successes are fairly rare).66 Sec. 2-5 _ The Poisson Distribution Example 1 The police ticket 5% of parked cars. (Assume cars are ticketed indepen- dently.) Find the probability of 1 ticket on a block with 7 parked cars. Each car is a Bernoulli trial with P(ticket) = .05, so P(1 ticket on block) = P(1 ticket in 7 trials) = (/) (.95)°(.05) Example 2 On the average, the police give out 2 tickets per block. Find the probability that a block gets 1 ticket. The cars are Bernoulli trials. We don’t know the number of cars on a block or P(ticket), but we can use the Poisson with \ = 2: P(1 ticket on block) = 2e~? Example 2 continued Find the probability that a 4-block strip gets a least 5 tickets. On the average, a 4-block strip gets 8 tickets, so use the Poisson with A = 8: P(at least 5 tickets on a 4-block strip) = 1 - P(0) — P(1) — P(2) — P(3) — P(4) Be~8_—B3e~8_— Be 8 =1-6-8 —ge-8— = - alae" Be 2 3t 4 Warning When you use (3) to find the probability of k successes in a bunch, you must use as \ the average number of successes in the bunch. In example 2, for parking tickets ina block, use A = 2, but for parking tickets in a 4-block strip, use \ = 8. Application of the Poisson to the Number of Arrivals in a Time Period As a telephone call arrives at a switchboard, the arrival time is noted and the switchboard is immediately ready to receive another call. Let \ be the average number of calls in an hour, the rate at which calls arrive. Assume that calls arrive independently. We'll show why it’s a good idea to use the Poisson as the model, that is, to use Ak 4) P(kcalls in an hour) ktChap.2 Independent Trials and 2-Stage Experiments 67 Divide the hour into a large number, n, of small time subintervals, so small that we can pretend that, at most, 1 call can arrive in a time subinterval. In other words, during each subinterval, either no call arrives or 1 call arrives (but it isn’t possible for 2 calls to arrive). With this pretense, the n time subintervals are Bernoulli trials where success means that a call has arrived. We don’t know nand p, but we do have A, so it makes good sense to use the Poisson in (4). Similarly, the Poisson distribution is the model for particles emitted, earth- quakes occurring, and arrivals in general: If arrivals are independent, the number of arrivals in a time period has a Poisson distribution: () P(k arrivals in a time period) = ~~ where the parameter ) is the arrival rate, the average number of arrivals in the time period. Example 3 Suppose particles arrive on the average twice a second, Find the probability of at most 3 particles in the next 5 seconds. The average number of particles in a 5-second period is 10, so use the Poisson distribution with = 10: P(at most 3 particles in 5 seconds) = P(none) + P(1) + P(2) + P(3) 10? | 10% = e710 =e (+104 a Warning The 2 in example 3 must be 10, the average number of particles arriving per 5 seconds, not the original 2, which is the average number per second. When you use (5) to find the probability of & arrivals in a time period, A must be the average number of arrivals in that time period. Problems for Section 2-5 1. On the average, a blood bank has 2 units of the rare type of blood, XYZ. (a) Find the prob that a bank can supply at least 3 units of XYZ. (6) Ifa community has two blood banks, find the prob that the com- munity can supply at least 6 units of XYZ.68__Review Problems for Chapters 1 and 2 2. 6. 1 La On the average there are 10 no-shows per airplane flight. If there are 5 flights scheduled, find the prob of (a) no no-shows (b) 4 no-shows (c) at most 4 no-shows . Assume drivers are independent. (a) If 5% of drivers fail to stop at the stop siga, find the prob that at least 2 of the next 100 drivers fail to stop. (b) Ifon the average 3 drivers fail to stop at the stop sign during each rush hour, find the prob that at least 2 fail to stop during tonight’s tush hour. On the average you get 2 speeding tickets a year. (a) Find the prob of getting 3 tickets this year. (b) Suppose you get 2 tickets in January. Find the prob of getting no tickets during the rest of the year (the other 11 months). Phone messages come to your desk at the rate of 2 per hour. Find the prob that if you take a 15-minute break you will miss (a) no calls (b) no more than 1 call On the average, in a year your town suffers through A; earthquakes, do lightning strikes, and 4g meteorites crashing to earth. Find the prob that there will be at least one of these natural disasters next year. On the average, you get 3 telephone calls a day. Find the prob that in 5 years there will be at least one day without a call. (This takes two steps. First, find the prob of no calls in a day.) . If P(H) =.01, then the prob of 1 H in 1000 tosses is (°°) (.99)9(.01). ‘What's the Poisson approximation to this answer? . The binomial distribution, and to a lesser extent the Poisson distribu- tion, involves Bernoulli trials. Do you remember what a Bernoulli trial is? Review Problems for Chapters 1 and 2 |. Draw 10 balls from a box containing 20 white, 30 black, 40 red, and 50 green. Find each prob twice, once if the drawing is with replacement and again if it is without replacement. (a) P(3W and 4R) (b) P(3W followed by 4R followed by 3 others) (©) P(AR followed by 3W followed by 3 others) Find the prob that among the first 9 digits from a random digit gener- ator, there are at least four 2’s. Draw from a deck without replacement. Find the prob that (a) the 10th draw is a king and the 11th is a non-kingoo 5. 10. i. 12. 14. 15. Chap.2 independent Trials and 2-Stage Experiments 69 (b) the first king occurs on the 10th draw (c) it takes 10 draws to get 3 kings (d) it takes at least 10 draws to get 3 kings Find the prob that a bridge hand contains at least one card lower than 6 given that it contains at least one card over 9. Form 12-symbol strings from the 26 letters and 10 digi that a string contains 3 vowels if repetition is (a) allowed (b) not allowed If the letters in ILLINOIS are rearranged at random, find the prob that the permutation begins or ends with L. (Just as you can assume ina probability problem that white balls are named Wi, ..., Wa, you can assume here that the word is I:LiL2I2NOIS, with all the letters distinguishable.) Find the prob that John and Mary are next to one another if eight people are seated at random (a) onabench (b) around a circular table Find the prob that a 3-letter word contains z (e.g., zzz, zab, bzc). Find the prob . You notice that 1 out of every 10 cars parked in a tow-away zone is actually towed away. Suppose you park in a tow-away zone every day for a year. Find the prob that you are towed at most once. At a banquet, m men and w women are introduced in random order to the audience. Find the prob that the last two introduced are men. Given j married couples, k single men, and n single women, pick a man and a woman at random. Find the prob that (a) both are married (b) only one is married (c) they are married to each other Find the prob of not getting any 3’s when you toss a die (a) 10 times (b) 100,000 times (©) forever . Your drawer contains 5 black, 6 blue, and 7 white socks. Pull out 2 at random. Find the prob that they match. If you get 6 heads and 4 tails in 10 tosses, find the prob that one of the heads was on the 8th toss. ‘Twenty-six ice cream flavors, A to Z, are available. Six orders are placed at random. Find the prob that the orders include (a) Aand B once each (b) A and Bat least once each (c) at least one of A and B (i.e., at least one A or at least one B) (d) two A’s and at least two B’s70___ Review Problems for Chapters 1 and 2 16. 17. 18. 19. 21. 22. (e) all different flavors ( all the same flavor Consider the probability of getting a void in bridge, a hand with at least one suit missing. (a) What's wrong with the following answer? The total number of hands is (°3). For the favorable hands: Pick a suit to be missing. Can be done in 4 ways. Pick 13 cards from the other 3 suits. Can be done in (32) ways. 39) 13 52) 13 Answer is (b) Find the right answer. A box contains 6 black (named By, ..., Bs), 5 white, and 7 red balls. Draw balls, Find the prob that (a) Bs is drawn before Bs (b) By is drawn before any of the whites One IRS office has three people to answer questions. Mr. X answers incorrectly 2% of the time, Ms. Y 3% of the time, and Ms. Z 4% of the time. Of all questions directed to this office, 60% are handled by X, 30% by Y, and 10% by Z. ‘What percentage of incorrect answers given by the office is due to Z? Toss a coin 10 times, Find the prob of getting no more than 5 heads given that there are at least 3 heads. Call the throw of a pair of dice lucky if the sum is 7 or 11. ‘Two players each toss a pair of dice (independently of one another) until each makes a lucky throw. Find the prob that they take the same number of throws. A basketball player has made 85% of her foul shots so far in her career. Find the prob that she will make at least 85% of her next 10 foul shots. Prizes are given out at random in a group of people. It’s possible for a person to get more than one prize. (a) If there are 10 prizes and 5 people, find the prob that no one ends up empty-handed. (b) If there are 5 prizes and 10 people, find the prob that no one gets two (or more) prizes. (c) If there are 5 prizes and 6 men and 4 women, find the prob that all the prizes go to men.Chap.2 independent Trials and 2-Stage Experiments 71 23. x 25. 26. 27. 8 29. 30. 31. ‘Toss a penny and a nickel 20 times each. For each coin, P(H) = .7. If the overall result is 17H and 23T, find the prob that 11 of those 17 heads were from the penny. Four hundred leaflets are dropped at random over 50 square blocks. (a) Find the prob that your block gets at least 3. (b) Find the Poisson approximation to the answer in (a). Three players toss coins simultaneously. For each player, P(H) = p, P(T) = gq. If the result is 2H and 1T or the result is 2T and 1H, then the player that is different from the other two is called the odd man out and the game is over. If the result is 3H or 3T, then the players toss again until they get an odd man out. Find the prob that the game lasts at least 6 rounds. Let P(A) = .5, P(B) = .2, P(C) = 1. Find P(A or Bor C) if (a) A,B,C are mutually exclusive (b) A, B,C are independent On the average, there is a power failure once every four months. (a) Find the prob of a power failure during exam week. (b) Find the prob that it will be at least a month before the next fail- ure. A bus makes 12 stops and no one stop is more popular than another. If 5 passengers travel on the bus independently, what's the prob that 3 get off at one stop and 2 get off at another (so that you have a full house of stops). nm Find (@) (1) ®) (°) (HP) @ Ca) ” ‘Teams A and B meet in the world series (the first to win 4 games is the series winner). Assume the teams are evenly matched and the games are independent events. Find the prob that the series ends in 6 games. Five shots are fired at random into a circle of radius R. The diagram shows an inscribed square and four other zones. Find the prob that the 5 shots end up (a) all in the same zone (b) in five different zones m Figure P3172___ Review Problems for Chapters 1 and 2 32. 33. 34, 35. 36. 37. 38. 39. Mary Smith has a 50-50 chance of carrying an XYZ gene. If she is a carrier, then any child has a 50-50 chance of inheriting the gene. Find the prob that her 4th child will not have the gene given that her first three children don’t have it. (the famous birthday problem) Find the prob that in a group of n peo- ple, at least two will have the same birthday. You have 5 dice and 3 chances with each die to get a 6. For example, if you toss the third die and get a 6, then you move on to the fourth die. But if the third die is non-6, you get to try again and then again if necessary. Find the prob of getting two 6’s overall with the 5 dice. Suggestion: First find the prob of getting a 6 from a single die in your 3 chances. A message is sent across a channel to a receiver. 6 that the message is zrrxz. 4 that the message is yyyyy. For each letter transmitted, the probability of error (that an z will be- come y, or vice versa) is .1. Find the probability that the message was zazzz if 2 z's and 3 y’s are received. John’s score is a number chosen at random between 0 and 3. Mary’s score is chosen at random between 0 and 1. The two scores are chosen independently. Find the prob that (a) his score is at least twice hers (b)_ the max of the two scores is < 1/2 (c) the min of the two scores is > 1/2 Five people are picked (without replacement) from a group of 20. Find the prob that John was chosen using these different methods. (a) fav committees/total committees (b) P(John was chosen 1st or 2nd or ... or chosen Sth) (©) 1= P(no John) Find the probability that, in a group of 30 people, at least 3 were born on the fourth of July. A class consists of 10 freshmen, 20 sophomores, 30 juniors, and 20 grads. (a) If grades A, B, C, D, E are assigned at random, find the prob that 4 freshmen get A’s. (b) If 6 class offices (president, vice president, etc.) are assigned at random, find the prob that 4 freshmen get offices.Chap. 2 Independent Trials and 2-Stage Experiments 73 40. (Computer Networks, Tannenbaun, Prentice Hall, 1989) A disadvan- tage of a broadcast subnet is the capacity wasted due to multiple hosts attempting to access the channel at the same time. Suppose a time period is divided into a certain number of discrete slots. During each time slot, the probability is p that a host will want to use the channel. If two or more hosts want to use a time slot, then a collision occurs, and the slot is wasted. If there are n hosts, what fraction of the slots is wasted due to collisions? 41. John will walk past a street corner some time between 10:00 and 11:00. Mary will pass the same street corner some time between 10:00 and 11:30. Find the prob that they meet at the corner if (a) each agrees to wait 10 minutes for the other (b) John will wait 10 (lovesick) minutes for Mary (but not vice versa) 42, Draw 20 times from the integers 1, 2, 3, ..., 100. Find the prob that your draws come out in increasing order (each draw is larger than the previous draw), if the drawing is (a) with replacement (b) without replacement ‘You can do it directly with fav/total (but most people get stuck on the fav).CHAPTER 3 Expectation SECTION 3-1 EXPECTED VALUE OF A RANDOM VARIABLE ‘You already know a lot about finding averages. If your grades for the year are 85% in a 3-point course 90% in a 4-point course 70% in a 5-point course then their (weighted) average is W (85 x 3) + (90% 4) + (70 x 5) In (1), each course grade in the numerator is weighted by its corresponding point value, and the denominator is the sum of the weights. Ina similar fashion, when the result of an experiment is a number, we'll find the average result. Example 1 A couple decides to have 3 children. But if none of the 3 is a girl, they'll try again, and if they still don’t get a girl, they'll try once more. On the average, how many children do they end up having? They can have 3 or 4 or 5 children, We'll average the three outcomes, a weighted average where the weights are the respective probabilities of the out- 4Chap.3 Expectation 75 comes (an outcome with greater likelihood counts more in the average). The sum of the weights, that is, the sum of the three probabilities, is 1, so we can leave out the denominator: average number of children (called the expected number of children) = 3x P(3 children) +4 x P(4 children) + 5 x P(5 children) = 3P(at least 1G in 3 tries) + 4P(BBBG) + 5P(BBBB) = 3[1 — P(BBB)] + 4P(BBBG) + 5P(BBBB) -+[-()] 4G) #0) 3 Ble We interpret this physically as meaning that if many couples have children according to this scheme, it is likely that the average family size would be near 33 children. Random Variables A numerical result of an experiment is called a random variable and is usually denoted with a capital letter like X,Y, Z. In example 1 the number of children the couple has is a random variable. Other random variables associated with the experiment are the number of boys, number of girls, number of older brothers, and so on. ‘As another example, consider Bernoulli trials where P(success) = p. Let X be the number of successes in n trials. Let Y be the number of trials it takes to get the first success. The random variable X has a binomial distribution, and Y has a geometric distribution. In particular, 2 P(X =k)= (t)ten* for k = 0,1,2,3,...,0 3) P(Y=k)=q'"p fork = 1,2,3,... The formulas in (2) and (3) are called the probability functions for X and Y. Expected Value of a Random Variable If X is the number of children in example 1, then X takes on values 3, 4, 5 and76 _Sec.3-1 __ Expected Value of a Random Variable expected value of X = 3P(X = 3) +4P(X = 4) +5P(X =5) In general, The expectation or expected value or mean of a random variable X is a weighted average of the values of X, where each value z is weighted by the probability of its occurrence. If the expectation is denoted by E(X) or EX, then E(X) =o 2P(X = 2) Note that if 0 is one of the values of X, the term 0 x P(X = 0) can be left out of the sum since it’s always 0 anyway. Expectation of a Poisson Random Variable Let X have a Poisson distribution with parameter A, so that ek kt P(X =k)= for k = 0,1,2,3,... We anticipate that E(X) =) since we use the Poisson as the model for the number of arrivals in a time period where A is physically the average number of arrivals per time period. Here’s the proof that the mathematical expectation agrees with the phys- ical interpretation: co co gAyk E(X) = Dewrx =kh)=>k 7 k=0 fee er 2X2 Ae Ke UtatatChap.3 Expectation 77 Problems for Section 3-1 1. A box of 5 items is known to contain 3 good and 2 defective. If you test the items successively (meaning you draw without replacement), find the expected number of tests needed to identify the D’s. Note that if you draw GGG, you are finished, since the remaining 2 items must be D’s. If you draw GGD, then it will take one more draw to locate both D’s. And it is never necessary to draw all 5 items. 2. An arrow is fired at random into a circle with radius 8. If it lands within 1 inch of the center, you win $10. If it lands between 1 and 3 inches from the center, you win $5. If it lands between 3 and 5 inches from the center, you win $2. Otherwise, you lose $4. Find your expected winnings. 3. A couple decides to have children until they get a girl, but they agree to stop with a maximum of 3 children even if they haven’t gotten a girl yet. Find the expected number of (a) children (b) girls (©) boys 4, In roulette, a wheel stops with equal probability at any of the 38 num- bers 0, 00, 1, 2, 3, ..., 36. If you bet $1 on a number, then you win $36 (net gain is $35) if the number comes up; otherwise, you lose your dollar. Find your expected winnings. 5. Let X be a random variable taking on values 0, 1, 2, 3, .... Show that EX = SPU >i) i= Suggestion: Write out the right-hand side in detail. SECTION 3-2 THE METHOD OF INDICATORS Some expectations are hard to find directly but are easy using the clever method of this section. Expectation of a Sum Before we get started we'll need this rule for E(X + Y): @ E(X +Y) = E(X) + E(Y) In other words, to find the expectation of a sum of random variables, just find all the separate expectations and add. (If the average number of girls in a78 Sec. 3-2 __ The Method of indicators family is 1.2 and the average number of boys is 1.3, then the average number of children is 2.5.) Here’s a justification. If a value of the random variable X is denoted by « and a value of ¥ is denoted by y, then the value of X + ¥ is @+y and E(X+Y)=> (ety) P(X =2,Y =y) zy =) P(X =2,¥ =y) + DyP(X =2,Y =y) v zy =D |e P(x =2,Y =) +d ly P(x =2,Y =y) z| 9 v| @ | P(X =2) PYY =y) =) eP(X=2)+) y PV =y) = v = E(X)+E(¥) The Method of Indicators and the Mean of the Binomial Distribution Let X have a binomial distribution with parameters n and p. You can think of X as the number of successes in n Bernoulli trials where P(success) = p. We want to find EX, the expected number of successes in n trials. If we try it directly, using the definition of expectation, we have EX= RPK =k)= S¥(7}eAc —p)r-* =O k=0 = ()por-2 n) 2un-24 n\n = (Jartea(eeetenee(t This answer is correct, but it would take a lot of effort and combinatorial identities to simplify. So, instead, we'll try another method that works more smoothly in this and many other problems. To find E(X), define n new random variables X; as follows. For i = 1,2,3,...,7, let 1 if the ith trial is a success X= a 0 otherwiseChap.3 Expectation 79 The X;’s are called indicator random variables or indicators. There’s an indi- cator for each trial, and each indicator signals (with a 1) if the trial is a success. For example, if n = 5 and the trials are SSFFS then X=3, X=1, X2=1, X3=0, X4=0, Xs=1 The total number of successes is the sum of all the signals, so X=Xytet Xn By (1), EX = EX, +-+-+EXn Now we need each of the E.X;’s, which is simple because X; takes on only two values. And, in this case, all the Z.X;’s are the same: EX, =0x P(X; =0) +1 x P(X;=1) P(X; =1) = P(success on ith trial) =P Q) So EX =EX,+---+EXy = sum of n terms each of which is p =np If X has a binomial distribution with parameters n and p, then [ B(X) = np This is a very intuitive result (which we spotted back in Section 2-5). It says that if P(heads) = 1/5, then the expected number of heads in 20 throws is} x 20=4, Here’s a summary of the method of indicators.90 Sec. 3-2 The Method of indicators Suppose X can be written as X=X+Xte. where the X;’s are indicators (random variables that take on just the values 0 and 1). Then EX = EX; + EX2 ++ Now alll you have to do is find the expectation of each indicator and add. An indicator expectation is usually easy to find (which is why this is a useful method). As in (2), it boils down to EX; = P(X; =1) = P(the indicator found what it was assigned to look for) Unfortunately, there is no rule for deciding whether or not to try indicators. And there is no rule for what to select as the indicators. Experience helps. Expected Number of Reds in a Sample Suppose 40% of the balls in a box are red. Draw n balls. Let X be the number of reds in the sample. Let's find BX. If the drawings are with replacement, then they are Bernoulli trials where P(success) = P(red) = .4. By (3), EX =np=nx 4 Expect 40% of the sample to be red. ‘We'll show that the expectation is the same if the drawings are without replacement. Let Xx, 1 if the ith ball is red + 10 otherwise X=X+...+ Xn and EX = EX,+...+ EX, Now we need the indicator expectations:Chap.3 Expectation _81 EX; (Xi = 1) = P(ith ball is red) (Ast ball is red) (by symmetry) 4 EX = sum of n terms each of which is 4 =n x .4= 40% xn If, say, 40% of the balls in a box are red, then the mathematical expectation is that 40% of a sample will be red. This holds for sampling with or without replacement. Mean of a Geometric Random Variable Til continue to illustrate the method of indicators and at the same time find another famous expectation. Consider Bernoulli trials where P(success) = p. Let X be the number of tries to get a success. Let’s find EX. Let x, = [1 if the first é trials are F **\0 otherwise Overall we're interested in the length of the F streak (if any) before the first S. For each trial there is an indicator assigned to see if the initial F streak is still alive. For example, if the outcome is FFFFS then X =5, X1 =1, Xp=1, X3=1, Xq=1, Xs =0, Xe = 0, X7 =0, For this outcome, of the infinitely many indicators, only X, ... , X4 kick in. The sum of the indicators counts the initial streak of F’s. Since X counts the number of wasted F’s plus the first S, we have X=14X14X4+-Xg4-- and E(X) = E(1) + E(X1) + E(X2) + E(X3) +++ By E(1) we mean the expectation of a random variable that always takes the value 1, so E(1) =182___Sec. 3-2 _The Method of Indicators Now we need the expectations of the indicators: EX; = P(X; = 1) = P(first trials are F) So (geometric series) Let P(success) = p. (4) | The expected number of Bernoulli trials to get the first success is Up. Again, this is intuitively reasonable: If P(head) = 1/5, then on the average it takes 5 tosses to get a head. If P(head) = 2/5, then on the average it takes 2 tosses to get a head. Example 1 (mean of the negative binomial) Suppose P(heads) = p. We'll find the expected number of tosses needed to get 3 heads. Let X be the number of tosses to get 3 heads. X = number of tosses needed to get the first H number of tosses needed after the first H to get the second H X3 = number of tosses needed after the second H to get the third H If the outcome is TTHHTTTH then In general, X =X, + Xot X3 The random variables X1, Xz, X3 are not indicators (they do not take on only the values 0,1) but we still have EX = BX; + BX2+EXsChap.3 Expectation 83 The tosses are independent, so waiting for the second H after the first H and waiting for the third H after the second H are like waiting for the first H in the sense that Xz and X3 have the same distribution as Xj. All three have a geometric distribution with parameter p. So each has mean 1/p and 1,1_3 ex=14141 Pp Ppp Pp For example, if P(heads) = 1/5, then on the average it takes 15 tosses to get 3 heads. More generally, if P(success) = p, then the expected number of Bernoulli trials to get k successes is k/p. (The result in (4) is the special case where k = 1.) Example 2 A box contains 10 white balls and 14 black balls. If balls are drawn with replacement, then the drawings are Bernoulli trials where P(W) = 10/24, so the expected number of trials to get a white is 24/10 (mean of the geometric distribution). Find the expected number of trials needed if the drawings are without replace- ment. Let X be the number of draws needed to get a white. Name the black balls By,..., Bra. Fori = 1,...,14, let X= 1 if B, is drawn before any white ~ \0 otherwise This assigns an indicator to each black ball to see if it turns up before any white. If the outcome is B,B, By Bg W then X=5, Xy=1, Xi=l, X7=1, Xg=1, other X;’sareO In general, X=1tX+ Xt + Xs and EX =14+EX,+EXo+-+:-+EXu = 14+ P(B; before any W) + P(B2 before any W) + ++++ P(Bu4 before any W)84 __ Problems for Section 3-2 The prob that B; is drawn before any Wis the same as the prob of getting B; in one draw from a new box containing just the 10 W’s and Bj, namely, 1/11. So 25 1 EX=1+4-D=7 (less than the expected value when the drawing is with replacement). Problems for Section 3-2 1. A box contains w whites and b blacks. Draw n without replacement. Let X be the number of whites in the sample. (a) Find EX immediately by quoting a known result. (b) For practice, find EX again using the following indicators: Let the white balls be named W1, Wo, ..., Ww. Define indicators to watch each white ball: for i= 1,2,...,w, let X; = 1if W; is in the sample 2. (the expected number of matches in the game of rencontre) A per- mutation of a1,...,a, has a fixed point if a; appears in the ith place (its “natural” place) in the permutation. For example, the permuta- tion aza,aga4agaz has two fixed points, a3 and ay. If a permutation of 4, ... ,@n is chosen at random, what is the expected number of fixed points? ‘Ten married couples are seated (a) in a circle and (b) in a line. Find the expected number of wives sitting next to their husbands. 4. A consecutive string of heads is called a run of heads. For example, if the outcome of coin tossing is 3. HH | TT HHHH/|T|H/T|H then there are 4 runs of heads (note that it’s possible for a run to con- tain only one head). If P(heads) = p, find the expected number of tuns of heads in n tosses. 5. Pick numbers at random between 0 and 1. The ith number chosen sets a record if it is larger than all the preceding choices. For example, if the numbers are .5, .2, .6, .34,.7, then three records are set, by the first number (which is always considered to set a record), and by .6 and .7. (The probability is 0 that a number is chosen twice, so ignore the possibility of tying a record.) Find the expected number of records in n trials. What happens to the expected number of records as n — 00?Chap.3 Expectation 85 6. Inside each box of Crunchies cereal the manufacturer has placed a picture of one of four Olympic athletes A, B,C, D. The pictures are distributed in equal numbers, so you are just as likely to get one as another. (Drawing cereal boxes without replacement from a large stock of boxes can be thought of as drawing with replacement. In other words, buying cereal is like drawing with replacement from an urn containing A’s, B’s, C’s, D’s in equal amounts.) (a) Ifyou buy n boxes, find the expected number of different pictures you get. (b) Find the expected number of boxes you have to buy to get a com- plete set of pictures. Suggestion: Define random variables X1, X2, X3, X4 so that X3, for instance, counts the number of draws to get a third letter after you already have two letters. SECTION 3-3 CONDITIONAL EXPECTATION AND THE THEOREM OF TOTAL EXPECTATION Definition of Conditional Expectation Let X be a random variable. Then E(Xlevent A) = > 2 P(X = 2A) The conditional expectation is really just an ordinary expectation, but within the new universe where event A has happened. Example 1 Draw one card from a deck. Let X be its face value (an ace has value 11, and every other picture has value 10). To find the (plain) expectation of X, average its 10 possible values: EX =2P(X = 2)+3P(X =3) +--+ 11P(X = 11) = 2P(deuce) + +--+ 10P(10, J, Q, or K) + 11P(ace) 1 1 4 1 95. at4..-49-4410-441.4-% 1) tig 43 2 gig To find the conditional expectation of X given that the card is a picture, aver- age the two possible values:86 __Sec.3-3 Conditional Expectation and the Theorem of Total Expectation B(X|picture) = 10P(X = 10|picture) + 11P(X = 11|picture) 3 1 =lOgte G7 The Theorem of Total Expectation Box I has 20 reds, 10 whites, and box II has 10 reds, 10 whites. Toss a coin where P(heads) = .4. If H, draw 3 balls from box I. IfT, draw 5 balls from box II. Let X be the number of reds in the sample. I'll find E(X). Remember that if 3 of the balls in an urn are red, then, on the average, $ of any sample is red (whether the drawing is with or without replacement). So E(number of reds in a sample of 3 from um I) = a “322 and g (number of reds in a sample of 5 from urn IT) = s 5s 2 In other words, E(X|H) = 2and &(X|T) = 5/2 (Fig. 1). Ho Eola 302 ; 3 If 40% of the time you expect 2 reds and 60% of the time you expect 5/2 reds, then, overall, BX =4x24.6x2=23 Here’s the general idea, called the theorem of total expectation.Chap.3 Expectation _ 87 Suppose a probability space can be divided into three mutually ex- clusive exhaustive events A, B, C (Fig. 2). Let X be a random variable defined on the space. Then E(X) = P(A) E(X|A) + P(B) E(X|B) + P(C) B(X|C) A exphereis £01 A) oo) . « ® et FO gp exphere ix £18) PO) Nc exphereis EOCIC) Figure 2 In other words, to get the overall expected value of X, find its expected value in the three new universes and then average those results using as weights the probabilities of the universes. Here’s the proof. EX=SoxP(X=2) (definition of EX) = Yo2[P(A)P(X = 2|A) + P(B)P(X = 2B) +P(C)P(X = 2|C)] (theorem of total prob) = P(A) DeP(x = xa +P(B) [Eerx = aa +P(C) [= 2P(X = 0 (algebra) = P(A) E(X|A) + P(B) E(X|B) + P(C) E(X|C) Warning Here are some results from earlier sections: Expected number of successes in n Bernoulli trials is np. Expected number of Bernoulli trials to get the first success is 1/p. Expected number of reds in n draws from a box is n x percentage of reds in box. You'll make extra work for yourself if you don’t take advantage of them.88 Sec.3-3_ Conditional Expectation and the Theorem of Total Expectation Example 1 In craps, a player makes a bet and then rolls a pair of dice. If the resulting sum is 7 or 11, the player wins. If the result is 2, 3, or 12, the player loses. If another sum, r, comes up, it’s called the player’s point. The player keeps tossing, and if r comes up before 7, the player wins; if 7 turns up before r, the player loses. Let X be the number of rolls in a game. I'll find E(X) using the theorem of total expectation, conditioning on the first roll (Fig. 3). BATNIZ E0237 = oer 4 Euxla) -148 25 s E19) <1 as 6 Bxt\o)=1 48 8 eusya19 8-2 9 EUX|9) 21438 0 0 EXI10=14%825 Figure 3 If the first roll is 2, 3, 7, 11, or 12, then the game is over with that 1 roll, so E(X|2,3,7, 11,12) =1 (first branch in Fig. 3) Suppose the first roll is 4. Now that the point is 4, the game will continue until 4 or 7 comes up. The continuation consists of Bernoulli trials where (success) = P(game-ending throw) = P(4 or 7) = 2 Remember that if P(S) = p, then the number of trials to get the first S is a geometric random variable with expectation 1/p. So the expected number of more rolls, after the initial roll comes up 4, is 36/9 and E(X|4) = the one initial roll + expected number of rolls in the continuation ore * (second branch in Fig. 3)Chap.3 Expectation 89 Similarly, if the first roll is 5, then the game continues until 5 or 7 comes up. The continuation consists of Bernoulli trials with P(success) = P(game-ending throw) = P(5 or 7) = 3 The expected number of more rounds after the roll of 5 is 36/10, and E(X|5) = the one initial roll + expected number of rolls in the continuation =1+ = (third branch in Fig. 3) The other conditional expectations in Fig. 3 are found similarly. Finally, pana Sed BS M8 AWD, oar 36 10 36 11 36 11 36 10 36 Example 2 A machine is erratic. When you push the button, one of three things happens. 60% of the time it goes into mode A and is done in 5 minutes. 30% of the time it goes into mode B, buzzes for 2 minutes, but produces nothing, so that you have to try again. 10% of the time it goes into mode C, buzzes uselessly for 3 minutes, after which time you must try again. Find the expected amount of time it takes to get done. Let X be the time it takes to get done. Let E(X) be denoted by E. We'll find E using the theorem of total expectation, conditioning on the first try (Fig. 4). If the machine goes into mode A on the first try, then X = 5, so we have E(X|A) = If the machine goes into mode B on the first try, then after 2 minutes we are ready to try again. The new situation is exactly the same as it was initially, so after the 2-minute waste of time, the new expected time to get done is E again. So E(X|B) = 2+. Similarly, E(X|C) = 3+ B. So B = (6)(6) + (.8)(2+ B) + (1)(8+B) Solve to get90 ___ Problems for Section 3-3. 39,1 B= 5 =65 A EMKIA)=S expisE B E(X|B)=2+E CXC) Ra +E Figure 4 Problems for Section 3-3 1, Roll a pair of dice. Find the expected value of the first die if the sum is4. 2. A student working on a problem has three available methods. Method A takes 5 hours but doesn’t solve the problem. Method B takes 2 hours and also leads nowhere. Method C leads to a solution after 4 hours of work. If the student has no reason to favor one untried method over another (but naturally a method that fails will not be tried again), what is the expected length of time it takes to solve the problem? 3. Toss a die 10 times. If you get six 1’s, find the expected number of 2’s. 4. Spin the indicated arrow. If it lands in quadrant k, toss k dice. Find the expected number of 5’s on the dice. Figure P4 5. A box contains 10 white and 20 black balls. Draw 8 balls without re- placement. Find the expected number of whites in the first 3 draws given that you got 1 white in the last 5 draws. 6. Toss a coin n times where P(H)= p. Find the expected number of H given that you got at least 2H.Chap.3 Expectation 91 7. Acoin has P(head)= p. Keep tossing until you finish a run of 5 heads (ie., until you get 5 heads in a row). Find the expected number of tosses it takes. Suggestion: Use the theorem of total expectation, conditioning on when the first tail occurs. SECTION 3-4 VARIANCE If John’s grades are 51 and 49 and Mary’s grades are 100 and 0, then each has a 50 average. But they seem like very different students (John is a dolt, while Mary is erratic). An average grade doesn’t tell the whole story. We should also measure the spread around the average, called the variance. Here’s an example to show how it’s done. Toss a 3-sided biased die where PQ)=.5, P(2)= Let X be the face value. Then E(X) = 1(.5) + 2(.1) + 3(.4) = 1.9 To find the variance of X, denoted by Var X, look at the differences between the possible values of X and the mean of X: 1-1.9= 9 (happens with probability .5) 2-1 (happens with probability .1) 3-1.9=1.1 (happens with probability .4) » P)= Square the differences (so that a positive over-the-mean doesn’t cancel a negative under-the-mean) and take a weighted average: Var X = (—.9)? (.5) + (1)? (.1) + (1.1)? (4) = .89 Here’s the general definition of the variance of a random variable: Let 1x denote the mean of X. Then Var X = E(X — x)? = E(X — EX)? If Var X is large, then it is likely that X will be far from the mean. If Var X is small, it is likely that X will be close to its mean. The standard deviation of X, denoted by a, is defined as /VarX. Variances are not usually computed directly as we just did for the die; there is a slicker method. All the details and theory and problems about vari- ance will be postponed until the chapter on continuous expectation. Then we'll save time by doing discrete and continuous variance simultaneously.Review Problems for Chapter 3 6. 9 Review Problems for Chapter 3 |. If 100 balls are tossed at random into 50 boxes, find the expected num- ber of empty boxes. ‘Toss a coin 10 times. Find the expected number of heads in the first 5 tosses, given 6 heads in the 10 tosses. In the game of chuck-a-luck, 3 dice are rolled. You can bet on any of the six faces 1, 2, ..., 6. Say you bet on 5. If exactly one of the 3 dice is a 5, then you win $1. If two of the 3 dice are 5’s, then you win $2. If all three of the dice are 5’s, you win $3. If none of the dice are 5’s, you lose $1. Find your expected winnings. Ifz1,..., 2, are permuted, find the expected number of items between z, and x2. . Toss a coin 10 times. Find the expected number of heads given that there are at least 9 heads. Ifyou have time, try it twice, with and with- out indicators. At the first stage of an experiment, a number X is selected so that X has a Poisson distribution with parameter \. Then if X = n, toss a coin n times where P(heads) = p. (If X = 0, don’t toss at all.) Find the expected number of heads. . You have n keys on your key ring. One of them unlocks the front door, but you forgot which one, so you keep trying keys until the door unlocks. Find the expected number of trials needed if (a) you sensibly try the keys without replacement (for practice, try it twice, with and without indicators) For reference: _n(nt1) 142434---4n a (b) you foolishly try the keys with replacement ‘Ten people each toss a coin 5 times. In each case, P(heads) = p. A person is shut out if she gets no heads. Find the expected number of shutouts. (a) Toss 100 balls at random into 50 boxes. Find the expected number of balls in the third box. (b) Toss balls at random into 50 boxes. Find the expected number of tosses required to get a ball into the third box.Chap.3 Expectation 93 10. A hospital handles 20 births a day. Ten percent require a special fetal monitor. Find the expected number of days out of the year when the hospital will need at least two monitors. The probability that a well is polluted is .1. Suppose 100 wells are tested as follows. Divide the wells into 5 groups of 20. For each group, water samples from each of the 20 wells are pooled and one test is performed. If a test is negative, then you know that all 20 wells are unpolluted. If a test is positive, then each of the 20 members is tested individually to see which ones are polluted. Find the expected number of tests needed with this method (as op- posed to the 100 tests it would take to forget about pooling and just test each one). 12. Find the expected number of suits in a poker hand. 13. Toss a die. 11 If the face is even, you win $1. If the face is 1 or 3, you lose $2. If the face is 5, you go to prison. When in prison keep tossing. If the next toss is even, you lose $1. If the next toss is odd, you get out of prison and begin the game again. (a) Find the expected winnings. (b) Find the expected number of tosses in a game (until you either win or lose money). 14, Noah’s ark contains a pair of Ay’s, a pair of Ay’s, ..., a pair of An’s. Draw m times from the ark (without replacement). Find the expected number of pairs left. 15. An urn contains 3 red and 7 black balls. Draw one ball. Then toss a coin 4 times, but use a fair coin if the ball was red and use a biased coin with P(heads) = 1/3 if the ball was black. Find the expected number of heads. 16. For a couple planning to have children, P(boy) = p, (git! (a) Ifthey keep trying until they have a girl, what is the expected num- ber of children? (b) Suppose they plan to have a minimum of 6 children. If the first 6 are all boys, they will continue until they get a girl. What is 1-p=q94 __ Review Problems for Chapter 3 the expected number of children? (This can be done nicely with indicators or with the theorem of total expectation.) 17. Acontest offers you (and 10 million others) the chance to win a million- dollar prize. No entry fee is required except for the price of a 25-cent stamp. Find your expected winnings to see if the contest is worth en- tering.Chap.3 Expectation 95 REVIEW FOR THE NEXT CHAPTER Some Integrals for Reference 1 Im! [ee (1—2)" de = —"im lo [retae=er( [serdone( i eo? dr=aVt lo 20 f ede =} = fora>0 2Va [ietas=va 0 [iewtae= 7 fora > 0 20 a [et de=nt for n = 0,1,2,3, oo nt f ae de= fora>0; n=0,1,2,3,... A ant Differentiating and Integrating Functions That Change Formula Suppose fa)={% Then re={ 3. and ff fede [odes [2 ae for2<2<3 for3 a nats =f) Figure 1 A typical piece has length dz and contains point x. The density within the interval varies, but if the interval is very small, its density is almost constant with value f(z) (its value at the particular point z in the interval). Then we can use (1) to find the mass (denoted dmass) of the little piece: dmass = length x almost constant mass density = dz x f(x) = f(x) dx The total mass of the interval is found by adding the dmass’s and making dz — 0 (so that the “almost” goes away). That's exactly what an integral does. So (2) total mass = [amass = [ see In other words, integrate the mass density to get the total mass. Now we’re ready to go back to probability. Continuous Random Variables Figure 2 shows a discrete random variable X which takes on values —7, 1,7,5 with P(X=-1)=4, P(X=1)=.1, P(X=n)=3, P(X =5)=2 We say that X has the probability function p(x), where 2-7) p(1) (4) p(n) = 3 (5) p(z)=0 otherwise a] 5 @) A Figure 2 Probs for a discrete random variable In general, for a discrete random variable taking on a few values, one unit of probability is split into chunks, which are assigned to the possible values.Chap.4 Continuous Random Variables 99 If a random variable can take on a continuum of values, then instead of placing chunks of probability at a few points, we mash up one unit of proba- bility and spread it out (not necessarily evenly) on the line (Fig. 3), creating a probability density (probability units per foot). Each indi100 Sec. 4-1__ Density Functions (5) | f(e)>0 forall (the graph never goes below the e-axis) © A ~ f(z) dz =1 (the area under the graph is 1) Note that (6) can’t hold unless we also have M f(-c0) =0 Figure 5 shows three typical densities. LLY fame ar Figure 5 Some densities Example 1 Let X have density fea for0<2<3 F(z) 0 otherwise (Fig. 6) 3 Figure 6 This is a legitimate density because f(z) > 0 and [ies [jeunk ThenChap.4 Continuous Random Variables 101 pasxsa= [leant ae eh gy ~ 2 pxsi= [adem [to d= = Px>3)= f Ode =0 Notation Instead of writing _ [2/9 for0 forasz0. (a) Find the prob that a component lasts at least 6 hours. (b) Suppose a machine contains 3 components. Find the prob that they all last at least 6 hours, assuming that the components func- tion independently of one another. 4. Let X have density f(2) = del. (a) Sketch the graph of f. (©) Find P(X? + X > 0). (b) Find P(|X| < 4). 5. Let X be a continuous random variable. True or False? (@) If the prob of an event involving X is 0, then the event is impossi- ble. (b) If the prob of an event involving X is 1, then the event is sure. 3.104__Sec. 4-2 __Distribution Functions 6. Pick a number at random between 2 and 7. (a) Find the probability that the number is 4. (b) Find the probability that the number is ~ 4, meaning that the number is in an interval of length dz around 4,Chap.4 Continuous Random Variables 105 P(X=2)=5, P(X=3)=.2, P(X=n)=3 so that the probability function for X is 5 if e=2 p(z)=% 2 if 2=3 3 if con (Fig. 2) 2 aus) Figure 2 Probability function for X The probability function in Fig. 2 is sometimes referred to as the discrete den- sity function for X. If you're familiar with delta functions, which allow area to be concentrated at a point (instead of spread out over an interval as with ordinary functions), then you recognize the function in Fig. 2 as, -56(x — 2) + .26(x — 3) + .36(x — 7) Let F(z) be the distribution function of X. Then F(z) represents cumulative probability in Fig. 2. The cumulative probability is 0 until z = 2 when it jumps to.5, Itstays.5 until 2 = 3 when it jumps to.7. It stays.7 until x when it jumps to 1 and thereafter stays 1. So 0 if r<2 _J 8 if 2<0<3 @ FO)=) 7 if Bn Figure 3 shows the graph of F(z). The solid dots are part of the graph; the open dots are points not in the graph. At x = 3 the solid dot is at .7, indicating that F(3) is.7, not .5. The correct value is .7 because, by the definition in (1), F(3) is the probability that has accumulated in Fig. 2 up to and including the chunk sitting at x = 3. In general, whenever there is a jump in a distribution function, the actual F value is at the top of the jump. The distribution function of a discrete random variable is a step function, rising from height 0to height 1. There are jumps at all the possible values of X, and the size of the jump at ap is P(X = ao).106 Sec. 4-2 __Distribution Functions Fo Incorrect but Useful Graphs and Notation Most computer graphing programs would produce the picture in Fig. 3’, not Fig. 3, for F(z). The vertical segments in Fig. 3’ make the graph ambiguous; for example, the value of F(3) looks as if it could be anything from .5 to .7. But even though Fig. 3 is the correct version, from now on in this book we’re going to draw in the style of Fig. 3’. And at the risk of further enraging math- ematicians, we'll refer to .7 as the upper F(3) and to .5 as the lower F(3). It’s simply more convenient that way. As long as we’re drawing ambiguous pictures, we'll also feel free to write ambiguous formulas to match. Instead of (2), it won’t hurt to write 0 if e<2 3B if 2<2<3 @) FO)=9) '7 if 3<2<0 1 if 22m Both (2) and (2’) convey a jump at x = 3 with upper value .7 and lower value 5. The Distribution Function of a Uniformly Distributed Random Variable Let X be uniformly distributed on [a, ]. Its density function f(z) is in Fig. 4. To find F(z), look at the cumulative area under f(z). F(z) isO until z = a. For z between a and 6, F(c) is the shaded area in Fig. 4, namely, (—a)-1/(b—a). For z > b, the accumulated area (probability) is 1, so F(z) = 1. So 0 if eb (Fig. 5)Chap.4 Continuous Random Variables 107 In other words, F(z) rises steadily from 0 to 1 on the interval (a, b). Example of a Mixed Random Variable ‘Toss a die. If the result is 2, then you win $7. If the result is a non-2, then your winnings are picked at random from the interval [3,5]. Let X be your winnings. Let’s find the distribution function of X. Remember that for a continuous random variable, P(X = z) is 0 for every x. But here, P(X = 7) = 1/6, so X is not continuous; that is, X doesn’t have a density function. Neither is X discrete, since X can take on any of the values in the interval [3,5], as well as 7. So X doesn’t have a probability function. But you can make a pseudodensity for X. Figure 6 shows a chunk of probability of size 1/6 at « = 7 and the remaining 5/6 units of probability spread evenly over the interval [3,5]. The corresponding density graph is in Fig. 7: the base of the rectangle is 2, and its area must be 5/6, so the height of the rectangle is 5/12. prob ants pr? fet 3 s 7 Figure 6 If you know about delta functions, you recognize Fig. 7 as the sum of an ordinary function and a delta function: = 12 The distribution function F(z) is the cumulative area in Fig. 7. Until = 3, F(z) is 0; as x goes from 3 to 5, F(z) rises steadily from 0 to 5/6; it stays at height 5/6 until x = 7 when it jumps by 1/6 to 1 (Fig. 8). Figure7 = (2) + zole =7) where (x)= for 3 xo) =1— P(X <2) =1-— upper F(zx0) P(X > x) =1— P(X < 29) =1- lower F(x0) If F jumps at a and at 6, then Pas X 2) = P(X > 20) =1-— F(z) Pas X and > or between < and < in describing events. Horizontal portions of the graph of F(x) (where probability is not accu- mulating) correspond to places where the density is 0 and events have proba- bility 0. Example 1 Let X have the distribution function 0 ife<-1 1 2. g@ti? if-1<2<1 _ ; if1<2<2 gets if 2<2<4 1 if2>4 (Fig. 11) Figure 11 Note that there is a jump of 1/4 at z = 1 because the F(z) formula for -1 <2 < 1 leaves off at il 1 geriagChap.4 Continuous Random Variables 111 but the F(x) formula for 1 < x < 2 picks up at 3/4. There are no jumps anywhere else. So feee ree eel P(X = 1) = jump size = 5 P(X $1) = upper F(1) = P(X <1) = lower F(1) = NIM ples P(X > 1) =1- upper FQ) =1- 3 = 5 P(X > 1) =1— lower F(1) =1- P(X =0)=0 P(X <0) = P(X <0) =F) = 3 P(X 20) = P(X >0)=1-F(0) = 7 1 POS X<3)=P24)=0 (the F graph is horizontal in these intervals) The Distribution Function of a Continuous Random Variable Every random variable has a distribution function. Not every random variable has a density function. Those that do are called continuous. A continuous random variable has the property that P(X = z) = 0 for all z, so its distribution function F(x) can’t jump. Conversely, it can be shown that for all practical purposes, a no-jump F(z) is an exclusive property of continuous random variables, so that all in all:112 Sec. 4-2 __ Distribution Functions A random variable is continuous (has a density) if and only if its distribution function has no jumps. Going from a Density to a Distribution Function You already know how to find the distribution function given a density func- tion: F(z) = cumulative area under the graph of f(z) (Fig. 1). In other words, 6) Fe)=P(X 0 (Fig, 12) Figure 12 Then F(a) = [ * #(e) de, but since f changes formula, you need cases to compute F(z). Casel. «<1 (Fig. 13). Case2. -1 <2 < 0 (Fig. 14). F(a) = [5 de= Het)Chap.4 Continuous Random Variables 113 = 0 Incase 2, use , not . 1 a 0 Ifyou use { you are getting F(0), not F(z). EA Case, x > 0 (Fig. 15). Fa)= [dacs [fee ae ~ S12 fo 2 Figure13 Case 1 Figure14 Case 2 Figure 15 Case 3 So all in all (Fig. 16),114 Sec. 42 Distribution Functions 0 if e<-1 1 1) if -l10 Figure 16 Asa check, note that F(—o0) = 0, F(oo) = 1- $e = 1-0 =1. Andas befits the distribution function of a continuous random variable, F(x) doesn’t jump: F(z) comes in 3 pieces, and the first piece leaves off at « = —1 with value 0 the second piece picks up at 3 (z+ »| 0 (nojump at z = 1) the second piece leaves off at ie + »| = ; Jz=0 the third piece picks up at (1 — je) 5 (nojump at « = 0) le=0 Warning Here are some things that F (2) is not: F(2) isnot f se) dz; 7 F(z) isnot [ f(z) de, even if you're in a case where 5 < x <7. s By (5), F(z) is always i * f(z) dz, which in practice becomes an integral from the first 2 in the universe up to an arbitrary «.hap. Continuous Random Variables Going from a Distribution Function with No Jumps to a Density It’s one of the fundamental theorems of calculus’ that if116 ___ Sec. 4-2 _Distribution Functions 0 if <-1 a 2 g@ti? if -1<2<1 F(z) = 5 if 1 if 4 4 Define a random variable X as follows. Toss a coin where P(heads) = 1/3. If the coin is heads, then set X = 5. If the coin is tails, then choose X at random between 3 and 6. X is a mixed random variable. It doesn’t have a legal density; it has a pseudodensity, which is part density and part probability function. Find the pseudodensity of X and the (legal) distribution function F(z) and draw a picture of each. Let X,Y, Z be independent continuous random variables, all with the same distribution function F. Consider the minimum and maximum of X,Y,Z. (a) Find P(max < 7) (in terms of F). (b) Find P(min > 5 and max < 7). (©) Put (a) and (6) together to find P(min < 5 and max < 7). For each distribution function F(z), find the corresponding density if a legal density exists (otherwise, find the pseudodensity). Sketch F and f. 1, = < Re te GR) 5 if O<2<2 (a) F(x) = 2 i+) if 25 je if 2x0 ) F(z) =120___Sec.43 The Exponential Distribution te it <0 © F(e)=4 4 1 if e20 10, For each density f(z), find the corresponding distribution function F(a). a 100 (@) f(e)=—y fore > 100 for |z| <1 4 if -1<@<. © fe ; if ices 11. Given this rough graph of a density f(z), sketch a rough graph of the distribution function F(z). a % Xe % SECTION 4-3 THE EXPONENTIAL DISTRIBUTION The Exponential Density Let d > 0. If X has density a f(z) =e" forz>0 (Fig.1) then we say that X has an exponential distribution with parameter 2. fe) a Figure 1 The exponential density