Probability Data Distributions in Data Science

To understand how variables relate to each other begins with looking at covariance and correlation. Once we see how they are connected the next step is to explore probability distribution as it helps to describe how data is spread out and reveal hidden patterns and provide better predictions.

Let's suppose if you roll a die then probability of rolling a 6 is 1/6 (16.67%). Hence we can say that probability distribution is a way to describe the chances of different outcomes that can happen. Now imagine if we apply this to complex data like customer purchases, stock prices or weather predictions to answer questions like:

• What is most likely to happen?
• What are the rare or unusual outcomes?
• Are the values close together or very different from each other?

By finding out the answer of these questions we can make better predictions and analyze uncertainty in data using probability distributions.

Why Are Probability Distributions Important?

Probability Distribution are said to be the backbone of data science because:

• Through probability distribution you can see how data behaves whether it clusters around certain values or spreads evenly.
• Many machine learning models are built on assumptions about how data is distributed.
• Statistical tests also use distributions to calculate things like p-value which tell you whether your results are meaningful or not.

Before learning about probability distributions we first need to understand random variables. A random variable is a way to use numbers to represent different outcomes of a random event like in rolling a die you can assign 1 to "even" and 0 to "odd".

Random variables can be classified into two types:

• Discrete Random Variables: It is one that takes values that you can count like whole numbers. For example the number of students in a class or the number of cars in a parking lot as these are always whole numbers.
• Continuous Random Variables: It can take any value within a range including decimals. These values come from measurements rather than counting. let's say a person’s height could be 5.7 or 6.2 feet and the temperature outside could be 27.3°C are the examples of Continuous Random variable.

Key Components of Probability Distributions

Now that we understand random variables let's explore how we describe their probabilities using three key concepts:

1. Probability Mass Function (PMF)

The PMF applies to discrete random variables like the number of products a customer buys per order. Let’s say after analyzing your customer data you find that 25% of customers buy exactly 3 products. This means you can predict future customer behavior based on this information as the PMF tells you the likelihood of each specific outcome.

2. Probability Density Function (PDF)

It is used for continuous random variables like how much money a customer spends. For example if you find that most customers spend around $50 but some customers spend much more the PDF helps you understand how customer spending is distributed.

It doesn’t give an exact probability for a specific value e.g. exactly $50 because spending is continuous and can have infinite possible values like $49.99, $50.25, or $51.00. Instead it shows how the probabilities are spread across a range of values

3. Cumulative Distribution Function (CDF)

It helps to determine probabilities for values less than or equal to a given number and is used for both continuous and discrete variables. For discrete data like the number of products bought the CDF tells us the probability of buying 3 or fewer products. For example CDF(3) = 0.75 means there's a 75% chance of buying 3 or fewer products.

For continuous data like spending the CDF shows the probability that a customer spends less than or equal to a certain amount like CDF($50) = 0.80 means 80% of customers spend $50 or less. To find the CDF we can use the formula given below:

\text{CDF: } F_X(x) = P(X \leq x) = \int_{-\infty}^x f(t) \, dt

where F(x) is the CDF and f(t) is the PDF.

Types of Probability Distributions

Probability distributions can be divided into two main types based on the nature of the random variables: discrete and continuous.

Discrete Data Distributions

A discrete distribution is used when the random variable can take on countable, specific values. For example, when predicting the number of products a customer buys in a single order the possible outcomes are whole numbers like 0, 1, 2, 3, etc. You can't buy 2.5 products so this is a discrete random variable.

It includes various distributions Let's understand them one by one:

1. Binomial Distribution

Imagine you're flipping a coin 10 times and you want to know how many heads (successes) you’ll get. You know that each flip has two possible outcomes: heads or tails. So the binomial distribution helps you to calculate the probability of getting a certain number of heads in those 10 flips.

In this case there are:

• The number of trials (flips) is fixed: 10.
• Each flip has two outcomes: heads (success) or tails (failure).
• The probability of heads is 0.5 and you want to know how many heads will show up.

This distribution is useful in situations where you have a set number of trials and you want to count how many times a specific outcome like success occurs. The graph of the binomial distribution would show a set of bars like a histogram representing how likely it is to get different numbers of heads (from 0 to 10) in those 10 flips.

2. Bernoulli Distribution

Now imagine you’re flipping a coin just once. You care about whether you get heads (success) or tails (failure). This is where the Bernoulli distribution comes in. It's the simplest form of a distribution because it deals with just one trial and two possible outcomes: success or failure.

• You only have one trial.
• Two possible outcomes: heads (success) or tails (failure).
• The probability of getting heads is 0.5.

The Bernoulli Distribution tells you the probability of getting either success or failure on a single trial. The graph of the Bernoulli distribution would just have two bars: one for success (1) and one for failure (0) each showing the probability 0.5 for each.

3. Poisson Distribution

Next let’s talk about the Poisson distribution. This distribution is used when you want to count the number of random events that happen in a fixed period of time or within a certain area.

For example let’s say you work at a coffee shop and on average 5 customers walk in every hour. It helps you calculate the probability of having exactly 3 customers, 6 customers or any other number of customers in an hour helps given that on average you get 5 customers per hour.

The Poisson Distribution helps answer questions like: "What’s the probability of seeing exactly 3 customers in one hour if the average rate is 5 per hour?". The graph of the Poisson distribution would be a curve where the most likely number of customers is around 5 and the curve gets lower as you move away from 5.

4. Geometric Distributions

The geometric distribution is used to model the number of trials it takes to get the first success in a sequence of independent trials each with a fixed probability of success.

Let’s say you're sending promotional emails to customers and you want to know how many emails you'll need to send before one customer makes a purchase. Each email you send has a fixed chance of resulting in a purchase but you’re interested in the number of emails it will take to get the first purchase.

• The trials (emails) are independent (each email is unrelated to the others).
• You’re counting how many trials it takes until the first success.

It helps us to answer questions like: “How many emails do I need to send before I get my first purchase?” The graph of the geometric distribution would show a decreasing curve where the probability of needing more emails decreases as the number of trials increases.

Continuous Data Distributions

A continuous distribution is used when the random variable can take any value within a specified range like when we analyze how much money a customer spends in a store then the amount can be any real number including decimals like $25.75, $50.23, etc.

In continuous distributions the Probability Density Function (PDF) shows how the probabilities are spread across the possible values. The area under the curve of this PDF represents the probability of the random variable falling within a certain range.

Now let's look at some types of continuous probability distributions that are commonly used in data science:

1. Normal Distribution

The normal distribution is one of the most common distributions and called the bell-shaped curve because it looks like a bell. In this distribution most of the data points are near the mean and the probability decreases as you move further away from the mean. This distribution is symmetrical means the left side looks just like the right side.

Let's think about the heights of people. Most people are around the average height with few people being very short or very tall. The normal distribution models this kind of data perfectly.

• The mean is the center of the curve.
• The standard deviation determines how spread out the data is. A smaller standard deviation means the data points are closer to the mean and a larger standard deviation means the data is more spread out.

2. Exponential Distribution

The normal distribution is useful for modeling naturally occurring data but what if we are interested in modeling time between events? Then we use the exponential distribution.

Suppose the average time between customers arriving at a store is 10 minutes. The exponential distribution can help you figure out how long you might wait for the next customer. Maybe you’ll wait 5 minutes or maybe you’ll wait 15 minutes but on average you expect 10-minute intervals. The rate parameter (λ) tells you how often the events happen. If customers arrive every 10 minutes on average λ is the rate of 1 customer per 10 minutes.

we can say that it models the time between events in a process where events can happen continuously and independently.

While the exponential distribution focuses on waiting times sometimes we just need to model situations where every outcome is equally likely. In that case we use the uniform distribution.

3. Uniform Distribution

The uniform distribution is a distribution where every outcome in a certain range is equally likely to happen. You can have a discrete uniform distribution like rolling a fair die or a continuous uniform distribution like picking a random number between 0 and 1.

Imagine you have a fair six-sided die. The chance of rolling any number from 1 to 6 is the same—1/6 for each outcome. This is a discrete uniform distribution.

For a continuous uniform distribution every number between a and b say 0 and 1 has the same chance of being picked.

4. Beta Distribution

However in many real-world problems probabilities are not uniform. Instead they may change based on prior knowledge. To handle uncertainty and update our beliefs as we gather more data we use the beta distribution

Let’s say you want to model the probability of a customer clicking on a new advertisement. The beta distribution helps you express your uncertainty especially when you have limited data. As you collect more data the beta distribution helps you update your belief about the probability of a click. The parameters of the beta distribution (α and β) control the shape of the distribution. They determine how confident you are about the probability.

It’s often used in Bayesian statistics to represent uncertainty about a probability before you observe new data. For example it’s used in A/B testing to compare the success rates of two different webpage designs which we study in upcoming articles.

5. Gamma Distribution

After studying the beta distribution which is useful for single probabilities sometimes we need to model the total time required for multiple independent events. This is where the gamma distribution used.

It is related to the exponential distribution but it’s used when you're modeling the total time it takes for multiple events to occur. It’s often used in scenarios like estimating the total duration of tasks when individual task times vary.

Suppose you have a project with three tasks and the time for each task is independent but varies. The gamma distribution can help you estimate how long the entire project will take by modeling the total time for the three tasks. The shape parameter (κ) controls the number of events and the scale parameter (θ) controls how long each event takes.

6. Chi-Square Distribution

The chi-square distribution is used in hypothesis testing particularly when you're testing the relationship between categorical variables. It's often used in the chi-square test to see if two variables are independent or not.

Imagine you're testing whether gender is related to whether people prefer coffee or tea. You collect data from a group of people and create a contingency table. It helps you calculate the probability that any differences between the groups coffee vs. tea, male vs. female are due to random chance. The degrees of freedom in the chi-square distribution depend on the number of categories in your data.

7. Log-Normal Distribution

If a stock price grows over time it usually grows in percentage terms rather than a fixed amount. This kind of growth is modeled by a log-normal distribution. and if you take the logarithm of the data and it becomes normally distributed then the original data follows a log-normal distribution.

This is used to model data that grows in a multiplicative way and cannot be negative. This happens when the data is the result of many small independent factors multiplying together like stock prices or income levels.

Now It is the time to summarize all the distributions that we have studied:

It looks like a bell and most data is around the middle and few values are at ends.

In this we learn about important probability distributions used for making predictions and understanding data. Next we’ll look at Inferential Statistics where we’ll learn how to make conclusions from it.