Source: Lecture 5 (9/15/22) | Course: Analytical Statistics (T), NYU
Tags: binomial distribution, Bayes' theorem, independent trials, success probability, conditional probability, posterior probability
Difficulty: Intermediate | Prerequisites: Discrete random variables, PMF basics, binomial coefficient, basic conditional probability
The binomial distribution is the workhorse model for counting successes. Whenever you have a fixed number of independent trials, each with the same probability of success, the binomial gives you the probability of any particular number of wins. This lecture builds the distribution from first principles (why the formula looks the way it does), then applies Bayes' theorem to update beliefs when new evidence arrives. You should already be comfortable with PMFs, the binomial coefficient, and basic conditional probability before working through this material.
The binomial distribution models the number of successes in n independent, identical trials, each with success probability p. Its PMF is P(X = x) = C(n, x) × pˣ × (1−p)ⁿ⁻ˣ, and we write X ~ B(n, p). When you observe data and want to revise a prior belief, Bayes' theorem flips the conditional: P(A|B) = P(B|A) × P(A) / P(B).
Binomial distribution
The probability distribution for the number of successes in n independent trials, each with the same success probability p. Written as X ~ B(n, p). Think of it as the "how many wins out of n attempts" distribution.
Bernoulli trial
A single experiment with exactly two outcomes: success (probability p) or failure (probability q = 1 − p). In simple terms, one coin flip, one yes/no question, one pass/fail test.
Success probability (p) and failure probability (q)
p is the probability of success on any single trial. q = 1 − p is the probability of failure. These two always sum to 1.
Independent trials
Trials where the outcome of one does not affect the outcome of any other. In simple terms, knowing the result of trial 3 tells you nothing about trial 7.
Bayes' theorem
A formula for updating the probability of a hypothesis after observing evidence: P(A|B) = P(B|A) × P(A) / P(B). Think of it as the "given what I just saw, how should I revise what I believed before?" formula.
Prior probability
Your initial belief about the probability of a hypothesis before seeing the data. In the bent-coin example, this is P(A) = 1/2 (the coin is equally likely to be bent or fair before you observe any flips).
Posterior probability
Your updated belief after incorporating the evidence. This is what Bayes' theorem calculates: P(A|B).
Start with n independent, identical trials. Each trial results in either "success" (probability p) or "failure" (probability q = 1 − p).
Step 1: Probability of one specific sequence
Because the trials are independent, the probability of any particular sequence containing exactly x successes and (n − x) failures, in a fixed order, is:
pˣ × (1 − p)ⁿ⁻ˣ
Step 2: Count the sequences
There are C(n, x) different orderings that contain exactly x successes among n trials.
Step 3: Combine
Multiply the probability of one sequence by the number of such sequences:
P(X = x) = C(n, x) × pˣ × (1 − p)ⁿ⁻ˣ
This is the binomial PMF.
A random variable X following the binomial distribution is written as X ~ B(n, p), where:
n = number of trials (a positive integer)
p = probability of success on each trial (between 0 and 1)
The PMF is defined for x ∈ {0, 1, 2, …, n} and equals 0 for all other values.
The binomial distribution serves as a model for whole families of experiments. Once you fix n and p, the entire probability table is determined.
This is a worked example combining the binomial distribution with Bayes' theorem.
Setup:
You have a coin that might be bent (comes up heads with probability p₁ = 3/4) or fair (p₂ = 1/2).
Prior belief: P(bent) = P(fair) = 1/2 (equally likely before any data).
You toss it 10 times and observe 8 heads.
Question: What is P(coin is bent | 8 heads out of 10)?
Define events:
A = "the remaining coin is bent"
B = "8 heads out of 10 tosses"
Step 1: Compute the likelihood under each hypothesis
If the coin is fair: P(B|Aᶜ) = C(10, 8) × (1/2)⁸ × (1/2)² = C(10, 8) / 2¹⁰
If the coin is bent: P(B|A) = C(10, 8) × (3/4)⁸ × (1/4)²
Step 2: Apply Bayes' theorem
P(A|B) = P(B|A) × P(A) / [P(B|A) × P(A) + P(B|Aᶜ) × P(Aᶜ)]
Plugging in:
P(A|B) = [C(10,8) × (3/4)⁸ × (1/4)² × 1/2] / [C(10,8) × (3/4)⁸ × (1/4)² × 1/2 + C(10,8) × (1/2)¹⁰ × 1/2]
The C(10, 8) terms cancel. After computing: P(A|B) ≈ 46.21%.
Step 3: Compute the updated probability of heads on the next flip
P(H|B) = P(H|bent) × P(bent|B) + P(H|fair) × P(fair|B)
P(H|B) = (3/4)(0.4621) + (1/2)(0.5379) ≈ 63.86%
The data (8 heads) shifted the probability of heads on the next toss from the naive 50/50 up to about 64%, because it became more plausible (though not certain) that the coin is bent.
Binomial PMF
P(X = x) = C(n, x) × pˣ × (1 − p)ⁿ⁻ˣ, for x ∈ {0, 1, 2, …, n}
Probability of one specific sequence of x successes and (n − x) failures
pˣ × (1 − p)ⁿ⁻ˣ
Bayes' theorem (two-hypothesis form)
P(A|B) = P(B|A) × P(A) / [P(B|A) × P(A) + P(B|Aᶜ) × P(Aᶜ)]
Law of total probability (used in the bent-coin example)
P(B) = P(B|A) × P(A) + P(B|Aᶜ) × P(Aᶜ)
Posterior predictive probability
P(H|B) = P(H|A) × P(A|B) + P(H|Aᶜ) × P(Aᶜ|B)
The binomial distribution appears any time you count how many times something happens across repeated, independent attempts: patients responding to a drug in a clinical trial, defective items in a manufacturing batch, clicks on an ad out of total impressions. Bayes' theorem is the foundation of spam filtering, medical diagnostic reasoning (updating disease probability after a test result), and machine learning classifiers that incorporate prior knowledge.
Students often apply the binomial formula when trials are not independent. If drawing cards from a deck without replacement, the trials are dependent and you need the hypergeometric distribution, not the binomial.
A frequent mistake is forgetting that p must be the same for every trial. If the success probability changes from trial to trial, the binomial does not apply.
Students sometimes confuse the prior and the posterior in Bayes' theorem. The prior is what you believed before seeing the data. The posterior is what you believe after. Mixing them up inverts the entire calculation.
In the bent-coin example, students often forget to use the law of total probability for the denominator P(B). You cannot just compute the numerator and call it the answer.
⚠️ Expect to be asked to derive or apply the binomial PMF. Be able to explain where C(n, x) comes from and why pˣ(1−p)ⁿ⁻ˣ appears.
⚠️ Bayes' theorem problems are a favourite. You will likely see a question structured like the bent-coin example: two hypotheses, compute likelihoods under each, apply the formula.
⚠️ Know how to compute a posterior predictive probability (the final step in the bent-coin example, where you weight the next-flip probability by the posterior).
⚠️ Be ready to check whether the binomial model is appropriate. The exam may describe a scenario where one of the assumptions (independence, constant p, fixed n) is violated.
True or False: In a binomial experiment, the probability of success can vary between trials. (False, p must be constant.)
Fill in the blank: X ~ B(10, 0.3) means ___ trials with success probability ___. (10 trials, 0.3)
True or False: Bayes' theorem lets you compute P(hypothesis | data) from P(data | hypothesis). (True)
Fill in the blank: If p = 0.4, then q = ___. (0.6)
True or False: The binomial PMF can give non-zero probabilities for x > n. (False, x can only range from 0 to n.)
Q: If X ~ B(8, 0.25), what is P(X = 0)?
A: P(X = 0) = C(8, 0) × (0.25)⁰ × (0.75)⁸ = 1 × 1 × (0.75)⁸ ≈ 0.1001
Q: A factory produces items with a 5% defect rate. In a batch of 20, what is the probability of exactly 2 defectives?
A: X ~ B(20, 0.05). P(X = 2) = C(20, 2) × (0.05)² × (0.95)¹⁸ = 190 × 0.0025 × 0.3972 ≈ 0.1887
Q: You believe there is a 60% chance a die is loaded (always rolls 6) and a 40% chance it is fair. You roll it once and get a 6. What is the updated probability that the die is loaded?
A: Let A = loaded, B = rolled a 6. P(B|A) = 1, P(B|Aᶜ) = 1/6, P(A) = 0.6, P(Aᶜ) = 0.4. By Bayes: P(A|B) = (1 × 0.6) / (1 × 0.6 + 1/6 × 0.4) = 0.6 / (0.6 + 0.0667) ≈ 0.90. So the posterior probability jumps to about 90%.
Q: Why does the C(n, x) term appear in the binomial PMF?
A: Because there are C(n, x) distinct orderings of x successes among n trials. Each ordering has the same probability pˣ(1−p)ⁿ⁻ˣ, so we multiply by the count of orderings.
Q: In the bent-coin example, why is the final answer for P(H|B) about 64% rather than 75%?
A: Because the posterior probability of the coin being bent is only about 46%, not 100%. The 64% is a weighted average: 75% (if bent) weighted by 46.21% plus 50% (if fair) weighted by 53.79%.
The binomial distribution is a special case of the more general multinomial distribution (when there are more than two outcome categories). As n grows large and p stays moderate, the binomial approximates the normal distribution (the basis for z-tests and confidence intervals later in the course). Bayes' theorem connects forward to Bayesian inference, which you may encounter in advanced statistics or machine learning courses.
Binomial distribution, B(n,p), Bernoulli trial, success-failure experiment, independent trials, binomial PMF, binomial probability, number of successes, Bayes' theorem, Bayes' rule, conditional probability, prior probability, posterior probability, law of total probability, likelihood, updating beliefs, bent coin problem, diagnostic testing, NYU Analytical Statistics, binomial model assumptions