Source: Principles of Statistics I, Texas A&M University

Tags: Bayes' theorem, Bayes' rule, conditional probability, law of total probability, prior probability, posterior probability, updating beliefs, disjoint union, weighted average, two-way probability table

TL;DR

Bayes' Rule lets you reverse a conditional probability: if you know P(B|A), it tells you P(A|B). The law of total probability gives you the denominator you need. In practice, the fastest approach for exam problems is to build a complete two-way table from the information given, then read the answer off as a cell divided by its column (or row) total.

Key Terms

Conditional probability, P(A|B)

The probability of A given that B has already occurred. Calculated as P(A ∩ B) / P(B). The "given" event (B) becomes the new denominator, restricting the universe.

Prior probability

Your initial belief about an event before observing new evidence. In a Bayes' problem, these are the marginal probabilities you start with.

Posterior probability

Your updated belief after incorporating new evidence. This is the output of Bayes' Rule.

Law of total probability

If events B₁, B₂, …, Bₙ partition the sample space, then P(A) = Σ P(A ∩ Bᵢ) = Σ P(Bᵢ) · P(A|Bᵢ). It expresses an overall probability as a weighted average of conditional probabilities.

Disjoint union

When events B₁, B₂, …, Bₙ don't overlap, the event A can be split into non-overlapping pieces: A = (A ∩ B₁) ∪ (A ∩ B₂) ∪ … ∪ (A ∩ Bₙ). Probabilities of disjoint events simply add.

Bayes' theorem (formula)

P(Bⱼ|A) = P(Bⱼ) · P(A|Bⱼ) / Σ P(Bᵢ) · P(A|Bᵢ). The numerator is the joint probability of your target; the denominator is the total probability of the observed evidence across all routes.

Marginal probability (in a table)

A row total or column total. In Bayes' problems, the marginals for each "view" of the data must each sum to 1.

Core Content

The Conceptual Idea

Bayesian reasoning is about updating beliefs with evidence. A useful analogy: imagine a child who sees the sun set for the first time and places one white marble (sunrise) and one black marble (no sunrise) in a bag. Each morning the sun rises, the child adds another white marble. After one sunrise the probability of drawing white is 2/3; after two sunrises, 3/4; and so on. The prior belief (50/50) gradually shifts toward near-certainty. That is the core of Bayes' Rule: new data reshapes what you believe.

The Formulaic Approach (Supplier Example)

Three suppliers feed a manufacturing plant:

• Supplier 1: 30% of parts, 1% defective rate

• Supplier 2: 50% of parts, 2% defective rate

• Supplier 3: 20% of parts, 3% defective rate

Step 1, law of total probability: find the overall defect rate.

P(D) = P(S₁)·P(D|S₁) + P(S₂)·P(D|S₂) + P(S₃)·P(D|S₃) P(D) = (0.30)(0.01) + (0.50)(0.02) + (0.20)(0.03) = 0.003 + 0.010 + 0.006 = 0.019

So 1.9% of all parts in inventory are defective. This is a weighted average of the individual defective rates, weighted by each supplier's share.

Step 2, Bayes' Rule: given a defective part, what is the probability it came from Supplier 3?

P(S₃|D) = P(S₃)·P(D|S₃) / P(D) = (0.20)(0.03) / 0.019 = 0.006 / 0.019 ≈ 0.3158

Similarly, P(S₁|D) ≈ 0.1579 and P(S₂|D) ≈ 0.5263. These three posterior probabilities sum to 1.000, as they must (the part came from exactly one supplier).

The Table Method (Application Approach)

For exam problems, building a two-way table is usually the fastest route. The process has a consistent pattern:

1. Identify the two "views" of the situation. Each view has mutually exclusive outcomes whose marginal probabilities sum to 1.

1. Place the given marginals along the edges of the table.

1. Fill in any cells you can calculate directly from the given conditional or joint information.

1. Complete the remaining cells by subtraction (each row and column must hit its marginal).

1. The Bayes' answer is a single cell divided by its row or column total.

Worked Example: Gas Station (Sports Cars and Fuel Type)

Given information:

• Fuel type marginals: Regular 70%, Mid-grade 20%, Premium 10%

• Vehicle marginals: Sports car 10%, Non-sports car 90%

• 69% of all purchases are Regular AND Non-sports car (a joint probability)

• Sports cars buy Premium 80% of the time (a conditional within the SC row)

Building the table:

How the cells were filled:

• SC ∩ Premium = 0.80 × 0.10 = 0.08 (80% of the SC row lands in Premium)

• Not SC ∩ Regular = 0.69 (given directly)

• SC ∩ Regular = 0.70 − 0.69 = 0.01 (column total minus the known cell)

• Not SC ∩ Premium = 0.10 − 0.08 = 0.02 (column total minus the known cell)

• The remaining two cells complete their respective rows.

The question: P(Sports car | Mid-grade) = ?

The denominator is the Mid-grade column total (0.20). The numerator is the SC cell in that column (0.01). So P(SC|M) = 0.01 / 0.20 = 0.05, or 5%.

Worked Example: Oil Wells (Depth and Profitability)

Given information:

• Depth marginals: Under 5k (20%), 5k–15k (50%), Over 15k (30%)

• Conditional "good" rates: 20% of under-5k wells are good, 30% of 5k–15k wells are good, 40% of over-15k wells are good

Table:

P(< 5k | Good) = 0.04 / 0.31 ≈ 0.129, or about 12.9%.

Worked Example: Star Trek Convention (Memorabilia and Uniforms)

Given information:

• Purchase marginals: Kirk 50%, Spock 30%, CD 20%

• Uniform marginal: 60% wore uniforms

• 70% of Kirk buyers wore uniforms; 20% of CD buyers wore uniforms

Table:

P(Uniform | Spock) = 0.21 / 0.30 = 0.70, or 70%.

Worked Example: Camera Warranty (Basic vs Deluxe)

Given information:

• 40% basic model, 60% deluxe model

• 30% of basic buyers get extended warranty; 50% of deluxe buyers do

Table:

P(Basic | Warranty) = 0.12 / 0.42 ≈ 0.2858, or about 28.6%.

Formulas

Conditional probability: P(A|B) = P(A ∩ B) / P(B)

Law of total probability: P(A) = P(B₁)·P(A|B₁) + P(B₂)·P(A|B₂) + … + P(Bₙ)·P(A|Bₙ)

Bayes' theorem: P(Bⱼ|A) = P(Bⱼ)·P(A|Bⱼ) / [P(B₁)·P(A|B₁) + P(B₂)·P(A|B₂) + … + P(Bₙ)·P(A|Bₙ)]

Why It Matters / Exam Flags

⚠️ The table method and the formula method give the same answer. The table is usually faster and less error-prone on timed exams because you can check your work (rows and columns must sum to their marginals, and the grand total is 1.00).

⚠️ The most common mistake is dividing by the wrong total. For P(A|B), the denominator is the total for B (the "given" event), not the grand total.

⚠️ Always verify that posterior probabilities across all mutually exclusive outcomes sum to 1.0. If they don't, a cell in the table is wrong.

⚠️ When filling the table, look for conditional statements ("80% of sports cars buy premium") versus joint statements ("69% of all purchases are regular and non-sports car"). Conditionals apply to a row or column subtotal; joints go directly into a cell.

⚠️ Problems often give you just enough information to fill 2–3 cells, then expect you to deduce the rest by subtraction. If you are stuck, write down every marginal first, then slot in whatever you can calculate.

Practice Q&A

Q: A factory has two machines. Machine A produces 60% of output with a 5% defect rate. Machine B produces 40% with a 3% defect rate. A defective item is found. What is the probability it came from Machine A?

A: P(D) = (0.60)(0.05) + (0.40)(0.03) = 0.030 + 0.012 = 0.042. P(A|D) = 0.030 / 0.042 ≈ 0.714, or about 71.4%.

Q: In the gas station example, what is P(Not sports car | Premium)?

A: The Premium column total is 0.10. The Not SC cell in that column is 0.02. So P(Not SC | Premium) = 0.02 / 0.10 = 0.20, or 20%.

Q: Why do the posterior probabilities P(S₁|D), P(S₂|D), and P(S₃|D) have to sum to 1?

A: Because S₁, S₂, and S₃ partition the sample space (every part comes from exactly one supplier). Given that the part is defective, it still must have come from one of the three, so the conditional probabilities over those three sources must be exhaustive.

Q: A student fills in a Bayes' table and gets a "Good" row total of 0.31 and a "Not Good" row total of 0.60. What went wrong?

A: The row totals should sum to 1.00. A total of 0.91 means at least one cell was calculated incorrectly. Recheck each cell against its column marginal and conditional rate.

Q: What is the conceptual difference between the law of total probability and Bayes' theorem?

A: The law of total probability calculates an overall (marginal) probability by summing weighted conditional contributions. Bayes' theorem reverses the direction of conditioning, using that total probability as the denominator.

Related Terms / Search Tags

Bayes' theorem, Bayes' rule, Bayesian probability, conditional probability, posterior probability, prior probability, law of total probability, total probability theorem, disjoint union, partition, weighted average, two-way table, joint probability table, contingency table, inverse probability, updating beliefs, P(A|B), STAT 301, Principles of Statistics I, Texas A&M