Difficulty: Intermediate | Prerequisites: Data Types and Sampling Methods notes
Probability is the mathematical language that connects data collection (sampling) to data analysis (inference). This topic covers the rules for combining probabilities, the key discrete and continuous distributions, and how density functions and cumulative distribution functions relate to one another. Almost every calculation-heavy exam question on this midterm draws on material from this section.
Independent events multiply: P(A and B) = P(A)P(B). Bayes' theorem flips conditional probabilities using priors. The binomial distribution models counted successes in fixed trials (mean = np, variance = np(1-p)). A PDF integrates to give the CDF; differentiating the CDF recovers the PDF. For independent random variables, the joint PDF is the product of the marginals, and E[XY] = E[X]E[Y].
Independent events
Two events A and B are independent if the occurrence of one does not affect the probability of the other. Formally, P(A and B) = P(A)P(B). In simple terms, knowing A happened tells you nothing new about B.
Mutually exclusive (disjoint) events
Two events that cannot occur at the same time: P(A and B) = 0. Think of it as "if one happens, the other is ruled out." Disjoint events with non-zero probabilities are never independent.
Conditional probability
The probability of event A given that event B has occurred: P(A|B) = P(A and B) / P(B). In simple terms, it is the probability recalculated after restricting the sample space to B.
Bayes' theorem
A formula that reverses a conditional probability: P(A|B) = P(B|A)P(A) / P(B). It lets you update a prior belief about A after observing evidence B.
Law of total probability
Links conditional probabilities to marginal probabilities by summing over a partition of the sample space: P(B) = sum of P(B|Ai)P(Ai) for all partitions Ai. In simple terms, it computes the overall probability of B by weighting each conditional scenario.
Combination (n choose k)
The number of ways to choose k items from n without regard to order: C(n,k) = n! / (k!(n-k)!). Used in the binomial formula and in counting problems.
Probability density function (PDF)
For a continuous random variable, f(x) gives the density of probability at each point. The probability of X falling in an interval is the area under f(x) over that interval. The PDF can exceed 1 at a point, but the total area under the curve must equal 1.
Cumulative distribution function (CDF)
F(x) = P(X <= x). It is always non-decreasing, right-continuous, approaches 0 as x goes to negative infinity and 1 as x goes to positive infinity. For a continuous variable, F(x) is the integral of the PDF from negative infinity to x.
Binomial distribution
Models the number of successes in n independent trials, each with success probability p. Mean = np, variance = np(1-p), standard deviation = sqrt(np(1-p)).
Joint PDF
For two continuous random variables X and Y, f(x,y) gives the density over the (x,y) plane. If X and Y are independent, f(x,y) = f_X(x) times f_Y(y).
Marginal PDF
The density of one variable obtained by integrating the joint PDF over the other variable. f_X(x) = integral of f(x,y) dy over all y.
If A and B are independent: P(A and B) = P(A)P(B).
The addition rule for any two events: P(A or B) = P(A) + P(B) - P(A and B). For mutually exclusive events the last term is zero, so P(A or B) = P(A) + P(B).
Independence does not mean disjoint. Disjoint events (with non-zero probability) are dependent: if one occurs, the other cannot.
A diagnostic test for a condition that affects 1% of the population has 95% sensitivity (P(positive | sick) = 0.95) and 90% specificity (P(negative | healthy) = 0.90). What is P(sick | positive)?
Step 1: Identify priors. P(C) = 0.01, P(not C) = 0.99.
Step 2: Identify conditionals. P(+|C) = 0.95, P(+|not C) = 1 - 0.90 = 0.10.
Step 3: Total probability of testing positive. P(+) = P(+|C)P(C) + P(+|not C)P(not C) = (0.95)(0.01) + (0.10)(0.99) = 0.0095 + 0.099 = 0.1085.
Step 4: Apply Bayes' theorem. P(C|+) = P(+|C)P(C) / P(+) = 0.0095 / 0.1085 = 0.0876, roughly 8.8%.
The low result despite a good test happens because the condition is rare: most positives come from the large healthy group.
When order does not matter, use combinations. C(10,3) = 10! / (3! times 7!) = (10 times 9 times 8) / (3 times 2 times 1) = 120. This appears in the binomial formula and in committee-selection questions.
Conditions: fixed number of trials n, each trial has two outcomes (success/failure), trials are independent, probability of success p is constant.
P(X = k) = C(n,k) times p^k times (1-p)^(n-k).
Worked example: n = 20, p = 0.05 (defective rate), k = 2. P(X = 2) = C(20,2) times (0.05)^2 times (0.95)^18 = 190 times 0.0025 times 0.3972 = 0.1887.
From PDF to CDF: integrate. F(x) = integral from negative infinity to x of f(t) dt.
From CDF to PDF: differentiate. f(x) = dF(x)/dx.
Students commonly get this backwards. Remember: integration accumulates (CDF), differentiation recovers the rate (PDF).
Worked example: f(x) = 2x for 0 <= x <= 1. F(x) = integral from 0 to x of 2t dt = t^2 evaluated from 0 to x = x^2. P(X <= 0.5) = F(0.5) = 0.25.
For independent X and Y, f(x,y) = f_X(x) times f_Y(y). For dependent variables, the joint PDF does not factor this way.
To find k in f(x,y) = k(x+y) over [0,1] x [0,1]: set the double integral to 1.
Integral from 0 to 1 of integral from 0 to 1 of k(x+y) dy dx = k times integral from 0 to 1 of (x + 1/2) dx = k times (1/2 + 1/2) = k times 1 = 1, so k = 1.
Marginal f_X(x) = integral from 0 to 1 of (x+y) dy = x + 1/2 for 0 <= x <= 1.
For independent variables, E[XY] = E[X]E[Y]. This property is tested directly on the exam.
Independence: P(A and B) = P(A)P(B)
Addition rule: P(A or B) = P(A) + P(B) - P(A and B)
Bayes' theorem: P(A|B) = P(B|A)P(A) / P(B)
Law of total probability: P(B) = sum over i of P(B|Ai)P(Ai)
Combinations: C(n,k) = n! / (k!(n-k)!)
Binomial PMF: P(X = k) = C(n,k) p^k (1-p)^(n-k)
Binomial mean: E[X] = np
Binomial variance: Var(X) = np(1-p)
Binomial standard deviation: SD(X) = sqrt(np(1-p))
CDF from PDF: F(x) = integral from negative infinity to x of f(t) dt
PDF from CDF: f(x) = dF(x)/dx
Joint PDF (independent): f(x,y) = f_X(x) times f_Y(y)
Marginal PDF: f_X(x) = integral of f(x,y) dy
Independence property: E[XY] = E[X]E[Y]
Students often assume that disjoint events are independent. They are the opposite: if A and B are disjoint and both have non-zero probability, knowing A occurred means B definitely did not, making them highly dependent.
Students sometimes think the PDF of a continuous variable cannot exceed 1. It can. Only the total area under the PDF must equal 1; the height at any single point is unconstrained.
Students confuse the direction of the CDF-PDF relationship. To go from CDF to PDF you differentiate (not integrate). To go from PDF to CDF you integrate.
Students read P(A|B) as "the probability of A and B" rather than "the probability of A given B." The conditioning bar restricts the sample space to B.
Bayes' theorem appears as a short-answer calculation problem. Practise the full setup: identify priors, compute total probability, then apply the formula.
The binomial distribution is tested both conceptually (which formula gives the variance?) and computationally (calculate P(X = k) for given n, p, k).
CDF properties (non-decreasing, right-continuous) appear in true/false questions. The CDF is always non-decreasing; this is a defining property.
Expect one question on joint density: either finding k to make the double integral equal 1, or computing a marginal PDF.
The F-distribution is used for comparing two variances; the chi-square is for testing one variance. Do not mix them up.
True or false: if two events are disjoint, they must be independent. (False)
True or false: a PDF can be greater than 1 at a specific point. (True)
True or false: E[X] for a binomial random variable is np. (True)
True or false: to find the PDF from the CDF, you must integrate the CDF. (False, you differentiate)
Fill in the blank: the variance of a Binomial(n, p) random variable is ____. (np(1-p))
Q: If A and B are independent events, which must be true: P(A or B) = P(A) + P(B), P(A and B) = P(A)P(B), P(A|B) = P(B), or P(A and B) = 0?
A: P(A and B) = P(A)P(B). This is the definition of independence.
Q: How many ways can a committee of 3 be chosen from 10 candidates if order does not matter?
A: C(10,3) = 120.
Q: For a Binomial distribution with n trials and success probability p, what is the variance?
A: np(1-p). The standard deviation is the square root of that: sqrt(np(1-p)).
Q: The CDF of any random variable is always ____.
A: Non-decreasing (and right-continuous).
Q: Given f(x,y) = k(x+y) for 0 <= x <= 1, 0 <= y <= 1, find k and the marginal f_X(x).
A: Setting the double integral to 1 gives k = 1. The marginal f_X(x) = x + 1/2 for 0 <= x <= 1.
Q: What distribution tests the ratio of two variances from independent normal populations?
A: The F-distribution.
Q: The law of total probability links what to what?
A: Conditional probabilities to marginal probabilities, by summing over a partition of the sample space.
Probability rules underpin every hypothesis test and confidence interval you will encounter later. Bayes' theorem reappears in diagnostic testing and decision analysis. The binomial distribution is the gateway to the normal approximation (via the Central Limit Theorem). Joint and marginal densities extend to covariance, correlation and regression.
Related Terms / Search Tags
independence, mutually exclusive, disjoint events, conditional probability, Bayes theorem, law of total probability, combination, permutation, binomial distribution, binomial PMF, PDF, CDF, cumulative distribution function, probability density function, joint density, marginal density, expected value, variance, F-distribution, chi-square, NYU analytical statistics midterm