Source: Principles of Statistics I, Texas A&M University / Tamhane-Dunlop Ch. 2
Tags: location parameter, scale parameter, location-scale family, standardisation, Bernoulli, binomial, Poisson, geometric, negative binomial, hypergeometric, discrete uniform, discrete distributions
Many distributions belong to families indexed by parameters. When those parameters are location and/or scale parameters, you can standardise the RV to a single "standard" member of the family and look up all probabilities from one table. The common discrete distributions (Bernoulli, binomial, Poisson, geometric, negative binomial, hypergeometric, discrete uniform) are all interconnected through Bernoulli trials, sampling schemes, and limiting arguments.
Location parameter
A parameter θ in a family of PDFs such that the distribution of W = Y − θ does not depend on θ. The standard member has θ = 0.
Scale parameter
A parameter θ such that the distribution of W = Y/θ does not depend on θ. The standard member has θ = 1.
Location-scale parameters
Parameters (θ₁, θ₂) such that W = (Y − θ₁)/θ₂ has a distribution free of both θ₁ and θ₂. The standard member has θ₁ = 0, θ₂ = 1.
Shape parameter
A parameter that changes the form of the distribution (not just shifting or stretching it). Cannot be removed by standardisation.
Bernoulli RV
Takes values 0 (failure) or 1 (success) with P(X = 1) = p. Mean = p, Variance = p(1 − p).
Binomial RV
Number of successes in n independent Bernoulli trials with common probability p. Notation: B(n, p). Mean = np, Variance = np(1 − p).
Discrete uniform RV
Takes values 1, 2, …, N each with probability 1/N. Mean = (N + 1)/2, Variance = (N + 1)(N − 1)/12.
Geometric RV
Number of trials until the first success in i.i.d. Bernoulli trials. Special case of negative binomial with r = 1. Mean = 1/p, Variance = (1 − p)/p².
Negative binomial RV
Number of trials until the r-th success in i.i.d. Bernoulli trials. Possible values: r, r + 1, r + 2, … Mean = r(1 − p)/p (using the "number of failures" form) or r/p (total trials form).
Hypergeometric RV
Number of Type A items in a sample of n drawn without replacement from N items containing M of Type A. Mean = KM/N, Variance = KM(N − M)(N − K)/[N²(N − 1)].
Poisson RV
Number of events in a fixed period of time or region of space, under Poisson postulates. Mean = λ, Variance = λ.
Memoryless property
P[X > s + t | X > s] = P[X > t]. Among discrete distributions, only the geometric has this property.
A family of distributions is indexed by parameters. Three important types:
Location family
If W = Y − θ has a distribution that does not depend on θ, then θ is a location parameter. Shifting Y by θ produces the standard member.
Example: Y ~ N(θ, 1). Then W = Y − θ ~ N(0, 1). The PDF of W is (1/√2π) e^(−w²/2), free of θ.
Scale family
If W = Y/θ has a distribution that does not depend on θ, then θ is a scale parameter. Rescaling Y by θ produces the standard member.
Example: Y ~ Exp(λ). Then W = Y/λ ~ Exp(1). The PDF of W is e^(−w), free of λ.
Location-scale family
If W = (Y − θ₁)/θ₂ is distribution-free of both parameters, then (θ₁, θ₂) are location-scale parameters.
Example: Y ~ N(θ₁, θ₂²). Then W = (Y − θ₁)/θ₂ ~ N(0, 1), free of both.
Not every parameter is location or scale. Example: Y ~ N(6, θ²). Setting W = Y/θ gives a PDF that still depends on θ. So θ is a shape parameter here, not a scale parameter. It would be a scale parameter only if the mean were also zero.
Probability lookup from one table. For a location-scale family with standard CDF F*, the CDF of any member is:
F(y) = F*((y − θ₁)/θ₂)
This is precisely what you do with the standard normal table: convert to a z-score and look up F*(z).
Example: Y ~ N(5, 2.3²). P[Y ≤ 1.7] = P[Z ≤ (1.7 − 5)/2.3] = F*(−1.435) = pnorm(−1.435) ≈ 0.0756.
Quantile computation from one table. If Q_Y*(u) is the quantile of the standard member:
Q_Y(u) = θ₁ + θ₂ · Q_Y*(u)
Example: 95th percentile of N(5, 2.3²): Q(.95) = 5 + 2.3 × 1.645 = 8.7835.
Not always.
Normal: location = mean, scale = standard deviation (yes)
Cauchy: has location and scale parameters, but the mean and variance do not exist
Double exponential (Laplace): location = mean, but SD = √2 × scale
3-parameter Weibull: mean and SD are functions of all three parameters
Discrete uniform
Equal probability 1/N for values 1 through N
Used in odds computation, card games
Historical use: WWII German tank problem (estimating total tanks from consecutive serial numbers)
Bernoulli
Single trial with two outcomes: success (1) with probability p, failure (0) with probability 1 − p
Building block for binomial, geometric, negative binomial
Binomial
Fixed number of trials n, random number of successes
Each trial independent with same p
Also models sampling with replacement from a finite population (p = M/N)
PMF: P(X = k) = C(n, k) p^k (1 − p)^(n−k)
Negative binomial
Fixed number of successes r, random number of trials
Trials until the r-th success in i.i.d. Bernoulli trials
PMF: P(B = k) = C(k−1, r−1) p^r (1 − p)^(k−r), k = r, r+1, …
Geometric
Special case of negative binomial with r = 1
Trials until the first success
Has the memoryless property among discrete distributions
Hypergeometric
Sampling without replacement from a population of N items (M of Type A, N − M of Type B)
Draw n items; X = number of Type A in the sample
PMF: P(X = k) = C(M, k) C(N−M, n−k) / C(N, n)
Possible values: max(0, M − (N − n)) to min(n, M)
Poisson
Counts events in a fixed window of time or space
Three postulates: (1) P(one event in Δt) ≈ λΔt, (2) P(more than one event in Δt) ≈ 0, (3) events in non-overlapping intervals are independent
λ is the average number of events per unit time/space
PMF: P(Y = y) = e^(−λ) λ^y / y!, y = 0, 1, 2, …
Arises as the limit of Binomial(n, p) as n → ∞, p → 0, np → λ
Bernoulli is Binomial with n = 1
Binomial counts successes in n trials; Geometric and Negative Binomial count trials to reach successes
Geometric is Negative Binomial with r = 1
Hypergeometric is the "without replacement" version of the Binomial
Binomial approximates Hypergeometric when N/20 > n (population much larger than sample)
Poisson is the limit of Binomial as n → ∞ with np → λ
Standardisation (location-scale): W = (Y − θ₁) / θ₂
CDF via standard member: F(y) = F*((y − θ₁) / θ₂)
Quantile via standard member: Q_Y(u) = θ₁ + θ₂ · Q_Y*(u)
Binomial PMF: P(X = k) = C(n, k) p^k (1 − p)^(n−k)
Poisson PMF: P(Y = y) = e^(−λ) λ^y / y!
Hypergeometric PMF: P(X = k) = C(M, k) C(N−M, n−k) / C(N, n)
Negative Binomial PMF: P(B = k) = C(k−1, r−1) p^r (1−p)^(k−r)
Geometric PMF: P(X = x) = p(1−p)^(x−1), x = 1, 2, …
⚠️ Know how to test whether a parameter is location, scale, or shape. The method: form W, compute its PDF, and check whether the parameter cancels out.
⚠️ Location ≠ mean in general. The Cauchy distribution is the classic counterexample (mean does not exist, but location parameter is well-defined).
⚠️ Binomial = fixed trials, random successes. Negative binomial = fixed successes, random trials. This distinction is a common exam question.
⚠️ Hypergeometric is for sampling without replacement; Binomial is for sampling with replacement (or independent trials). If the population is much larger than the sample, Binomial approximates Hypergeometric.
⚠️ Poisson postulates: know all three. Independence of non-overlapping intervals is often the one students forget.
⚠️ "No model is correct, but many are useful" (George Box). Be prepared to discuss model selection and goodness-of-fit conceptually.
Q: Y has a N(6, θ²) distribution. Is θ a scale parameter?
A: No. Setting W = Y/θ gives a PDF that still depends on θ (because the mean 6 does not cancel). θ is a shape parameter in this family. It would be scale only if the mean were 0.
Q: In a series of independent Bernoulli trials with p = 0.3, what distribution describes the number of trials needed to observe the 4th success?
A: Negative binomial with r = 4 and p = 0.3.
Q: What is the Poisson distribution the limit of, and under what conditions?
A: The Poisson is the limit of the Binomial(n, pₙ) as n → ∞ and pₙ → 0, with npₙ → λ.
Q: A population has N = 200 items, M = 50 of Type A. You draw n = 10 without replacement. What distribution models the number of Type A items in your sample?
A: Hypergeometric with parameters N = 200, M = 50, n = 10.
Q: How do you find P[Y ≤ 1.7] for Y ~ N(5, 2.3²) using the standard normal table?
A: Standardise: z = (1.7 − 5)/2.3 = −1.435. Look up F*(−1.435) ≈ 0.0756.
location parameter, scale parameter, location-scale family, shape parameter, standardisation, z-score, standard normal, Bernoulli trial, binomial distribution, Poisson distribution, geometric distribution, negative binomial, hypergeometric, discrete uniform, sampling with replacement, sampling without replacement, German tank problem, Poisson postulates, Poisson limit theorem, memoryless property, STAT 301, principles of statistics