PDFs, Histograms, and Density Functions
Difficulty: Intermediate | Prerequisites: basic probability, discrete random variables, introductory calculus (integration).
Up to this point in Analytical Statistics, you have worked with discrete random variables, where outcomes are countable and each outcome carries its own probability. Continuous random variables are the next logical step: they model measurements (weights, lengths, incomes, time) that can land anywhere on an interval of the real line. Because the number of possible values is uncountable, the way we assign and calculate probabilities changes fundamentally. You need to be comfortable with basic integration before diving in, since probability is now computed as area under a curve rather than a sum of individual values.
A continuous random variable takes values across an entire interval, so the probability of hitting any single exact value is zero. Instead, we use a probability density function (PDF), where probability equals the area under the curve over an interval. The PDF must be non-negative everywhere and integrate to 1 over the entire real line.
Continuous random variable
A random variable X that can take on any value in some interval (bounded or unbounded) of the real line. The number of possible outcomes is uncountable.
In simple terms, this means X is a measurement, not a count. Height, weight, temperature, time.
Probability density function (PDF)
A non-negative function f_X(x) such that for any interval A in the real numbers, P(X in A) = the integral of f_X(t) dt over A. Must satisfy f_X(x) >= 0 for all x and the integral from negative infinity to positive infinity of f_X(x) = 1.
Think of it as the curve whose area gives you probability. The function itself is not a probability; the area under it is.
Histogram (density histogram)
A way of approximating a continuous distribution by grouping possible values into bins. The height of each bar is chosen so that the area of the bar equals the probability of landing in that bin.
In simple terms, this means chopping the number line into intervals and plotting how likely each interval is. As the bins get narrower, the histogram approaches the true PDF.
Continuous uniform distribution
A distribution where X is equally likely to fall anywhere in a fixed interval [a, b]. Written X ~ U[a, b]. Its PDF is f_X(x) = 1/(b - a) for a <= x <= b, and 0 otherwise.
Think of it as the "flat" distribution: every sub-interval of the same width has the same probability.
Data types that are measurements (weights, lengths, incomes) can take any value on an interval
The number of possible outcomes is uncountable, so assigning a positive probability to every single value would sum to more than 1
Consequence: for any continuous random variable X and any specific value x, P(X = x) = 0
This is not a flaw. It simply means probability lives in intervals, not at points
To approximate a continuous distribution, we "discretise" it by grouping values into bins
Instead of P(X = x), we look at P(X_0 <= X <= X_1), P(X_1 <= X <= X_2), and so on
For each bin [X_{i-1}, X_i], define a function h_n(x) = P(X_{i-1} <= X <= X_i) / (X_i - X_{i-1})
The area of each bar (height times width) equals the probability for that bin
As the number of bins increases and their widths shrink, the histogram converges to a smooth curve: the PDF
If X has a continuous distribution, its PDF f_X(x) is a non-negative function defined so that for any interval A contained in the real numbers:
P(X in A) = integral over A of f_X(t) dt
Two required properties:
Non-negativity: f_X(x) >= 0 for all x in the real numbers
Normalisation: the integral from negative infinity to positive infinity of f_X(x) = 1
Critical point: f_X(x) is not itself a probability. It can exceed 1 at specific values of x. Only the area under the curve over an interval gives a probability
Because P(X = x) = 0 for continuous distributions, the following are all identical: P(a < X < b) = P(a <= X < b) = P(a < X <= b) = P(a <= X <= b)
X ~ U[a, b] means X is equally likely to fall anywhere in the interval [a, b]
PDF: f_X(x) = 1/(b - a) for a <= x <= b, and 0 otherwise
The density is a flat, constant function, which is why it is called "uniform"
Example: if X ~ U[0, 10], then P(3 < X < 4) = integral from 3 to 4 of (1/10) dt = 1/10
Note: P(X = 3) = 0 = P(X = 4), so P(3 <= X <= 4) also equals 1/10
Suppose f_X(x) = ax^2 for 0 < x < 3, and 0 otherwise. What must a be?
The PDF must integrate to 1: integral from 0 to 3 of at^2 dt = a * [t^3 / 3] from 0 to 3 = a * 9 = 1, so a = 1/9
With a = 1/9, compute P(1 < X < 2): integral from 1 to 2 of (t^2 / 9) dt = (1/9) * [t^3 / 3] from 1 to 2 = (8/27 - 1/27) = 7/27
Compute P(1 < X): integral from 1 to 3 of (t^2 / 9) dt = (1/9) * [t^3 / 3] from 1 to 3 = (27/27 - 1/27) = 26/27
PDF definition
P(X in A) = integral over A of f_X(t) dt
Two PDF requirements
f_X(x) >= 0 for all x
Integral from negative infinity to positive infinity of f_X(x) dx = 1
Probability over an interval
P(a <= X <= b) = integral from a to b of f_X(t) dt
Continuous uniform PDF
f_X(x) = 1/(b - a) for a <= x <= b, and 0 otherwise. Notation: X ~ U[a, b]
Histogram bar height
h_n(x) = P(X_{i-1} <= X <= X_i) / (X_i - X_{i-1}) for x in the bin [X_{i-1}, X_i]
Continuous random variables model almost every physical measurement you encounter: the exact weight of a package, the precise time between arrivals at a service desk, the voltage reading from a sensor. Engineers use continuous distributions to design quality control systems; if a machine produces bolts whose diameter follows a continuous distribution, they calculate the probability that a bolt falls outside tolerance by integrating the PDF over the rejection region.
The uniform distribution specifically appears in simulation and random number generation. Pseudo-random number generators produce values from U[0, 1], which are then transformed into other distributions for Monte Carlo simulations, cryptographic protocols, and randomised algorithms.
Students often think f_X(x) is the probability that X equals x. It is not. f_X(x) is a density, not a probability. The probability comes from integrating the density over an interval. A PDF value can exceed 1 without violating any rules
Students sometimes believe P(X = x) = 0 means the event "X equals x" is impossible. It does not. It means the event has zero probability, which is a distinct concept for continuous distributions. The event can still occur; it just has no measurable probability mass
When computing probabilities for a continuous variable, students occasionally forget that strict and non-strict inequalities give the same answer. P(3 < X < 4) = P(3 <= X <= 4) when X is continuous. Worrying about whether the endpoint is included wastes time on exams
Students sometimes confuse the PDF with the CDF. The PDF gives density (area under it equals probability). The CDF gives accumulated probability up to a point. These are related by integration, but they are different functions with different interpretations
⚠️ You will almost certainly be asked to find a normalising constant (like the a = 1/9 example). Set the integral of the PDF equal to 1, solve for the unknown constant
⚠️ Computing P(a < X < b) by integrating the PDF over [a, b] is a staple exam question. Be fluent with it
⚠️ Know the uniform distribution cold: its PDF, how to compute interval probabilities, and why P(X = c) = 0
⚠️ Be prepared to explain why P(X = x) = 0 for a continuous variable. This appears as a conceptual short-answer question
⚠️ Histogram-to-PDF reasoning (why bar area, not bar height, represents probability) is a common multiple-choice or true/false target
True or false: If X is a continuous random variable, P(X = 5) could be 0.3. (False. P(X = any exact value) = 0 for continuous distributions.)
True or false: A PDF value f_X(x) can be greater than 1. (True. The density is not a probability; only the area under the curve must equal 1.)
Fill in the blank: For X ~ U[2, 8], the PDF equals ___ on the interval [2, 8]. (1/6)
True or false: P(1 < X < 3) and P(1 <= X <= 3) are different when X is continuous. (False. They are identical.)
Fill in the blank: The two requirements for a valid PDF are that it is ___ everywhere and integrates to ___ over the entire real line. (Non-negative; 1)
Q: Let f_X(x) = cx^3 for 0 < x < 2, and 0 otherwise. Find the value of c that makes this a valid PDF.
A: Integrate cx^3 from 0 to 2: c * [x^4 / 4] from 0 to 2 = c * 4 = 1, so c = 1/4.
Q: For the PDF above (c = 1/4), compute P(1 < X < 2).
A: Integral from 1 to 2 of (x^3 / 4) dx = (1/4) * [x^4 / 4] from 1 to 2 = (1/4)(16/4 - 1/4) = (1/4)(15/4) = 15/16.
Q: If X ~ U[3, 7], what is P(5 <= X <= 6)?
A: The PDF is 1/(7 - 3) = 1/4 on [3, 7]. P(5 <= X <= 6) = (6 - 5) * (1/4) = 1/4.
Q: Explain why, for a continuous random variable, P(X = 3) = 0 does not mean the event is impossible.
A: The value x = 3 is within the support of the distribution, so it can occur. However, a single point has zero width, and probability is area under the PDF curve. An interval of zero width has zero area, hence zero probability. "Zero probability" and "impossible" are not the same thing in continuous probability theory.
Q: A histogram divides the range [0, 10] into 5 equal bins. The probability of falling in the bin [2, 4] is 0.3. What is the height of that bar in the density histogram?
A: Height = probability / bin width = 0.3 / 2 = 0.15.
This material connects directly to the cumulative distribution function (CDF), which is covered in Part 2 of these notes. The CDF is obtained by integrating the PDF, so everything here is foundational for that next step.
The uniform distribution U[0, 1] is the building block for generating random numbers from any distribution, a technique you will encounter in simulation and computational statistics courses.
If you have studied discrete random variables, the shift from summation to integration is the single biggest conceptual leap. The logic is the same (probabilities must be non-negative and sum/integrate to 1), but the mechanics change entirely.
continuous random variable, probability density function, PDF, density function, continuous distribution, uniform distribution, U[a,b], histogram, density histogram, bin width, area under the curve, non-negative density, normalisation condition, integrate to 1, P(X=x)=0, continuous probability, measurement data, uncountable outcomes, interval probability, analytical statistics, NYU statistics