Statistics Concepts and Definitions Overview, ST101 – Study Notes
Source: ST101 Course Material, University of Minnesota Twin Cities
Tags: statistics, probability, ST101, intro to statistics, descriptive statistics, inferential statistics, standard deviation, binomial experiment, normal distribution, central limit theorem, PDF, cumulative relative frequency
Difficulty: Beginner
Prerequisites: Basic algebra. Familiarity with fractions, percentages and simple equations is helpful but not essential.
This material covers the foundational vocabulary and concepts for an introductory probability and statistics course. If you are new to the subject, this is where you start: it defines the building blocks (types of statistics, probability rules, key distributions) that every later topic in ST101 depends on. If you have missed early lectures, these notes will bring you up to speed on what the course assumes you already know. The concepts here reappear in hypothesis testing, regression and data analysis later in the term.
Statistics splits into two camps: descriptive (summarising data you have) and inferential (drawing conclusions about a larger group from a sample). The core toolkit includes standard deviation for measuring spread, probability rules for independent events and binomial experiments, the normal distribution as a modelling workhorse, and the central limit theorem, which explains why larger samples produce more reliable results.
Descriptive statistics
Methods used to summarise and organise data so it is easy to understand. Includes averages (mean), percentages and charts that describe a data set.
In simple terms, this means: the numbers and pictures you use to say "here is what the data looks like" before you try to draw any bigger conclusions.
Inferential statistics
Methods that use probability, hypothesis testing and confidence intervals to make predictions or draw conclusions about a larger group (population) based on a smaller group (sample).
Think of it as: using a spoonful of soup to judge the whole pot.
Cumulative relative frequency
Applied to an ordered set of observations from smallest to largest. The cumulative relative frequency for a given value is the sum of the relative frequencies for all values less than or equal to that value.
In simple terms, this means: for any data point, what proportion of the data set sits at that value or below it.
Standard deviation
A number equal to the square root of the variance. It measures how far data values are from their mean. Notation: s for the sample standard deviation, σ (sigma) for the population standard deviation.
Think of it as: a single number that tells you how spread out the data is. A small standard deviation means values cluster near the mean; a large one means they are more scattered.
Independent events
Two events are independent when the occurrence of one has no effect on the probability of the other occurring.
In simple terms, this means: knowing the outcome of one event tells you nothing new about the other. Flipping a coin and rolling a die are independent, because the coin result does not change the die probabilities.
Binomial experiment
A statistical experiment that satisfies three conditions:
There is a fixed number of trials, n.
Each trial has exactly two possible outcomes, labelled "success" (probability p) and "failure" (probability q = 1 − p).
The n trials are independent and conducted under identical conditions.
Think of it as: flipping a coin a set number of times and counting how many heads you get. Any repeated yes/no experiment with a constant probability fits this model.
Probability distribution function (PDF)
A mathematical description of a discrete random variable (RV), given either as an equation or as a table listing every possible outcome and its associated probability.
In simple terms, this means: the complete map of "what can happen" and "how likely is each outcome."
Standard normal distribution
A continuous random variable X that follows N(0, 1), meaning it has a mean of 0 and a standard deviation of 1. Often written as Z ~ N(0, 1).
Think of it as: the "default" bell curve. Any normal distribution can be converted to this one through a z-score transformation, which is why it turns up everywhere.
Central limit theorem (CLT)
As the sample size increases, the distribution of the sample mean approaches a normal distribution (bell shape), regardless of the shape of the original population distribution.
In simple terms, this means: take big enough samples and the averages will form a bell curve, even if the raw data does not. This is the reason so many statistical methods assume normality.
Descriptive statistics: summarising what you have
Purpose: organise raw data into something readable.
Tools: mean, median, mode, percentages, frequency tables, bar charts, histograms.
Output: a snapshot of the data set, nothing more. You are not making claims about anything outside the data you collected.
Example: "The average exam score for this class was 74%." That tells you about this class, not all classes.
Inferential statistics: drawing conclusions beyond the data
Purpose: use a sample to say something about a larger population.
Tools: probability, hypothesis tests, confidence intervals, regression.
Key requirement: the sample must be representative, otherwise your conclusions are unreliable.
Example: "Based on a poll of 1,000 voters, we estimate 52% of the electorate supports the measure, with a margin of error of ±3%." You are reaching beyond your sample to say something about millions of people.
How they relate
Descriptive statistics come first. You summarise and explore your data before running inferential tests.
Inferential statistics build on descriptive ones. The mean and standard deviation of your sample become inputs for hypothesis tests and confidence intervals.
Independent events
Two events A and B are independent if P(A | B) = P(A), meaning knowing B happened does not change A's probability.
The multiplication rule for independent events: P(A and B) = P(A) × P(B).
Classic example: rolling a 4 on a die and flipping heads on a coin. One result does not affect the other.
Contrast with dependent events: drawing two cards from a deck without replacement. The first draw changes the probabilities for the second.
Binomial experiments
Fixed number of trials (n), two outcomes per trial (success/failure), constant probability of success (p), and trials are independent.
The probability of getting exactly k successes in n trials is given by the binomial formula (see Formulas section).
Everyday example: a manufacturer tests 20 light bulbs off the line. Each either passes or fails. If the defect rate is constant and bulbs are tested independently, this is a binomial experiment.
Cumulative relative frequency
Order your data from smallest to largest.
For each value, add up the relative frequencies of all values at or below it.
The result tells you what proportion of the data falls at or below any given point.
Useful for answering questions like: "What percentage of students scored 70 or below?"
Probability distribution function (PDF)
Describes every possible outcome of a discrete random variable and the probability of each.
Can be expressed as a formula or as a table.
All probabilities must be between 0 and 1, and they must sum to 1.
Example: for a fair six-sided die, the PDF assigns probability 1/6 to each of the outcomes {1, 2, 3, 4, 5, 6}.
Standard normal distribution
A special case of the normal (bell curve) distribution with mean = 0 and standard deviation = 1.
Written as Z ~ N(0, 1).
Any normal distribution can be converted to the standard normal using z-scores: z = (x − μ) / σ.
Z-tables (or software) then give you the probability of being above, below or between any two values.
This is the reference curve against which all normal distributions are compared.
Central limit theorem (CLT)
Regardless of the shape of the underlying population, the distribution of sample means approaches a normal distribution as the sample size grows.
Rule of thumb: a sample size of n ≥ 30 is usually large enough for the CLT to apply.
Why it matters: the CLT is the reason so many statistical tests assume normality. Even if individual data points are skewed, the averages of those data points will be approximately normal for large samples.
Practical implication: the larger your sample, the tighter the sampling distribution clusters around the true population mean, which is why bigger samples give more reliable estimates.
Standard deviation (population)
σ = √[ Σ(xᵢ − μ)² / N ]
Where μ is the population mean and N is the population size.
Standard deviation (sample)
s = √[ Σ(xᵢ − x̄)² / (n − 1) ]
Where x̄ is the sample mean and n is the sample size. The denominator is n − 1 (not n) to correct for bias in estimating the population variance from a sample.
Binomial probability
P(X = k) = C(n, k) × p^k × q^(n − k)
Where C(n, k) = n! / [k!(n − k)!], p is the probability of success on one trial, and q = 1 − p.
Z-score transformation
z = (x − μ) / σ
Converts any value from a normal distribution to its equivalent on the standard normal distribution.
Key notation
Symbol | Meaning |
|---|---|
μ | Population mean |
x̄ | Sample mean |
σ | Population standard deviation |
s | Sample standard deviation |
n | Sample size or number of trials |
p | Probability of success (binomial) |
q | Probability of failure (= 1 − p) |
Z | Standard normal random variable |
Standard deviation is used in finance to measure the volatility of stock returns. A fund with a high standard deviation is a bumpier ride.
Binomial experiments underpin quality control in manufacturing. If a factory knows 2% of its widgets are defective, the binomial model predicts how many defectives to expect in a batch of 500.
The normal distribution is the basis for grading on a curve, setting insurance premiums and modelling measurement error in engineering.
The central limit theorem is why political polls work: even though individual voter preferences are not normally distributed, the average of a large random sample will be, allowing pollsters to calculate margins of error.
Inferential statistics drive clinical trials. Researchers test a drug on a sample of patients and use hypothesis tests to decide whether the effect is real or likely due to chance.
"Standard deviation and variance are the same thing." They are not. Variance is the average of the squared deviations from the mean. Standard deviation is the square root of the variance. They measure the same idea (spread), but in different units. Standard deviation is in the original units of the data, which is why it is used more often in practice.
"Independent events means the events cannot happen at the same time." That describes mutually exclusive events, not independent events. Independent events can and often do occur together; the point is that one does not affect the probability of the other.
"The central limit theorem says the data becomes normally distributed." It does not. The raw data keeps whatever shape it has. The CLT says the distribution of sample means approaches normality as the sample size increases.
"A PDF gives you the probability of a specific value for any random variable." For discrete random variables, yes. For continuous random variables, the PDF gives probability density, not probability. The probability of a continuous variable equalling any single exact value is technically zero; you need to integrate over an interval.
⚠️ Know the difference between descriptive and inferential statistics cold. Exam questions often present a scenario and ask you to classify the analysis as one or the other.
⚠️ Be able to identify whether a scenario fits the binomial model. Check all three conditions: fixed n, two outcomes, independent trials with constant p. If any condition fails, it is not binomial.
⚠️ Standard deviation questions frequently test whether you can distinguish between the population formula (divide by N) and the sample formula (divide by n − 1). Know which to use and why.
⚠️ Z-score calculations come up repeatedly. Practise converting raw scores to z-scores and looking up probabilities using a z-table.
⚠️ The central limit theorem is a favourite exam topic. Expect questions asking what happens to the sampling distribution as n increases, and why n ≥ 30 is the conventional threshold.
⚠️ Independent vs mutually exclusive events is a classic trap question. Be ready to explain the difference with an example.
True or false: Descriptive statistics can be used to make predictions about a population.
False. Descriptive statistics summarise the data you have. Predictions about a population require inferential statistics.
Fill in the blank: The standard deviation is the square root of the ______.
Variance.
True or false: If two events are independent, they cannot occur at the same time.
False. That describes mutually exclusive events. Independent events can occur together.
Fill in the blank: A binomial experiment requires a fixed number of trials, exactly ___ possible outcomes per trial, and independent trials with a constant probability of success.
Two.
True or false: The central limit theorem states that the raw data in a sample becomes normally distributed as the sample size increases.
False. The CLT says the distribution of sample means approaches normality, not the raw data itself.
Q: A researcher surveys 200 university students to estimate the average sleep hours of all students at the university. Is this an example of descriptive or inferential statistics?
A: Inferential statistics, because the researcher is using a sample (200 students) to draw a conclusion about a larger population (all students at the university).
Q: A coin is flipped 10 times. Each flip is independent, and the probability of heads is 0.5. Does this qualify as a binomial experiment? Why or why not?
A: Yes. It meets all three conditions: fixed number of trials (n = 10), two outcomes per trial (heads or tails), and independent trials with constant probability of success (p = 0.5).
Q: If the population mean is 80 and the population standard deviation is 5, what is the z-score for a data value of 92?
A: z = (92 − 80) / 5 = 12 / 5 = 2.4. The value 92 is 2.4 standard deviations above the mean.
Q: Explain in one or two sentences why the central limit theorem is important for hypothesis testing.
A: The CLT guarantees that the sampling distribution of the mean will be approximately normal for large samples, regardless of the population's shape. This allows us to use normal-distribution-based tests (like z-tests) even when the underlying data is not normally distributed.
Q: A data set of exam scores has a variance of 64. What is the standard deviation?
A: The standard deviation is √64 = 8.
Q: Give an example of two events that are mutually exclusive but not independent.
A: Rolling a 3 and rolling a 5 on a single die. They cannot both happen on the same roll (mutually exclusive), but they are not independent, because knowing one occurred tells you the other did not (the probability of the other drops to zero).
The descriptive vs inferential distinction runs through the entire course. Every later chapter builds on it: when you reach hypothesis testing and confidence intervals, you are doing inferential statistics using the descriptive measures (mean, standard deviation) you learned here.
The binomial distribution connects forward to the Poisson distribution (for rare events) and backward to basic counting rules and combinatorics. If you are comfortable with the binomial formula, the Poisson is a short step.
The central limit theorem is the bridge between probability theory and applied inference. Without it, most of the confidence interval and hypothesis testing machinery would not work, because those methods assume the sampling distribution is normal.
statistics definitions, ST101 study guide, intro to probability and statistics, descriptive vs inferential statistics, standard deviation formula, population vs sample standard deviation, independent events probability, mutually exclusive vs independent, binomial distribution, binomial probability formula, probability distribution function, discrete random variable, standard normal distribution, z-score, z-table, bell curve, normal distribution, central limit theorem, CLT, sampling distribution, cumulative relative frequency, variance vs standard deviation, University of Minnesota statistics