Source: Statistics – Key Concepts and Applications in Various Fields
Difficulty: Introductory
Prerequisites: Basic algebra. Familiarity with summation notation (Σ) is helpful but not essential.
Tags: statistics, descriptive statistics, inferential statistics, probability, sampling, data types, statistical models, mean, median, mode, standard deviation, hypothesis testing, confidence intervals, regression, correlation, Bayes' theorem, nominal ordinal interval ratio, linear regression, logistic regression, time series
Statistics sits at the foundation of nearly every quantitative course you will take, from econometrics to biostatistics to machine learning. It is the toolkit for turning raw data into conclusions you can defend. This set of notes covers the broad landscape: what the main branches of statistics are, how data is classified, and which models crop up most often. If you are starting a mathematical statistics module at university level, this is the lay of the land before the formal proofs begin.
Statistics is about collecting data, summarising it (descriptive), and drawing conclusions about larger populations from samples (inferential). Probability underpins the inferential side. The type of data you have (nominal, ordinal, interval, ratio) determines which tools are appropriate, and the most common modelling tools are linear regression, logistic regression, and time series analysis.
Mean (average)
The sum of all data points divided by the number of points. In simple terms, it is the "centre of gravity" of your dataset.
Median
The middle value when data points are arranged in order. Think of it as the value that splits your dataset into two equal halves, and unlike the mean it is not pulled around by extreme values.
Mode
The most frequently occurring value in the dataset. In simple terms, it is the answer to "which value shows up the most?"
Range
The difference between the highest and lowest values in a dataset. It gives you a quick, rough sense of how spread out the data is, though it tells you nothing about what happens between those two extremes.
Standard deviation
A measure of how spread out values are around the mean. Think of it as the average distance each data point sits from the centre. A small standard deviation means the data clusters tightly; a large one means it is dispersed.
Descriptive statistics
The branch of statistics concerned with summarising and organising data so it can be understood at a glance, using tools such as the mean, median, mode, range, and standard deviation.
Inferential statistics
The branch of statistics that uses sample data to draw conclusions or make predictions about a larger population. This is where hypothesis tests, confidence intervals, and regression live.
Hypothesis testing
A formal procedure for deciding whether sample data provides enough evidence to support a specific claim (hypothesis) about a population. In simple terms, it is a structured way of asking "could this result have happened by chance?"
Confidence interval
A range of values, calculated from sample data, that is likely to contain the true population parameter at a stated level of confidence (e.g. 95%). Think of it as a net around your estimate that expresses how precise that estimate is.
Regression analysis
A method for modelling the relationship between variables, typically to predict the value of one variable (the dependent variable) from one or more others (independent variables).
Correlation
A measure of the strength and direction of the linear relationship between two variables. Values range from −1 (perfect negative) to +1 (perfect positive), with 0 indicating no linear relationship.
Probability
A number between 0 and 1 representing the likelihood of an event occurring. It is the mathematical foundation of all inferential statistics.
Events and outcomes
The different possible results of a random experiment. An outcome is a single result; an event is a set of outcomes you are interested in.
Probability distribution
A function that describes how probabilities are spread across the possible values of a random variable. Common examples include the normal (bell curve) distribution and the binomial distribution.
Bayes' theorem
A rule for updating the probability of a hypothesis after observing new evidence. In simple terms, it tells you how to revise what you believed before in light of what you have just learned.
Random sampling
A sampling method where every member of the population has an equal chance of being selected. This is the gold standard for avoiding selection bias.
Stratified sampling
Dividing the population into meaningful subgroups (strata) and then sampling from each subgroup. This ensures every important segment of the population is represented.
Convenience sampling
Selecting whichever subjects are easiest to reach. It is fast and cheap, but prone to bias because the sample may not represent the population.
Nominal data
Categorical data with no inherent order. Colours, country names, and types of fruit are all nominal. You can count them but you cannot meaningfully rank or average them.
Ordinal data
Categorical data with a meaningful order, but where the gaps between categories are not necessarily equal. Satisfaction ratings (poor, fair, good, excellent) are a typical example.
Interval data
Numeric data with equal spacing between values but no true zero point. Temperature in Celsius is the classic case: the difference between 10 °C and 20 °C is the same as between 20 °C and 30 °C, but 0 °C does not mean "no temperature."
Ratio data
Numeric data with equal intervals and a true zero point. Height, weight, and income are ratio data. Zero means the absence of the quantity, and ratios are meaningful (someone who weighs 80 kg is twice as heavy as someone who weighs 40 kg).
Linear regression
A model that fits a straight line to the relationship between a dependent variable and one or more independent variables. It is the workhorse model of statistics and often the first one taught.
Logistic regression
A model used when the outcome is binary (yes/no, pass/fail). Instead of predicting a continuous value, it estimates the probability of an event occurring.
Time series analysis
Methods for analysing data points collected at regular time intervals. Stock prices, monthly sales figures, and daily temperatures are all time series.
The goal is to reduce a large dataset to a handful of numbers that capture its essential character.
Measures of central tendency (mean, median, mode) tell you where the data clusters.
Measures of spread (range, standard deviation) tell you how much variability there is.
The mean is sensitive to outliers; the median is resistant. Choosing between them depends on the shape of your distribution.
Example: household income data is typically right-skewed, so the median is a more representative "typical" value than the mean.
You almost never have data on the entire population, so you work with a sample and use probability theory to generalise.
Hypothesis testing provides a decision framework: set up a null hypothesis, collect data, and decide whether the evidence is strong enough to reject it.
Confidence intervals give you a range rather than a single point estimate, which is more informative and more honest about uncertainty.
Regression and correlation are the main tools for studying relationships between variables.
Correlation quantifies the relationship; regression lets you use one variable to predict another.
Correlation does not imply causation. Two variables can move together because of a third, unmeasured factor.
Every inferential method relies on probability to say how likely an observed result would be under certain assumptions.
A probability distribution maps every possible value of a random variable to its likelihood.
The normal distribution (bell curve) appears constantly because of the central limit theorem: the average of a large enough sample tends toward a normal distribution regardless of the population's shape.
The binomial distribution models the number of successes in a fixed number of independent trials (e.g. coin flips).
Bayes' theorem reverses the direction: rather than asking "given the model, how likely is this data?", it asks "given the data, how likely is this model?"
The quality of your inference depends directly on the quality of your sample.
Random sampling is the ideal because it removes systematic bias.
Stratified sampling improves precision when the population has known subgroups that differ from one another (e.g. age brackets, income bands).
Convenience sampling is common in practice (think: surveying students who happen to be in the cafeteria) but the results may not generalise.
The type of data you have constrains which statistical tools are valid.
The hierarchy runs: nominal → ordinal → interval → ratio, with each level inheriting the properties of the one below and adding a new one.
Nominal: categories only. You can compute a mode but not a mean.
Ordinal: categories with order. You can compute a median.
Interval: order plus equal spacing. You can compute a mean, but ratios are not meaningful.
Ratio: order, equal spacing, and a true zero. All arithmetic operations are meaningful.
Applying a tool designed for a higher level of data to a lower level (e.g. computing the mean of ordinal ratings) is a common source of misleading results.
A statistical model is a mathematical equation that describes how variables relate to one another, plus assumptions about the error.
Linear regression is the starting point. For a single predictor, the model is: ŷ = β₀ + β₁x.
Logistic regression wraps a linear model inside a logistic function to output probabilities between 0 and 1, which is what you need when the outcome is binary.
Time series models add a time dimension and account for trends, seasonality, and autocorrelation (the tendency of nearby time points to be similar).
Mean: x̄ = (Σxᵢ) / n
Standard deviation (population): σ = √[ Σ(xᵢ − μ)² / N ]
Standard deviation (sample): s = √[ Σ(xᵢ − x̄)² / (n − 1) ]
Note the denominator: N for a population, n − 1 for a sample (Bessel's correction).
Simple linear regression: ŷ = β₀ + β₁x
Bayes' theorem: P(A|B) = [ P(B|A) · P(A) ] / P(B)
Descriptive statistics appear on every dashboard and business report you will ever read. When a company says "average order value is £45," that is the mean in action.
Inferential statistics power clinical trials: researchers test whether a new drug performs better than a placebo by running a hypothesis test on sample data.
Probability distributions underpin insurance pricing, where actuaries model the likelihood and size of future claims.
Regression is how economists estimate the effect of education on earnings, how meteorologists build weather forecasts, and how tech companies predict user churn.
Sampling design determines whether a political poll is trustworthy or misleading.
Students often confuse the mean with the median and use them interchangeably. They measure different things, and in skewed data they can be far apart.
Correlation is frequently treated as evidence of causation. A strong correlation only means two variables move together; it says nothing about whether one causes the other.
A 95% confidence interval does not mean "there is a 95% probability that the true value lies in this range." It means that if you repeated the sampling process many times, 95% of the resulting intervals would contain the true value.
Students sometimes assume convenience sampling is acceptable for inferential work. It is quick, but results drawn from a convenience sample should be treated as suggestive, not conclusive.
The nominal/ordinal/interval/ratio hierarchy is often memorised without understanding its practical consequence: using the wrong analysis for your data type produces numbers that look precise but mean nothing.
⚠️ Be able to distinguish descriptive from inferential statistics and give an example of each.
⚠️ Know the difference between population parameters (μ, σ) and sample statistics (x̄, s), and why the sample standard deviation uses n − 1.
⚠️ Expect a question asking you to classify variables as nominal, ordinal, interval, or ratio. Have a clear example ready for each.
⚠️ Understand that correlation ≠ causation. This is tested in nearly every introductory statistics exam.
⚠️ Be able to state Bayes' theorem and describe what it does in plain language.
⚠️ Know the three sampling methods (random, stratified, convenience) and when each is appropriate.
True or false: The median is always equal to the mean.
Fill in the blank: A _______ interval gives a range of plausible values for a population parameter.
True or false: Ordinal data has equal spacing between categories.
Fill in the blank: In the sample standard deviation formula, we divide by _______ instead of n.
True or false: Logistic regression is used when the outcome variable is continuous.
Answers: 1. False (they are equal only in perfectly symmetrical distributions). 2. Confidence. 3. False (it has meaningful order but not necessarily equal spacing). 4. n − 1. 5. False (it is used for binary outcomes).
Q: What is the difference between descriptive and inferential statistics?
A: Descriptive statistics summarises and organises data (e.g. computing the mean or standard deviation). Inferential statistics uses sample data to draw conclusions about a larger population (e.g. hypothesis testing, confidence intervals).
Q: A researcher measures customer satisfaction on a scale of 1 to 5 (very dissatisfied to very satisfied). What type of data is this?
A: Ordinal. The categories have a meaningful order, but the intervals between them are not necessarily equal.
Q: Why does the sample standard deviation formula use n − 1 in the denominator rather than n?
A: Dividing by n − 1 (Bessel's correction) corrects for the bias that arises because a sample tends to underestimate the true population variance. It produces an unbiased estimate of the population variance.
Q: State Bayes' theorem and describe in one sentence what it does.
A: P(A|B) = [P(B|A) · P(A)] / P(B). It allows you to update the probability of a hypothesis (A) after observing new evidence (B).
Q: Give one advantage and one disadvantage of convenience sampling.
A: Advantage: it is quick and inexpensive to carry out. Disadvantage: it is prone to selection bias, so results may not be representative of the wider population.
Q: A variable records temperature in degrees Celsius. Is this interval or ratio data, and why?
A: Interval. The spacing between degrees is equal, but 0 °C does not represent the absence of temperature, so there is no true zero point, which rules out ratio.
Q: What does a correlation coefficient of −0.85 tell you?
A: It indicates a strong negative linear relationship between the two variables: as one increases, the other tends to decrease.
Probability theory connects directly to combinatorics and set theory, which you will encounter (or have encountered) in a discrete mathematics course.
Regression analysis is the gateway to machine learning. Linear and logistic regression are, respectively, the simplest supervised learning models for continuous and categorical outcomes.
Sampling and data types feed into research methods and experimental design courses, where you learn to plan studies that produce valid statistical conclusions.
statistics, descriptive statistics, inferential statistics, probability, sampling methods, data types, nominal ordinal interval ratio, mean, median, mode, range, standard deviation, variance, hypothesis testing, confidence interval, regression analysis, correlation, correlation coefficient, Bayes' theorem, posterior probability, random sampling, stratified sampling, convenience sampling, linear regression, logistic regression, time series analysis, population vs sample, parameter vs statistic, Bessel's correction, central tendency, measures of spread, variability, bell curve, normal distribution, binomial distribution, UCF, mathematical statistics