Source: Principles of Statistics I, Texas A&M University
Difficulty: Intermediate | Prerequisites: Basic probability, random variables, sampling distributions, confidence intervals
Tags: dependent data, paired data, longitudinal data, time-series data, spatial data, covariance, correlation structures, independence, statistical dependence, Cov(U V), autoregressive, Gaussian correlation, paired t-test, linear combinations
Most of introductory statistics assumes your data points are independent of each other. This topic is about what happens when they are not, and why it matters. Dependence between observations crops up constantly in practice: repeated measurements on the same person, data collected over time, or readings taken at nearby locations. If you ignore it, your confidence intervals, hypothesis tests, and variance estimates can all go wrong. You should already be comfortable with basic probability rules, expected value, variance, and the mechanics of a two-sample t-test before working through this material.
When data points are related to each other (paired, repeated, spatial, or temporal), standard statistical methods that assume independence can produce misleading results. Recognising and modelling dependence, using tools like covariance and correlation structures, is essential for valid inference.
Dependent data
Observations that are statistically related to one another, so the value of one data point gives information about another. Arises in paired, longitudinal, time-series, and spatial settings. In simple terms, this means your data points are not free-standing: knowing one tells you something about another.
Independence (of random variables)
Two random variables U and V are independent if knowing the value of one provides no information about the other. Formally, their joint distribution equals the product of their marginals. Think of it as: learning V happened tells you absolutely nothing new about U.
Covariance, Cov(U, V)
A numerical measure of how two random variables move together. Positive covariance means they tend to increase together; negative means one tends to decrease when the other increases; zero means no linear association. In simple terms, this means it quantifies the direction and rough strength of the linear link between two variables.
Uncorrelatedness
Two variables are uncorrelated when their covariance is zero. This is weaker than independence: independent variables are always uncorrelated, but uncorrelated variables can still be dependent. Think of it as: zero covariance rules out a linear relationship, but a non-linear one could still be lurking.
Paired data
Data collected so that each observation in one group is naturally matched with exactly one observation in the other group, often because both measurements come from the same subject. In simple terms, this means each data point has a partner, and the pair share something in common (the same person, the same location, etc.).
Correlation structure
A mathematical model describing the pattern of correlations among a set of observations. Different structures suit different data types. Think of it as: a template that tells you which observations in your dataset are likely to be correlated, and by how much.
Gaussian (spatial) correlation structure
A correlation model where the strength of association between data points decreases as the physical distance between them increases, following a Gaussian (bell-curve-shaped) decay. In simple terms, this means nearby measurements are strongly correlated; far-apart ones are nearly independent.
Autoregressive (AR) correlation structure
A correlation model for ordered data (usually time series) where each observation depends most strongly on its immediate predecessor, and the dependence decays exponentially with lag. In simple terms, this means today's value is most closely tied to yesterday's, less to the day before, and so on.
Population parameter θ (theta)
The true, fixed quantity a researcher is trying to estimate, such as a population mean or proportion. The entire point of sampling and inference is to say something credible about θ. Think of it as: the answer you would get if you could measure the whole population, which you almost never can.
The standard statistical framework assumes a representative sample of n independent observations drawn from a population.
Independence simplifies variance calculations, justifies the standard errors behind confidence intervals, and underpins most common hypothesis tests.
When that assumption fails, the machinery still runs, but it can give you the wrong answer.
If U and V are independent, then Cov(U, V) = 0. The converse is not guaranteed.
Independence implies uncorrelatedness, but uncorrelatedness does not imply independence. This is a one-way street.
The distinction matters because some dependence is non-linear and will not show up in covariance at all.
When U and V are independent, the conditional distribution of U given V equals the marginal distribution of U. Knowing V changes nothing about U.
When dependence exists, conditional probabilities and conditional expectations shift, and failing to account for this leads to incorrect inferences.
For independent variables: Var(aU + bV) = a²Var(U) + b²Var(V).
For dependent variables, a cross-term appears: Var(aU + bV) = a²Var(U) + b²Var(V) + 2ab·Cov(U, V).
Ignoring the covariance term when it is non-zero means your variance estimate (and therefore your standard errors, confidence intervals, and test statistics) will be wrong.
In a paired design, each subject receives two treatments, creating a natural correlation between the two measurements.
The conventional two-sample t-based confidence interval assumes independence between the groups, which is violated here.
Using the wrong method can inflate the width of confidence intervals, reducing the precision of your estimate.
The correct approach is a paired t-test (or paired confidence interval), which works with the differences within each pair and accounts for the within-subject correlation.
Gaussian structure: suited for spatial data, where correlation decays with physical distance.
Autoregressive (AR) structure: suited for time-series or distance-ordered data, where correlation decays with lag or separation.
Choosing the right structure matters because it determines how the model accounts for dependence among observations.
Covariance definition:
Cov(U, V) = E[(U − E[U])(V − E[V])]
Variance of a linear combination (general case):
Var(aU + bV) = a²Var(U) + b²Var(V) + 2ab·Cov(U, V)
Independence special case:
If U, V independent → Cov(U, V) = 0, so the cross-term drops out.
Orthodontic measurements over time: repeated measurements of the same child's jaw development are inherently correlated across ages, making longitudinal models necessary.
Brick field pollution in Bangladesh: environmental and health data collected near brick kilns exhibit spatial dependence; measurements at nearby locations are more similar than those far apart.
Pesticide drift: the concentration of pesticide residue at different distances from the application site follows a spatial pattern, and treating those measurements as independent would understate uncertainty.
"Uncorrelated means independent." It does not. Zero covariance rules out linear dependence only. Two variables can be uncorrelated yet strongly dependent through a non-linear relationship.
"Paired data can just be analysed as two independent samples." This ignores the within-subject correlation and typically gives wider confidence intervals or the wrong test statistic. Use a paired method.
"Dependence only matters in time-series data." Spatial data, paired experiments, and longitudinal studies all introduce dependence. It is not exclusive to time series.
"If I have a large sample, dependence does not matter." Sample size does not fix a violated independence assumption. The structure of the dependence still biases standard errors regardless of n.
⚠️ You will likely be asked to identify when data are dependent and explain why a standard (independent-samples) method is inappropriate in that setting.
⚠️ The distinction between independence and uncorrelatedness is a classic exam question. Know the direction of the implication and be ready to give or recognise a counterexample.
⚠️ Be prepared to write the variance formula for a linear combination with the covariance cross-term included, and to explain what goes wrong when you leave it out.
⚠️ Paired data problems are common. Know when to use a paired t-test versus a two-sample t-test, and be able to explain why the choice matters.
True or false: If Cov(U, V) = 0, then U and V must be independent. False. Zero covariance means no linear association. Non-linear dependence can still exist.
Fill in the blank: Var(aU + bV) = a²Var(U) + b²Var(V) + _______ when U and V are dependent. 2ab·Cov(U, V)
True or false: A paired t-test is used when the two groups of data are collected from entirely different, unrelated subjects. False. A paired t-test is for data where each observation in one group is matched with one in the other (e.g., same subject measured twice).
Fill in the blank: An autoregressive correlation structure is typically used for _______ data. Time-series (or distance-ordered) data.
True or false: Independence always implies uncorrelatedness. True.
Q: What is dependent data, and why does it matter in statistical analysis?
A: Dependent data are observations that are statistically related to one another. It matters because most standard statistical methods assume independence; violating that assumption can produce incorrect standard errors, confidence intervals, and hypothesis test results.
Q: Explain the relationship between independence and uncorrelatedness. Which implies which?
A: Independence implies uncorrelatedness (if U and V are independent, Cov(U, V) = 0). The reverse does not hold: two variables can have zero covariance yet still be dependent, typically through a non-linear relationship.
Q: A researcher measures blood pressure before and after a drug treatment on the same 30 patients, then uses a two-sample t-test to compare the groups. What is wrong with this approach?
A: The data are paired (same patients measured twice), so the observations are not independent across the two groups. A paired t-test should be used instead, because it accounts for the within-subject correlation. Using a two-sample t-test here ignores that correlation and can produce wider confidence intervals or an incorrect test statistic.
Q: Write the formula for Var(aU + bV) when U and V are dependent. What happens to this formula when they are independent?
A: Var(aU + bV) = a²Var(U) + b²Var(V) + 2ab·Cov(U, V). When U and V are independent, Cov(U, V) = 0, so the cross-term drops out and the variance simplifies to a²Var(U) + b²Var(V).
Q: Name two types of correlation structure and the kind of data each is suited for.
A: Gaussian correlation structure, suited for spatial data (correlation decays with physical distance). Autoregressive (AR) correlation structure, suited for time-series data (correlation decays with temporal lag).
This material connects directly to hypothesis testing and confidence intervals, because the validity of those tools depends on whether the independence assumption holds. It also links to regression, where correlated residuals (e.g., in time-series regression) violate the standard OLS assumptions and require corrections such as generalised least squares. Understanding correlation structures here lays groundwork for mixed-effects models and repeated-measures ANOVA later in the course.
dependent data, independent data, data dependence, paired data, paired t-test, longitudinal data, time-series data, spatial data, covariance, Cov(U V), correlation, uncorrelated vs independent, correlation structure, Gaussian correlation, autoregressive correlation, AR(1), variance of linear combinations, cross-term, conditional probability, conditional expectation, population parameter theta, repeated measures, within-subject correlation, two-sample t-test vs paired t-test, STAT principles of statistics, Texas A&M