20Latent Variables: Confirmatory Factor Analysis and SEM
Every model so far has related variables we can see - a test score, a reaction time, a GPA. But the things social scientists most want to study cannot be seen at all. Nobody has ever directly observed depression, curiosity, or intelligence. We observe only their shadows: answers to questionnaire items, behaviors, symptoms. A construct is latent - inferred from many imperfect indicators, never measured directly.
Structural Equation Modeling (SEM) is the framework built for exactly this: it models latent variables and the relationships among them. Its measurement half - Confirmatory Factor Analysis (CFA) - is how you establish that a set of indicators really does reflect one underlying construct. This is home turf for the authors, whose published work leans heavily on factor-analytic and covariance-structure methods; this chapter shows the machinery underneath.
20.1 Learning Objectives
State the reflective measurement model: a latent factor as the common cause of its indicators.
Fit and read a factor analysis, and understand loadings and uniquenesses.
See that fitting a factor model means matching a model-implied covariance matrix to the observed one.
Understand how SEM disattenuates relationships by modeling measurement error - and where it connects to reliability.
20.2 The Reflective Measurement Model
Why do the items on a good depression scale correlate with one another? The reflective model’s answer is causal: they share a common cause - the latent factor. Depression makes you sadder, and less energetic, and more pessimistic, so those items rise and fall together. The factor is the reason for the correlations; each indicator is a noisy reflection of it:
\[x_i = \lambda_i \, \xi + \varepsilon_i\]
where \(\xi\) is the latent factor, \(\lambda_i\) is the loading (how strongly item \(i\) reflects the factor), and \(\varepsilon_i\) is that item’s unique part (its reliability gap plus whatever else is specific to it). This is just the regression idea from earlier chapters - but the predictor is unobserved.
20.3 Factor Analysis, Executable
R’s base factanal() fits a factor model by maximum likelihood - no extra packages. Build eight items secretly generated by two correlated factors (four indicators each), and watch the analysis recover the structure blind:
* ML factor analysis as a CFA proxy; a true CFA uses AMOS or the CFA/OMS approach.
FACTOR
/VARIABLES a1 a2 a3 a4 b1 b2 b3 b4
/EXTRACTION ML
/CRITERIA FACTORS(2)
/ROTATION PROMAX.
usingMultivariateStats# ML factor analysis on the eight items (variables in columns)items =Matrix(df[:, [:a1, :a2, :a3, :a4, :b1, :b2, :b3, :b4]])fit(FactorAnalysis, items'; maxoutdim =2)
The pattern is unmistakable: a1–a4 load on one factor (~0.75), b1–b4 on the other, and the cross-loadings are essentially zero. The analysis reconstructed the hidden blueprint from nothing but the correlations. And the two factors are themselves related, just as we built them:
round(fa$correlation[1:2, 1:2], 2) # estimated correlation between the two factors
a1 a2
a1 1.00 0.58
a2 0.58 1.00
20.4 What “Fit” Actually Means: The Model-Implied Covariance
This is the engine of all of SEM, and it is simpler than its reputation suggests. A factor model does not just describe the data - it predicts the entire covariance matrix. If each item is \(\lambda_i \xi + \varepsilon_i\), then the model implies that the correlation matrix must equal \(\Lambda\Lambda^{\top} + \Psi\) (loadings times loadings, plus the unique variances on the diagonal). Fitting the model means finding loadings that make this implied matrix match the observed one. Watch it work on a single-factor block:
fa1 <-factanal(X[, 1:4], factors =1)lambda <-as.numeric(fa1$loadings) # the four loadingspsi <- fa1$uniquenesses # each item's unique varianceimplied <- lambda %*%t(lambda) +diag(psi) # the covariance the model PREDICTSmax(abs(cor(X[, 1:4]) - implied)) # how far off is the prediction?
[1] 0.02021382
The largest discrepancy between the observed correlations and the ones the model predicts is about 0.02 - a near-perfect reproduction. That gap, summarized across the whole matrix, is exactly what fit indices measure. A model that reproduces the observed covariances well fits well; one that cannot is rejected. This is why CFA is confirmatory: you specify the structure, and the data get to say whether that structure could have produced the correlations you actually see.
ImportantEFA explores, CFA confirms - and fit has thresholds
Exploratory factor analysis asks “how many factors, and what loads where?” - it discovers structure. CFA fixes the structure in advance (item 1 loads on factor A, not B) and tests it, ideally on a fresh sample. You judge the model-implied-vs-observed gap against conventional standards (Hu & Bentler): CFI/TLI ≥ .95, RMSEA ≤ .06, SRMR ≤ .08. Good fit is the model earning the right to be believed.
20.5 From Measurement to Structure
CFA is the measurement half. The “structural” in SEM is the other half: once you have clean latent variables, you can regress them on each other - does latent curiosity predict latent well-being? - with the measurement model and the structural model estimated together. For that you reach for a dedicated engine like lavaan:
# A full SEM: measurement model + structural paths (needs lavaan — shown, not run here)library(lavaan)model <-' # measurement model (CFA) curiosity =~ a1 + a2 + a3 + a4 wellbeing =~ b1 + b2 + b3 + b4 # structural model (a regression among latent variables) wellbeing ~ curiosity'fit <-sem(model, data = X)summary(fit, fit.measures =TRUE, standardized =TRUE)
The path wellbeing ~ curiosity is an ordinary regression - but between two variables neither of which was ever observed, each purified of its measurement error first.
20.6 Why Bother? SEM Fixes Attenuation
Recall from the reliability chapter that unreliable measures attenuate correlations - a real relationship looks weaker than it is because the sum scores are noisy. Regressing one sum score on another inherits that bias; the slope is pulled toward zero by measurement error. SEM’s real strength is that it separates the true score from the error before estimating the relationship. The regression among latent variables is disattenuated - it estimates the relationship between the constructs themselves, not between their noisy stand-ins. That is the reason SEM is worth its added complexity: it gives you the relationship you actually wanted, the one between the constructs, not between their noisy measures.
NoteFrom the Field
The authors’ factor-analytic and covariance-structure work runs throughout their research - the political-curiosity measure was built with exactly this EFA→CFA→invariance pipeline. And Voelkle and McKnight (Voelkle and McKnight 2012) showed by Monte-Carlo simulation that classic repeated-measures (M)ANOVA is itself a latent curve model in disguise - the same “everything is one model” lesson from the GLM chapter, pushed all the way into structural models. The methods showcase lists more.
20.7 Causal Structure: SEM Is a DAG With Numbers
SEM is often used to make causal claims: a structural path from curiosity to well-being looks like an arrow of cause. But an SEM is only as causal as the assumptions behind its arrows, and those assumptions deserve to be made explicit. That is exactly what a directed acyclic graph (DAG) does. A DAG is the same causal object as an SEM path diagram, drawn before you attach any numbers - the arrows are identical, only the notation differs. An SEM is a DAG with its arrows estimated. Drawing the DAG first is how you decide which variables to control (confounders) and which to leave alone (mediators, and above all colliders, which create spurious associations the moment you condition on them). Fitting the SEM never verifies the causal story; it estimates coefficients given the structure you assumed. We develop DAGs, the four elemental confounds, and the back-door criterion in the causal inference chapter. The workflow is always the same two steps: draw the graph to choose the structure, then fit the model to put numbers on it.
20.8 What This Chapter Teaches
Constructs are latent; we model them through multiple imperfect indicators, not a single score.
A factor is a common cause of its indicators - which is why they correlate, and why the loadings mean something.
Fitting = matching a model-implied covariance (\(\Lambda\Lambda^{\top}+\Psi\)) to the observed one. Fit indices measure the gap.
SEM disattenuates: by modeling measurement error, it estimates relationships among constructs, correcting the bias that sinks sum-score regression.
Factor models find dimensions along which people vary. But a factor model raises an immediate question before you use it to compare groups: does it mean the same thing for everyone? The next chapter tests exactly that - measurement invariance across groups - and shows how a group difference can be an artifact of the ruler rather than a real difference in the trait.
Voelkle, Manuel C., and Patrick E. McKnight. 2012. “One Size Fits All? A Monte-Carlo Simulation on the Relationship Between Repeated-Measures (m)ANOVA and Latent Curve Modeling.”Methodology 8 (1): 23–38.