The General Linear Model (GLM) is a statistical model that is used to model the relationship between a single dependent variable and one or more independent variables. Multiple regression and ANOVA are specific applications of the GLM. In fact, the t-test, correlation, regression, ANOVA, and ANCOVA are really one technique seen from different angles. Learn the one, and the rest come with it.
13.1 Learning Objectives
State the General Linear Model in one equation.
See that covariance is what drives it.
Watch a t-test, an ANOVA, and a regression return the same answer.
Understand what “general” does and does not mean.
13.2 One Equation to Rule Them All
Strip away the vocabulary and every model in this section is:
An outcome \(Y\) is modeled as a weighted sum of predictors plus error \(e\). That is it. Whether the \(X\)’s are continuous (regression), categorical (ANOVA), or a mix (ANCOVA) changes only how we code them - never the underlying machine. And, as we saw in the covariance chapter, the weights themselves are built from covariances and variances. Covariance is the fuel; the GLM is the engine.
13.3 The Same Answer, Three Ways
Let’s take one simple question, do two groups differ on an outcome, and answer it with three “different” methods. Watch the p-value:
NoteWorking in SPSS, Julia, or Python?
The code tabs below assume this chapter’s data is already loaded. Grab the one-file setup for your language from Getting the Book’s Data, run it once, then load what you need by name - this chapter uses ch12-groups. For example, book_data("ch12-groups") in R, Julia, or Python, or !bookdata name = "ch12-groups". in SPSS. Every language reads the same shipped files, so your numbers will match the ones printed here exactly.
# not merely close when rounded - identical to full machine precision:all.equal(p_ttest, p_anova) &&all.equal(p_ttest, p_regr)
[1] TRUE
INSERT FILE='data/sim/ch12-groups.sps'.
* Same question, three ways: t-test, ANOVA, regression.
T-TEST GROUPS=group('control' 'treatment') /VARIABLES=y.
ONEWAY y BY group /STATISTICS DESCRIPTIVES.
REGRESSION /STATISTICS COEFF R ANOVA /DEPENDENT y /METHOD=ENTER group.
Group Statistics
+-----------+--+----+--------------+---------+
| Group | N|Mean|Std. Deviation|S.E. Mean|
+-----------+--+----+--------------+---------+
|y control |40|-.12| .89| .14|
| treatment|40| .65| .90| .14|
+-----------+--+----+--------------+---------+
Independent Samples Test
+-----------+----------+------------------------------------------------------+
| | Levene's | |
| | Test for | |
| | Equality | |
| | of | |
| | Variances| T-Test for Equality of Means |
| +----+-----+-----+-----+--------+----------+----------+-----------+
| | | | | | | | | 95% |
| | | | | | | | | Confidence|
| | | | | | | | |Interval of|
| | | | | | | | | the |
| | | | | | | | | Difference|
| | | | | |Sig. (2-| Mean |Std. Error+-----+-----+
| | F | Sig.| t | df | tailed)|Difference|Difference|Lower|Upper|
+-----------+----+-----+-----+-----+--------+----------+----------+-----+-----+
|y Equal | .03| .859|-3.83|78.00| .000| -.77| .20|-1.16| -.37|
| variances| | | | | | | | | |
| assumed | | | | | | | | | |
| Equal | | |-3.83|78.00| .000| -.77| .20|-1.16| -.37|
| variances| | | | | | | | | |
| not | | | | | | | | | |
| assumed | | | | | | | | | |
+-----------+----+-----+-----+-----+--------+----------+----------+-----+-----+
Descriptives
+-----------+--+----+-----------+-------+---------------------+-------+-------+
| | | | | | 95% Confidence | | |
| | | | | | Interval for Mean | | |
| | | | | +----------+----------+ | |
| | | | Std. | Std. | Lower | Upper | | |
| group | N|Mean| Deviation | Error | Bound | Bound |Minimum|Maximum|
+-----------+--+----+-----------+-------+----------+----------+-------+-------+
|y control |40|-.12| .89| .14| -.40| .17| -2.00| 2.07|
| treatment|40| .65| .90| .14| .36| .94| -1.55| 2.62|
| Total |80| .27| .97| .11| .05| .48| -2.00| 2.62|
+-----------+--+----+-----------+-------+----------+----------+-------+-------+
ANOVA
+----------------+--------------+--+-----------+-----+----+
| |Sum of Squares|df|Mean Square| F |Sig.|
+----------------+--------------+--+-----------+-----+----+
|y Between Groups| 11.71| 1| 11.71|14.65|.000|
| Within Groups | 62.35|78| .80| | |
| Total | 74.06|79| | | |
+----------------+--------------+--+-----------+-----+----+
/tmp/RtmpoBLMPG/file32bb31fb5115.sps:5.65-5.69: warning: REGRESSION: group is
not a numeric variable. It will not be included in the variable list.
5 | REGRESSION /STATISTICS COEFF R ANOVA /DEPENDENT y /METHOD=ENTER group.
| ^~~~~
Two sample t-test (equal variance)
----------------------------------
Population details:
parameter of interest: Mean difference
value under h_0: 0
point estimate: -0.765134
95% confidence interval: (-1.163, -0.3671)
Test summary:
outcome with 95% confidence: reject h_0
two-sided p-value: 0.0003
Details:
number of observations: [40,40]
t-statistic: -3.8270867244982787
degrees of freedom: 78
empirical standard error: 0.19992590277277825
F-test against the null model:
F-statistic: 14.65 on 80 observations and 1 degrees of freedom, p-value: 0.0003
──────────────────────────────────────────────────────────────────────────────
Coef. Std. Error t Pr(>|t|) Lower 95% Upper 95%
──────────────────────────────────────────────────────────────────────────────
(Intercept) -0.116769 0.141369 -0.83 0.4113 -0.398213 0.164675
group: treatment 0.765134 0.199926 3.83 0.0003 0.367112 1.16316
──────────────────────────────────────────────────────────────────────────────
import pandas as pddf = pd.read_csv("data/sim/ch12-groups.csv")from scipy import statsimport statsmodels.formula.api as smffrom statsmodels.stats.anova import anova_lmstats.ttest_ind(df.y[df.group =="control"], df.y[df.group =="treatment"])fit = smf.ols("y ~ group", data=df).fit()anova_lm(fit) # the ANOVAfit.summary() # the regression
TtestResult(statistic=np.float64(-3.8270867244982787), pvalue=np.float64(0.00026008567897510494), df=np.float64(78.0))
df sum_sq mean_sq F PR(>F)
group 1.0 11.708594 11.708594 14.646593 0.00026
Residual 78.0 62.353772 0.799407 NaN NaN
<class 'statsmodels.iolib.summary.Summary'>
"""
OLS Regression Results
==============================================================================
Dep. Variable: y R-squared: 0.158
Model: OLS Adj. R-squared: 0.147
Method: Least Squares F-statistic: 14.65
Date: Tue, 08 Sep 2026 Prob (F-statistic): 0.000260
Time: 21:28:34 Log-Likelihood: -103.55
No. Observations: 80 AIC: 211.1
Df Residuals: 78 BIC: 215.9
Df Model: 1
Covariance Type: nonrobust
======================================================================================
coef std err t P>|t| [0.025 0.975]
--------------------------------------------------------------------------------------
Intercept -0.1168 0.141 -0.826 0.411 -0.398 0.165
group[T.treatment] 0.7651 0.200 3.827 0.000 0.367 1.163
==============================================================================
Omnibus: 1.034 Durbin-Watson: 2.012
Prob(Omnibus): 0.596 Jarque-Bera (JB): 0.596
Skew: 0.191 Prob(JB): 0.742
Kurtosis: 3.182 Cond. No. 2.62
==============================================================================
Notes:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
"""
Three identical p-values. The t-test, the ANOVA, and the regression are literally the same model here - the t-test is a two-group ANOVA, the two-group ANOVA is a regression with one dummy-coded predictor, and (as a bonus) \(F = t^2\):
t_stat <-t.test(y ~ group, data = d, var.equal =TRUE) |>tidy() |>pull(statistic)F_stat <-aov(y ~ group, data = d) |>tidy() |>slice(1) |>pull(statistic)tibble(t_squared = t_stat^2, F = F_stat) |>round2() # F = t^2
t_squared
F
14.65
14.65
Once you see this, a huge amount of statistics stops being a pile of disconnected recipes and becomes a single, learnable idea.
13.4 What “General” Means (and What It Doesn’t)
The GLM is general because it swallows so many named procedures. But it does carry assumptions - chiefly that the errors are roughly normal and that the relationship is linear in the parameters. When the outcome is not continuous-and-normal (a yes/no, a count), we need the generalized linear model, logistic and Poisson regression, which we met briefly at the end of the ANOVA & GLM chapter. Do not confuse the two: the general linear model is the unifying frame for this whole section; the generalized linear model extends it to other kinds of outcomes.
ImportantThe Roadmap From Here
Everything remaining in this book is a special case of the equation above:
Basic ANOVA - partitioning variance, the classical view.
Same model, different predictors. Keep the one equation in your head and you will not get lost.
13.5 Challenge
TipDo One Yourself
Generate your own two-group data. Confirm that t.test(..., var.equal = TRUE), aov(), and lm() give the identical p-value, and that \(F = t^2\).
Add a continuous covariate to the outcome and fit lm(y ~ group + covariate). You have just run an ANCOVA without learning a new technique - explain to a friend why that is not cheating.
In one sentence, state the difference between the general linear model and the generalized linear model.
13.6 Where We Go Next
Let’s build the bridge that makes the “categorical predictors are just regression” claim concrete, using the simplest possible categorical predictor - a single yes/no. That is the point-biserial correlation.