COLUMN | How statistics became econometricsWritten on: 26 October 2026
TITLE:

How Statistics Became Econometrics

SUBJECT:A short history of the ideas behind regression, testing and time-series models, from Bayes to Hamilton
AUTHOR:Muhammed İkbal Yılmaz

Every regression an economist runs today rests on a simple idea. What we observe is treated as a systematic part plus an error. One way to read the history of statistics and econometrics is as the history of that error term, of where it comes from, what shape it has and what happens to our conclusions when it behaves differently from what was assumed. Many of the tools used in this history were originally developed for other fields. Least squares was developed to compute the orbits of comets and planets, regression to study heredity, the t-test to serve a brewery and randomised experiments to improve agricultural trials. This piece tries to follow, in chronological order, how these tools were brought into economics and reshaped there.

Error and probability, 1763–1898

An early step was to treat uncertainty itself as something that can be calculated. In an essay published after his death by his friend Richard Price, Thomas Bayes asked how, after seeing an event happen a given number of times and fail a given number of times, one should assess the probability that its unknown chance lies between two limits (Bayes, 1763). This "inverse" question, reasoning from data back to the cause that produced them, is often seen as the starting point of Bayesian statistics.

The next step came from astronomy, where observations often produce more equations than unknowns and some error always remains (Legendre, 1805, p. 72). Adrien-Marie Legendre published the "method of least squares" in an appendix to a book on the orbits of comets. In this method the unknowns of a system of equations are chosen so that the sum of the squared errors is as small as possible (Legendre, 1805, pp. 72–73).1 Four years later Carl Friedrich Gauss gave the method a probabilistic justification. Assuming that observation errors follow what we now call the normal distribution, he showed that least squares gives the most probable values of the unknowns (Gauss, 1809). He obtained the error law itself from the assumption that the arithmetic mean of repeated observations is the most probable value (Gauss, 1809, p. 212).2 Gauss was already well known for an earlier success. In 1801 the newly discovered dwarf planet Ceres was lost in the glare of the Sun, and his orbit calculation allowed astronomers to find it again.

Least squares dealt with errors of measurement. Siméon Denis Poisson turned to a different question, namely how often rare events occur in a large number of trials. In a book on the probability of judicial decisions he derived the distribution that now carries his name (Poisson, 1837). A well-known application of this distribution came about sixty years later, when Ladislaus von Bortkiewicz showed that the yearly number of soldiers killed by horse kicks in Prussian army corps followed it closely (Bortkiewicz, 1898).

Heredity, correlation and small samples, 1886–1908

In the late nineteenth century these ideas moved from the sky to living populations, and the word "regression" itself comes from biology. Comparing the heights of parents and their adult children, Francis Galton found that the children of unusually tall or short parents were, on average, closer to the population average than their parents. He called this "regression towards mediocrity" (Galton, 1886). What Galton described as a law of heredity is today usually understood as a general statistical property of two imperfectly correlated variables, and the name stayed with the method.

Karl Pearson gave Galton's ideas a more formal mathematical treatment. In a long paper on regression and heredity he set out what he argued was the best value of the correlation coefficient, and noted that the same value had already been given by Bravais, but without a proof that it is the best (Pearson, 1896, p. 265).3 Four years later he introduced the chi-square test, which asks whether the differences between observed and expected frequencies are too large to be due to chance alone (Pearson, 1900, pp. 160, 163).4

These methods assumed large samples, while practical work often had only a few observations. William Sealy Gosset, who worked for the Guinness brewery in Dublin, often had to judge results from a small number of experiments. With so few observations the usual large-sample rules could become "altogether misleading" (Student, 1908, p. 2). Because the brewery restricted publication by its staff, he wrote under the pen name "Student". He derived the small-sample distribution of the distance between the sample mean and the population mean divided by the sample standard deviation (Student, 1908, pp. 3, 7).5 Today's t-statistic is a rescaled version of this quantity, and its distribution underlies the t-statistics reported in regression tables.

The foundations of inference, 1922–1935

By the 1920s the field had accumulated many separate results. Much of the common framework that later joined them is associated with Ronald A. Fisher, who worked at the Rothamsted agricultural research station. In one paper he separated the population, described by unknown parameters, from the sample used to estimate them, and introduced the ideas of consistency, efficiency, sufficiency and maximum likelihood (Fisher, 1922). His handbook for working scientists helped make significance tests a routine part of research (Fisher, 1925/1934), and his book on experiments argued that treatments should be assigned at random, so that differences between groups can be attributed to the treatment (Fisher, 1935/1937).

Fisher's significance tests can be read as asking how surprising the data are under a given hypothesis. Jerzy Neyman and Egon Pearson, Karl Pearson's son, approached testing from a different angle. They treated a test as a decision rule that can go wrong in two ways, by rejecting a true hypothesis or by failing to reject a false one. They then showed how to construct the test with the smallest chance of the second error for a given chance of the first (Neyman & Pearson, 1933). The paper calls the first of these "errors of the first kind". Today the two are taught in statistics courses as Type I and Type II errors.

The birth of econometrics, 1926–1944

Most of these tools were developed with independent observations in mind. Economic data raise a difficulty that controlled experiments can largely avoid, because observations come in time order and usually depend on one another. George Udny Yule showed that two time series whose values change only slowly from one period to the next can show a very high correlation even when the two series are unrelated, and he called these "nonsense correlations" (Yule, 1926). A year later, studying sunspot numbers, he proposed explaining a series by its own past values plus random disturbances, the idea behind the autoregressive model. He illustrated it with a pendulum that boys pelt with peas from both sides. The motion stays smooth, but its amplitude and phase keep changing. In his experiment the disturbances were generated by throwing dice (Yule, 1927, pp. 268–269).6 Eugen Slutzky showed that simply adding up purely random numbers can produce smooth waves that look like business cycles (Slutzky, 1937).

These results suggested that random shocks might lie behind economic fluctuations. Ragnar Frisch played a central role in bringing these threads together. In the first issue of Econometrica, the journal of the Econometric Society, which was founded in 1930, he set out the aims of the new field (Frisch, 1933). In the same year he separated the "propagation" mechanism of an economy, the way it responds to a disturbance, from the "impulses", the random shocks that keep it moving (Frisch, 1933/1995).

Testing such ideas required statistical models of whole economies. Jan Tinbergen built some of the first of these models. For the League of Nations he tested business-cycle theories with regression equations, including a model of the United States for 1919–1932 (Tinbergen, 1939/1968). John Maynard Keynes was critical of the work. He asked whether all relevant causes could be listed and measured, whether the relations were really linear, and whether coefficients estimated on past data would remain stable in the future (Keynes, 1939). Many of these questions are still asked today.

Alongside this debate, the theory of time series was also advancing. In Sweden, Herman Wold proved that any stationary time series can be written as a deterministic part plus a weighted sum of current and past random shocks (Wold, 1938). This result is often cited as a theoretical basis for the ARMA (autoregressive moving-average) models used later.

Keynes's doubts concerned the foundations of the approach, and one influential response is often attributed to Trygve Haavelmo. He first showed that when prices and quantities are determined together, as supply and demand are, estimating one equation alone by least squares gives biased results (Haavelmo, 1943). He then argued that economic relations should be written as probability models from the start, so that the tools of Fisher and of Neyman and Pearson could be applied to them (Haavelmo, 1944). This "probability approach" can be read as the point at which statistics became econometrics in a fuller sense, because economic models now came with their own error terms and their own assumptions to be tested.

Structural models and tests, 1950–1958

After the Second World War, the Cowles Commission in the United States developed Haavelmo's programme. Its monograph on dynamic economic models dealt with two questions. The first was when the parameters of a system of equations can be recovered from data, known as the identification problem, and the second was how they can be estimated (Koopmans, 1950). Lawrence Klein and Arthur Goldberger used these methods to build an annual model of the United States economy for 1929–1952 (Klein & Goldberger, 1955).

As such models spread, checking their assumptions became more important. James Durbin and Geoffrey Watson addressed the assumption that regression errors are independent over time. Because the true errors remain unobserved, they built a test on the regression residuals. The exact distribution of their statistic depends on the regressors, so they derived lower and upper bounds for its significance points (Durbin & Watson, 1950) and published tables of these bounds in a second paper. A value of their statistic below the lower bound indicates significant positive serial correlation (correlation between successive error terms), a value above the upper bound indicates the absence of significant serial correlation, and a value in between is inconclusive (Durbin & Watson, 1951, p. 161).7 In the same decade Emil Julius Gumbel, working on a different problem, collected the theory of the largest and smallest values of a sample, such as the highest flood in a century, in a single book (Gumbel, 1958).

The time-series revolution, 1969–1989

From the late 1960s, time-series methods became increasingly prominent. Clive Granger proposed a definition of causality that can be tested. In this definition Y is "causing" X if X can be predicted better using all available past information than using all past information apart from Y, and it is known today as Granger causality (Granger, 1969, p. 428).8 George Box and Gwilym Jenkins, in a book first published in 1970, set out a practical three-step procedure for ARIMA (autoregressive integrated moving-average) models, in which a model is first identified, then estimated and finally checked through its residuals (Box et al., 2016).

The new interest in time series also brought old problems back into view. Granger and Paul Newbold noted that published time-series regressions often combined a high R² with a very low Durbin–Watson statistic. In a simulation in which one random walk was regressed on another, independent one, the usual t-test at the 5% level wrongly rejected the true hypothesis that the two series are unrelated on about three-quarters of occasions (Granger & Newbold, 1974, pp. 111, 115). The problem echoed Yule's nonsense correlations, now in regression form.

Other criticisms were aimed at the large models themselves. Robert Lucas argued that if people change their expectations when policy changes, the estimated coefficients of an econometric model will change too, so the model may be unreliable for predicting the effects of a new policy (Lucas, 1976). Christopher Sims argued that the restrictions used to identify large macroeconomic models lacked credibility, and proposed vector autoregressions (VARs), in which every variable is explained by the past of all variables in the system (Sims, 1980).

The spurious-regression problem also called for a test. A widely used one came from David Dickey and Wayne Fuller, who studied the hypothesis that a series has a unit root (Dickey & Fuller, 1979, p. 427).9 They showed that, when the series has a unit root, the usual t-statistic follows a different distribution from the t-distribution, and they tabulated the correct critical values (Dickey & Fuller, 1979, 1981). Charles Nelson and Charles Plosser applied such tests to long United States macroeconomic series and found most of them consistent with a unit root (Nelson & Plosser, 1982).

The error term also came under closer study in other ways. Halbert White proposed standard errors that remain valid when the error variance differs across observations, a situation known as heteroscedasticity (White, 1980), and Whitney Newey and Kenneth West extended them to errors that are also correlated over time (Newey & West, 1987). Lars Peter Hansen developed the generalised method of moments, which estimates parameters from conditions that the model implies must hold on average (Hansen, 1982). David Hendry, in a public lecture at the London School of Economics titled "Econometrics—Alchemy or Science?", argued for testing econometric models rigorously against the data (Hendry, 1980).

Robert Engle went a step further and let the variance of the error itself change over time. In his ARCH (autoregressive conditional heteroscedasticity [an error variance that changes from one observation to another]) model, the variance of the error in each period depends on the square of the previous period's value, so that a large value in this period raises the variance expected for the next period (Engle, 1982, p. 988).10 He applied the model to United Kingdom inflation (Engle, 1982). Tim Bollerslev later generalised the model as GARCH (Bollerslev, 1986).

Engle and Granger then offered an answer to a question raised by spurious regressions. Two series that each have a unit root may still move together in the long run, in which case they are "co-integrated" and can be modelled with an error-correction model (Engle & Granger, 1987, p. 253).11 Søren Johansen developed a maximum likelihood method for systems with several such long-run relations (Johansen, 1988). Finally, James Hamilton allowed an economy to switch between unobserved regimes, such as expansion and recession, and estimated, for each period, the probability that the economy was in each regime (Hamilton, 1989).

These models also had limits. Richard Meese and Kenneth Rogoff found that, in their comparisons, a simple random walk forecast out of sample at least as well as the structural exchange-rate models of the 1970s (Meese & Rogoff, 1983).

Afterword

The story continued after 1989. James Heckman had already pointed out that analysing only a non-random sample, such as people who chose to work, biases the estimates (Heckman, 1979). In the 1990s much attention turned to the credibility of causal estimates from microeconomic data. Guido Imbens and Joshua Angrist clarified what an instrumental-variable estimate (one that uses a variable correlated with the explanatory variable but uncorrelated with the error term) measures when people respond to treatment differently (Imbens & Angrist, 1994). David Card and Alan Krueger used a natural experiment, a minimum-wage increase in New Jersey compared with neighbouring Pennsylvania, to estimate its effect on employment (Card & Krueger, 1994).

More recently, the growth of data has raised new questions. Hal Varian noted three issues that big datasets bring. First, their size may require more powerful tools for handling data. Second, there may be more potential predictors than estimation can accommodate, so some form of variable selection is needed. Third, large datasets may allow for more flexible relationships than simple linear models. For such relationships he pointed to machine-learning methods such as decision trees, support vector machines and neural networks, and wrote that his standard advice to graduate students was to take a machine-learning class in a computer science department (Varian, 2014, p. 3).

Looking back, one common thread is the error term. Gauss assumed it was normal, Student showed what happens when its spread is estimated from a few observations, Durbin and Watson tested whether it is independent over time, Engle let its variance move, and Engle and Granger asked when a combination of non-stationary series leaves a stationary error. In this reading, statistics became econometrics when economists began to treat the error as part of the model rather than as a nuisance. Data grow and methods multiply, yet the error term keeps its place. It will most likely write the next chapter of this history as well.

References

Bayes, T. (1763). An essay towards solving a problem in the doctrine of chances. Philosophical Transactions of the Royal Society of London, 53, 370–418.

Bollerslev, T. (1986). Generalized autoregressive conditional heteroskedasticity. Journal of Econometrics, 31(3), 307–327.

Bortkiewicz, L. von. (1898). Das Gesetz der kleinen Zahlen. B. G. Teubner.

Box, G. E. P., Jenkins, G. M., Reinsel, G. C., & Ljung, G. M. (2016). Time series analysis: Forecasting and control (5th ed.). Wiley.

Card, D., & Krueger, A. B. (1994). Minimum wages and employment: A case study of the fast-food industry in New Jersey and Pennsylvania. The American Economic Review, 84(4), 772–793.

Dickey, D. A., & Fuller, W. A. (1979). Distribution of the estimators for autoregressive time series with a unit root. Journal of the American Statistical Association, 74(366), 427–431.

Dickey, D. A., & Fuller, W. A. (1981). Likelihood ratio statistics for autoregressive time series with a unit root. Econometrica, 49(4), 1057–1072.

Durbin, J., & Watson, G. S. (1950). Testing for serial correlation in least squares regression. I. Biometrika, 37(3/4), 409–428.

Durbin, J., & Watson, G. S. (1951). Testing for serial correlation in least squares regression. II. Biometrika, 38(1/2), 159–178.

Engle, R. F. (1982). Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation. Econometrica, 50(4), 987–1007.

Engle, R. F., & Granger, C. W. J. (1987). Co-integration and error correction: Representation, estimation, and testing. Econometrica, 55(2), 251–276.

Fisher, R. A. (1922). On the mathematical foundations of theoretical statistics. Philosophical Transactions of the Royal Society of London, Series A, 222, 309–368.

Fisher, R. A. (1934). Statistical methods for research workers (5th ed.). Oliver and Boyd. (Original work published 1925)

Fisher, R. A. (1937). The design of experiments (2nd ed.). Oliver and Boyd. (Original work published 1935)

Frisch, R. (1933). Editor's note. Econometrica, 1(1), 1–4.

Frisch, R. (1995). Propagation problems and impulse problems in dynamic economics. In D. F. Hendry & M. S. Morgan (Eds.), The foundations of econometric analysis (pp. 333–346). Cambridge University Press. (Original work published 1933)

Galton, F. (1886). Regression towards mediocrity in hereditary stature. Journal of the Anthropological Institute of Great Britain and Ireland, 15, 246–263.

Gauss, C. F. (1809). Theoria motus corporum coelestium in sectionibus conicis solem ambientium. Perthes et Besser.

Granger, C. W. J. (1969). Investigating causal relations by econometric models and cross-spectral methods. Econometrica, 37(3), 424–438.

Granger, C. W. J., & Newbold, P. (1974). Spurious regressions in econometrics. Journal of Econometrics, 2(2), 111–120.

Gumbel, E. J. (1958). Statistics of extremes. Columbia University Press.

Haavelmo, T. (1943). The statistical implications of a system of simultaneous equations. Econometrica, 11(1), 1–12.

Haavelmo, T. (1944). The probability approach in econometrics. Econometrica, 12(Suppl.), iii–vi, 1–115.

Hamilton, J. D. (1989). A new approach to the economic analysis of nonstationary time series and the business cycle. Econometrica, 57(2), 357–384.

Hansen, L. P. (1982). Large sample properties of generalized method of moments estimators. Econometrica, 50(4), 1029–1054.

Heckman, J. J. (1979). Sample selection bias as a specification error. Econometrica, 47(1), 153–161.

Hendry, D. F. (1980). Econometrics—Alchemy or science? Economica, 47(188), 387–406.

Imbens, G. W., & Angrist, J. D. (1994). Identification and estimation of local average treatment effects. Econometrica, 62(2), 467–475.

Johansen, S. (1988). Statistical analysis of cointegration vectors. Journal of Economic Dynamics and Control, 12(2–3), 231–254.

Keynes, J. M. (1939). Professor Tinbergen's method. The Economic Journal, 49(195), 558–568.

Klein, L. R., & Goldberger, A. S. (1955). An econometric model of the United States, 1929–1952. North-Holland.

Koopmans, T. C. (Ed.). (1950). Statistical inference in dynamic economic models (Cowles Commission Monograph No. 10). Wiley.

Legendre, A.-M. (1805). Nouvelles méthodes pour la détermination des orbites des comètes. Firmin Didot.

Lucas, R. E., Jr. (1976). Econometric policy evaluation: A critique. Carnegie-Rochester Conference Series on Public Policy, 1, 19–46.

Meese, R. A., & Rogoff, K. (1983). Empirical exchange rate models of the seventies: Do they fit out of sample? Journal of International Economics, 14(1–2), 3–24.

Nelson, C. R., & Plosser, C. I. (1982). Trends and random walks in macroeconomic time series: Some evidence and implications. Journal of Monetary Economics, 10(2), 139–162.

Newey, W. K., & West, K. D. (1987). A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix. Econometrica, 55(3), 703–708.

Neyman, J., & Pearson, E. S. (1933). On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London, Series A, 231, 289–337.

Pearson, K. (1896). Mathematical contributions to the theory of evolution. III. Regression, heredity, and panmixia. Philosophical Transactions of the Royal Society of London, Series A, 187, 253–318.

Pearson, K. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 50(302), 157–175.

Poisson, S.-D. (1837). Recherches sur la probabilité des jugements en matière criminelle et en matière civile. Bachelier.

Sims, C. A. (1980). Macroeconomics and reality. Econometrica, 48(1), 1–48.

Slutzky, E. (1937). The summation of random causes as the source of cyclic processes. Econometrica, 5(2), 105–146.

Student. (1908). The probable error of a mean. Biometrika, 6(1), 1–25.

Tinbergen, J. (1968). Statistical testing of business-cycle theories (Vols. 1–2). Agathon Press. (Original work published 1939)

Varian, H. R. (2014). Big data: New tricks for econometrics. Journal of Economic Perspectives, 28(2), 3–28.

White, H. (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica, 48(4), 817–838.

Wold, H. (1938). A study in the analysis of stationary time series. Almqvist & Wiksell.

Yule, G. U. (1926). Why do we sometimes get nonsense-correlations between time-series? A study in sampling and the nature of time-series. Journal of the Royal Statistical Society, 89(1), 1–63.

Yule, G. U. (1927). On a method of investigating periodicities in disturbed series, with special reference to Wolfer's sunspot numbers. Philosophical Transactions of the Royal Society of London, Series A, 226, 267–298.

Notes


  1. Legendre writes each equation of the system as $E = a + bx + cy + fz + \&c.$, where $E$ is the error and $x$, $y$ and $z$ are the unknowns, and chooses the unknowns so that $E^2 + E'^2 + E''^2 + \&c.$ is a minimum (Legendre, 1805, pp. 72–73). ↑

  2. Gauss writes the error law as $\varphi\Delta = \frac{h}{\sqrt{\pi}}\, e^{-hh\Delta\Delta}$, where $\Delta$ is the error and $h$ is a constant (Gauss, 1809, p. 212). ↑

  3. Pearson's expression is $r = S(xy)/(n\sigma_1\sigma_2)$, where $S$ denotes summation, $\sigma_1^2 = S(x^2)/n$ and $\sigma_2^2 = S(y^2)/n$ (Pearson, 1896, p. 265). ↑

  4. With $m'$ the observed and $m$ the theoretical frequency of a group, and $e = m' - m$, the statistic is $\chi^2 = S(e^2/m)$ (Pearson, 1900, pp. 160, 163). ↑

  5. Student studies $z = x/s$, where $x$ is the distance of the sample mean from the population mean and $s$ is the sample standard deviation computed with divisor $n$, so that $s^2 = S(x_1^2)/n - (S(x_1)/n)^2$, with $x_1$ standing for each observation in the sample (Student, 1908, pp. 3, 7). Today's t-statistic is $t = z\sqrt{n-1}$. ↑

  6. In Yule's notation the model is $u_x = (2 - \mu)u_{x-1} - u_{x-2} + \varepsilon$, where $\varepsilon$ is the disturbance (Yule, 1927, p. 269). ↑

  7. With $z$ the regression residuals, the statistic is $d = \Sigma(\Delta z)^2/\Sigma z^2$. The lower and upper bounds are written $d_L$ and $d_U$ (Durbin & Watson, 1951, p. 161). ↑

  8. With $U$ all the information in the universe and a bar that, unlike its common use for a mean, denotes past values, $Y$ causes $X$ if $\sigma^2(X \mid \overline{U}) < \sigma^2(X \mid \overline{U - Y})$, where $\sigma^2$ is the variance of the prediction error (Granger, 1969, p. 428). ↑

  9. The model is $Y_t = \rho Y_{t-1} + e_t$, and the hypothesis tested is $\rho = 1$ (Dickey & Fuller, 1979, p. 427). ↑

  10. The model is $y_t \mid \psi_{t-1} \sim N(0, h_t)$ with $h_t = \alpha_0 + \alpha_1 y_{t-1}^2$, where $\psi_{t-1}$ is the information available at time $t-1$. A large value raises the expected variance when $\alpha_1 > 0$ (Engle, 1982, p. 988). ↑

  11. The components of a vector $x_t$ are co-integrated of order $d, b$, written $x_t \sim CI(d, b)$, if all of them are $I(d)$ (integrated of order $d$) and there is a vector $\alpha \neq 0$ such that $z_t = \alpha' x_t \sim I(d-b)$, with $b > 0$ (Engle & Granger, 1987, p. 253). ↑

Cite as: Yılmaz, M. İ. (2026). How Statistics Became Econometrics. https://muhammedikbalyilmaz.com/posts/how-statistics-became-econometrics

Column · 3734 words · 17 min read