OLS Regression
SPSS
EViews
Data Analysis
Research Methods
Tobit Research Consulting | Statistics & Data Analysis Series | Reading time: ~15 minutes
What you will learn: What Ordinary Least Squares (OLS) regression actually estimates; when OLS is appropriate; how to specify a regression model; how to run OLS regression step by step in SPSS and EViews; how to interpret R-squared, the F-statistic, regression coefficients, t-statistics and p-values; which assumptions must be checked before relying on the results; and how to report OLS findings correctly in a thesis, dissertation or journal article.
1. What Is OLS Regression?
Ordinary Least Squares, usually abbreviated as OLS, is one of the most widely used methods for estimating linear regression models. It examines how a dependent variable changes in relation to one or more independent variables and estimates the coefficients that best describe those relationships.
The term “least squares” refers to how the regression line is fitted. Every observation has a residual — the difference between its observed value and the value predicted by the regression model. OLS chooses the set of regression coefficients that minimises the sum of the squared residuals.
Simple Example
Suppose a researcher wants to determine whether employee training predicts employee productivity. Productivity is the dependent variable (Y), while training is the independent variable (X). OLS estimates the regression line that produces the smallest possible total squared prediction error across the observations.
With several predictors, OLS becomes multiple linear regression. A researcher might, for example, estimate employee productivity using training, compensation, work experience and employee engagement simultaneously. Each regression coefficient then represents the estimated relationship between one predictor and the dependent variable while holding the other predictors in the model constant.
The important distinction: OLS is an estimation method. “Regression analysis” is the broader analytical process that includes model specification, estimation, diagnostics, interpretation and statistical inference. Clicking Run in SPSS or EViews therefore does not by itself complete a sound regression analysis.
2. When Should You Use OLS?
OLS is generally suitable when the outcome variable is continuous and the researcher is estimating a relationship that can reasonably be represented by a linear regression model. Common dependent variables include profitability, sales revenue, test scores, productivity, household income, financial performance and other quantitative outcomes.
| Research Situation |
Is Standard OLS Appropriate? |
| Continuous dependent variable |
Potentially yes |
| One or several independent variables |
Yes |
| Binary dependent variable such as Yes/No |
Usually no — logistic/probit models are generally more appropriate |
| Count outcome such as number of events |
Often no — consider count-data models |
| Panel data with firm- or individual-specific effects |
Pooled OLS may be only a starting point; fixed or random-effects models may be required |
| Non-stationary time-series data |
OLS in levels can be misleading unless the time-series properties and possible cointegration are addressed |
Do not select OLS merely because your research has independent and dependent variables. The measurement of the dependent variable, structure of the data, theoretical relationship, sampling design and properties of the errors all affect which regression technique is appropriate.
3. Understanding the OLS Regression Model
A standard multiple linear regression model can be written as:
Y = β₀ + β₁X₁ + β₂X₂ + β₃X₃ + ε
| Symbol |
Meaning |
| Y |
Dependent or outcome variable |
| β₀ |
Intercept or constant |
| β₁, β₂, β₃ |
Population regression coefficients |
| X₁, X₂, X₃ |
Independent or predictor variables |
| ε |
Error term containing influences on Y not captured by the regressors |
Suppose financial performance is predicted using liquidity, leverage and firm size. The empirical model could be expressed as:
Financial Performance = β₀ + β₁(Liquidity) + β₂(Leverage) + β₃(Firm Size) + ε
If the estimated coefficient for liquidity is 0.35, the interpretation is that a one-unit increase in liquidity is associated with an estimated 0.35-unit increase in financial performance, holding leverage and firm size constant. Whether that estimated relationship is statistically distinguishable from zero is then assessed using its standard error, t-statistic, p-value and preferably its confidence interval.
4. OLS Assumptions You Must Check
A common mistake in dissertations is to present the regression table first and perform diagnostics almost as an afterthought. In practice, the validity of standard OLS inference depends on the model and error structure satisfying important conditions.
Assumption 1
Linearity and Correct Functional Form
The conditional relationship being estimated should be adequately represented by the specified linear model. Residual plots can reveal curvature or systematic patterns suggesting that the functional form is inadequate.
Assumption 2
No Perfect Multicollinearity
No independent variable should be an exact linear combination of other predictors. Severe, though imperfect, multicollinearity can also make coefficient estimates unstable and inflate their standard errors. VIF and tolerance statistics are commonly used as diagnostics.
Assumption 3
Homoskedasticity
The conditional variance of the error should be constant when conventional OLS standard errors are used. Heteroskedasticity does not automatically invalidate the coefficient point estimates under the other standard conditions, but conventional standard errors and hypothesis tests can become unreliable.
Assumption 4
Independence / No Problematic Serial Correlation
Error terms should not exhibit unmodelled dependence. This issue is particularly important with time-series and longitudinal data, where residuals may be correlated across periods.
Assumption 5
Exogeneity
A crucial econometric requirement is that regressors should not be systematically related to the error term. Omitted variables, simultaneity and some forms of measurement error can create endogeneity. If endogeneity is present, simply obtaining a significant OLS coefficient does not establish a valid causal effect.
Assumption 6
Residual Distribution and Inference
Normality is particularly relevant to exact small-sample t- and F-inference under the classical normal linear model. In applied research, residual distributions should be inspected alongside the sample size, outliers and other diagnostics rather than treating a single normality test as an automatic pass/fail rule.
Important: A p-value above .05 on one diagnostic test does not prove that an assumption is perfectly satisfied. Diagnostic tests should be interpreted together with residual graphs, theoretical considerations and the structure of the dataset.
5. How to Run OLS Regression in SPSS
IBM SPSS Statistics provides OLS regression through the Linear Regression procedure. Before running the model, ensure that the dependent and independent variables have been coded correctly, missing values have been examined and categorical predictors have been represented appropriately, such as through dummy variables where necessary.
Step-by-Step SPSS Procedure
- Open the dataset in SPSS and confirm that the relevant variables are correctly defined.
- Go to Analyze → Regression → Linear.
- Move your outcome variable into the Dependent box.
- Move your predictor variables into the Independent(s) box.
- For a standard theory-driven multiple regression, select Enter as the entry method.
- Click Statistics and request Estimates, Model Fit, Confidence Intervals and Collinearity Diagnostics. Durbin-Watson may also be selected when relevant.
- Click Plots and plot standardized residuals against standardized predicted values. A histogram and normal probability plot of residuals can also be requested.
- Under Save, save residuals, predicted values and influence measures when more detailed diagnostics are required.
- Click OK to estimate the model.
Useful SPSS Syntax
REGRESSION
/DEPENDENT performance
/METHOD=ENTER training compensation engagement
/STATISTICS COEFF OUTS R ANOVA COLLIN TOL CI(95)
/RESIDUALS HISTOGRAM(ZRESID) NORMPROB(ZRESID)
/SCATTERPLOT=(*ZPRED,*ZRESID).
The default SPSS output normally includes the Model Summary, ANOVA and Coefficients tables. Additional diagnostic output depends on the options selected.
| SPSS Output |
What It Tells You |
| Model Summary |
R, R², adjusted R² and standard error of estimate |
| ANOVA |
Overall F-test for the regression model |
| Coefficients |
B coefficients, standard errors, standardized Beta where requested, t-statistics and significance levels |
| Collinearity Statistics |
Tolerance and VIF |
| Residual Plots |
Evidence concerning linearity, unequal variance and unusual observations |
Do not confuse B with Beta in SPSS. The unstandardized coefficient B is expressed in the original units of the variables and is generally the coefficient used to write the estimated regression equation. Standardized Beta expresses relationships in standard-deviation units and may be useful for certain comparisons, but it is not interchangeable with B.
6. How to Run OLS Regression in EViews
EViews is particularly common in economics, finance and other research involving time-series or panel data. The software refers to standard OLS estimation as Least Squares (LS).
Step-by-Step EViews Procedure
- Create or open an EViews workfile with the appropriate data structure and frequency.
- Import or enter the variables and verify the observation range.
- Choose Quick → Estimate Equation, or create a new Equation object.
- Select LS – Least Squares as the estimation method.
- Enter the dependent variable first, followed by C and then the independent variables.
- Specify the estimation sample where necessary.
- Click OK to estimate the equation.
- Use the equation window’s View menu to inspect residual and coefficient diagnostics.
For example, to estimate financial performance as a function of liquidity, leverage and firm size, enter:
performance c liquidity leverage firmsize
The equivalent EViews command is:
equation eq1.ls performance c liquidity leverage firmsize
EViews-specific point: unlike software that inserts an intercept automatically, EViews requires you to include C explicitly when you want a constant in a list-specified least-squares regression.
After estimation, the standard equation output presents coefficient estimates, standard errors, t-statistics and probabilities together with model-level information such as R-squared, adjusted R-squared, the F-statistic and the Durbin-Watson statistic.
Useful EViews Diagnostics
| Issue |
Typical EViews Route |
| Residual normality |
View → Residual Diagnostics → Histogram-Normality Test |
| Serial correlation |
View → Residual Diagnostics → Serial Correlation LM Test |
| Residual autocorrelation pattern |
View → Residual Diagnostics → Correlogram-Q-statistics |
| Heteroskedasticity |
View → Residual Diagnostics → Heteroskedasticity Tests |
| Coefficient restrictions |
View → Coefficient Diagnostics → Wald Test |
EViews supports several heteroskedasticity tests, including Breusch-Pagan-Godfrey and White tests. For time-series regression, EViews also provides the Breusch-Godfrey serial-correlation LM test, which can test serial correlation beyond the first order.
7. How to Interpret OLS Regression Output
7.1 R-Squared (R²)
R-squared measures the proportion of sample variation in the dependent variable accounted for by the fitted regression model. If R² = .62, the regressors in the model collectively account for 62% of the observed variation in the dependent variable in that sample.
Do not write that R² = .62 means the independent variables “cause 62% of the dependent variable.” R² is a goodness-of-fit measure. It does not establish causality.
7.2 Adjusted R-Squared
Adjusted R² modifies R² to account for the number of regressors relative to the available sample information. It is particularly useful when comparing nested specifications with different numbers of predictors because ordinary R² cannot decrease merely from adding another reg
“`
