An introduction to the F-test

 Let's assume that we have a linear regression such as:

If we wanted to work with the t statistic, our null hypothesis would be something along the lines of:






 The above is ok for testing just one regression coefficient, but we can't use the current framework to test multiple coefficients.

Multiple coefficient test

This null hypothesis is basically telling us that in the population, all the effects of each variable are jointly equal to 0. We are therefore testing for the joint non significance of all the variables in our model.





The alternative hypothesis states that if any of the coefficients are not equal to 0, this would result in us rejecting the null hypothesis.

What we should expect:

•  If we have a regression where some of our coefficients have very high t-stats, it is unlikely that we would fail to reject the null hypothesis. This is because even if one of the variables is significant, all the variables are not jointly insignificant.

•  A high t-stat means that we are likely to reject the null hypothesis under multiple regression.

• Typically, for the F-stat, it normally has a critical value ranging from 3 to 5.

So far, we've only discussed the t-stat, how do we conduct an F-test?

We go about this by first thinking about unrestricted regression (UR):

This is essentially saying that if our model fits the data well, the sum of squared residuals (SSR) is likely to be low as our errors will be low.

After we get the SSR from our UR, we have to perform a restricted regression (R) :

 We restrict the model by assuming that the dependent variable does not depend on the independent variables.

 Therefore, any sort of extra variation in the independent variables helps to reduce the error associated with predicted the dependent variables.


In order to know if the restricted SSR is significantly greater than the unrestricted version, we need to form the F-statistic:

•  If the numerator is particularly large relative to the denominator, when we move from R to UR, we explain more than the R model. This leads to the unrestricted SSR being significantly lower than the restricted version.

• The larger the F-stat, the more likely we are to reject the null hypothesis.

We will reject the null hypothesis if the value of our F stat > critical value.