How to Evaluate a Regression Model in Machine Learning

João Barros 05 de July de 2026 4 min read

Once you have trained a model that predicts continuous values — house prices, temperatures or sales — the next question is always the same: are the predictions any good? Evaluating a regression model in Machine Learning with the right metrics gives you a numeric answer, showing how far the predictions are from the real values. The three most common metrics for this are MAE, RMSE and R², and that is exactly what you will learn to calculate and interpret.

Prerequisites

Python 3 installed on your computer.
The scikit-learn and pandas libraries (install them with pip install scikit-learn pandas).
Knowing how to split data into training and test sets with train_test_split.
Basic knowledge of how to train a model (we use LinearRegression as an example).

Step 1: Train a model to evaluate

To have predictions to measure, we first train a simple model. We use the California Housing dataset that ships with scikit-learn, split the data into training and test sets, and train a linear regression. The predictions come from the test set — that is, from data the model has never seen.

from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression

dados = fetch_california_housing()
X_treino, X_teste, y_treino, y_teste = train_test_split(
    dados.data, dados.target, test_size=0.2, random_state=42)

modelo = LinearRegression()
modelo.fit(X_treino, y_treino)
previsoes = modelo.predict(X_teste)

Step 2: Calculate the MAE (mean absolute error)

The MAE (Mean Absolute Error) is the average of the absolute differences between the actual and predicted values. It is the easiest metric to explain: it is in the same unit as the target and tells you, on average, how far off the model is. The lower, the better.

from sklearn.metrics import mean_absolute_error

mae = mean_absolute_error(y_teste, previsoes)
print(f"MAE: {mae:.3f}")

Step 3: Calculate the RMSE (root mean squared error)

The RMSE (Root Mean Squared Error) is also in the target's unit, but it squares the errors before adding them up. In practice, this makes large errors weigh far more than small ones: if the model is badly wrong in a few cases, the RMSE rises quickly. Since scikit-learn 1.4 there is the root_mean_squared_error function, which is the current way to compute it.

from sklearn.metrics import root_mean_squared_error

rmse = root_mean_squared_error(y_teste, previsoes)
print(f"RMSE: {rmse:.3f}")

Older versions used mean_squared_error(y_teste, previsoes, squared=False), but that parameter has been deprecated, so prefer the new function.

Step 4: Calculate the R² (coefficient of determination)

The R² answers a different question: what percentage of the variation in the data is explained by the model? It usually ranges from 0 to 1, where 1 is a perfect prediction. A value close to 0 means the model adds little over simply guessing the mean; negative values indicate a model that is worse than that mean.

from sklearn.metrics import r2_score

r2 = r2_score(y_teste, previsoes)
print(f"R2: {r2:.3f}")

Verify the result

When you run the script you should see three printed values. With this dataset and a linear regression, the MAE is usually around 0.53, the RMSE about 0.75 and the R² around 0.58 (the target is in hundreds of thousands of dollars). The exact values may vary slightly depending on the library versions. Notice that the RMSE is larger than the MAE: whenever that gap is wide, it signals that a few large errors are pulling the average up. An R² of 0.58, in turn, tells you the model explains about 58% of the variation in prices — there is room to improve.

Conclusion

With the MAE, RMSE and R² you get a complete picture of a regression model's performance: the average error, the weight of the large errors and the overall quality of the fit. The natural next step is to use these same three metrics to compare different models — for example, swapping LinearRegression for a RandomForestRegressor and seeing which numbers improve. Tip: always compute these metrics on the test data, never on the training data — only then do they reflect real performance. And for your own problem, what matters more: heavily penalising large errors (RMSE) or treating all errors equally (MAE)?