How to create a text classification model in Machine Learning

João Barros 05 de September de 2026 6 min read

This tutorial shows how to turn text into features and train a Machine Learning classification model to categorize messages or reviews. It's useful to automate support triage, sentiment analysis or content classification with a practical example in Python and scikit-learn. I will explain why each step is taken and give practical tips to improve real-world results.

Prerequisites

Python 3.8+ installed
Packages: scikit-learn, pandas, numpy (pip install scikit-learn pandas numpy)
Basic knowledge of Python and pandas

I recommend working in a virtual environment (venv) to avoid version conflicts. For real datasets, ideally you have a few hundred to thousands of examples to obtain robust models — with only 50-100 samples results are very unstable.

Step 1: Prepare a simple dataset

We start with a small set of labeled examples (for example, training messages). Keep text and label in a pandas DataFrame. Well-structured data makes it easier to expand later to CSV or databases.

import pandas as pd

data = [
    ("Adorei o produto, chegou rápido e funciona bem", "positivo"),
    ("Produto avariado, quero reembolso", "negativo"),
    ("Entrega atrasada, má experiência", "negativo"),
    ("Muito satisfeito com a qualidade", "positivo"),
    ("Suporte lento e pouco útil", "negativo"),
    ("Excelente, recomendo", "positivo")
]

df = pd.DataFrame(data, columns=["text", "label"]) 
print(df.head())

This dataframe is just a didactic example. In production, also store an id, date and metadata (channel, product) for later analysis.

Step 2: Basic text cleaning

We normalize case and remove extra whitespace. Depending on the case you can remove punctuation, URLs or phone numbers; however, TF-IDF tends to ignore very frequent words if configured correctly. Avoid over-preprocessing: sometimes excessive removal loses important signals (e.g., "não gosto"). Here we do a simple and readable pre-processing:

def clean_text(s):
    return s.lower().strip()

df["text_clean"] = df["text"].apply(clean_text)
print(df[["text", "text_clean"]].head())

If you have imbalanced classes (e.g., 90% positive), consider techniques like resampling or class weights in the classifier.

Step 3: Split data into train and test

Use train_test_split to evaluate the model on new data. Stratification is important when there are few samples per class. For small datasets use test_size=0.2-0.33; for large datasets you can reserve only 10%.

from sklearn.model_selection import train_test_split

X = df["text_clean"]
y = df["label"]

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.33, random_state=42, stratify=y
)

With stratify=y we ensure the class proportion in train/test is preserved. For more robust validation, use Cross-Validation instead of a single split.

Step 4: Convert text to vectors with TF-IDF

TF-IDF converts text into numeric vectors weighting terms by relative importance. Here we choose ngram_range=(1,2) to capture unigrams and bigrams (e.g., "muito bom", "não gostei") and use max_df/min_df to reduce noise.

from sklearn.feature_extraction.text import TfidfVectorizer

vect = TfidfVectorizer(ngram_range=(1,2), max_df=0.9, min_df=1)
X_train_tfidf = vect.fit_transform(X_train)
X_test_tfidf = vect.transform(X_test)

print("Number of features:", X_train_tfidf.shape[1])

In a small sample you'll have few features (10-100). In medium datasets (1000+ texts) you typically have thousands of features; in that case consider dimensionality reduction (TruncatedSVD) or larger min_df limits.

Step 5: Train a simple classifier (Logistic Regression)

Logistic Regression works well for text and is fast. The parameter max_iter=1000 avoids non-convergence in cases with many features.

from sklearn.linear_model import LogisticRegression

model = LogisticRegression(max_iter=1000)
model.fit(X_train_tfidf, y_train)

y_pred = model.predict(X_test_tfidf)

Alternatives to try: SVM (good for sparse, high-dimensional data) or RandomForest (slower, may require dense vectors).

Step 6: Evaluate the model

Compute simple metrics: precision, recall and confusion matrix to understand common errors (false positives/negatives). In imbalanced problems, the f1-score is more informative than accuracy.

from sklearn.metrics import accuracy_score, classification_report, confusion_matrix

acc = accuracy_score(y_test, y_pred)
print("Accuracy:", acc)
print(classification_report(y_test, y_pred))
print("Confusion matrix:\n", confusion_matrix(y_test, y_pred))

In a toy dataset accuracy can vary a lot — values between 50% and 90% are common depending on the split. In real datasets, aim for >70-80% for simple sentiment tasks; for more complex tasks you may need more advanced models or more data.

Step 7: Interpret the most important features

Check which terms contribute most to each class using the model coefficients. This helps understand errors and improve preprocessing (for example, handling negations).

import numpy as np

feature_names = vect.get_feature_names_out()
coefs = model.coef_[0]

top_pos = np.argsort(coefs)[-10:][[::-1]][0]
top_neg = np.argsort(coefs)[:10]

print("Top positive terms:")
for i in top_pos:
    print(feature_names[i], round(coefs[i], 3))

print("Top negative terms:")
for i in top_neg:
    print(feature_names[i], round(coefs[i], 3))

If you find ambiguous terms, consider adjusting ngram_range or normalizing expressions (e.g., keeping the negation in "não gostei").

Check the result

Confirm you have a coherent accuracy and inspect the confusion matrix. Test the model with new sentences to see if predictions make sense:

samples = [
    "Quero devolução, chegou danificado",
    "Produto excelente, muito bom",
    "A entrega demorou mas o produto é bom"
]

samples_tfidf = vect.transform([s.lower().strip() for s in samples])
print(model.predict(samples_tfidf))

Also perform stress tests with 50-100 varied sentences to evaluate stability. If the model systematically fails on sentences with negations, add preprocessing rules or specific features.

Conclusion

You now have a complete pipeline: basic cleaning, TF-IDF, training and evaluation with Logistic Regression for text classification. Practical next steps: increase the dataset to hundreds of examples, try models like RandomForest or SVM, and add advanced cleaning (stopword removal, stemming, lemmatization). Use Cross-Validation and GridSearchCV to choose hyperparameters and reduce overfitting. Observe which terms are causing the most errors and iterate on preprocessing until you achieve the desired robustness.