How to validate and clean emails in Python for data: step by step

Learning to validate and clean emails in Python for data is useful to improve database quality, reduce failures when sending notifications and ensure better matching between records. This tutorial shows how to detect invalid formats, normalize common domains and remove duplicates in a practical way.

Prerequisites

• Python 3.8+ installed.
• Libraries: pandas, email-validator (installation via pip).
• Text editor or Jupyter Notebook.

Step 1: Install dependencies

Install the required libraries. The email-validator library performs syntactic validation and some basic corrections; pandas makes data manipulation easier.

pip install pandas email-validator

Step 2: Load sample data

Create a simple DataFrame with emails that contain common errors: spaces, uppercase, duplicated dots, domains with typos and empty entries.

import pandas as pd

df = pd.DataFrame({
    'id': [1,2,3,4,5,6],
    'email': [
        'Joao.Silva@Exemplo.COM ',
        'maria..souza@@gmail.com',
        'ana@gnail.con',
        '  ',
        'pedro@exemplo.com',
        'PEDRO@Exemplo.com'
    ]
})
print(df)

Step 3: Normalize basic strings

Remove surrounding whitespace, convert to lowercase and reduce duplicated dots in the local part of the email. This reduces trivial differences between addresses.

def normalize_basic(email):
    if not isinstance(email, str):
        return ''
    e = email.strip().lower()
    # reduce duplicated dots in the part before @
    if '@' in e:
        local, domain = e.split('@', 1)
        while '..' in local:
            local = local.replace('..', '.')
        e = f"{local}@{domain}"
    return e

df['email_norm'] = df['email'].apply(normalize_basic)
print(df[['email','email_norm']])

Step 4: Validate syntax with email-validator

Use email-validator to check if the format is valid and receive suggestions. The function below returns a status and the version normalized by the validator when possible.

from email_validator import validate_email, EmailNotValidError

def validate_email_safe(email):
    if not email:
        return {'valid': False, 'reason': 'empty', 'email': ''}
    try:
        v = validate_email(email, check_deliverability=False)
        return {'valid': True, 'reason': '', 'email': v.email}
    except EmailNotValidError as e:
        return {'valid': False, 'reason': str(e), 'email': ''}

res = df['email_norm'].apply(validate_email_safe)
res_df = pd.json_normalize(res)
df = pd.concat([df, res_df.add_prefix('val_')], axis=1)
print(df[['email_norm','val_valid','val_reason','val_email']])

Step 5: Fix common domains with typos

Many errors come from incorrectly written domains (e.g.: gnail -> gmail). Create a corrections dictionary to apply after syntactic validation.

COMMON_DOMAINS = {
    'gnail.com': 'gmail.com',
    'gnail.con': 'gmail.com',
    'exemplo.com': 'exemplo.com',  # intended example
}

def fix_common_domain(email):
    if not email or '@' not in email:
        return email
    local, domain = email.split('@', 1)
    domain_fix = COMMON_DOMAINS.get(domain, domain)
    return f"{local}@{domain_fix}"

# apply only where there is a validated email in val_email or try using email_norm
df['email_fixed'] = df['val_email'].where(df['val_email'] != '', df['email_norm']).apply(fix_common_domain)
print(df[['email_norm','val_email','email_fixed']])

Step 6: Revalidate and mark invalids

Revalidate the corrected emails to separate those that remain invalid. Keep a column with the final state: valid/invalid/unknown.

final_res = df['email_fixed'].apply(validate_email_safe)
final_df = pd.json_normalize(final_res)
df = pd.concat([df, final_df.add_prefix('final_')], axis=1)

def status(row):
    if row['final_valid']:
        return 'valid'
    if not row['email_fixed']:
        return 'invalid'
    return 'invalid'

df['status'] = df.apply(status, axis=1)
print(df[['email','email_fixed','final_valid','status']])

Step 7: Remove duplicates and consolidate

When normalizing and fixing, duplicates often appear (e.g.: pedro@exemplo.com and PEDRO@Exemplo.com). Use pandas to deduplicate keeping the first record or aggregating IDs.

# create key to deduplicate
clean = df[df['status']=='valid'].copy()
clean['email_canonical'] = clean['email_fixed']
# keep the first id per email
clean_dedup = clean.drop_duplicates(subset=['email_canonical'], keep='first')
print(clean_dedup[['id','email','email_canonical']])

Verify the result

Confirm that the valid emails are normalized, invalids identified and duplicates removed. Check the numbers: original total, final valids, invalids and duplicates removed.

total = len(df)
validos = df['status'].eq('valid').sum()
invalidos = df['status'].eq('invalid').sum()
deduped = len(clean_dedup)
print(f"Total: {total}, Válidos: {validos}, Inválidos: {invalidos}, Após deduplicação: {deduped}")

Conclusion

You now have a simple process to normalize, validate and clean emails in Python for data, including domain corrections and deduplication. Next steps: integrate this pipeline into an ETL, use check_deliverability=True with caution or enrich with SMTP validation. Tip: keep a log of corrections to audit changes to emails.