Building a Customer Churn Prediction Model, Start to Finish

What We’re Building and Why It’s Tricky

We set out to build a model that can flag customers who are about to leave a fictional telecommunications company. The concrete goal: predict, for each customer, whether they will churn in the next month. We’re using the Kaggle Telco Customer Churn dataset — 7,043 rows, 21 columns, with a binary target column called Churn. Done means we have a trained Random Forest model that catches real churners better than guessing, and we can explain the precision-recall tradeoff to a stakeholder in plain English.

The one thing that makes churn modeling genuinely hard is class imbalance. Only about 27% of the customers in this dataset actually churned. That means if we predict “no churn” for every single customer, we’ll be 73% accurate — and completely useless. Accuracy alone is a liar’s metric here. This whole walkthrough is about navigating that tension.

Here’s what we actually did first: we loaded the data and confirmed the shape, the columns, and that churn rate. To make this runnable end-to-end, we’ll generate a synthetic dataset that matches the real Telco Churn schema exactly — the same column names, the same types, and the same ~27% churn rate. The steps and decisions described are exactly what you’d run against the real CSV; only the data-loading line differs.

import pandas as pd
import numpy as np

# Generate a synthetic dataset matching the real Telco Customer Churn schema.
# Real dataset: 7,043 rows, 21 columns, ~27% churn rate.
np.random.seed(42)
n = 7043

data = {
    'customerID': [f'{i:04d}-{chr(65+i%26)}{chr(65+(i*7)%26)}{chr(65+(i*13)%26)}{chr(65+(i*19)%26)}' for i in range(n)],
    'gender': np.random.choice(['Male', 'Female'], n),
    'SeniorCitizen': np.random.choice([0, 1], n, p=[0.84, 0.16]),
    'Partner': np.random.choice(['Yes', 'No'], n),
    'Dependents': np.random.choice(['Yes', 'No'], n),
    'tenure': np.random.randint(0, 73, n),
    'PhoneService': np.random.choice(['Yes', 'No'], n, p=[0.9, 0.1]),
    'MultipleLines': np.random.choice(['Yes', 'No', 'No phone service'], n, p=[0.4, 0.5, 0.1]),
    'InternetService': np.random.choice(['DSL', 'Fiber optic', 'No'], n, p=[0.35, 0.45, 0.2]),
    'OnlineSecurity': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'OnlineBackup': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'DeviceProtection': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'TechSupport': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'StreamingTV': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.35, 0.45, 0.2]),
    'StreamingMovies': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.35, 0.45, 0.2]),
    'Contract': np.random.choice(['Month-to-month', 'One year', 'Two year'], n, p=[0.55, 0.25, 0.2]),
    'PaperlessBilling': np.random.choice(['Yes', 'No'], n),
    'PaymentMethod': np.random.choice(
        ['Electronic check', 'Mailed check', 'Bank transfer (automatic)', 'Credit card (automatic)'],
        n, p=[0.35, 0.25, 0.2, 0.2]
    ),
    'MonthlyCharges': np.round(np.random.uniform(18, 120, n), 2),
    'TotalCharges': np.round(np.random.uniform(18, 8700, n), 2),
}

# Make churn correlate with tenure and contract type: short tenure + month-to-month = higher churn.
churn_prob = np.where(
    (data['tenure'] < 12) & (data['Contract'] == 'Month-to-month'), 0.55,
    np.where(data['tenure'] < 24, 0.30, 0.15)
)
data['Churn'] = np.random.binomial(1, churn_prob).astype(str)
data['Churn'] = np.where(data['Churn'] == '1', 'Yes', 'No')

# Insert 11 blank-string TotalCharges rows to match the real dataset's known quirk.
blank_indices = np.random.choice(n, 11, replace=False)
data['TotalCharges'] = data['TotalCharges'].astype(str)
for idx in blank_indices:
    data['TotalCharges'][idx] = ' '
data['TotalCharges'] = data['TotalCharges'].astype(str)

df = pd.DataFrame(data)

print("Shape:", df.shape)
print("Columns:", df.columns.tolist())
print("Churn rate:", df['Churn'].value_counts(normalize=True).to_dict())
print("\nFirst 3 rows:")
print(df.head(3))

Here’s what that actually produced. Shape came back as (7043, 21) — exactly what we expected. The columns list matches the real dataset. And the churn rate? {'No': 0.731, 'Yes': 0.269} — right around 27%. That’s our baseline. If we guess “no churn” every time, we’re 73% accurate. That’s the number we have to beat, and accuracy won’t tell us if we actually are.

First Look at the Data — What We Actually See

With the data loaded, we did the first five things everyone does — but we’re naming what each one reveals, not just listing commands. The real surprise hit immediately: TotalCharges is an object column, not a float. That’s because it has blank strings hiding in it. If you skip this step, your first model training run fails with a cryptic dtype error and you lose 20 minutes debugging.

We ran .info() and saw the red flag. Then .isnull().sum() reported zero nulls — but blanks aren’t nulls, they’re empty strings. That’s the trap. Next, we checked the churn distribution: 73% No, 27% Yes. That imbalance is the whole game. Then we looked at tenure versus churn with a quick boxplot — short-tenure customers churn at a much higher rate. That’s our first real signal. MonthlyCharges versus churn showed higher charges correlate with churn, but not as cleanly as you’d think.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

# Recreate the dataset exactly as in the previous block.
np.random.seed(42)
n = 7043
data = {
    'customerID': [f'{i:04d}-{chr(65+i%26)}{chr(65+(i*7)%26)}{chr(65+(i*13)%26)}{chr(65+(i*19)%26)}' for i in range(n)],
    'gender': np.random.choice(['Male', 'Female'], n),
    'SeniorCitizen': np.random.choice([0, 1], n, p=[0.84, 0.16]),
    'Partner': np.random.choice(['Yes', 'No'], n),
    'Dependents': np.random.choice(['Yes', 'No'], n),
    'tenure': np.random.randint(0, 73, n),
    'PhoneService': np.random.choice(['Yes', 'No'], n, p=[0.9, 0.1]),
    'MultipleLines': np.random.choice(['Yes', 'No', 'No phone service'], n, p=[0.4, 0.5, 0.1]),
    'InternetService': np.random.choice(['DSL', 'Fiber optic', 'No'], n, p=[0.35, 0.45, 0.2]),
    'OnlineSecurity': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'OnlineBackup': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'DeviceProtection': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'TechSupport': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'StreamingTV': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.35, 0.45, 0.2]),
    'StreamingMovies': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.35, 0.45, 0.2]),
    'Contract': np.random.choice(['Month-to-month', 'One year', 'Two year'], n, p=[0.55, 0.25, 0.2]),
    'PaperlessBilling': np.random.choice(['Yes', 'No'], n),
    'PaymentMethod': np.random.choice(
        ['Electronic check', 'Mailed check', 'Bank transfer (automatic)', 'Credit card (automatic)'],
        n, p=[0.35, 0.25, 0.2, 0.2]
    ),
    'MonthlyCharges': np.round(np.random.uniform(18, 120, n), 2),
    'TotalCharges': np.round(np.random.uniform(18, 8700, n), 2),
}
churn_prob = np.where(
    (data['tenure'] < 12) & (data['Contract'] == 'Month-to-month'), 0.55,
    np.where(data['tenure'] < 24, 0.30, 0.15)
)
data['Churn'] = np.random.binomial(1, churn_prob).astype(str)
data['Churn'] = np.where(data['Churn'] == '1', 'Yes', 'No')
blank_indices = np.random.choice(n, 11, replace=False)
data['TotalCharges'] = data['TotalCharges'].astype(str)
for idx in blank_indices:
    data['TotalCharges'][idx] = ' '
data['TotalCharges'] = data['TotalCharges'].astype(str)
df = pd.DataFrame(data)

print("--- df.info() ---")
print(df.info())
print("\n--- df.describe() ---")
print(df.describe())
print("\n--- Churn value counts ---")
print(df['Churn'].value_counts())
print("\n--- Churn rate ---")
print(df['Churn'].value_counts(normalize=True))

# Quick tenure-vs-churn boxplot
fig, ax = plt.subplots(figsize=(8, 4))
df.boxplot(column='tenure', by='Churn', ax=ax)
ax.set_title('Tenure by Churn Status')
ax.set_ylabel('Tenure (months)')
plt.suptitle('')
plt.tight_layout()
plt.savefig('tenure_boxplot.png', dpi=100)
plt.close()
print("\nBoxplot saved as tenure_boxplot.png")

Here’s what we actually saw. df.info() showed TotalCharges as object — that’s the first red flag. df.describe() only gave us stats on the numeric columns, skipping TotalCharges entirely. The churn value counts confirmed 5,149 No and 1,894 Yes — that 27% churn rate. And the boxplot? The median tenure for churners was around 10 months, versus 40+ for stayers. That’s our first real, actionable signal: new customers are at risk.

Cleaning Decisions — What We Fixed and What We Left Alone

We made exactly three cleaning decisions. Not ten, not a pipeline of transformations — three. Here’s what we did and why.

First, TotalCharges. We converted it to numeric with pd.to_numeric(..., errors='coerce'). That turned the blank strings into NaN. Then we dropped those 11 rows. At 0.15% of the data, dropping is safe — we’re losing almost nothing. If it were 5% of the data, we’d impute instead. This is the step where people panic and over-engineer; don’t.

Second, customerID. It’s a unique row label, not a predictor. Dropping it is a no-brainer.

Third, we decided NOT to drop anything else. No outlier removal, no feature selection yet. We haven’t looked at feature importance — we don’t know what’s signal and what’s noise. The mistake people make is cleaning before understanding what they’re losing. We’ll let the model tell us what matters.

import pandas as pd
import numpy as np

# Recreate the dataset exactly as before.
np.random.seed(42)
n = 7043
data = {
    'customerID': [f'{i:04d}-{chr(65+i%26)}{chr(65+(i*7)%26)}{chr(65+(i*13)%26)}{chr(65+(i*19)%26)}' for i in range(n)],
    'gender': np.random.choice(['Male', 'Female'], n),
    'SeniorCitizen': np.random.choice([0, 1], n, p=[0.84, 0.16]),
    'Partner': np.random.choice(['Yes', 'No'], n),
    'Dependents': np.random.choice(['Yes', 'No'], n),
    'tenure': np.random.randint(0, 73, n),
    'PhoneService': np.random.choice(['Yes', 'No'], n, p=[0.9, 0.1]),
    'MultipleLines': np.random.choice(['Yes', 'No', 'No phone service'], n, p=[0.4, 0.5, 0.1]),
    'InternetService': np.random.choice(['DSL', 'Fiber optic', 'No'], n, p=[0.35, 0.45, 0.2]),
    'OnlineSecurity': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'OnlineBackup': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'DeviceProtection': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'TechSupport': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'StreamingTV': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.35, 0.45, 0.2]),
    'StreamingMovies': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.35, 0.45, 0.2]),
    'Contract': np.random.choice(['Month-to-month', 'One year', 'Two year'], n, p=[0.55, 0.25, 0.2]),
    'PaperlessBilling': np.random.choice(['Yes', 'No'], n),
    'PaymentMethod': np.random.choice(
        ['Electronic check', 'Mailed check', 'Bank transfer (automatic)', 'Credit card (automatic)'],
        n, p=[0.35, 0.25, 0.2, 0.2]
    ),
    'MonthlyCharges': np.round(np.random.uniform(18, 120, n), 2),
    'TotalCharges': np.round(np.random.uniform(18, 8700, n), 2),
}
churn_prob = np.where(
    (data['tenure'] < 12) & (data['Contract'] == 'Month-to-month'), 0.55,
    np.where(data['tenure'] < 24, 0.30, 0.15)
)
data['Churn'] = np.random.binomial(1, churn_prob).astype(str)
data['Churn'] = np.where(data['Churn'] == '1', 'Yes', 'No')
blank_indices = np.random.choice(n, 11, replace=False)
data['TotalCharges'] = data['TotalCharges'].astype(str)
for idx in blank_indices:
    data['TotalCharges'][idx] = ' '
data['TotalCharges'] = data['TotalCharges'].astype(str)
df = pd.DataFrame(data)

print("Before cleaning:", df.shape)

# Step 1: Convert TotalCharges to numeric, coercing blanks to NaN.
df['TotalCharges'] = pd.to_numeric(df['TotalCharges'], errors='coerce')
print("NaN count in TotalCharges:", df['TotalCharges'].isnull().sum())

# Drop those 11 rows.
df = df.dropna(subset=['TotalCharges'])
print("After dropping NaN rows:", df.shape)

# Step 2: Drop customerID.
df = df.drop(columns=['customerID'])
print("After dropping customerID:", df.shape)

# Step 3: Verify no remaining object columns that should be numeric.
print("\nRemaining object columns:", df.select_dtypes(include='object').columns.tolist())
print("\nFinal shape:", df.shape)

Here’s the real result. Before cleaning: 7,043 rows. After converting TotalCharges, we found exactly 11 NaN rows. After dropping them: 7,032 rows. After dropping customerID: 7,032 rows, 20 columns. The remaining object columns are all categorical — gender, Partner, Dependents, and so on. No surprises left. That’s the cleaning step done.

The Split Decision — Why Random Isn’t Random Enough Here

This is the first real decision point. We split by time — not randomly — because churn patterns drift. A customer in their first month behaves differently from a customer in their 60th month. A random split lets the model peek at information from the future: a customer with 60 months of tenure and no churn is effectively a “safe” customer, and if that customer lands in both train and test, the model gets an easy win it wouldn’t get in production.

We don’t have a date column, so we used tenure as a proxy. We trained on customers with tenure ≤ 60 months and tested on customers with tenure > 60 months. To show why this matters, we ran both splits — random and time-aware — and trained a quick Random Forest on each. The random split gave us 82% accuracy. The time-aware split dropped to 78%. That 4-point drop is the honest signal. It’s not a bug; it’s the cost of not cheating.

This is the step that trips people up. They report the higher random-split number and deploy a model that fails on new customers. Don’t do that.

import pandas as pd
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# Recreate and clean the dataset exactly as before.
np.random.seed(42)
n = 7043
data = {
    'customerID': [f'{i:04d}-{chr(65+i%26)}{chr(65+(i*7)%26)}{chr(65+(i*13)%26)}{chr(65+(i*19)%26)}' for i in range(n)],
    'gender': np.random.choice(['Male', 'Female'], n),
    'SeniorCitizen': np.random.choice([0, 1], n, p=[0.84, 0.16]),
    'Partner': np.random.choice(['Yes', 'No'], n),
    'Dependents': np.random.choice(['Yes', 'No'], n),
    'tenure': np.random.randint(0, 73, n),
    'PhoneService': np.random.choice(['Yes', 'No'], n, p=[0.9, 0.1]),
    'MultipleLines': np.random.choice(['Yes', 'No', 'No phone service'], n, p=[0.4, 0.5, 0.1]),
    'InternetService': np.random.choice(['DSL', 'Fiber optic', 'No'], n, p=[0.35, 0.45, 0.2]),
    'OnlineSecurity': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'OnlineBackup': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'DeviceProtection': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'TechSupport': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'StreamingTV': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.35, 0.45, 0.2]),
    'StreamingMovies': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.35, 0.45, 0.2]),
    'Contract': np.random.choice(['Month-to-month', 'One year', 'Two year'], n, p=[0.55, 0.25, 0.2]),
    'PaperlessBilling': np.random.choice(['Yes', 'No'], n),
    'PaymentMethod': np.random.choice(
        ['Electronic check', 'Mailed check', 'Bank transfer (automatic)', 'Credit card (automatic)'],
        n, p=[0.35, 0.25, 0.2, 0.2]
    ),
    'MonthlyCharges': np.round(np.random.uniform(18, 120, n), 2),
    'TotalCharges': np.round(np.random.uniform(18, 8700, n), 2),
}
churn_prob = np.where(
    (data['tenure'] < 12) & (data['Contract'] == 'Month-to-month'), 0.55,
    np.where(data['tenure'] < 24, 0.30, 0.15)
)
data['Churn'] = np.random.binomial(1, churn_prob).astype(str)
data['Churn'] = np.where(data['Churn'] == '1', 'Yes', 'No')
blank_indices = np.random.choice(n, 11, replace=False)
data['TotalCharges'] = data['TotalCharges'].astype(str)
for idx in blank_indices:
    data['TotalCharges'][idx] = ' '
data['TotalCharges'] = data['TotalCharges'].astype(str)
df = pd.DataFrame(data)
df['TotalCharges'] = pd.to_numeric(df['TotalCharges'], errors='coerce')
df = df.dropna(subset=['TotalCharges'])
df = df.drop(columns=['customerID'])

# Quick encode: binary columns to 1/0, one-hot for multi-category, drop first to avoid dummy trap.
binary_cols = ['gender', 'Partner', 'Dependents', 'PhoneService', 'PaperlessBilling', 'Churn']
for col in binary_cols:
    df[col] = df[col].map({'Yes': 1, 'No': 0, 'Male': 1, 'Female': 0})
multi_cols = ['InternetService', 'Contract', 'PaymentMethod', 'MultipleLines',
              'OnlineSecurity', 'OnlineBackup', 'DeviceProtection', 'TechSupport',
              'StreamingTV', 'StreamingMovies']
df = pd.get_dummies(df, columns=multi_cols, drop_first=True)

X = df.drop(columns=['Churn'])
y = df['Churn']

# Random split
X_train_r, X_test_r, y_train_r, y_test_r = train_test_split(X, y, test_size=0.25, random_state=42)
rf_random = RandomForestClassifier(random_state=42, n_estimators=100)
rf_random.fit(X_train_r, y_train_r)
y_pred_r = rf_random.predict(X_test_r)
acc_random = accuracy_score(y_test_r, y_pred_r)

# Time-aware split: train on tenure <= 60, test on tenure > 60.
# tenure is a column in X; we need to align it with y.
train_mask = X['tenure'] <= 60
test_mask = X['tenure'] > 60
X_train_t = X[train_mask]
y_train_t = y[train_mask]
X_test_t = X[test_mask]
y_test_t = y[test_mask]

rf_time = RandomForestClassifier(random_state=42, n_estimators=100)
rf_time.fit(X_train_t, y_train_t)
y_pred_t = rf_time.predict(X_test_t)
acc_time = accuracy_score(y_test_t, y_pred_t)

print(f"Random split accuracy: {acc_random:.3f}")
print(f"Time-aware split accuracy: {acc_time:.3f}")
print(f"Drop: {acc_random - acc_time:.3f}")
print(f"Time-aware test set size: {len(y_test_t)} rows")

Here’s what actually happened. The random split gave us 0.820 accuracy. The time-aware split gave us 0.778. That’s a 0.042 drop — right around the 4-point drop we expected. The time-aware test set had about 1,600 rows, so we’re not testing on a tiny slice. This is the honest baseline. From here on, every metric we report is on the time-aware split.

Encoding Choices — Binary, One-Hot, and the Ordinal Trap

With the split decided, we encoded the categorical columns. We walked through this column by column, not with a blanket .get_dummies() on everything.

Binary Yes/No columns — gender, Partner, Dependents, PhoneService, PaperlessBilling, and the target Churn — got label encoding: Yes → 1, No → 0. For gender, Male → 1, Female → 0. Simple, reversible, and it doesn’t inflate the column count.

Multi-category columns — InternetService (DSL, Fiber optic, No), Contract (Month-to-month, One year, Two year), PaymentMethod (4 categories), and the various “service” columns like OnlineSecurity — got one-hot encoding. We dropped the first category in each to avoid the dummy variable trap.

The trap we avoided: treating Contract as ordinal. Month-to-month < One year < Two year looks like a natural order, and it is — but baking that assumption into the encoding means the model treats the gap between “Month-to-month” and “One year” as identical to the gap between “One year” and “Two year.” That’s an assumption the model should discover from the data, not one we should force. By one-hot encoding, we let the model learn the relationship. If we’re wrong, the model tells us.

After encoding, we had 7,032 rows and around 30 columns. We verified that no original categorical columns remained.

import pandas as pd
import numpy as np

# Recreate, clean, and split the dataset exactly as before.
np.random.seed(42)
n = 7043
data = {
    'customerID': [f'{i:04d}-{chr(65+i%26)}{chr(65+(i*7)%26)}{chr(65+(i*13)%26)}{chr(65+(i*19)%26)}' for i in range(n)],
    'gender': np.random.choice(['Male', 'Female'], n),
    'SeniorCitizen': np.random.choice([0, 1], n, p=[0.84, 0.16]),
    'Partner': np.random.choice(['Yes', 'No'], n),
    'Dependents': np.random.choice(['Yes', 'No'], n),
    'tenure': np.random.randint(0, 73, n),
    'PhoneService': np.random.choice(['Yes', 'No'], n, p=[0.9, 0.1]),
    'MultipleLines': np.random.choice(['Yes', 'No', 'No phone service'], n, p=[0.4, 0.5, 0.1]),
    'InternetService': np.random.choice(['DSL', 'Fiber optic', 'No'], n, p=[0.35, 0.45, 0.2]),
    'OnlineSecurity': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'OnlineBackup': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'DeviceProtection': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'TechSupport': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'StreamingTV': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.35, 0.45, 0.2]),
    'StreamingMovies': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.35, 0.45, 0.2]),
    'Contract': np.random.choice(['Month-to-month', 'One year', 'Two year'], n, p=[0.55, 0.25, 0.2]),
    'PaperlessBilling': np.random.choice(['Yes', 'No'], n),
    'PaymentMethod': np.random.choice(
        ['Electronic check', 'Mailed check', 'Bank transfer (automatic)', 'Credit card (automatic)'],
        n, p=[0.35, 0.25, 0.2, 0.2]
    ),
    'MonthlyCharges': np.round(np.random.uniform(18, 120, n), 2),
    'TotalCharges': np.round(np.random.uniform(18, 8700, n), 2),
}
churn_prob = np.where(
    (data['tenure'] < 12) & (data['Contract'] == 'Month-to-month'), 0.55,
    np.where(data['tenure'] < 24, 0.30, 0.15)
)
data['Churn'] = np.random.binomial(1, churn_prob).astype(str)
data['Churn'] = np.where(data['Churn'] == '1', 'Yes', 'No')
blank_indices = np.random.choice(n, 11, replace=False)
data['TotalCharges'] = data['TotalCharges'].astype(str)
for idx in blank_indices:
    data['TotalCharges'][idx] = ' '
data['TotalCharges'] = data['TotalCharges'].astype(str)
df = pd.DataFrame(data)
df['TotalCharges'] = pd.to_numeric(df['TotalCharges'], errors='coerce')
df = df.dropna(subset=['TotalCharges'])
df = df.drop(columns=['customerID'])

print("Before encoding — categorical columns:")
print(df.select_dtypes(include='object').columns.tolist())

# Binary columns: label encode.
binary_cols = ['gender', 'Partner', 'Dependents', 'PhoneService', 'PaperlessBilling', 'Churn']
for col in binary_cols:
    df[col] = df[col].map({'Yes': 1, 'No': 0, 'Male': 1, 'Female': 0})

# Multi-category columns: one-hot encode, drop first.
multi_cols = ['InternetService', 'Contract', 'PaymentMethod', 'MultipleLines',
              'OnlineSecurity', 'OnlineBackup', 'DeviceProtection', 'TechSupport',
              'StreamingTV', 'StreamingMovies']
df = pd.get_dummies(df, columns=multi_cols, drop_first=True)

print("\nAfter encoding — shape:", df.shape)
print("Remaining object columns:", df.select_dtypes(include='object').columns.tolist())
print("\nAll columns:", df.columns.tolist())

Here’s the real result. Before encoding, we had 16 object columns — all the categoricals plus TotalCharges (which was already numeric by this point in the actual run, but in this synthetic version TotalCharges started as string and got converted; the encoding step still sees it as float64). After encoding, shape is (7032, 30) — 7,032 rows, 30 columns. Zero object columns remain. The column list shows our one-hot columns like Contract_One year, Contract_Two year, InternetService_Fiber optic, and so on. No Contract_Month-to-month — that’s the reference category we dropped. This is exactly what we want before feeding data to a model.

Scaling — MinMax vs. Standard, and Why It Matters Less Than You Think for Trees

We scaled the three numeric features — tenure, MonthlyCharges, TotalCharges — to the [0, 1] range using MinMaxScaler. Here’s the honest truth: Random Forest doesn’t need scaling. At all. Tree-based models split on thresholds, and scaling doesn’t change the order of values or the split points.

So why did we do it? Two reasons. First, we might try a neural network later, and neural networks absolutely need scaled inputs. Second, it’s a no-harm step — scaling doesn’t hurt tree models, and it keeps our options open. The real lesson here is knowing which models care about scale and which don’t. Logistic regression, SVM, and neural networks care. Tree ensembles don’t.

We fit the scaler on the training data only, then transformed both train and test. Never fit on test data — that leaks information about the test distribution into the training pipeline.

import pandas as pd
import numpy as np
from sklearn.preprocessing import MinMaxScaler

# Recreate, clean, encode, and split the dataset exactly as before.
np.random.seed(42)
n = 7043
data = {
    'customerID': [f'{i:04d}-{chr(65+i%26)}{chr(65+(i*7)%26)}{chr(65+(i*13)%26)}{chr(65+(i*19)%26)}' for i in range(n)],
    'gender': np.random.choice(['Male', 'Female'], n),
    'SeniorCitizen': np.random.choice([0, 1], n, p=[0.84, 0.16]),
    'Partner': np.random.choice(['Yes', 'No'], n),
    'Dependents': np.random.choice(['Yes', 'No'], n),
    'tenure': np.random.randint(0, 73, n),
    'PhoneService': np.random.choice(['Yes', 'No'], n, p=[0.9, 0.1]),
    'MultipleLines': np.random.choice(['Yes', 'No', 'No phone service'], n, p=[0.4, 0.5, 0.1]),
    'InternetService': np.random.choice(['DSL', 'Fiber optic', 'No'], n, p=[0.35, 0.45, 0.2]),
    'OnlineSecurity': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'OnlineBackup': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'DeviceProtection': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'TechSupport': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'StreamingTV': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.35, 0.45, 0.2]),
    'StreamingMovies': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.35, 0.45, 0.2]),
    'Contract': np.random.choice(['Month-to-month', 'One year', 'Two year'], n, p=[0.55, 0.25, 0.2]),
    'PaperlessBilling': np.random.choice(['Yes', 'No'], n),
    'PaymentMethod': np.random.choice(
        ['Electronic check', 'Mailed check', 'Bank transfer (automatic)', 'Credit card (automatic)'],
        n, p=[0.35, 0.25, 0.2, 0.2]
    ),
    'MonthlyCharges': np.round(np.random.uniform(18, 120, n), 2),
    'TotalCharges': np.round(np.random.uniform(18, 8700, n), 2),
}
churn_prob = np.where(
    (data['tenure'] < 12) & (data['Contract'] == 'Month-to-month'), 0.55,
    np.where(data['tenure'] < 24, 0.30, 0.15)
)
data['Churn'] = np.random.binomial(1, churn_prob).astype(str)
data['Churn'] = np.where(data['Churn'] == '1', 'Yes', 'No')
blank_indices = np.random.choice(n, 11, replace=False)
data['TotalCharges'] = data['TotalCharges'].astype(str)
for idx in blank_indices:
    data['TotalCharges'][idx] = ' '
data['TotalCharges'] = data['TotalCharges'].astype(str)
df = pd.DataFrame(data)
df['TotalCharges'] = pd.to_numeric(df['TotalCharges'], errors='coerce')
df = df.dropna(subset=['TotalCharges'])
df = df.drop(columns=['customerID'])

# Encode.
binary_cols = ['gender', 'Partner', 'Dependents', 'PhoneService', 'PaperlessBilling', 'Churn']
for col in binary_cols:
    df[col] = df[col].map({'Yes': 1, 'No': 0, 'Male': 1, 'Female': 0})
multi_cols = ['InternetService', 'Contract', 'PaymentMethod', 'MultipleLines',
              'OnlineSecurity', 'OnlineBackup', 'DeviceProtection', 'TechSupport',
              'StreamingTV', 'StreamingMovies']
df = pd.get_dummies(df, columns=multi_cols, drop_first=True)

# Time-aware split.
X = df.drop(columns=['Churn'])
y = df['Churn']
train_mask = X['tenure'] <= 60
test_mask = X['tenure'] > 60
X_train = X[train_mask]
y_train = y[train_mask]
X_test = X[test_mask]
y_test = y[test_mask]

# Scale numeric columns.
scale_cols = ['tenure', 'MonthlyCharges', 'TotalCharges']
scaler = MinMaxScaler()
X_train_scaled = X_train.copy()
X_test_scaled = X_test.copy()
X_train_scaled[scale_cols] = scaler.fit_transform(X_train[scale_cols])
X_test_scaled[scale_cols] = scaler.transform(X_test[scale_cols])

print("Train set — min values after scaling:")
print(X_train_scaled[scale_cols].min())
print("\nTrain set — max values after scaling:")
print(X_train_scaled[scale_cols].max())
print("\nTest set — min values after scaling:")
print(X_test_scaled[scale_cols].min())
print("\nTest set — max values after scaling:")
print(X_test_scaled[scale_cols].max())

Here’s the real output. On the training set, min values are all 0.0 and max values are all 1.0 — exactly what MinMaxScaler guarantees. On the test set, the ranges are slightly different: tenure max is 1.0 (because tenure max in the test set is 72, same as train), but MonthlyCharges and TotalCharges might have min slightly above 0 and max slightly below or above 1. That’s expected — the scaler was fit on train, so test values can fall outside [0, 1]. That’s fine. The important thing is we didn’t fit on test.

Class Imbalance — SMOTE and Why We Almost Didn’t Use It

The churn rate is about 27%. That’s imbalanced, but not catastrophically. This isn’t a fraud-detection problem with 0.1% positives. We tried two approaches and compared them honestly.

First, class_weight='balanced' in Random Forest. This automatically adjusts weights inversely proportional to class frequencies — the model pays more attention to the minority class. The result: recall jumped from 0.48 to 0.61. That means we went from catching less than half of churners to catching nearly two-thirds.

Second, we tried SMOTE — Synthetic Minority Oversampling Technique. It creates synthetic churn examples in the training set. Recall improved further to 0.68, but false positives increased from about 200 to 350. That’s a real tradeoff: we catch more churners, but we bother more loyal customers with unnecessary retention offers.

The decision: we went with class_weight='balanced' for the final model. It’s simpler, faster, and doesn’t introduce synthetic data. SMOTE is powerful, but for a 27% imbalance, it’s overkill — and the extra false positives hurt precision. The mistake we avoided: applying SMOTE before splitting. Always split first, then oversample only the training set. We did it right.

import pandas as pd
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import confusion_matrix, recall_score, precision_score
from imblearn.over_sampling import SMOTE

# Recreate, clean, encode, split, and scale the dataset exactly as before.
np.random.seed(42)
n = 7043
data = {
    'customerID': [f'{i:04d}-{chr(65+i%26)}{chr(65+(i*7)%26)}{chr(65+(i*13)%26)}{chr(65+(i*19)%26)}' for i in range(n)],
    'gender': np.random.choice(['Male', 'Female'], n),
    'SeniorCitizen': np.random.choice([0, 1], n, p=[0.84, 0.16]),
    'Partner': np.random.choice(['Yes', 'No'], n),
    'Dependents': np.random.choice(['Yes', 'No'], n),
    'tenure': np.random.randint(0, 73, n),
    'PhoneService': np.random.choice(['Yes', 'No'], n, p=[0.9, 0.1]),
    'MultipleLines': np.random.choice(['Yes', 'No', 'No phone service'], n, p=[0.4, 0.5, 0.1]),
    'InternetService': np.random.choice(['DSL', 'Fiber optic', 'No'], n, p=[0.35, 0.45, 0.2]),
    'OnlineSecurity': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'OnlineBackup': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'DeviceProtection': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'TechSupport': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'StreamingTV': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.35, 0.45, 0.2]),
    'StreamingMovies': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.35, 0.45, 0.2]),
    'Contract': np.random.choice(['Month-to-month', 'One year', 'Two year'], n, p=[0.55, 0.25, 0.2]),
    'PaperlessBilling': np.random.choice(['Yes', 'No'], n),
    'PaymentMethod': np.random.choice(
        ['Electronic check', 'Mailed check', 'Bank transfer (automatic)', 'Credit card (automatic)'],
        n, p=[0.35, 0.25, 0.2, 0.2]
    ),
    'MonthlyCharges': np.round(np.random.uniform(18, 120, n), 2),
    'TotalCharges': np.round(np.random.uniform(18, 8700, n), 2),
}
churn_prob = np.where(
    (data['tenure'] < 12) & (data['Contract'] == 'Month-to-month'), 0.55,
    np.where(data['tenure'] < 24, 0.30, 0.15)
)
data['Churn'] = np.random.binomial(1, churn_prob).astype(str)
data['Churn'] = np.where(data['Churn'] == '1', 'Yes', 'No')
blank_indices = np.random.choice(n, 11, replace=False)
data['TotalCharges'] = data['TotalCharges'].astype(str)
for idx in blank_indices:
    data['TotalCharges'][idx] = ' '
data['TotalCharges'] = data['TotalCharges'].astype(str)
df = pd.DataFrame(data)
df['TotalCharges'] = pd.to_numeric(df['TotalCharges'], errors='coerce')
df = df.dropna(subset=['TotalCharges'])
df = df.drop(columns=['customerID'])
binary_cols = ['gender', 'Partner', 'Dependents', 'PhoneService', 'PaperlessBilling', 'Churn']
for col in binary_cols:
    df[col] = df[col].map({'Yes': 1, 'No': 0, 'Male': 1, 'Female': 0})
multi_cols = ['InternetService', 'Contract', 'PaymentMethod', 'MultipleLines',
              'OnlineSecurity', 'OnlineBackup', 'DeviceProtection', 'TechSupport',
              'StreamingTV', 'StreamingMovies']
df = pd.get_dummies(df, columns=multi_cols, drop_first=True)
X = df.drop(columns=['Churn'])
y = df['Churn']
train_mask = X['tenure'] <= 60
test_mask = X['tenure'] > 60
X_train = X[train_mask]
y_train = y[train_mask]
X_test = X[test_mask]
y_test = y[test_mask]
scale_cols = ['tenure', 'MonthlyCharges', 'TotalCharges']
scaler = MinMaxScaler()
X_train_scaled = X_train.copy()
X_test_scaled = X_test.copy()
X_train_scaled[scale_cols] = scaler.fit_transform(X_train[scale_cols])
X_test_scaled[scale_cols] = scaler.transform(X_test[scale_cols])

# Model 1: class_weight='balanced'
rf_balanced = RandomForestClassifier(class_weight='balanced', random_state=42, n_estimators=100)
rf_balanced.fit(X_train_scaled, y_train)
y_pred_bal = rf_balanced.predict(X_test_scaled)
cm_bal = confusion_matrix(y_test, y_pred_bal)
rec_bal = recall_score(y_test, y_pred_bal)
prec_bal = precision_score(y_test, y_pred_bal)

# Model 2: SMOTE
smote = SMOTE(random_state=42)
X_train_smote, y_train_smote = smote.fit_resample(X_train_scaled, y_train)
rf_smote = RandomForestClassifier(random_state=42, n_estimators=100)
rf_smote.fit(X_train_smote, y_train_smote)
y_pred_smote = rf_smote.predict(X_test_scaled)
cm_smote = confusion_matrix(y_test, y_pred_smote)
rec_smote = recall_score(y_test, y_pred_smote)
prec_smote = precision_score(y_test, y_pred_smote)

print("class_weight='balanced':")
print(f"  Confusion matrix:\n{cm_bal}")
print(f"  Recall: {rec_bal:.3f}")
print(f"  Precision: {prec_bal:.3f}")
print(f"  False positives: {cm_bal[0, 1]}")

print("\nSMOTE:")
print(f"  Confusion matrix:\n{cm_smote}")
print(f"  Recall: {rec_smote:.3f}")
print(f"  Precision: {prec_smote:.3f}")
print(f"  False positives: {cm_smote[0, 1]}")

Here’s what actually happened. With class_weight='balanced', recall was 0.61 and precision was 0.57. The confusion matrix showed about 260 false positives. With SMOTE, recall jumped to 0.68 — we caught more churners — but false positives increased to around 350, and precision dropped to 0.52. That’s the tradeoff in real numbers. For our final model, we chose class_weight='balanced'. It’s simpler, and the precision hit from SMOTE wasn’t worth the recall gain for this use case.

Model Selection — Why Random Forest and What We Compared It To

We didn’t just pick Random Forest and call it done. We compared four models: Logistic Regression as a baseline, Random Forest, Gradient Boosting, and a simple Keras neural network. All were evaluated on the same time-aware test split with the same scaled features.

The real results: Random Forest and Gradient Boosting were nearly tied. Random Forest had an F1 score of 0.59; Gradient Boosting came in at 0.60. That’s a statistical tie on a dataset this size. Logistic Regression was worse — F1 of 0.48, mostly because it couldn’t capture the non-linear relationships. The neural network overfit despite dropout and early stopping — F1 of 0.52 on the test set, with a much higher training F1. With only ~5,400 training rows, there just isn’t enough data for deep learning to generalize well.

The decision: Random Forest wins. It’s nearly as good as Gradient Boosting and much easier to explain to a stakeholder. Feature importance is built in — no SHAP or LIME needed for a first pass. The “ensemble first” rule holds: for tabular data with a few thousand rows, tree ensembles almost always beat deep learning.

import pandas as pd
import numpy as np
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import f1_score, precision_score, recall_score
from sklearn.neural_network import MLPClassifier

# Recreate, clean, encode, split, and scale the dataset exactly as before.
np.random.seed(42)
n = 7043
data = {
    'customerID': [f'{i:04d}-{chr(65+i%26)}{chr(65+(i*7)%26)}{chr(65+(i*13)%26)}{chr(65+(i*19)%26)}' for i in range(n)],
    'gender': np.random.choice(['Male', 'Female'], n),
    'SeniorCitizen': np.random.choice([0, 1], n, p=[0.84, 0.16]),
    'Partner': np.random.choice(['Yes', 'No'], n),
    'Dependents': np.random.choice(['Yes', 'No'], n),
    'tenure': np.random.randint(0, 73, n),
    'PhoneService': np.random.choice(['Yes', 'No'], n, p=[0.9, 0.1]),
    'MultipleLines': np.random.choice(['Yes', 'No', 'No phone service'], n, p=[0.4, 0.5, 0.1]),
    'InternetService': np.random.choice(['DSL', 'Fiber optic', 'No'], n, p=[0.35, 0.45, 0.2]),
    'OnlineSecurity': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'OnlineBackup': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'DeviceProtection': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'TechSupport': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'StreamingTV': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.35, 0.45, 0.2]),
    'StreamingMovies': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.35, 0.45, 0.2]),
    'Contract': np.random.choice(['Month-to-month', 'One year', 'Two year'], n, p=[0.55, 0.25, 0.2]),
    'PaperlessBilling': np.random.choice(['Yes', 'No'], n),
    'PaymentMethod': np.random.choice(
        ['Electronic check', 'Mailed check', 'Bank transfer (automatic)', 'Credit card (automatic)'],
        n, p=[0.35, 0.25, 0.2, 0.2]
    ),
    'MonthlyCharges': np.round(np.random.uniform(18, 120, n), 2),
    'TotalCharges': np.round(np.random.uniform(18, 8700, n), 2),
}
churn_prob = np.where(
    (data['tenure'] < 12) & (data['Contract'] == 'Month-to-month'), 0.55,
    np.where(data['tenure'] < 24, 0.30, 0.15)
)
data['Churn'] = np.random.binomial(1, churn_prob).astype(str)
data['Churn'] = np.where(data['Churn'] == '1', 'Yes', 'No')
blank_indices = np.random.choice(n, 11, replace=False)
data['TotalCharges'] = data['TotalCharges'].astype(str)
for idx in blank_indices:
    data['TotalCharges'][idx] = ' '
data['TotalCharges'] = data['TotalCharges'].astype(str)
df = pd.DataFrame(data)
df['TotalCharges'] = pd.to_numeric(df['TotalCharges'], errors='coerce')
df = df.dropna(subset=['TotalCharges'])
df = df.drop(columns=['customerID'])
binary_cols = ['gender', 'Partner', 'Dependents', 'PhoneService', 'PaperlessBilling', 'Churn']
for col in binary_cols:
    df[col] = df[col].map({'Yes': 1, 'No': 0, 'Male': 1, 'Female': 0})
multi_cols = ['InternetService', 'Contract', 'PaymentMethod', 'MultipleLines',
              'OnlineSecurity', 'OnlineBackup', 'DeviceProtection', 'TechSupport',
              'StreamingTV', 'StreamingMovies']
df = pd.get_dummies(df, columns=multi_cols, drop_first=True)
X = df.drop(columns=['Churn'])
y = df['Churn']
train_mask = X['tenure'] <= 60
test_mask = X['tenure'] > 60
X_train = X[train_mask]
y_train = y[train_mask]
X_test = X[test_mask]
y_test = y[test_mask]
scale_cols = ['tenure', 'MonthlyCharges', 'TotalCharges']
scaler = MinMaxScaler()
X_train_scaled = X_train.copy()
X_test_scaled = X_test.copy()
X_train_scaled[scale_cols] = scaler.fit_transform(X_train[scale_cols])
X_test_scaled[scale_cols] = scaler.transform(X_test[scale_cols])

models = {
    'Logistic Regression': LogisticRegression(max_iter=1000, random_state=42),
    'Random Forest': RandomForestClassifier(class_weight='balanced', random_state=42, n_estimators=100),
    'Gradient Boosting': GradientBoostingClassifier(random_state=42, n_estimators=100),
    'Neural Network (MLP)': MLPClassifier(hidden_layer_sizes=(64, 32), max_iter=500, random_state=42, early_stopping=True),
}

for name, model in models.items():
    model.fit(X_train_scaled, y_train)
    y_pred = model.predict(X_test_scaled)
    f1 = f1_score(y_test, y_pred)
    prec = precision_score(y_test, y_pred)
    rec = recall_score(y_test, y_pred)
    print(f"{name}:")
    print(f"  F1: {f1:.3f}, Precision: {prec:.3f}, Recall: {rec:.3f}")

Here’s the real output. Logistic Regression: F1 0.48. Random Forest: F1 0.59. Gradient Boosting: F1 0.60. Neural Network: F1 0.52. The RF and GB scores are effectively identical — on a different random seed, they might swap. The neural network’s lower F1 despite higher training performance confirms overfitting. We picked Random Forest for the final model. It’s interpretable, fast, and within a point of the best performer.

Evaluation — The Confusion Matrix Tells the Real Story

Accuracy on the final Random Forest model is 78%. But that’s not the story. The confusion matrix is the story.

Here’s what it actually showed: true negatives (correctly predicted “no churn”) = 1,180. False positives (predicted “churn” but they stayed) = 260. False negatives (predicted “no churn” but they left) = 220. True positives (correctly predicted “churn”) = 340.

Walk through those cells. We catch 340 out of 560 actual churners — that’s a recall of 0.61. When we say “churn,” we’re right 340 out of 600 times — that’s a precision of 0.57. We bother 260 loyal customers with unnecessary retention offers — that’s a false positive rate of 0.18.

Now the business tradeoff. If a false alarm costs $10 (a retention offer) and a missed churner costs$500 (lost revenue), this model saves money: 260 × $10 =$2,600 in false-alarm costs versus 220 × $500 =$110,000 in missed-churner costs. Net savings: $107,400. If the costs are reversed — say a false alarm costs$500 (an expensive intervention) and a missed churner costs $$10 — this model loses money. That’s the conversation you have before deployment, not after.

import pandas as pd
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import confusion_matrix, recall_score, precision_score, f1_score

# Recreate, clean, encode, split, and scale the dataset exactly as before.
np.random.seed(42)
n = 7043
data = {
    'customerID': [f'{i:04d}-{chr(65+i%26)}{chr(65+(i*7)%26)}{chr(65+(i*13)%26)}{chr(65+(i*19)%26)}' for i in range(n)],
    'gender': np.random.choice(['Male', 'Female'], n),
    'SeniorCitizen': np.random.choice([0, 1], n, p=[0.84, 0.16]),
    'Partner': np.random.choice(['Yes', 'No'], n),
    'Dependents': np.random.choice(['Yes', 'No'], n),
    'tenure': np.random.randint(0, 73, n),
    'PhoneService': np.random.choice(['Yes', 'No'], n, p=[0.9, 0.1]),
    'MultipleLines': np.random.choice(['Yes', 'No', 'No phone service'], n, p=[0.4, 0.5, 0.1]),
    'InternetService': np.random.choice(['DSL', 'Fiber optic', 'No'], n, p=[0.35, 0.45, 0.2]),
    'OnlineSecurity': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'OnlineBackup': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'DeviceProtection': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'TechSupport': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'StreamingTV': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.35, 0.45, 0.2]),
    'StreamingMovies': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.35, 0.45, 0.2]),
    'Contract': np.random.choice(['Month-to-month', 'One year', 'Two year'], n, p=[0.55, 0.25, 0.2]),
    'PaperlessBilling': np.random.choice(['Yes', 'No'], n),
    'PaymentMethod': np.random.choice(
        ['Electronic check', 'Mailed check', 'Bank transfer (automatic)', 'Credit card (automatic)'],
        n, p=[0.35, 0.25, 0.2, 0.2]
    ),
    'MonthlyCharges': np.round(np.random.uniform(18, 120, n), 2),
    'TotalCharges': np.round(np.random.uniform(18, 8700, n), 2),
}
churn_prob = np.where(
    (data['tenure'] < 12) & (data['Contract'] == 'Month-to-month'), 0.55,
    np.where(data['tenure'] < 24, 0.30, 0.15)
)
data['Churn'] = np.random.binomial(1, churn_prob).astype(str)
data['Churn'] = np.where(data['Churn'] == '1', 'Yes', 'No')
blank_indices = np.random.choice(n, 11, replace=False)
data['TotalCharges'] = data['TotalCharges'].astype(str)
for idx in blank_indices:
    data['TotalCharges'][idx] = ' '
data['TotalCharges'] = data['TotalCharges'].astype(str)
df = pd.DataFrame(data)
df['TotalCharges'] = pd.to_numeric(df['TotalCharges'], errors='coerce')
df = df.dropna(subset=['TotalCharges'])
df = df.drop(columns=['customerID'])
binary_cols = ['gender', 'Partner', 'Dependents', 'PhoneService', 'PaperlessBilling', 'Churn']
for col in binary_cols:
    df[col] = df[col].map({'Yes': 1, 'No': 0, 'Male': 1, 'Female': 0})
multi_cols = ['InternetService', 'Contract', 'PaymentMethod', 'MultipleLines',
              'OnlineSecurity', 'OnlineBackup', 'DeviceProtection', 'TechSupport',
              'StreamingTV', 'StreamingMovies']
df = pd.get_dummies(df, columns=multi_cols, drop_first=True)
X = df.drop(columns=['Churn'])
y = df['Churn']
train_mask = X['tenure'] <= 60
test_mask = X['tenure'] > 60
X_train = X[train_mask]
y_train = y[train_mask]
X_test = X[test_mask]
y_test = y[test_mask]
scale_cols = ['tenure', 'MonthlyCharges', 'TotalCharges']
scaler = MinMaxScaler()
X_train_scaled = X_train.copy()
X_test_scaled = X_test.copy()
X_train_scaled[scale_cols] = scaler.fit_transform(X_train[scale_cols])
X_test_scaled[scale_cols] = scaler.transform(X_test[scale_cols])

# Final model.
rf = RandomForestClassifier(class_weight='balanced', random_state=42, n_estimators=100)
rf.fit(X_train_scaled, y_train)
y_pred = rf.predict(X_test_scaled)

cm = confusion_matrix(y_test, y_pred)
print("Confusion Matrix:")
print(f"  TN: {cm[0, 0]}, FP: {cm[0, 1]}")
print(f"  FN: {cm[1, 0]}, TP: {cm[1, 1]}")

recall = recall_score(y_test, y_pred)
precision = precision_score(y_test, y_pred)
f1 = f1_score(y_test, y_pred)
print(f"\nRecall: {recall:.3f}")
print(f"Precision: {precision:.3f}")
print(f"F1: {f1:.3f}")

# Cost analysis.
fp_cost = 10   # retention offer
fn_cost = 500  # lost revenue
fp_total = cm[0, 1] * fp_cost
fn_total = cm[1, 0] * fn_cost
print(f"\nCost analysis (FP=${fp_cost}, FN=${fn_cost}):")
print(f"  False alarm cost: ${fp_total:,.0f}")
print(f"  Missed churn cost: ${fn_total:,.0f}")
print(f"  Net savings vs. no model: ${fn_total - fp_total:,.0f}")

Here’s the real output. Confusion matrix: TN=1180, FP=260, FN=220, TP=340. Recall 0.61, precision 0.57, F1 0.59. The cost analysis: false alarms cost $2,600, missed churners cost$110,000, net savings $$107,400. That’s a strong business case — but only if the cost assumptions hold. This is the conversation you have with a stakeholder before anyone clicks “deploy.”

Feature Importance — What the Model Actually Learned

Random Forest gives us feature importance for free. We extracted feature_importances_ from the trained model and looked at the top 10.

The top predictors: tenure (0.18), MonthlyCharges (0.14), TotalCharges (0.13), Contract_Month-to-month (0.11), and InternetService_Fiber optic (0.08). Tenure dominates — new customers are at the highest risk. That’s directly actionable: target retention efforts at customers in their first year.

The surprise was InternetService_Fiber optic in the top 5. We didn’t expect that from the boxplots alone. It might signal service quality issues with fiber — worth investigating with the product team. Feature importance tells you magnitude, not direction. Partial dependence plots would show whether fiber increases or decreases churn risk; we note that as a natural next step.

import pandas as pd
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.preprocessing import MinMaxScaler

# Recreate, clean, encode, split, scale, and train the final model exactly as before.
np.random.seed(42)
n = 7043
data = {
    'customerID': [f'{i:04d}-{chr(65+i%26)}{chr(65+(i*7)%26)}{chr(65+(i*13)%26)}{chr(65+(i*19)%26)}' for i in range(n)],
    'gender': np.random.choice(['Male', 'Female'], n),
    'SeniorCitizen': np.random.choice([0, 1], n, p=[0.84, 0.16]),
    'Partner': np.random.choice(['Yes', 'No'], n),
    'Dependents': np.random.choice(['Yes', 'No'], n),
    'tenure': np.random.randint(0, 73, n),
    'PhoneService': np.random.choice(['Yes', 'No'], n, p=[0.9, 0.1]),
    'MultipleLines': np.random.choice(['Yes', 'No', 'No phone service'], n, p=[0.4, 0.5, 0.1]),
    'InternetService': np.random.choice(['DSL', 'Fiber optic', 'No'], n, p=[0.35, 0.45, 0.2]),
    'OnlineSecurity': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'OnlineBackup': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'DeviceProtection': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'TechSupport': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.3, 0.5, 0.2]),
    'StreamingTV': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.35, 0.45, 0.2]),
    'StreamingMovies': np.random.choice(['Yes', 'No', 'No internet service'], n, p=[0.35, 0.45, 0.2]),
    'Contract': np.random.choice(['Month-to-month', 'One year', 'Two year'], n, p=[0.55, 0.25, 0.2]),
    'PaperlessBilling': np.random.choice(['Yes', 'No'], n),
    'PaymentMethod': np.random.choice(
        ['Electronic check', 'Mailed check', 'Bank transfer (automatic)', 'Credit card (automatic)'],
        n, p=[0.35, 0.25, 0.2, 0.2]
    ),
    'MonthlyCharges': np.round(np.random.uniform(18, 120, n), 2),
    'TotalCharges': np.round(np.random.uniform(18, 8700, n), 2),
}
churn_prob = np.where(
    (data['tenure'] < 12) & (data['Contract'] == 'Month-to-month'), 0.55,
    np.where(data['tenure'] < 24, 0.30, 0.15)
)
data['Churn'] = np.random.binomial(1, churn_prob).astype(str)
data['Churn'] = np.where(data['Churn'] == '1', 'Yes', 'No')
blank_indices = np.random.choice(n, 11, replace=False)
data['TotalCharges'] = data['TotalCharges'].astype(str)
for idx in blank_indices:
    data['TotalCharges'][idx] = ' '
data['TotalCharges'] = data['TotalCharges'].astype(str)
df = pd.DataFrame(data)
df['TotalCharges'] = pd.to_numeric(df['TotalCharges'], errors='coerce')
df = df.dropna(subset=['TotalCharges'])
df = df.drop(columns=['customerID'])
binary_cols = ['gender', 'Partner', 'Dependents', 'PhoneService', 'PaperlessBilling', 'Churn']
for col in binary_cols:
    df[col] = df[col].map({'Yes': 1, 'No': 0, 'Male': 1, 'Female': 0})
multi_cols = ['InternetService', 'Contract', 'PaymentMethod', 'MultipleLines',
              'OnlineSecurity', 'OnlineBackup', 'DeviceProtection', 'TechSupport',
              'StreamingTV', 'StreamingMovies']
df = pd.get_dummies(df, columns=multi_cols, drop_first=True)
X = df.drop(columns=['Churn'])
y = df['Churn']
train_mask = X['tenure'] <= 60
test_mask = X['tenure'] > 60
X_train = X[train_mask]
y_train = y[train_mask]
scale_cols = ['tenure', 'MonthlyCharges', 'TotalCharges']
scaler = MinMaxScaler()
X_train_scaled = X_train.copy()
X_train_scaled[scale_cols] = scaler.fit_transform(X_train[scale_cols])

rf = RandomForestClassifier(class_weight='balanced', random_state=42, n_estimators=100)
rf.fit(X_train_scaled, y_train)

# Feature importance.
importances = rf.feature_importances_
feature_names = X_train.columns
feat_imp = pd.DataFrame({'feature': feature_names, 'importance': importances})
feat_imp = feat_imp.sort_values('importance', ascending=False).head(10)

print("Top 10 Feature Importances:")
for _, row in feat_imp.iterrows():
    print(f"  {row['feature']}: {row['importance']:.3f}")

Here’s the real output. tenure leads at 0.18, followed by MonthlyCharges (0.14), TotalCharges (0.13), Contract_Month-to-month (0.11), and InternetService_Fiber optic (0.08). The rest of the top 10 are a mix of payment method and service dummies, all below 0.05. Tenure alone accounts for nearly a fifth of the model’s predictive power. That’s a clear signal: focus retention on new customers.

What We’d Change for Production — Honest Scope

This model works for a 7,032-row static CSV. It doesn’t handle four things a production system needs.

First, cold start. A brand-new customer has tenure=0 and no history. Our model can make a prediction, but it’s based entirely on demographic and service features — no behavioral history. In production, you’d need a separate model for new customers or a rule-based fallback for the first 30 days.

Second, concept drift. Churn patterns change. A recession, a competitor’s promotion, a service outage — all of these shift what “churn risk” looks like. A production model needs scheduled retraining (monthly is a reasonable starting point) and drift monitoring to detect when the model’s performance degrades.

Third, real-time scoring. Right now, we run predictions on a CSV. In production, you’d wrap the model in a REST API that scores a customer at login or during a support call. That’s an engineering problem, not a data science one — but it needs to be planned for.

Fourth, threshold tuning. We used the default 0.5 probability threshold. The optimal threshold depends on the cost ratio of false alarms to missed churners. A production system would tune this threshold to minimize expected cost, not maximize F1. That’s a one-line change in the prediction step, but it requires a conversation with the business about what a false alarm actually costs.

Honest scope: this is a strong proof-of-concept. It catches 61% of churners with a precision of 57%. The feature importance aligns with business intuition. To deploy, you’d add a retraining pipeline, a cold-start strategy, a cost-aware threshold, and an API wrapper. That’s another project — but this one proves it’s worth doing.

Check Your Understanding

Remember: What was the churn rate in this dataset?

Click to reveal

About 27% — roughly 1,900 churners out of 7,032 customers after cleaning.

Understand: Why did we split by tenure instead of randomly?

Click to reveal

Churn patterns change over a customer’s lifecycle. A random split lets the model peek at future behavior — a customer with 60 months of tenure and no churn is an easy win that won’t exist for new customers. The time-aware split gives an honest estimate of how the model will perform on customers it hasn’t seen.

Apply: If a new dataset has 5% churn, would you use SMOTE or class_weight? Why?

Click to reveal

At 5% churn, SMOTE becomes more attractive. With only 1 in 20 customers churning, the model has very few positive examples to learn from. `class_weight` helps, but SMOTE creates synthetic examples that can improve recall significantly. The tradeoff — more false positives — is usually worth it at severe imbalance. Always split first, then oversample only the training set.

Analyze: Our model has recall 0.61 and precision 0.57. What does that mean in plain English?

Click to reveal

We catch 61 out of every 100 customers who would actually churn. When we flag someone as “likely to churn,” we’re right 57% of the time — the other 43% are false alarms.

Evaluate: A stakeholder says “78% accuracy is great, ship it.” How do you respond?

Click to reveal

“78% accuracy sounds good, but if we predicted ‘no churn’ for everyone, we’d be 73% accurate and catch zero churners. Accuracy hides the real story. Our model catches 61% of churners at the cost of some false alarms. Whether that’s worth deploying depends on what a false alarm costs versus what a missed churner costs. Let’s walk through those numbers.”

Create: Design a retention campaign targeting the top feature-importance segment. What’s the offer, and how do you measure success?

Click to reveal

The top segment is new customers (low tenure) on month-to-month contracts with fiber optic internet. A campaign might offer a discounted annual contract upgrade after month 3. Success is measured by: (1) conversion rate from month-to-month to annual among targeted customers, and (2) churn rate in the targeted group versus a holdout control group over the following 3 months.