Pearson vs Spearman vs Kendall: Picking the Right Correlation for Your Data

The ‘Ice Cream and Sunburn’ Problem: Why One Number Isn’t Enough

Imagine tracking ice cream sales and sunburn cases at a beach. Temperature rises, both go up. Draw a graph and you’ll see a clear pattern. Correlation measures how much two variables move together.

Most people think correlation is one thing. It isn’t. It’s a lens. Use the wrong one and your data looks like a blurry mess even when a perfect relationship sits underneath.

Pearson is the straight-line method. It wants a constant rate of change. But life doesn’t always move in straight lines. Some things grow exponentially — a viral video, compound interest. Pearson will call that relationship “weak” just because it curves. Spearman and Kendall take a different approach. They rank values instead of measuring them directly, caring about order rather than exact amounts.

Now look at a relationship that’s perfectly consistent but not a straight line.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

# Create a non-linear but perfectly 'upward' relationship
x = np.linspace(1, 10, 100)
y = np.exp(x)  # Exponential growth

df = pd.DataFrame({'X': x, 'Y': y})

plt.figure(figsize=(8, 4))
plt.scatter(df.X, df.Y, alpha=0.6)
plt.title("A Perfect Relationship that Isn't a Straight Line")
plt.xlabel("X (Time)")
plt.ylabel("Y (Growth)")
plt.show()

print(f"Pearson Correlation: {df.X.corr(df.Y, method='pearson'):.3f}")
print(f"Spearman Correlation: {df.X.corr(df.Y, method='spearman'):.3f}")

The Pearson score comes out to roughly 0.71. Sounds decent, but it’s misleading. The Spearman score hits a perfect 1.0. Why? Because every time X goes up, Y always goes up. Spearman sees the perfection; Pearson only sees the curve.

Pearson: The Ruler for Straight Lines

Think of Pearson’s correlation as a ruler. It measures how tightly your data points hug a single straight line. Draw a clean line through your dots, and Pearson will tell you so.

In other words, Pearson looks for a constant rate of change. If X goes up by 1, Y should go up by a fixed amount — say, 2. The catch: Pearson is very sensitive to outliers. One stray point can sink the whole score.

# Linear data with one extreme outlier -- the outlier is bigger in
# magnitude, but it's still the largest value, so it never breaks the
# increasing order of the data
x_lin = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
y_lin = np.array([2, 4, 6, 8, 10, 12, 14, 16, 18, 200])

df_outlier = pd.DataFrame({'X': x_lin, 'Y': y_lin})
pearson_val = df_outlier.X.corr(df_outlier.Y, method='pearson')
spearman_val = df_outlier.X.corr(df_outlier.Y, method='spearman')

print(f"Pearson with one outlier: {pearson_val:.3f}")
print(f"Spearman with one outlier: {spearman_val:.3f}")

Pearson drops to 0.59. Ninety percent of the data is a perfect straight line. Yet that single jump to 200 at the end drags the score down, because Pearson uses the actual distances — the raw values — to compute the score. Spearman, computed on the exact same data, stays at a perfect 1.0: the outlier is bigger, but it’s still the largest value in the series, so the ranking (1st, 2nd, 3rd, … 10th) never changes. Spearman doesn’t feel the jump because it never looks at the size of the jump — only the order.

But that isolation only holds when the outlier doesn’t scramble the order. Watch what happens if the same kind of extreme value lands somewhere that breaks the ranking instead of just stretching it:

# Counterexample: an outlier that also flips the rank order
y_flip = np.array([2, 4, 6, 8, 10, 12, 14, 16, 18, 2])  # last point drops back down

df_flip = pd.DataFrame({'X': x_lin, 'Y': y_flip})
pearson_flip = df_flip.X.corr(df_flip.Y, method='pearson')
spearman_flip = df_flip.X.corr(df_flip.Y, method='spearman')

print(f"Pearson, rank-flip case: {pearson_flip:.3f}")
print(f"Spearman, rank-flip case: {spearman_flip:.3f}")

Now Pearson reads 0.54, and Spearman reads 0.51 — lower than Pearson. Ranking the data doesn’t save you here, because the last point isn’t just a large value anymore; it’s now the smallest value in the series, so it’s out of order relative to everything before it. Spearman is robust to how far an outlier jumps, not to whether it jumps out of order.

Spearman: The ‘Rank’ Specialist

Spearman works differently. Think of it as Pearson’s cousin who only cares about who came in first, second, and third. The key difference: Spearman doesn’t look at your numbers — it looks at their ranks.

If you have heights of 5’0”, 5’1”, and 6’10”, Spearman just sees 1st, 2nd, and 3rd. The gap between 2nd and 3rd doesn’t matter. It converts your data into a leaderboard.

That makes it a good fit for ‘curvy’ (monotonic) data. As long as the values keep moving in the same direction, Spearman handles it fine.

# Let's check the ranks of our exponential data from earlier
df['X_rank'] = df['X'].rank()
df['Y_rank'] = df['Y'].rank()

# Now calculate Pearson on the RANKS
rank_corr = df['X_rank'].corr(df['Y_rank'], method='pearson')
print(f"Pearson on Ranks (which is Spearman): {rank_corr:.3f}")

The result is 1.0. By turning values into ranks, we flattened the curve into a straight line of ranks. With skewed data or outliers, Spearman is usually more honest than Pearson.

Kendall’s Tau: The Logic of Pairs

Kendall’s Tau works differently. Rather than scanning the whole line or a leaderboard, it acts like a voting system. It examines every possible pair of points and asks: do these two agree on direction?

Take two points, A and B. If Point B has a higher X value than Point A, check whether it also has a higher Y value. If yes, they’re ‘concordant’ — they agree. If not, they’re ‘discordant’ — they disagree. Kendall tallies the agreements and disagreements to compute the score.

This is the most robust method when you have very small datasets or lots of ties (where multiple people have the same score).

# Small dataset example
x_small = [1, 2, 3, 4, 5]
y_small = [1, 3, 2, 5, 4]

df_small = pd.DataFrame({'X': x_small, 'Y': y_small})
kendall_val = df_small.X.corr(df_small.Y, method='kendall')

print(f"Kendall's Tau on small data: {kendall_val:.3f}")

The result is 0.6 — moderate positive agreement. Most, though not all, pairs move in the same direction. In a small sample of 5 points, Kendall gives a more conservative, reliable estimate. Spearman works well for general use. But when the sample size is tiny, researchers often prefer Kendall — it has better statistical properties in that range.

The Ultimate Cheat Sheet: Which One Do I Use?

Choosing the right tool doesn’t have to be a headache. Here’s the plain decision tree:

1. Use Pearson if: Your data looks like a straight line, you have no major outliers, and your data is ‘normal’ (bell-curve shaped).
2. Use Spearman if: Your data is curvy but always goes up (or down), you have outliers, or your data is ordinal (like ‘Satisfied’, ‘Neutral’, ‘Unsatisfied’).
3. Use Kendall if: You have a very small number of data points or many tied values.

When Pearson and Spearman disagree: don’t just default to whichever is higher — run all three and diagnose why they disagree. As the examples above show, a low Pearson relative to Spearman can mean curvature or a magnitude-only outlier (Spearman is the safer read). But if Spearman is also low, or lower than Pearson, suspect an outlier that flipped the rank order rather than just stretching a value — ranking won’t rescue you from that.

Let’s look at all three on a messy dataset to see the difference:

# Messy data: monotonic trend with one point that's an extreme value,
# but still doesn't break the ordering
data = {
    'A': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
    'B': [2, 4, 6, 8, 10, 12, 14, 16, 18, 1000] # A huge spike, but still the largest value
}
df_messy = pd.DataFrame(data)

results = {
    'Pearson': df_messy.A.corr(df_messy.B, method='pearson'),
    'Spearman': df_messy.A.corr(df_messy.B, method='spearman'),
    'Kendall': df_messy.A.corr(df_messy.B, method='kendall')
}

for name, val in results.items():
    print(f"{name}: {val:.3f}")

In this messy result, Pearson shows a 0.54 — weak, dragged down by that one huge value. Spearman shows a perfect 1.0. The spike is still the largest value in the series, so it never breaks the “always goes up” pattern. Spearman sees a perfectly consistent relationship; Pearson only sees a wrecked straight line. This is the same magnitude-only robustness the first outlier example showed, just with a bigger spike. Here’s the key distinction, worth repeating: Spearman is robust to outliers in magnitude, but not to outliers that flip the rank order — as the counterexample above demonstrated, if that extreme value had landed out of order instead, it would hurt Spearman too.

Next time you run a correlation, don’t just settle for the default. Ask yourself: ‘Am I looking for a straight line, or just a relationship?’

Check Your Understanding

Questions below move from simple recall up to open-ended design, roughly following Bloom’s Taxonomy.

Remember What does Spearman’s correlation actually compute its score from — the raw values, or something else?

Understand In your own words, explain why Pearson gives a “weak” 0.71 score for the exponential (X, exp(X)) relationship even though the relationship is perfectly consistent — what specifically is Pearson penalizing?

Apply Using the article’s corrected messy-data example (B = [2,4,…,18,1000]), explain why Spearman still reports a perfect 1.0 despite the extreme value 1000 — what property of the data makes it robust to that specific kind of outlier?

Analyze The article’s final note explains that Spearman is “robust to outliers in magnitude, but not to outliers that flip the rank order.” Walk through what would happen to the Spearman score if that same extreme value (1000) had instead been placed at position 5 instead of position 10 — would Spearman still report a perfect correlation?

Evaluate The article’s guidance says Spearman is usually the safer read when Pearson looks surprisingly low. Critique this as a default: what real information does Spearman throw away by converting values to ranks, and can you think of an analysis (not just a correlation coefficient) where that lost magnitude information would actually matter?

Create Design a correlation analysis for a new scenario: a company wants to know if hours_of_onboarding_training relates to 90_day_retention, where the data has 3 employees who left almost immediately (outliers) and a mostly monotonic trend for everyone else. Walk through which correlation method(s) from the article’s cheat sheet you’d run and in what order, to build confidence that any signal you see is real and not an artifact of those 3 outliers.

---

Apply What You Learned

Scenario: You ran a correlation analysis on a dataset with a monotonic-but-nonlinear trend and one extreme magnitude outlier. Pearson returned 0.54; Spearman returned 1.0. A VP on the review panel asks: “The default Pearson score looks weak — are you sure there’s a real relationship here?”

Deliverable: A 200–400 word stakeholder memo (prose, not a notebook cell) defending your choice of Spearman over Pearson for this dataset. Write it as if it will be read aloud at the review.

Rubric (checklist):

• Cites the article’s exponential example (Pearson ≈ 0.71 vs Spearman = 1.0) to show that a “low” Pearson can mean curvature, not weakness
• Explains that the 0.54 here is driven by the single outlier’s leverage on the straight-line fit — not by absence of a relationship
• Names the mechanism: Spearman computes Pearson on ranks, which is why a magnitude spike like 1000 leaves the score unaffected as long as rank order is preserved
• Volunteers at least one caveat a sharp reviewer would catch: Spearman discards magnitude information, and would also break if the outlier flipped the rank order (e.g., 1000 landed mid-list)
• Written in stakeholder-accessible prose — no unexplained jargon, no code blocks, no “you’ll see in the notebook”