Speeding Up Python With Numba Jit Compilation With
You’ve got a loop that processes 10 million numbers. It’s a simple calculation — sum of squares — but it takes over two seconds in pure Python. You know that in C, it would finish in a few milliseconds. You’ve thought about learning C, but you have deadlines. You’ve thought about rewriting it with NumPy vectorization, but your algorithm is a recurrence relation — it can’t be vectorized. You’re stuck.
What if I told you that adding one line — a single decorator — could make your loop run 50x faster, without leaving Python? That’s exactly what Numba’s JIT compiler does.
Let’s feel the pain first, so we can appreciate the cure.
import time
import random
# Generate 10 million random numbers
data = [random.random() for _ in range(10_000_000)]
# Pure Python loop: sum of squares
def sum_of_squares_py(arr):
total = 0.0
for x in arr:
total += x * x
return total
start = time.perf_counter()
result_py = sum_of_squares_py(data)
elapsed_py = time.perf_counter() - start
print(f"Pure Python: {elapsed_py:.4f}s, result = {result_py:.4f}")
On a typical laptop, this takes about 1.5–2.5 seconds. That’s an eternity in computing. And it’s not doing anything complicated — just multiplication and addition. The problem is Python’s overhead: dynamic type checking, interpretation, and the GIL (Global Interpreter Lock) all add up.
Now let’s see what happens when we add one line.
What Is a JIT Compiler, Really? (Intuition, Not Theory)
Before we write any code, let’s understand what’s happening under the hood. A JIT (Just-In-Time) compiler is like a translator who listens to you speak, then writes a perfect script in the audience’s language. The first time you give your speech, it takes a while because the translator is writing the script. But the second time? Instant.
Here’s what’s actually going on with Python vs. Numba:
- Python is interpreted: Every line is read and executed on the fly. Every time your loop runs
total += x * x, Python checks: “Istotala number? Isxa number? What type of number?” It does this for every single iteration — 10 million times. - Numba’s JIT compiler watches the types: On the first call to your function, Numba sees that
totalis a float andxis a float. It then generates specialized machine code that works only for those types. No type checks, no interpretation — just raw machine instructions. - On the second call, it’s cached: The compiled machine code is stored. The function runs at C-like speed. No compilation overhead, no type checks.
This is the hardest part: the first call is slow. Many users panic. But that’s normal — the speedup comes on the second, third, and millionth call.
Numba uses LLVM under the hood — the same compiler infrastructure that powers Rust, Swift, and Clang. But you don’t need to know that to use it. Think of it as “magic that works best on loops and NumPy arrays.”
Let’s see the syntax before we dive into the timing.
from numba import jit
@jit
def sum_of_squares_numba(arr):
total = 0.0
for x in arr:
total += x * x
return total
That’s it. A single @jit decorator. Now let’s see what happens when we run it.
Your First @jit: The One-Liner That Changes Everything
First, install Numba if you haven’t:
pip install numba
Now let’s benchmark the same function with and without the decorator. Remember, the first call includes compilation time, so we’ll call it twice to see the real speed.
import time
import random
from numba import jit
# Generate data
data = [random.random() for _ in range(10_000_000)]
# Pure Python version (no decorator)
def sum_of_squares_py(arr):
total = 0.0
for x in arr:
total += x * x
return total
# Numba version
@jit
def sum_of_squares_numba(arr):
total = 0.0
for x in arr:
total += x * x
return total
# Warm-up call for Python (to exclude any system-level caching)
_ = sum_of_squares_py(data[:10])
# Time pure Python
start = time.perf_counter()
result_py = sum_of_squares_py(data)
elapsed_py = time.perf_counter() - start
print(f"Pure Python: {elapsed_py:.4f}s")
# First call to Numba (includes compilation)
start = time.perf_counter()
result_numba_1 = sum_of_squares_numba(data)
elapsed_numba_1 = time.perf_counter() - start
print(f"Numba (first call, includes compilation): {elapsed_numba_1:.4f}s")
# Second call to Numba (cached, no compilation)
start = time.perf_counter()
result_numba_2 = sum_of_squares_numba(data)
elapsed_numba_2 = time.perf_counter() - start
print(f"Numba (second call, cached): {elapsed_numba_2:.4f}s")
# Speedup factor
print(f"\nSpeedup (second call vs. Python): {elapsed_py / elapsed_numba_2:.1f}x")
You’ll see something like:
Pure Python: 1.8523s
Numba (first call, includes compilation): 0.2841s
Numba (second call, cached): 0.0152s
Speedup (second call vs. Python): 121.9x
Let’s interpret these numbers:
- Pure Python took 1.85 seconds. That’s the baseline pain.
- Numba’s first call took 0.28 seconds. That includes compilation time. It’s already faster than Python, but not by much.
- Numba’s second call took 0.015 seconds. That’s 122x faster than pure Python. The compilation cost is paid once and amortized over every subsequent call.
Now here’s an important detail: I used @jit without any arguments. In modern Numba (version 0.57+), @jit defaults to nopython mode — the mode that gives you the big speedup. If Numba can’t compile your function in nopython mode, it falls back to object mode (which is often slower than plain Python). You might not even notice the fallback unless you check.
To be safe, use @njit (a shortcut for @jit(nopython=True)). This forces Numba to raise an error if it can’t compile in nopython mode, so you know immediately.
from numba import njit
@njit # same as @jit(nopython=True)
def sum_of_squares_safe(arr):
total = 0.0
for x in arr:
total += x * x
return total
But Wait — Numba Doesn’t Work on Everything. Here’s the Catch.
Numba is not a universal Python accelerator. It’s a race car engine — amazing on a track (numerical loops), useless off-road (Pandas DataFrames).
What Numba compiles well:
- Loops over NumPy arrays
- Mathematical functions (sin, cos, exp, etc.)
- Linear algebra operations
- Custom reductions
- Simulations and iterative algorithms
What Numba cannot compile (in nopython mode):
- Pandas operations (DataFrame, Series)
- Most string methods
- Arbitrary Python objects (classes, dicts with mixed types)
try/exceptblocks- Generators
- Most of the Python standard library (file I/O, networking)
If you pass unsupported types, Numba falls back to object mode — which is often slower than plain Python. You must check.
Let’s see this in action.
import numpy as np
import pandas as pd
from numba import njit
import time
# This works: loop over a NumPy array
@njit
def loop_over_numpy(arr):
total = 0.0
for i in range(len(arr)):
total += arr[i] * arr[i]
return total
# This fails: using a Pandas Series inside
@njit
def loop_over_pandas(series):
total = 0.0
for i in range(len(series)):
total += series.iloc[i] * series.iloc[i]
return total
# Test the NumPy version
arr = np.random.randn(100_000)
start = time.perf_counter()
result = loop_over_numpy(arr)
elapsed = time.perf_counter() - start
print(f"NumPy array with @njit: {elapsed:.6f}s, result = {result:.4f}")
# Test the Pandas version — this will raise an error
print("\nTrying Pandas Series with @njit...")
try:
series = pd.Series(np.random.randn(100_000))
result = loop_over_pandas(series)
except Exception as e:
print(f"Error: {type(e).__name__}: {e}")
You’ll see something like:
NumPy array with @njit: 0.000152s, result = 100234.5678
Trying Pandas Series with @njit...
Error: TypingError: Failed in nopython mode pipeline...
The error tells you exactly which operation is unsupported. The fix is to extract the underlying NumPy array from the Pandas Series before passing it to Numba:
series = pd.Series(np.random.randn(100_000))
arr = series.to_numpy() # Extract NumPy array
result = loop_over_numpy(arr) # This works
The sweet spot for Numba: loops over NumPy arrays, mathematical functions, linear algebra, custom reductions, and simulations. If your code uses Pandas, strings, or complex Python objects, Numba probably isn’t the right tool.
Leveling Up: @vectorize for Element-Wise Operations
You’ve probably used np.vectorize before. It’s a lie — it’s just a for loop in disguise, not actually vectorized. It gives you the convenience of calling a scalar function on an array, but it’s still slow.
Numba’s @vectorize is different. It takes a scalar function (one that operates on a single number) and compiles it into a true NumPy ufunc that runs on the whole array at C speed.
Let’s see this with a custom mathematical function — a piecewise activation function that NumPy doesn’t have built-in.
import numpy as np
import time
from numba import vectorize
# Custom scalar function: a piecewise activation
def custom_activation_scalar(x):
if x > 0:
return x * 1.05
elif x > -0.5:
return x * 0.5 + 0.25
else:
return x * 0.1 - 0.2
# Apply with list comprehension (baseline)
def apply_list_comp(arr):
return np.array([custom_activation_scalar(x) for x in arr])
# Apply with np.vectorize (still slow)
np_vec_activation = np.vectorize(custom_activation_scalar)
# Apply with Numba @vectorize
@vectorize(['float64(float64)'])
def activation_vectorized(x):
if x > 0:
return x * 1.05
elif x > -0.5:
return x * 0.5 + 0.25
else:
return x * 0.1 - 0.2
# Generate data
arr = np.random.randn(10_000_000)
# Warm-up calls
_ = apply_list_comp(arr[:100])
_ = np_vec_activation(arr[:100])
_ = activation_vectorized(arr[:100])
# Time list comprehension
start = time.perf_counter()
result_lc = apply_list_comp(arr)
elapsed_lc = time.perf_counter() - start
print(f"List comprehension: {elapsed_lc:.4f}s")
# Time np.vectorize
start = time.perf_counter()
result_npv = np_vec_activation(arr)
elapsed_npv = time.perf_counter() - start
print(f"np.vectorize: {elapsed_npv:.4f}s")
# Time @vectorize (first call includes compilation)
start = time.perf_counter()
result_vec = activation_vectorized(arr)
elapsed_vec_1 = time.perf_counter() - start
print(f"@vectorize (first call): {elapsed_vec_1:.4f}s")
# Time @vectorize (second call, cached)
start = time.perf_counter()
result_vec = activation_vectorized(arr)
elapsed_vec_2 = time.perf_counter() - start
print(f"@vectorize (second call): {elapsed_vec_2:.4f}s")
# Speedup
print(f"\nSpeedup (second call vs. list comp): {elapsed_lc / elapsed_vec_2:.1f}x")
print(f"Speedup (second call vs. np.vectorize): {elapsed_npv / elapsed_vec_2:.1f}x")
You’ll see something like:
List comprehension: 1.2345s
np.vectorize: 1.1987s
@vectorize (first call): 0.0891s
@vectorize (second call): 0.0082s
Speedup (second call vs. list comp): 150.5x
Speedup (second call vs. np.vectorize): 146.2x
Let’s interpret these numbers:
- List comprehension took 1.23 seconds. It’s pure Python overhead per element.
- np.vectorize took 1.20 seconds. Almost identical — it’s still a Python loop under the hood.
- @vectorize took 0.008 seconds. That’s 150x faster. It’s a real ufunc, running at C speed.
The @vectorize decorator takes a signature string like 'float64(float64)' which means “takes one float64, returns one float64.” You can also specify multiple input types: 'float64(float64, float64)' for a two-input function.
You can even target parallelism or GPU by changing the target argument:
@vectorize(['float64(float64)'], target='parallel') # Multi-core CPU
@vectorize(['float64(float64)'], target='cuda') # GPU (if you have CUDA)
This is great for custom activation functions, distance metrics, or any element-wise math that NumPy doesn’t have built-in.
Going Parallel: prange and Automatic Parallelization
So far, we’ve been using a single CPU core. But your laptop probably has 4, 6, or 8 cores. Numba can use them all with prange — a parallel version of range.
Here’s the catch: prange only works for loops where iterations are independent — no shared state, no order dependence. If you’re computing a running sum (where each iteration depends on the previous one), you can’t parallelize it. But for embarrassingly parallel tasks like Monte Carlo simulations, it’s a game-changer.
Let’s estimate pi using a Monte Carlo method — a classic embarrassingly parallel problem.
import random
import time
import math
from numba import jit, prange, njit
# Pure Python version
def estimate_pi_py(n_points):
inside = 0
for _ in range(n_points):
x = random.random()
y = random.random()
if x*x + y*y <= 1.0:
inside += 1
return 4.0 * inside / n_points
# Numba single-threaded version
@njit
def estimate_pi_numba(n_points):
inside = 0
for i in range(n_points):
x = random.random()
y = random.random()
if x*x + y*y <= 1.0:
inside += 1
return 4.0 * inside / n_points
# Numba parallel version with prange
@njit(parallel=True)
def estimate_pi_parallel(n_points):
inside = 0
# prange distributes iterations across cores
for i in prange(n_points):
x = random.random()
y = random.random()
if x*x + y*y <= 1.0:
inside += 1
return 4.0 * inside / n_points
n = 10_000_000
# Warm-up calls
_ = estimate_pi_py(1000)
_ = estimate_pi_numba(1000)
_ = estimate_pi_parallel(1000)
# Time pure Python
start = time.perf_counter()
pi_py = estimate_pi_py(n)
elapsed_py = time.perf_counter() - start
print(f"Pure Python: {elapsed_py:.4f}s, pi ≈ {pi_py:.6f}")
# Time Numba single-threaded (second call, cached)
start = time.perf_counter()
pi_numba = estimate_pi_numba(n)
elapsed_numba = time.perf_counter() - start
print(f"Numba (single-threaded): {elapsed_numba:.4f}s, pi ≈ {pi_numba:.6f}")
# Time Numba parallel (second call, cached)
start = time.perf_counter()
pi_parallel = estimate_pi_parallel(n)
elapsed_parallel = time.perf_counter() - start
print(f"Numba (parallel with prange): {elapsed_parallel:.4f}s, pi ≈ {pi_parallel:.6f}")
# Speedups
print(f"\nSpeedup (Numba single vs. Python): {elapsed_py / elapsed_numba:.1f}x")
print(f"Speedup (Numba parallel vs. Python): {elapsed_py / elapsed_parallel:.1f}x")
print(f"Speedup (parallel vs. single): {elapsed_numba / elapsed_parallel:.1f}x")
On a 4-core laptop, you’ll see something like:
Pure Python: 8.2341s
Numba (single-threaded): 0.2156s
Numba (parallel with prange): 0.0589s
Speedup (Numba single vs. Python): 38.2x
Speedup (Numba parallel vs. Python): 139.8x
Speedup (parallel vs. single): 3.7x
Let’s interpret these numbers:
- Pure Python took 8.2 seconds. Monte Carlo is slow in Python because of the overhead per iteration.
- Numba single-threaded took 0.22 seconds. That’s 38x faster — just from JIT compilation on one core.
- Numba parallel took 0.059 seconds. That’s 140x faster than Python, and 3.7x faster than single-threaded Numba. On a 4-core machine, that’s near-perfect scaling.
Important caveat: prange only works for independent iterations. If you write to the same list or dictionary from multiple iterations, you’ll get race conditions — Numba doesn’t protect you. Always double-check that your loop is truly parallelizable.
Why Isn’t My Numba Code Faster? (Troubleshooting)
You’ve tried @jit, but your code isn’t any faster. Maybe it’s even slower. Here are the top three reasons, and how to fix them.
Reason 1: Fallback to Object Mode
This is the most common issue. If Numba can’t compile your function in nopython mode, it silently falls back to object mode — which is often slower than plain Python. You might not even notice.
Fix: Use @njit (or @jit(nopython=True)) instead of @jit. This forces an error if compilation fails, so you know immediately.
from numba import njit
@njit # Will raise an error if it can't compile
def my_function(arr):
# ... your code
Reason 2: Timing the First Call
The first call includes compilation time. If you’re only timing one call, you’re measuring compilation + execution, not just execution.
Fix: Always call the function once before timing (a “warm-up” call), then time the second call.
# Warm-up call (compilation happens here)
_ = my_function(data)
# Timed call (no compilation)
start = time.perf_counter()
result = my_function(data)
elapsed = time.perf_counter() - start
Reason 3: The Function Is Too Simple
Numba shines on loops, not on single NumPy calls. If your function is just np.sum(arr), Numba won’t help — NumPy already calls C code. Numba’s advantage is in complex loops where Python’s overhead dominates.
Fix: Profile your code. If the bottleneck is a single NumPy call, Numba won’t help. If it’s a loop over many elements, Numba will.
Bonus: Use NUMBA_DISABLE_JIT for Debugging
You can set the environment variable NUMBA_DISABLE_JIT=1 to temporarily disable Numba. This lets you compare the Numba version against the plain Python version to check for correctness.
import os
from numba import njit
# Disable JIT for debugging
os.environ['NUMBA_DISABLE_JIT'] = '1'
@njit
def my_function(arr):
# ... your code
# This will run as plain Python when JIT is disabled
Using inspect_types() to Diagnose
Numba provides inspect_types() to show you what it compiled and whether it’s in nopython mode.
from numba import njit
@njit
def my_function(arr):
total = 0.0
for i in range(len(arr)):
total += arr[i] * arr[i]
return total
# Inspect the compiled types
my_function.inspect_types()
This prints a detailed report showing the types Numba inferred for each variable. If you see “object” anywhere in the types, you’re in object mode (slow).
Numba vs. the Alternatives: Cython, C Extensions, and PyPy
How does Numba compare to other acceleration tools? Let’s break it down.
Cython
Cython requires learning a C-like syntax and writing .pyx files, then compiling. It’s more flexible (works with classes, custom types), but the learning curve is steeper.
Compare the same function in Numba vs. Cython:
Numba (one decorator):
from numba import njit
@njit
def sum_of_squares(arr):
total = 0.0
for x in arr:
total += x * x
return total
Cython (separate .pyx file with type declarations):
def sum_of_squares(double[:] arr):
cdef double total = 0.0
cdef int i
for i in range(arr.shape[0]):
total += arr[i] * arr[i]
return total
Numba is clearly easier. But Cython gives you more control and can handle more complex Python features.
C Extensions
Writing C extensions gives you full control, but requires C knowledge and manual memory management. It’s overkill for most data science tasks.
PyPy
PyPy is a JIT-compiled Python interpreter. It works on any Python code without changes — but it doesn’t support NumPy well (or at all, in practice). For pure Python numerical code, PyPy can be fast, but for data science work, Numba is usually the better choice.
The Benchmarks
According to benchmarks from verified sources:
- Numba with prange: up to 221x faster than pure Python
- Cython: up to 264x faster
They’re very close in performance, but Numba required far less code. Neither consistently beats the other — choose based on your specific needs (ease vs. flexibility).
When to choose Numba:
- You have numerical loops over NumPy arrays
- You want C-like speed with zero C knowledge
- Your code is already pure Python and you want a quick speedup
When to choose Cython:
- You need to work with Python classes or custom types
- You need fine-grained control over memory management
- You’re already comfortable with C-like syntax
Putting It All Together: A Real-World Worked Example
Let’s apply everything we’ve learned to a realistic data science task: computing a rolling custom statistic on a large time series.
The problem: Compute the rolling median absolute deviation (MAD) with a window of 1000 on a time series of 1 million points. NumPy doesn’t have a built-in rolling MAD function. The pure Python version is correct but painfully slow.
Let’s build three implementations and compare them.
import numpy as np
import time
from numba import njit
# Generate a time series (1 million points)
np.random.seed(42)
data = np.random.randn(1_000_000) + 0.01 * np.arange(1_000_000) # Random walk with trend
window = 1000
# ---- Pure Python version ----
def rolling_mad_py(arr, window):
"""Rolling median absolute deviation, pure Python."""
result = np.empty(len(arr))
result[:] = np.nan
for i in range(window - 1, len(arr)):
window_data = arr[i - window + 1 : i + 1]
median = np.median(window_data)
mad = np.median(np.abs(window_data - median))
result[i] = mad
return result
# ---- Numba version ----
@njit
def rolling_mad_numba(arr, window):
"""Rolling median absolute deviation, Numba JIT."""
n = len(arr)
result = np.empty(n)
result[:] = np.nan
for i in range(window - 1, n):
# Extract window
window_data = arr[i - window + 1 : i + 1]
# Compute median
sorted_window = np.sort(window_data)
mid = window // 2
if window % 2 == 1:
median = sorted_window[mid]
else:
median = (sorted_window[mid - 1] + sorted_window[mid]) / 2.0
# Compute MAD
abs_devs = np.abs(window_data - median)
sorted_abs = np.sort(abs_devs)
if window % 2 == 1:
mad = sorted_abs[mid]
else:
mad = (sorted_abs[mid - 1] + sorted_abs[mid]) / 2.0
result[i] = mad
return result
# ---- Benchmark ----
print("Benchmarking rolling MAD implementations...")
print(f"Data size: {len(data):,}, Window: {window}")
# Warm-up calls (use a small subset to avoid long wait)
_ = rolling_mad_py(data[:1000], 100)
_ = rolling_mad_numba(data[:1000], 100)
# Time pure Python (use smaller data for sanity)
small_data = data[:100_000]
small_window = 100
print("\nPure Python (100k points, window=100):")
start = time.perf_counter()
result_py = rolling_mad_py(small_data, small_window)
elapsed_py = time.perf_counter() - start
print(f" Time: {elapsed_py:.4f}s")
# Time Numba (full 1M points, window=1000)
print("\nNumba (1M points, window=1000):")
start = time.perf_counter()
result_numba = rolling_mad_numba(data, window)
elapsed_numba = time.perf_counter() - start
print(f" Time: {elapsed_numba:.4f}s")
# Correctness check (on small data)
result_numba_small = rolling_mad_numba(small_data, small_window)
# Remove NaN positions for comparison
mask = ~np.isnan(result_py)
assert np.allclose(result_py[mask], result_numba_small[mask], atol=1e-10), "Results differ!"
print("\n✓ Correctness check passed: both implementations produce identical results.")
# Estimate pure Python time for full dataset
# Rolling MAD is O(n * window * log(window)) in both cases
# So scaling is roughly linear in n
estimated_full_time = elapsed_py * (len(data) / len(small_data)) * (window / small_window)
print(f"\nEstimated pure Python time for full dataset: {estimated_full_time:.1f}s")
print(f"Actual Numba time for full dataset: {elapsed_numba:.4f}s")
print(f"Estimated speedup: {estimated_full_time / elapsed_numba:.0f}x")
You’ll see something like:
Benchmarking rolling MAD implementations...
Data size: 1,000,000, Window: 1000
Pure Python (100k points, window=100):
Time: 4.5231s
Numba (1M points, window=1000):
Time: 0.4217s
✓ Correctness check passed: both implementations produce identical results.
Estimated pure Python time for full dataset: 452.3s
Actual Numba time for full dataset: 0.4217s
Estimated speedup: 1073x
Let’s interpret these numbers:
- Pure Python on 100k points took 4.5 seconds. Extrapolating to 1 million points with a larger window, it would take about 7.5 minutes.
- Numba on the full dataset took 0.42 seconds. That’s the difference between a coffee break and a blink.
- The estimated speedup is over 1000x — but that’s because the pure Python version also uses
np.medianinside, which has overhead per iteration. Numba compiles the entire loop, including the sorting and median computation, into optimized machine code.
This is the real power of Numba: you write the same simple, readable loop logic, and it runs at C-like speed. No rewriting in C, no learning a new language, no complex compilation steps.
What You Learned (and Where to Go Next)
Let’s recap what you’ve learned:
- Numba is a JIT compiler that turns Python loops into machine code with one decorator. The first call is slow (compilation), but subsequent calls are fast.
- It works best on NumPy arrays and numerical loops. Pandas, strings, and complex Python objects don’t work in nopython mode.
@vectorizeturns scalar functions into fast ufuncs — great for custom element-wise operations.prangegives multi-core parallelism — but only for independent iterations.- If you’re not seeing speedup, check for object mode fallback (use
@njit), warm-up calls, and whether your function is too simple for Numba to help.
You’ve gone from a 2-second pure Python loop to a 0.015-second Numba loop — a 122x speedup. You’ve parallelized Monte Carlo simulations across all your CPU cores. You’ve turned a 7.5-minute rolling statistic computation into a 0.4-second operation.
Next in the series: “Scaling Beyond a Single Core: Parallel Processing with Dask” — when your data doesn’t fit in memory or you need a cluster. Dask complements Numba perfectly: Numba speeds up individual computations, while Dask distributes them across multiple machines.
Check Your Understanding
Remember:
- What does
@njitdo, and how is it different from@jit? - What is the purpose of a warm-up call when benchmarking Numba?
Understand:
- Explain why Numba’s first call is slower than subsequent calls.
- Why does Numba fail to compile Pandas operations in nopython mode?
Apply:
- Given a function that computes the Euclidean distance between two points, write a
@vectorizeversion that operates on arrays of points. - Convert a pure Python loop that computes a running average into a Numba
@njitfunction.
Analyze:
- A colleague reports that Numba didn’t speed up their code. What three things would you check first?
- Compare the performance characteristics of
np.vectorizevs. Numba’s@vectorize. Why are they different?
Evaluate:
- For a data science pipeline that involves Pandas data cleaning followed by a numerical simulation, where would you apply Numba? Where would you not?
- When would you choose Cython over Numba, despite the steeper learning curve?
Create:
- Design a benchmark that compares pure Python, NumPy vectorized, and Numba JIT versions of a custom statistical function (e.g., rolling skewness). What metrics would you report?
Apply What You Learned is for Supporter and Insider subscribers.
Subscribe to unlock the exercises on this post.
See plansRelated articles
- Python Engineering Under review
Why is My Data Pipeline Crashing? A Friendly Guide to Python Memory Profiling
Learn to diagnose and fix Python MemoryError crashes in data pipelines using memory_profiler, Fil, and chunking to handle massive datasets on limited RAM.
- Python Engineering Under review
Dask for Out-of-Core Data: When Your Dataset Doesn't Fit in Memory
You’ve just received a 20GB CSV file. Your laptop has 8GB of RAM. You fire up a Jupyter notebook and type `pd.read_csv('huge_file.csv')`. The kernel churns for a few seconds, then you see it: `MemoryError`.
- Python Engineering Under review
Web Scraping For Data Science Beautifulsoup Vs Scr
Let's get the definition straight first. Web scraping is the process of fetching a web page and extracting specific pieces of information from it. That's it. Two steps: fetch, then extract.
- Python Engineering Under review
SQLAlchemy for Data Scientists: Querying Databases Without Raw SQL Strings
Picture this: You're a data scientist, and you've just spent three hours building a beautiful data pipeline in a Jupyter notebook. You've got your SQL queries working perfectly in your local SQLite database.
Looking for something else?
Search every article by title, summary or topic.