Feature Engineering for Time Series: Lags, Rolling Windows, and Seasonality

Last time, Nora confirmed that simpler models like ARIMA often win when time-series data is limited — a lesson that held up on the airline-passengers benchmark just as it did on the bakery’s own sourdough sales. Now she needs to give her model better raw material, which means engineering proper time-aware features for the bakery’s daily sales data.

Have you ever tried to predict tomorrow’s stock price or next week’s sales with a standard regression model, only to get a flat line or a total mess? Nora hit exactly this wall as the demand planner for a regional bakery chain — her first regression model couldn’t see the weekly pattern at all.

If you’ve built models for tabular data, say predicting house prices from square footage, you’re used to each row being independent. Time series breaks that assumption. The past is a prologue here. Without teaching your model how to look backward, it has no way to pick up the patterns that matter.

This guide shows how to turn time itself into features a model can actually use.

1. Why Your Regular Features Don’t Work for Time Series

Most machine learning models assume your data points are independent. The model thinks the price of a house today has nothing to do with what the house next door sold for yesterday.

In time series, tomorrow’s value is almost always correlated with today’s. This is autocorrelation. Feed a model a raw timestamp like “2023-10-01” and it sees a unique string or a large number. It doesn’t inherently know that this day follows a Saturday. Or that it’s colder than July.

So what happens when we fit a standard model to time series data without proper features? The synthetic dataset below — a gentle trend with a weekly sinusoidal cycle — stands in for the bakery’s daily pastry sales.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression

# Create a synthetic sales dataset with a trend and weekly seasonality
np.random.seed(42)
time = np.arange(100)
sales = 10 + 0.5 * time + 5 * np.sin(2 * np.pi * time / 7) + np.random.normal(0, 1, 100)
df = pd.DataFrame({'day': time, 'sales': sales})

# Naive model: Try to predict sales using only the 'day' number
X = df[['day']]
y = df['sales']
model = LinearRegression().fit(X, y)
df['pred'] = model.predict(X)

print(f"Model R-squared: {model.score(X, y):.4f}")
# Interpretation: An R-squared of 0.9426 looks great, but look at the plot...

plt.figure(figsize=(10, 5))
plt.plot(df['day'], df['sales'], label='Actual Sales')
plt.plot(df['day'], df['pred'], label='Naive Prediction', linestyle='--')
plt.legend()
plt.title("The Failure of Raw Timestamps")
plt.show()

• np.random.seed(42) — fixes the random number generator so the synthetic dataset is identical every run, making results reproducible.
• time = np.arange(100) — creates an array [0, 1, 2, ..., 99] representing 100 consecutive days.
• sales = 10 + 0.5 * time + 5 * np.sin(2 * np.pi * time / 7) + np.random.normal(0, 1, 100) — builds the synthetic series from four components: a baseline of 10, a linear trend (0.5 units/day), a weekly sinusoidal cycle (period = 7 days), and Gaussian noise.
• LinearRegression().fit(X, y) — fits an ordinary least-squares regression using only the raw day number as the predictor — no lag, no seasonality, no rolling statistics.
• model.score(X, y) — returns the R² value, which measures how much of the variance in sales is explained by the day number alone. Running the block prints 0.9426: the trend dominates this series (its variance is roughly 17x the seasonal wiggle’s), so a straight line through the day number alone already looks like a great fit by R² — even though, as the plot shows, it completely misses the weekly wave.

What this actually means: The model captured the general upward trend but completely missed the “wiggles” — the weekly ups and downs. It was off by a significant margin on specific days because it didn’t know that “Day 7” relates to “Day 0.” A 0.94 R² sounds like a success story; the plot is what tells you it isn’t.

2. The Core Idea: Features From the Past

The secret to time series is a concept called a Lag. Think of it this way: if you want to know whether it will rain tomorrow, the most important piece of info is usually whether it rained today.

A lag is just a previous value, shifted forward. Lag-1 is yesterday’s value. Lag-2 is the day before that. By adding these columns, we turn a sequence of numbers into a table that a regular regression model can handle.

# Creating a simple lag table
df_lags = pd.DataFrame({'Target_Today': [10, 12, 15, 14, 18]})
df_lags['Lag_1'] = df_lags['Target_Today'].shift(1)
df_lags['Lag_2'] = df_lags['Target_Today'].shift(2)

print(df_lags)

• df_lags['Target_Today'].shift(1) — shifts the column down by one row so that each row now shows yesterday’s value alongside today’s; the first row gets NaN because there is no prior value.
• shift(2) — shifts by two rows, placing the value from two days ago on the current row; the first two rows become NaN.
• The resulting table has three columns: today’s value, the 1-step lag, and the 2-step lag — the format a standard regression or tree-based model needs.

Notice the NaN values at the top? On the first day, there is no “yesterday.” This trips up beginners — you always lose a bit of data at the start of your series when you create lags.

3. Building Lag Features in Practice

Let’s build a function to automate this. The goal is to give our model a “window” into the past.

def create_lags(data, n_lags):
    df = data.copy()
    for i in range(1, n_lags + 1):
        df[f'lag_{i}'] = df['sales'].shift(i)
    return df.dropna()

# Let's use 3 lags
df_with_lags = create_lags(df[['sales']], 3)

X = df_with_lags.drop('sales', axis=1)
y = df_with_lags['sales']

# The 3 lags cost us 3 rows (100 -> 97); split chronologically:
# train on the first 77 rows, test on the last 20
X_train, X_test = X[:77], X[77:]
y_train, y_test = y[:77], y[77:]

model = LinearRegression().fit(X_train, y_train)
preds = model.predict(X_test)

print(f"Test MAE with Lags: {np.mean(np.abs(y_test - preds)):.2f}")
# Interpretation: The error is now much lower because the model sees recent history.

• def create_lags(data, n_lags) — defines a reusable function that takes a DataFrame and the number of lag columns to create.
• df[f'lag_{i}'] = df['sales'].shift(i) — inside the loop, creates lag_1, lag_2, lag_3 by shifting the sales column by 1, 2, and 3 positions respectively.
• df.dropna() — removes the first n_lags rows where lag values don’t exist yet, so the model never trains on NaN inputs. With 3 lags on 100 rows, that leaves 97.
• X[:77], X[77:] — splits chronologically rather than randomly: the first 77 of those 97 rows become training data, the last 20 become the test set.
• np.mean(np.abs(y_test - preds)) — computes Mean Absolute Error (MAE) by averaging the absolute differences between actual and predicted sales on the held-out test period.

4. Rolling Windows: Capturing Local Patterns

Lags capture specific points in time. But the overall shape of the last week often matters more than any single day. Rolling Windows address this.

A rolling mean (or moving average) smooths out the noise. Say sales spiked yesterday from a one-time promotion — a 7-day rolling mean dampens that spike and reveals the underlying trend.

# Creating rolling features
df['rolling_mean_7'] = df['sales'].shift(1).rolling(window=7).mean()
df['rolling_std_7'] = df['sales'].shift(1).rolling(window=7).std()

# Note: We shift(1) before rolling to avoid 'Data Leakage' 
# (using today's value to predict today's value!)

plt.figure(figsize=(10, 4))
plt.plot(df['sales'], alpha=0.3, label='Raw')
plt.plot(df['rolling_mean_7'], label='7-Day Rolling Mean', color='red')
plt.legend()
plt.show()

• df['sales'].shift(1).rolling(window=7).mean() — first shifts the series by 1 (so the window never includes the current day’s value), then computes the average of the 7 most recent past values.
• rolling(window=7).std() — computes the rolling standard deviation over the same 7-day shifted window, giving the model a sense of how volatile recent sales have been.
• The shift(1) before rolling() is critical: without it, the window would include today’s actual value, which is the target you’re trying to predict — a textbook case of data leakage.
• plt.plot(df['sales'], alpha=0.3) — renders the raw series at 30% opacity so the smoothed rolling-mean line (in red) stands out against the noisy original.

The rolling-window statistics used throughout this article can be formalized as:

$\bar{x}_t^{(k)} = \frac{1}{k}\sum_{i=1}^{k} x_{t-i} \qquad s_t^{(k)} = \sqrt{\frac{1}{k-1}\sum_{i=1}^{k}\left(x_{t-i} - \bar{x}_t^{(k)}\right)^2}$

Plain English	Statistical symbol	Python equivalent
Rolling mean over window of size $k$ (excludes today)	$\bar{x}_t^{(k)} = \frac{1}{k}\sum_{i=1}^{k} x_{t-i}$	df['sales'].shift(1).rolling(window=k).mean()
Rolling std dev over window of size $k$	$s_t^{(k)} = \sqrt{\frac{1}{k-1}\sum_{i=1}^{k}(x_{t-i} - \bar{x}_t^{(k)})^2}$	df['sales'].shift(1).rolling(window=k).std()
Lag of order $p$ (value from $p$ steps ago)	$x_{t-p}$	df['sales'].shift(p)

5. Seasonality: Repeating Patterns Across Time

Does your data repeat every Monday, or every December? That’s Seasonality. We can pull it straight from the date index — tell the model “today is a Monday” and it learns that Mondays tend to be slow. We also use Seasonal Lags. If the cycle repeats weekly, lag_7 is the feature you want.

# Simulating a date range for our synthetic data
df['date'] = pd.date_range(start='2023-01-01', periods=len(df))
df['day_of_week'] = df['date'].dt.dayofweek
df['month'] = df['date'].dt.month
df['seasonal_lag_7'] = df['sales'].shift(7)

# Let's see how day_of_week correlates with sales
print(df.groupby('day_of_week')['sales'].mean())
# Interpretation: If these means are different, the feature is useful!

• pd.date_range(start='2023-01-01', periods=len(df)) — generates a sequence of daily timestamps starting January 1, 2023, one per row in the DataFrame.
• df['date'].dt.dayofweek — extracts the day-of-week as an integer (0 = Monday, 6 = Sunday), letting the model learn weekday-level patterns like slower Friday pastry sales. Running this against the synthetic series shows a real spread — Monday averages around 39, Friday around 31 — which is exactly the “these means are different” signal the print statement is looking for.
• df['date'].dt.month — extracts the month number (1–12). On this particular 100-day dataset (Jan 1 – Apr 10, 2023) that only produces 4 distinct values and there’s no yearly component in the generator, so month carries no real signal here — it’s included to show the technique, not because this synthetic series has a monthly effect to find. On a real multi-year sales history it would be the feature that catches things like holiday-season baking spikes.
• df['sales'].shift(7) — creates a seasonal lag: the value from exactly one week (7 days) ago, which is often the single most predictive feature for weekly-cyclic data.
• df.groupby('day_of_week')['sales'].mean() — groups all rows by day of week and computes the average sales for each, a quick diagnostic for whether the weekday feature carries signal.

6. Putting It Together: A Complete Feature Set

Now we can bring the pieces together — recent lags, rolling averages, and seasonal context.

def engineer_features(data):
    df = data.copy()
    # Lags
    for i in [1, 2, 3]:
        df[f'lag_{i}'] = df['sales'].shift(i)
    # Rolling
    df['roll_mean'] = df['sales'].shift(1).rolling(7).mean()
    # Seasonal
    df['seasonal_lag_7'] = df['sales'].shift(7)
    # Calendar
    df['day_of_week'] = pd.to_datetime(df['day'], unit='D', origin='2023-01-01').dt.dayofweek
    return df.dropna()

final_df = engineer_features(df[['day', 'sales']])
print(f"Total features created: {final_df.shape[1] - 1}")

• for i in [1, 2, 3]: — creates three lag features (lag_1, lag_2, lag_3) covering the last three days of sales history.
• df['sales'].shift(1).rolling(7).mean() — computes a 7-day rolling mean on the shifted series, giving the model a smoothed “recent average” signal.
• df['sales'].shift(7) — adds the seasonal lag, capturing the value from exactly one week ago — the most important feature for weekly-cyclic bakery sales.
• pd.to_datetime(df['day'], unit='D', origin='2023-01-01') — converts the integer day index back into a datetime object, anchored at the same start date (2023-01-01) used to build date in Section 5, so .dt.dayofweek lines up with the weekday labels the rest of the article already established. Leaving off origin defaults it to the Unix epoch (1970-01-01, a Thursday) instead of the series’ actual start date (2023-01-01, a Sunday) — a 3-day offset that would silently relabel every row’s weekday.
• final_df.shape[1] - 1 — counts the total number of feature columns (excluding the target sales column), so you can verify how many predictors the model will see.

Not all time-series features are interchangeable — each captures a different kind of temporal signal.

Feature Type	What It Captures	Best For	Risk
Lag features (lag_1, lag_2, lag_3)	Specific past points at fixed offsets	Short-term momentum (“yesterday was high, today will be too”)	No context beyond the exact lag day; a single anomalous value propagates directly
Rolling-window statistics (roll_mean, roll_std)	Local trend and volatility over a window	Smoothing noise, detecting regime shifts, giving the model a “vibe” of recent days	Lagging signal — a rolling mean reacts slowly to sudden turns; window size is a hyperparameter to tune
Calendar / seasonality features (day_of_week, month, seasonal_lag_7)	Recurring cyclical patterns tied to the calendar	Weekly cycles, monthly effects, holiday proximity	Useless if the cycle itself is shifting (e.g., a store changes its opening days); calendar features assume the pattern is stable

In practice, you rarely choose just one type — the engineer_features function above combines all three. Lags give precision, rolling stats give context, and calendar features give structure. The art is choosing the right lag distances and window sizes for your specific series.

7. Common Pitfalls and How to Avoid Them

Here’s what trips people up: it’s easy to cheat in time series without meaning to. The term is Data Leakage.

1. The Look-Ahead Bias: If your rolling mean for “Today” includes “Today’s” value, your model looks perfect in training and falls apart in the real world. Always .shift(1) your features.
2. The Shuffle Trap: Don’t use train_test_split(shuffle=True). If you train on Wednesday to predict Tuesday, you’ve leaked the future into the past.

8. When to Use Which Features

• Short-term (Predicting tomorrow): Stick with lag_1, lag_2, and a short rolling mean (3–5 days).
• Long-term (Predicting next month): Reach for seasonal_lag_30, month, and longer rolling means (30–90 days).
• High Volatility: Add rolling_std so the model knows how turbulent the market is right now.

9. Automating Feature Engineering

You don’t have to do this by hand for every project. Two libraries cover different ends of the job:

• feature-engine’s LagFeatures/WindowFeatures transformers are the right pick when you already know which lags and windows you want (say, lag_1, lag_7, lag_30) and just want to skip writing the shift() loop yourself — they slot into a scikit-learn Pipeline like any other transformer. See the feature-engine time series docs.
• tsfresh is a different tool for a different stage: it extracts hundreds of statistical features per series (autocorrelation, entropy, peak counts, and more) with no need to specify which ones matter up front, and expects a feature-selection step afterward to cut that pile back down. Reach for it during exploration, when you don’t yet know which shape of feature will be predictive — not as a drop-in replacement for a few hand-picked lags. See the tsfresh docs.

# Example logic (conceptual)
# from feature_engine.timeseries.forecasting import LagFeatures
# transformer = LagFeatures(variables=['sales'], periods=[1, 7, 30])
# df_auto = transformer.fit_transform(df)

• LagFeatures(variables=['sales'], periods=[1, 7, 30]) — a conceptual call to the feature-engine library that would automatically create lag_1, lag_7, and lag_30 columns in one step, replacing the manual shift() loop.
• transformer.fit_transform(df) — applies the transformation and returns a new DataFrame with the lag columns appended; this is illustrative code (commented out) rather than something meant to run as-is.

10. Next Steps: From Features to Forecasts

You’ve turned a single column of numbers into a feature matrix.

Recap:

• Lags hand the model specific past points.
• Rolling windows capture the recent trend.
• Seasonal features encode the cycles.

Next up: choosing between Linear Regression, XGBoost, or Prophet to generate forecasts from these features. So explore your data, build your lags, and watch the predictions improve. With these in hand, Nora’s ready to try a different model family — gradient boosting — which is where we head next.

Check Your Understanding

Questions below move from simple recall up to open-ended design, roughly following Bloom’s Taxonomy.

Remember What is a “Lag” feature, and why does creating one always produce some NaN values at the start of the series?

Understand In your own words, explain why the article shifts the data by 1 (.shift(1)) before computing a rolling mean, instead of computing the rolling mean directly on the raw column.

Apply Using the article’s create_lags logic, if you had a series [5, 8, 13, 21, 34] and created a single lag_1 column, what would the resulting lag_1 values be (before dropping the NaN row)?

Analyze The article warns that shuffling time series data before a train/test split (“The Shuffle Trap”) causes leakage, because “you’ve leaked the future into the past.” Walk through exactly what a lag-1 feature would contain for a shuffled-in test row that originally came from a Wednesday, if the row placed before it in the shuffled order happened to be from the following Thursday.

Evaluate The article recommends seasonal_lag_30 and long rolling means (30-90 days) for long-term forecasts. Critique this recommendation for a business that experienced a major disruption 45 days ago (e.g., a supply chain issue that temporarily crashed sales): what would a 90-day rolling mean feature currently reflect, and how might that mislead a model trying to forecast next month?

Create Design a feature set (lags, rolling windows, seasonal features) for a new time series: hourly website traffic, which has both a daily cycle (busy at lunch) and a weekly cycle (quiet on weekends). Specify which lag periods and rolling window sizes you’d choose and why, following the article’s “short-term vs. long-term” framework.

---

Related articles

• Classical Forecasting vs. Machine Learning: When Does Simpler Win?) — the previous article: Nora confirms that ARIMA often beats LSTM on limited time-series data, then heads back to build proper features.
• Gradient Boosting for Time Series Using LightGBM) — the next article: Nora takes the engineered feature set and tries a completely different model family — tree-based gradient boosting.

References & Further reading

• Hyndman, R. J. & Athanasopoulos, G. (2021). Forecasting: Principles and Practice (3rd ed.), Chapter 5 — “Time Series Regression and Time Series Decomposition.” otexts.com/fpp3/ — the canonical textbook treatment of lag features, rolling windows, and seasonal decomposition.
• pandas rolling-window documentation: pandas.pydata.org/docs/reference/api/pandas.DataFrame.rolling.html — the DataFrame.rolling() API used throughout this article’s code blocks.
• Kaggle “Store Sales — Time Series Forecasting” competition: kaggle.com/competitions/store-sales-time-series-forecasting — a real-world retail benchmark where lag and rolling-window features are essential for competitive performance.

---