Time Series Forecasting
Understand temporal patterns and build honest forecasts for Indian retail sales or electricity demand using baselines, statistical models, lag features, and rolling backtests.
A forecast must be made from information available at its forecast origin—not from a shuffled snapshot of the past and future.
An electricity utility needs to estimate tomorrow's hourly demand for grid planning. A retailer needs to forecast daily sales for replenishment. Both are supervised prediction problems, but time imposes an ordering: observations from the future cannot be used to train a model that claims to predict them. Promotions, holidays, weather, prices, and stock availability may be useful covariates, but only when their values would actually be known at prediction time.
This lesson builds an end-to-end forecasting mindset. We will identify trend and seasonality, consider stationarity and differencing, construct lag features without leakage, compare ARIMA, Prophet, and gradient boosting, and evaluate forecasts using chronological backtesting and scale-aware metrics. The examples use an Indian retail or electricity-demand setting; they are templates, not claims about any particular dataset.
Trend, seasonality, calendar effects, and irregular variation are different signals.
Trend is a longer-run increase or decrease, such as growth in customer demand. Seasonality repeats at a known calendar period: hourly electricity use can have daily and weekly patterns; retail sales may have weekly patterns and annual festivals. Calendar effects such as public holidays, payday timing, and one-off promotions can create structured deviations. Residual variation contains unpredictable shocks, measurement noise, and patterns not yet represented.
Plot the series at several scales and annotate missing timestamps, stockouts, holidays, and changes in data collection. For a store-level daily series in India, a weekly pattern can reflect weekday-versus-weekend shopping; annual patterns may include regional festivals with different dates across states. A national holiday flag alone may not represent a local event or store-specific campaign.
Check the time index before modelling
Confirm sorting, duplicate timestamps, frequency, timezone, missing periods, and aggregation rules. A missing row is not always zero demand: the store may have been closed, the sensor may be offline, or the data may be absent. Preserve the difference between observed zero and unavailable measurement. For multiple stores or products, define each series key and make sure lag calculations never cross from one entity into another.
Stationarity is a useful modelling assumption, not a requirement that every real series be flat.
A weakly stationary process has stable mean and variance over time and autocovariance that depends on lag rather than absolute time. Trend and changing seasonal patterns violate this simple picture. ARIMA models combine autoregressive terms, differencing, and moving-average terms to represent temporal dependence after suitable transformations. Seasonal ARIMA adds seasonal structure for a chosen period. Differencing can remove some trend, but excessive differencing can discard signal and create artificial dependence.
Inspect plots and residuals; do not select differencing solely from one test. Augmented Dickey–Fuller and KPSS tests assess different null hypotheses and should be interpreted with domain context. A statistical test is evidence, not an automatic model-selection oracle. Transformations such as a log transform can stabilize variance for positive values, but zeros and negative values require careful handling.
The order shown is an illustrative specification, not a universally correct choice. Select orders and seasonal terms using training-period diagnostics and rolling validation. The statsmodels library provides ARIMA model interfaces; its fit results and forecasting methods should be checked against the installed documentation and the data's frequency/index behavior.
Residuals reveal what the model left behind
After fitting, inspect residual mean, distribution, autocorrelation, and plots over time. Structured residual patterns suggest omitted seasonality, interventions, changing variance, or a mismatch between the model and data. A model's in-sample fit is not a forecast score. Compare predictions at realistic horizons with a validation period and with a simple baseline.
A seasonal-naive forecast is a strong first check for repeating demand.
A naive forecast repeats the most recent observed value. A seasonal-naive forecast repeats the value from the corresponding prior season—for example, last week's same weekday for a daily series with weekly seasonality, or the same hour yesterday for an hourly load series with a daily cycle. These baselines are cheap, transparent, and often surprisingly hard to beat.
Add known-in-advance features such as day of week, month, public-holiday indicators, planned price, and scheduled promotion. A weather forecast can be a valid feature if the production system will have that forecast at prediction time; using the subsequently observed weather creates an unrealistically optimistic evaluation. For a holiday calendar, verify regional applicability and how dates are represented in the source.
Tabular models can learn useful forecasting patterns when temporal features are created without crossing the forecast boundary.
For each store-product-date, features might include sales one day, one week, and four weeks earlier; rolling mean and standard deviation; day of week; promotion plan; and price. A gradient-boosting model such as a tree ensemble can learn nonlinear interactions—for example, a promotion may matter more on a weekend. It does not automatically understand time: the engineer constructs the lag and calendar representation.
The shift before rolling is essential. A rolling mean that includes today's target leaks the answer into a feature for today's prediction. Grouping by series prevents a product's final observations from becoming another product's lags. For multi-step forecasts, decide whether later horizons can use actual observations arriving after the first forecast; if not, generate those lag values recursively from predictions or train direct horizon-specific models.
Use Prophet when its additive structure is a good fit
Prophet is a forecasting library that represents trend, seasonality, and holiday effects in an additive framework with configurable components. It can be useful for interpretable business series with strong calendar structure and is convenient for modelling multiple seasonalities. It is not automatically superior to ARIMA, a tree model, or a baseline; changepoints, holidays, outliers, and forecast horizon still need validation. Use its documented input format and avoid fitting future observations during model selection.
For Indian retail demand, consider whether annual festival dates shift year to year and whether regional observance matters. For electricity demand, calendar and weather effects can be strong, but operational constraints and intraday patterns may require more specialized features or models. Begin with a forecasting objective and information set, then choose a model class.
A single random train/test split destroys the question a forecast is meant to answer.
Train on earlier periods and evaluate on later periods. A final holdout can simulate deployment once, but it does not show whether performance is stable across seasons. Rolling-origin backtesting creates several forecast origins: train on an initial history, predict the next horizon, move the origin forward, and repeat. Depending on the use case, training windows can expand or roll over a fixed duration. Keep each fold's target window strictly after its training observations.
If the target has delayed reporting, define when each value becomes available. If model retraining occurs weekly, evaluate weekly forecast origins. Compare the same horizons and series across models. A model that is excellent one day ahead but poor two weeks ahead may not solve the inventory-planning problem. For panel data, prevent entity and date leakage in a way that reflects how the system will generalize.
Fit transformations inside each fold
Imputation, scaling, target encoding, feature selection, and any learned seasonal adjustment must be fit using only the training segment of each fold. Reusing statistics computed from the full dataset leaks future distribution information. Deterministic known calendar features can be generated for future dates, but their values must be truly known in advance.
Report more than one metric when zero demand and asymmetric costs matter.
Mean absolute error (MAE) expresses average absolute error in the target's units. Root mean squared error (RMSE) penalizes large errors more heavily. MAPE averages absolute percentage error, but is undefined when actual values are zero and can become unstable near zero. This is a material issue for sparse retail products or low-load periods. Never silently divide by zero or drop those observations without reporting the rule.
Symmetric MAPE (sMAPE) uses a denominator based on the magnitudes of actual and forecast values. Several formulas are called sMAPE and differ in scaling or zero handling, so state the exact definition in reports. One common form below returns a percentage from 0 to 200 when its denominator is nonzero; an alternative convention divides by two and reports 0 to 100. Define and use one convention consistently.
A percentage metric can overweight small series. Report aggregate and segment-level performance: high-volume versus low-volume products, store regions, weekdays versus holidays, and forecast horizons. If over-forecasting and under-forecasting have different inventory costs, include a metric or decision analysis that reflects that asymmetry instead of optimizing a symmetric average alone.
A forecast becomes a production system when its assumptions are monitored.
Store the forecast origin, horizon, series identifiers, prediction, model version, and feature snapshot. Monitor input freshness, missing data, changes in series coverage, forecast errors after actuals arrive, and drift in error by important segment. Keep a fallback baseline for outages or series with too little history. For electricity demand, extreme weather and grid events can move demand outside historical patterns; for retail, stockouts can make observed sales a censored view of customer demand.
Prediction intervals communicate uncertainty and support decisions better than one point estimate. Their calibration should be evaluated on time-aware backtests. A narrow interval is not automatically useful if it misses actual outcomes; a wide interval can be honest but operationally uninformative. Match uncertainty communication to the decision and the cost of being wrong.
Forecasting predicts what may happen; recommenders help decide what to show next.
The next lesson moves from temporal demand to ranking items for a user. We compare popularity, content-based and collaborative approaches, matrix factorization, implicit feedback, retrieval, cold-start strategies, and ranking metrics.
Rank relevant movies or products for each user.
🎯 Key Takeaways
- ✓Separate trend, repeating seasonal patterns, known calendar effects, and irregular variation before selecting a model.
- ✓Stationarity and differencing are modelling considerations; inspect the series and residuals rather than treating a test as an automatic answer.
- ✓Start with naive and seasonal-naive baselines. ARIMA, Prophet, and gradient boosting with lag features are alternatives whose value must be shown on time-aware validation.
- ✓Construct lag and rolling features within each series, shift before rolling, and use only values available at the forecast origin.
- ✓Split chronologically and use rolling-origin backtesting to measure performance across forecast origins and horizons. Fit learned preprocessing inside each training fold.
- ✓MAPE is undefined at zero and unstable near zero. State the sMAPE convention and inspect segment-level metrics and asymmetric business costs.
- ✓Sales can be censored by stockouts; monitor freshness, drift, model versions, forecast errors, and fallback behavior in production.
HELP US IMPROVE
How was this lesson?
Your note is shared with the Chaduvuko team for review. Please don’t include personal or sensitive information.
Spotted a mistake or something unclear? Suggest a fix on GitHub or email us. Select the text first and it is quoted for you.
Discussion
0Have a better approach? Found something outdated? Share it — your knowledge helps everyone learning here.
