← Showcase 4.2 Β· Showcase

Technical Deep-Dives: Methodological Rigour

Two production case studies β€” graph neural networks for fraud detection and hierarchical forecast reconciliation for retail β€” examined at the level of detail required for serious implementation.

πŸ“š 6 min readβ€’Updated: October 2025
Opening

These deep-dives lead with technical challenges and solutions, not business context. Each follows a problem β†’ formulation β†’ solution β†’ evaluation structure, with honest discussion of what worked, what did not, and why.

Deep-Dive 1

Graph Neural Networks at Scale

Fraud detection on millions of daily transactions β€” from graph construction and message passing to sub-100 ms production serving.

πŸ•ΈοΈ

Graph Construction

  • Transaction records: sender, receiver, amount, timestamp, merchant category, location
  • Bipartite graph β€” accounts and transactions as nodes; directed edges (sender β†’ transaction β†’ receiver)
  • Edge features: amount, time, category; node features: account age, history, velocity
  • Graph Attention Networks weight neighbours by learned relevance β€” not equal weighting
βš™οΈ

Training Strategy

  • Neighbourhood explosion: 2-hop = thousands of nodes β†’ full-batch infeasible
  • GraphSAGE samples 15–25 neighbours per layer, bounding cost
  • Importance sampling weighted by amount & recency β†’ +8 pp precision
  • Focal loss counters 0.2% fraud rate; standard cross-entropy predicts "legitimate" always
  • Temporal split: months 1–18 train Β· 19–21 validate Β· 22–24 test (prevents leakage)
πŸš€

Deployment & Monitoring

  • Target: sub-100 ms latency for real-time scoring
  • Pre-computed embeddings for established accounts, cached nightly in Redis
  • Transaction-time: shallow GNN forward pass only
  • Mini-batch inference Β· quantisation Β· ONNX Runtime Β· Kubernetes horizontal scaling
  • Daily alerts on data drift, prediction drift, concept drift, precision/recall/F1
  • Automated retraining when drift thresholds are exceeded
Feature Engineering
Transaction
  • Amount (log-transformed)
  • Merchant category codes
  • Cyclical time encoding (sine/cosine hour & day)
  • Distance from previous transaction
  • Travel-time feasibility
Graph
  • Degree centrality
  • PageRank
  • Clustering coefficient
Temporal
  • Days since account creation
  • Rolling 7-day transaction velocity
  • Spending deviation from personal baseline
Hyperparameter Optimisation

GNNs introduce many hyperparameters beyond standard networks: layers, hidden dims, sampling sizes, attention heads, learning rates, dropout, and focal-loss parameters. Grid search is infeasible. Bayesian optimisation builds a probabilistic model of the objective and guides search towards promising regions. After approximately 50 iterations it substantially outperformed initial random-search results.

Explainability & Regulatory Compliance

GNNExplainer identifies the important subgraph and features for each prediction. Investigators receive a risk score plus a visual network map of the suspicious pattern β€” clusters of linked accounts, unusual transaction sequences, connections to known fraud rings β€” satisfying regulatory explainability requirements.

Lessons Learned

  • Neighbourhood sampling is crucial β€” naive implementations quickly become infeasible
  • Attention helps when neighbour relevance varies substantially, but adds training complexity
  • Combining GNNs with domain knowledge (feature engineering, temporal validation, interpretability) beats either alone
Deep-Dive 2

Hierarchical Forecast Reconciliation

Retail demand forecasting at 200 stores Γ— 50,000 products β€” ensuring mathematical coherence across all aggregation levels whilst improving accuracy.

πŸ“

Reconciliation Methods

  • Constraint: store forecasts must sum to regional, regional to total β€” represented via summing matrix S
  • Bottom-up: coherent but noisy for sparse series
  • Top-down: coherent but loses lower-level detail
  • MinT: minimises trace of reconciled error variance using all levels β€” typically best
  • Covariance options: identity β†’ structural scaling β†’ sample β†’ mint_shrink (shrinkage compromise)
πŸ—οΈ

Large Hierarchies

  • 200 stores Γ— 50,000 products = 10 M series
  • Full covariance inversion is infeasible at this scale
  • Sparse matrix operations: only non-zero entries between series sharing hierarchy relationships
  • Reduces storage from quadratic to linear
  • Very large hierarchies: structural approach (parameter-free, needs only hierarchy structure)
πŸ”„

Production Pipeline

  • Nightly batch β€” forecasts ready before Monday planning meetings
  • Extract: POS, promotional calendars, product hierarchies, weather, external sources
  • Transform: cleaning, validation, feature engineering, SPC anomaly detection
  • Load: CSV exports to planning systems with defined schemas
  • ~4 hours for full catalogue on compute cluster, parallelised by category
  • Weekly granularity Β· 52-week horizon Β· Monday dashboards with drill-down
Feature Engineering
Calendar
  • Day-of-week & month (sine/cosine encoding)
  • Holidays: Christmas, Easter, school terms
  • Special days: Black Friday, Boxing Day
Promotional
  • Depth (discount %), breadth (# products), loyalty bonuses
  • Promotion cadence (frequency & regularity)
  • Category & customer-segment interaction effects
External
  • Weather: temperature, rainfall
  • Economic: fuel prices, unemployment
  • Competitor activity (legally scraped)

Prophet provides additive decomposition (trend + seasonal + holidays + regressors), Bayesian uncertainty intervals, and automatic changepoint detection β€” interpretable by merchandising teams and robust to trend shifts.

Intermittent Demand

Croston's method separately forecasts demand size (when non-zero) and inter-demand interval, then combines them. For very sparse individual series, strength is borrowed from higher hierarchy levels β€” a product's behaviour across all stores, or the category within the store, provides stable patterns via reconciliation.

22%
MAPE reduction across all hierarchy levels
15%
Safety stock reduction
12%
Product availability improvement
18%
Obsolescence reduction
Β£8 M
Annual value created
<2%
Coherence gap post-reconciliation (was 15–20%)
Key Insight β€” Mathematical rigour combined with domain knowledge produces the best resultsβ€”interpretable models with sophisticated post-processing can match complex models whilst providing better maintainability and user trust.

Lessons Learned

  • Hierarchical reconciliation dramatically improves coherence β€” incoherent forecasts undermine business planning trust
  • MinT with appropriate covariance typically outperforms simpler approaches; magnitude depends on hierarchy structure and error patterns
  • Base forecast quality matters more than sophisticated reconciliation β€” reconciliation cannot fully compensate for poor base models
  • Interpretable models (Prophet) + sophisticated post-processing (MinT) match complex models with far better maintainability

Key Takeaways

  • Production ML requires careful engineering beyond research implementations
  • Scalability, interpretability, and monitoring are crucial for deployment
  • Mathematical rigour combined with domain knowledge produces best results
  • Open source implementations enable reproducibility and community advancement