03

Statistical Learning Fundamentals

Pythonscikit-learnLogistic RegressionPR AUCjoblib

Overview

Binary classification model to predict push notification engagement on a heavily imbalanced dataset. The positive class represents users who engage with a notification, a rare event by design.

Modelling Approach

  • arrow_forwardTemporal split by cumulative order volume (70/20/10) rather than fixed dates, reflecting real production conditions and preserving seasonality
  • arrow_forwardBaseline built on global item popularity
  • arrow_forwardLogistic Regression with L1 and L2 regularisation, wrapped in sklearn Pipelines with StandardScaler
  • arrow_forwardOptimised for PR AUC given class imbalance; ROC AUC used as secondary metric
  • arrow_forwardFeature importance extracted from Lasso coefficients
  • arrow_forwardFinal model trained on a 4-feature subset that matched full-feature performance
  • arrow_forwardModel serialised with joblib

Key Findings

  • arrow_forwardBoth Ridge and Lasso clearly outperform the baseline
  • arrow_forwardHeavy regularisation improves ROC AUC slightly but has limited impact on PR AUC
  • arrow_forwardThe shift in the PR curve above precision 0.4 under strong regularisation suggests further threshold validation is needed before deployment

PDF Resources