03
Statistical Learning Fundamentals
Pythonscikit-learnLogistic RegressionPR AUCjoblib
Overview
Binary classification model to predict push notification engagement on a heavily imbalanced dataset. The positive class represents users who engage with a notification, a rare event by design.
Modelling Approach
- arrow_forwardTemporal split by cumulative order volume (70/20/10) rather than fixed dates, reflecting real production conditions and preserving seasonality
- arrow_forwardBaseline built on global item popularity
- arrow_forwardLogistic Regression with L1 and L2 regularisation, wrapped in sklearn Pipelines with StandardScaler
- arrow_forwardOptimised for PR AUC given class imbalance; ROC AUC used as secondary metric
- arrow_forwardFeature importance extracted from Lasso coefficients
- arrow_forwardFinal model trained on a 4-feature subset that matched full-feature performance
- arrow_forwardModel serialised with joblib
Key Findings
- arrow_forwardBoth Ridge and Lasso clearly outperform the baseline
- arrow_forwardHeavy regularisation improves ROC AUC slightly but has limited impact on PR AUC
- arrow_forwardThe shift in the PR curve above precision 0.4 under strong regularisation suggests further threshold validation is needed before deployment