Komorebi
Advertiser churn prediction for Sumauto vehicle classifieds.
Predict churn one month ahead
Komorebi is a Madrid-based AI consultancy. They brought this case to Zrive's 2025 cohort from their client Sumauto, a group of car advertising portals. Sumauto loses advertisers without warning, each costing roughly €1,000 in foregone revenue. The goal was a model that flags at-risk advertisers one month before they leave.
0.72
ROC-AUC
0.18
PR-AUC
0.05
Std dev
Project Details
- ClientSumauto via Komorebi
- Team4 Zrive students
- ModelXGBoost
- Lead time1 month ahead
crisis_alertProblem
Sumauto is a group of car advertising portals that connect car dealerships with buyers. Advertisers leave the platform both explicitly through cancellation requests and implicitly by letting contracts expire without renewal. The existing churn model, built by Komorebi for an earlier engagement, was limited in scope and lacked the depth needed for reliable early warning.
Komorebi proposed the case to Zrive's 2025 cohort as a real business problem. The team received anonymised, scaled data and full freedom in modelling approach, with the constraint that every technical decision had to be justified and validated against appropriate evaluation metrics.
query_statsApproach
The problem was framed as binary classification: predict whether an advertiser will churn in the following month. Two churn signals were defined. Explicit churn came from the withdrawal dataset, filtering for total cancellations that were not administrative contract changes. Implicit churn came from contract end dates where the advertiser had been active the prior month but did not renew.
Evaluation used a sliding window scheme where training and validation windows advance month by month. This mirrors real deployment conditions and surfaces concept drift early. Business performance was measured alongside technical metrics using a retention cost simulation. The operating threshold was chosen based on expected ROI rather than classification accuracy alone.
account_treeSteps Taken
Data definition and labelling
Churn was extracted from two sources: explicit withdrawals (advertisers filing formal cancellations) and implicit inactivity (contracts ending without renewal and no subsequent activity). Post-first-churn rows were removed to keep the dataset focused on pre-departure behaviour.
Exploratory analysis and cleaning
Several columns had up to 40% null values and were dropped entirely. Rows with no active contract and no published ads were removed. The goal was a clean, temporally consistent dataset without introducing bias through imputation.
Feature engineering
Three feature families: temporal (tenure, months since last contract, renewal flag), ratio-based (leads per visit, invoice per lead, ad utilisation rate), and 3-month rolling aggregates with delta metrics to capture short-term trend changes.
Model selection
Progression from logistic regression to random forest to XGBoost. Logistic regression outperformed the naive baseline but modestly. Random forest improved further but added complexity without proportional gain. XGBoost, tuned with a low learning rate, was the final choice for its stability across validation windows.
Business threshold analysis
A retention simulation modelled churn cost (€1,000 per lost advertiser), intervention cost (€100 per action), and 70% effectiveness. Thresholds between 0.14 and 0.16 produced the best balance: positive ROI, low month-to-month dispersion, and manageable false positive rate.
developer_modeTechnologies and Tools
bar_chart_4_barsResults and Conclusions
XGBoost reached a mean ROC-AUC of 0.72 and PR-AUC of 0.18 across validation months, stable across five random seeds (std dev 0.05). The model consistently outperformed both the naive baseline and earlier linear approaches. Performance degraded predictably as the validation window moved further from the training period, confirming concept drift as expected in time-dependent data.
Feature importance pointed to tenure and months since the last contract as the strongest predictors. Both were removed from the final model as a precaution against leakage, though they may reflect real contractual patterns (such as discount expiry windows) worth reintroducing after stakeholder validation.
Business threshold analysis identified 0.14 to 0.16 as the optimal operating range: positive ROI, low month-to-month dispersion, and an acceptable false positive rate. Retraining monthly is recommended to combat drift and maintain prediction quality over time.