A major food delivery company was operating across 23 countries. Growth was strong, but so was churn. Customers would order regularly for weeks, then quietly disappear. The marketing team had budget for retention campaigns, but no way of knowing who to target. They were spending blind.
The obvious approach (send everyone a discount) burns money on customers who were never going to leave, while missing the ones who were. They needed a way to identify at-risk customers before they churned, so the retention budget could be focused where it would actually make a difference.
Why churn prediction is harder than it looks
Customer purchasing data is inherently noisy. A customer who stops ordering hasn't necessarily churned. They might be on holiday, cutting spending temporarily, or trying a competitor before coming back. There's no moment where someone formally "leaves." You have to infer it from behaviour, and the signals are subtle.
We started by identifying the patterns that actually mattered for food delivery. Pricing sensitivity, limited restaurant options in their area, gaps in regular ordering habits (a customer who orders every Friday and then misses two weeks is a very different signal to someone who orders sporadically). Each market had its own dynamics, and what predicted churn in one country didn't necessarily apply in another.
Choosing the right model, and why we rejected deep learning
We tested several approaches, including a hybrid neural network combining LSTM and CNN architectures designed to extract patterns from temporal customer data. It performed marginally better on raw prediction metrics.
We chose XGBoost instead.
The reason was interpretability. In churn modelling, knowing why a customer is predicted to leave is as important as knowing that they will. If the model flags a spike in churn probability after a price increase, the marketing team can act on that signal specifically. A black-box neural network gives you a number. A transparent model gives you a story. For a team making daily decisions about where to spend retention budget, the story matters more than the last fraction of a percent of accuracy.
Making the predictions trustworthy
A model that says "this customer has a 30% chance of churning" is only useful if that number means what it says. If you take every customer the model rates at 30%, roughly 30% of them should actually churn. This is called calibration, and most models get it wrong out of the box.
We validated our predictions against real outcomes across every probability band. When the model said 20% risk, we checked that roughly 20% of those customers actually churned. When it said 80%, we checked that too. This gave the marketing team confidence to act on the scores. They could set thresholds ("target everyone above 60% risk") knowing the numbers were reliable, not just directionally correct.
Proving it worked, not just in theory
The model went into a proper A/B test before anyone called it shipped.
One group received promotions targeted at random, the way the team had always done it. The other received promotions targeted by the churn model, focusing spend on the customers most likely to leave. Both groups had the same budget. We calculated the required sample sizes and test duration upfront to ensure statistical significance.
How it shipped
Predicted churn scores write to a database table the marketing team queries from their campaign tooling. They set thresholds against the score ("everyone above 60% risk gets the targeted promo") and run their existing campaign workflow against the segment. No new tooling for the marketing team to learn. The model just turned a noisy customer list into a ranked one.
Per-market retraining keeps the model current as buying behaviour shifts. What predicted churn in one country last quarter doesn't necessarily apply this quarter, or in the next.
The results
A 2% churn reduction doesn't sound dramatic on paper. At the scale of millions of orders across 23 countries, the same retention budget spent on the right people delivered twice the impact, and that compounds every cycle. Retention stopped being guesswork and became a number on a dashboard the marketing team trusted enough to spend behind.