A major food delivery company was operating across 12 countries. Growth was strong, but so was churn — customers would order regularly for weeks, then quietly disappear. The marketing team had budget for retention campaigns, but no way of knowing who to target. They were spending blind.
The obvious approach — send everyone a discount — burns money on customers who were never going to leave, while missing the ones who were. They needed a way to identify at-risk customers before they churned, so the retention budget could be focused where it would actually make a difference.
Why churn prediction is harder than it looks
Customer purchasing data is inherently noisy. A customer who stops ordering hasn't necessarily churned — they might be on holiday, cutting spending temporarily, or trying a competitor before coming back. There's no moment where someone formally "leaves." You have to infer it from behaviour, and the signals are subtle.
We started by identifying the patterns that actually mattered for food delivery: pricing sensitivity, limited restaurant options in their area, gaps in regular ordering habits (a customer who orders every Friday and then misses two weeks is a very different signal to someone who orders sporadically). Each market had its own dynamics — what predicted churn in one country didn't necessarily apply in another.
Choosing the right model — and why we rejected deep learning
We tested several approaches, including a hybrid neural network combining LSTM and CNN architectures designed to extract patterns from temporal customer data. It performed marginally better on raw prediction metrics.
We chose XGBoost instead.
The reason was interpretability. In churn modelling, knowing why a customer is predicted to leave is as important as knowing that they will. If the model flags a spike in churn probability after a price increase, the marketing team can act on that signal specifically. A black-box neural network gives you a number — a transparent model gives you a story. For a team making daily decisions about where to spend retention budget, the story matters more than the last fraction of a percent of accuracy.
Making the predictions trustworthy
A model that says "this customer has a 30% chance of churning" is only useful if that number means what it says. If you take every customer the model rates at 30%, roughly 30% of them should actually churn. This is called calibration, and most models get it wrong out of the box.
We validated our predictions against real outcomes across every probability band. When the model said 20% risk, we checked that roughly 20% of those customers actually churned. When it said 80%, we checked that too. This gave the marketing team confidence to act on the scores — they could set thresholds ("target everyone above 60% risk") knowing the numbers were reliable, not just directionally correct.
Proving it worked — not just in theory
We didn't ship the model and call it done. We ran a proper A/B test.
One group received promotions targeted at random — the way the team had always done it. The other received promotions targeted by the churn model, focusing spend on the customers most likely to leave. Both groups had the same budget. We calculated the required sample sizes and test duration upfront to ensure statistical significance.
The results
A 2% churn reduction might not sound dramatic on paper. But at the scale of a company processing millions of orders across 12 countries, small percentage improvements translate into significant revenue. The same budget, spent on the right people, delivered twice the impact — and that compounds with every cycle. The model didn't just predict churn — it turned retention from guesswork into a system.