Forecasting Football: a hedgehog amongst the foxes.
“I never make predictions and I never will.” – Paul Gascoigne
For football fans, making predictions is half the fun. It’s also big business: the global sports betting market is estimated to be worth roughly half a trillion pounds, and 70% of the trade is thought to come from football.
We all make predictions, but some of us are better than others. In his book, The Signal and the Noise, Nate Silver describes two categories of predictor: hedgehogs and foxes[1]. Hedgehogs tend to have strong pre-conceived notions of how the future will pan out, a view of the natural order of things that they can be reluctant to change. They are confident and assertive. Foxes, on the other hand, are not anchored to a particular world-view: they are more willing to change their position as new information arrives. Foxes try to weigh up all the available evidence; often their predictions are probabilistic in nature.
In football, TV pundits are hedgehogs: ex-players and managers chosen to provide us with their insider perspective. But who are the Foxes? I think the closest example is the bookmakers, many of whom now rely on statistical models to forecast the outcome of matches and set their odds.
How do pundits and bookmakers measure up? In this post I compare the forecasts of a well-known football pundit with some of the UK’s largest bookmakers. We'll see who comes out on top.
It’s difficult to collect up a large sample of predictions for pundits. Fortunately, Mark Lawrenson comes to the rescue. For those that are not familiar with him, Lawrenson is a former Liverpool player who now makes regular TV appearances as a pundit for the BBC. For the last six seasons he has also provided them with his predictions for the outcome of almost every EPL game.[2]
So how do we compare Lawrenson’s intuition with bookmakers betting odds? I think the simplest answer is also the most fun: run an experiment in which we imagine that I had bet on every prediction he made from the 2011/12 season through the 2015/16 season and see how much money I would have won or lost along the way.
Here’s how the experiment works: for each game I collected the odds offered by four bookmakers on a home win, away win and draw[3]. I identified the bookmaker that offered the best odds for Lawrenson’s predicted outcome and bet £1 on that outcome. I then tracked the cumulative profit/loss over the 1856 games that he attempted to predict (he misses a few each season). The results are shown in Figure 1.
After a shaky first season, Lawrenson’s predictions do pretty well. If you had bet £1 on each of his predictions since the beginning of the 11/12 season you would have made a £105 profit by the end of last season; this equates to a return of 6% per match. On a season-by-season basis, you’d make a profit in all but the first season, with 12/13 being the most profitable year. He had particularly good stretches at the beginning of the 12/13 and 14/15 seasons, with runs of 11 and 14 correct outcomes, respectively.
Lawrenson correctly predicted the outcome in 938 of the 1856 matches, a success rate of 51%. To put this in context, 44% of the matches ended in a home win, 31% in an away win and 25% in a draw. If you just bet on a home win in every game you’d be down £51 by the end of the five seasons.
How significant is this? What is the probability that you could be at least £105 in profit after 5 years by pure luck alone? This is an important question that requires a technical answer.
I reran the full 5-year experiment 10,000 times, randomly assigning a prediction of home win, away win or draw for every game. The rate of home wins, away wins and draws in each season were fixed to be the same as in Lawrenson’s forecasts; I basically just shuffled his forecasts around between matches. The result: in only 156 of the 10,000 simulations did the profits exceed £105, indicating that it would be unlikely to make this level of profit by chance. So he demonstrated real skill in his forecasts.
Has Lawrenson beat the casino? I think the answer is yes. That does not necessarily mean that he is a superior predictor though. If you were to perform the experiment again, this time selecting the outcome with the shortest betting odds as your prediction, you’d be right in 53% in matches – a higher success rate than Lawrenson. However, if you had placed bets on those outcomes you would have lost £47[4]. So Lawrenson’s predictions make their money on the less favoured outcomes – those with longer odds. Indeed, he only picks the outcome with the shortest odds about two-thirds of the time.
Finally, I’m keeping track of Lawrenson’s hypothetical P&L for the 16/17 season. You can find it here. If I can get hold of the data, I’ll add other pundits to this, too.
--------
Thanks to David Shaw and Brent Strickland for comments.
[1] This originates back to the essay The Hedgehog and the Fox by the philosopher Isiah Berlin.
[2] These guys look like they have collected data for other pundits, too. I’ll see if I can get hold of it and add it to this analysis
[3] The four bookmakers are Ladbrokes, William Hill, Bet365 and Bet Victor.
[4] Presumably because bookmakers shorten the odds they offer to generate a profit.
For football fans, making predictions is half the fun. It’s also big business: the global sports betting market is estimated to be worth roughly half a trillion pounds, and 70% of the trade is thought to come from football.
We all make predictions, but some of us are better than others. In his book, The Signal and the Noise, Nate Silver describes two categories of predictor: hedgehogs and foxes[1]. Hedgehogs tend to have strong pre-conceived notions of how the future will pan out, a view of the natural order of things that they can be reluctant to change. They are confident and assertive. Foxes, on the other hand, are not anchored to a particular world-view: they are more willing to change their position as new information arrives. Foxes try to weigh up all the available evidence; often their predictions are probabilistic in nature.
In football, TV pundits are hedgehogs: ex-players and managers chosen to provide us with their insider perspective. But who are the Foxes? I think the closest example is the bookmakers, many of whom now rely on statistical models to forecast the outcome of matches and set their odds.
How do pundits and bookmakers measure up? In this post I compare the forecasts of a well-known football pundit with some of the UK’s largest bookmakers. We'll see who comes out on top.
Lawrenson vs the Bookmakers
It’s difficult to collect up a large sample of predictions for pundits. Fortunately, Mark Lawrenson comes to the rescue. For those that are not familiar with him, Lawrenson is a former Liverpool player who now makes regular TV appearances as a pundit for the BBC. For the last six seasons he has also provided them with his predictions for the outcome of almost every EPL game.[2]
So how do we compare Lawrenson’s intuition with bookmakers betting odds? I think the simplest answer is also the most fun: run an experiment in which we imagine that I had bet on every prediction he made from the 2011/12 season through the 2015/16 season and see how much money I would have won or lost along the way.
Here’s how the experiment works: for each game I collected the odds offered by four bookmakers on a home win, away win and draw[3]. I identified the bookmaker that offered the best odds for Lawrenson’s predicted outcome and bet £1 on that outcome. I then tracked the cumulative profit/loss over the 1856 games that he attempted to predict (he misses a few each season). The results are shown in Figure 1.
 |
| Figure 1: The cumulative profit/loss generated from betting £1 on each of Mark Lawrenson’s match predictions over the last 5 EPL seasons (nearly 2000 matches). |
After a shaky first season, Lawrenson’s predictions do pretty well. If you had bet £1 on each of his predictions since the beginning of the 11/12 season you would have made a £105 profit by the end of last season; this equates to a return of 6% per match. On a season-by-season basis, you’d make a profit in all but the first season, with 12/13 being the most profitable year. He had particularly good stretches at the beginning of the 12/13 and 14/15 seasons, with runs of 11 and 14 correct outcomes, respectively.
Lawrenson correctly predicted the outcome in 938 of the 1856 matches, a success rate of 51%. To put this in context, 44% of the matches ended in a home win, 31% in an away win and 25% in a draw. If you just bet on a home win in every game you’d be down £51 by the end of the five seasons.
How significant is this? What is the probability that you could be at least £105 in profit after 5 years by pure luck alone? This is an important question that requires a technical answer.
I reran the full 5-year experiment 10,000 times, randomly assigning a prediction of home win, away win or draw for every game. The rate of home wins, away wins and draws in each season were fixed to be the same as in Lawrenson’s forecasts; I basically just shuffled his forecasts around between matches. The result: in only 156 of the 10,000 simulations did the profits exceed £105, indicating that it would be unlikely to make this level of profit by chance. So he demonstrated real skill in his forecasts.
Has Lawrenson beat the casino? I think the answer is yes. That does not necessarily mean that he is a superior predictor though. If you were to perform the experiment again, this time selecting the outcome with the shortest betting odds as your prediction, you’d be right in 53% in matches – a higher success rate than Lawrenson. However, if you had placed bets on those outcomes you would have lost £47[4]. So Lawrenson’s predictions make their money on the less favoured outcomes – those with longer odds. Indeed, he only picks the outcome with the shortest odds about two-thirds of the time.
Finally, I’m keeping track of Lawrenson’s hypothetical P&L for the 16/17 season. You can find it here. If I can get hold of the data, I’ll add other pundits to this, too.
Thanks to David Shaw and Brent Strickland for comments.
[1] This originates back to the essay The Hedgehog and the Fox by the philosopher Isiah Berlin.
[2] These guys look like they have collected data for other pundits, too. I’ll see if I can get hold of it and add it to this analysis
[3] The four bookmakers are Ladbrokes, William Hill, Bet365 and Bet Victor.
[4] Presumably because bookmakers shorten the odds they offer to generate a profit.
Comments
Post a Comment