Just Deserves

Whose play suggests they deserved more from this season and whose less?

If you’ve read my book, Attacking the Space, you’ll know that I consider 22-Entry Difference (that is the number of times you get into the opposition 22 minus the number of times they get into yours) as the most meaningful stat in the game. It tells you who created the most chances and who limited their opponent’s. Theoretically then, it can also tell us who should have done well and who outperformed.

Using the excellent Oval Insights data we can create a basic but effective Expected Points (xPTS) model based purely on 22-Entries Difference. If your 22-entries difference was greater than +5 you get awarded 4 xPTS, if it’s between +2 and +5 you get 3 xPTS, +2 to -2 is 2 xPTS, -2 to -5 is 1 xPTS, anything worse than that is 0 xPTS. You’ll note that we don’t have bonus points here so when we compare our xPTS to the real world we are comparing them to ‘True Points’ those are 4pts for a win, 2 for a draw, 0 for a loss.

Let’s start by looking at the Prem this season. Three of the play-off teams are at the top of our xPTS table. Bath and Leicester Tigers far outstrip the competition and you can see that just 6 xPTS cover third to Exeter Chiefs in eighth. If you are wondering what is going on here, it’s important to note that both Northampton Saints and Exeter Chiefs won two games when the xPTS model would’ve suggested they should have received 0pts. In the case of Exeter those games were a 27-26 win versus Sale Sharks in November when they had six entries to eleven for Sale and a 38-14 victory over Newcastle Red Bulls when the Red Bulls had 14 entries and Exeter had just eight. Sale Sharks on the other hand got 0 True Points from five games where the xPTS model said they should have received 12 PTS.

All in though I am happy with what we are seeing. It’s showing us interesting results without completely overturning what really happened. If you are working on something like this and the results don’t chime with the real world then double check everything.

We can look at what happened in the Champ, a season with more games where theoretically we should see less volatility. Unsurprisingly, Ealing remain top of the charts, the only game they played all season with fewer xPTS thought they would lose was their narrow 34-33 win against eventual victors Worcester Warriors where they had ten (!!) fewer 22-entries than the Warrior’s 19. Elsewhere, there is some concern for Bedford Blues who over performed their xPTS by 15pts, though their xPTS total was still enough to suggest they were not lucky to make the play-offs. Chinnor also under performed their xPTS, as did Cambridge by a remarkable 26pts!

Again, the Top 14 has plenty of games so we see less volatility with the exception of Racing 92, who enormously out performed their xPTS and La Rochelle, who ‘should’ have finished second. Racing beat Perpignan, Pau, Bordeaux, and Toulouse when their 22-entry difference was at least five worse than their opponents. They also won four of the five games with a 22-entry difference of between +2 and 0. Is that sustainable? We’ll discuss that later. La Rochelle on the other hand were rarely bad, just two games had 0 xPTS (and they lost them both) but they also lost five of the eight games where their 22-entry difference was between +2 and -2. They also lost two of the twelve games where they were expected to get 4 PTS.

Finally, the URC. The top four are clearly ahead of everyone based on both types of points but we should have our eye drawn to the bottom right and the Scarlets. Both they are the Dragons have 20 True PTS, so what is different for them? The Scarlets are an odd choice. They have dominated very few games, they’ve not had more than a +5 22-entry difference. But, the seven games with a positive 22-entry difference should have yielded 22pts but instead they brought just six; a win against Zebre and a draw against the Dragons. Meanwhile, there were three matches where xPTS gave them 0 PTS, they won two of them. One was a 27-22 victory over Ulster when dealing with a -9 22-entry difference and the other was an absurd 21-17 victory over Cardiff when dealing with a -17!! difference. Of the 79 games with a 22-difference of -10 or worse, just six were won. This was the only one of 34 games where a team had a 22-difference of worse than -13 and won.

We can how much a team over performed their xPTS per game in this graph. A team like Northampton Saints, 1.3pts per game more than their xPTS would suggest whereas Sale Sharks lost 1.1pts per game compared to their xPTS. The question is whether this is a skill.

Let’s consider something like kicking at goal. That is a skill, so you’d expect that a good player last year would be a good player this year. It’s not a coin flip where one year Jonny Wilkinson is great, the next he misses everything, the next he’s fine, the next he is bad, the next he’s amazing etc. The key is understanding if something is a skill or if it’s random. The question is whether being able to out perform your xPTS is a skill (and equally if under performing it is something you can consistently do). The answer is that I don’t yet know but it seems highly unlikely.

To understand why, let’s break down what 22-entries is actually covering. It looks just at your 22-entries and your opponent’s 22-entries. You might say that therefore an incredibly efficient team might be able to score a load of points on their own entries and limit the opposition on theirs. That’s true to a point, a good side would score more points on their entries than the opposition. But, to out perform your xPTS you need to be both efficient on your entries but also not very good at limiting the opposition at getting entries which is a curious and unlikely combination of skills. More likely, you are getting lucky. That’s an ugly word I know, but luck plays a massive part in sport. Were Sale Sharks a better team than Exeter Chiefs this year? On the face of it, that is a totally mad thing to say given that the difference in the league table was 29pts. So is there any logic in it?

Let’s start by removing 9pts from that difference that have come from bonus points and we are therefore removing from our measurements. So Exeter are 20pts better still in the league. There were eleven games between the two teams which had an xPTS of three or more; five for Sale and six for Exeter. Sale’s xPTS from these games was 20 PTS and they came away with 16 PTS. Exeter’s xPTS from these games was 21 PTS and they came away with 16 PTS. In the games they were meant to win, they were more or less just as good as one another.

Between the two teams there were 14 matches with an xPTS of 1 or zero; seven for each team. Sale were expected to come away with 3 PTS from these games but came away with 0 and Exeter were expected to come away with 2 PTS but instead came away with ten. One conclusion you might reach is that therefore Exeter are just better at winning games when the chips are down. There is some evidence to suggest that, last year they got 8 PTS from their nine matches last season where they had an xPTS of one or less (they were expected to get four points from those matches). Though, on the flip side, they were much worse at winning games they should win. There were six matches with an xPTS of 3 or more (totaling 22 xPTS) and they won just eight actual points from them.

If you feel you’ve got lost in the numbers, here is a quick summary. While it is tempting to believe that teams can either win games they aren’t expected to win or win games they are expected to win, there isn’t any evidence to show that. The real skill is limiting the number of games where you’re not expected to win. Which basically means having a positive 22-entry difference as often as you can. And, if your True Points is out performing your xPTS significantly, then be aware about what might happen next season. Unless you improve then expect lady luck to be less favourable to you next time.