Statistics can conceal more than they reveal, as illustrated by SANZAR’s tackle ratios in the case of Kurtley Beale.

Beale’s ratio gained currency in rugby circles with the news that he ended the Super Rugby rounds with 47 successful and 29 missed tackles, yielding a success ratio of 62 per cent in a competition where teams boast averages around 85 per cent.

He’s a ‘liability’ wrote one rugby fan. ‘Terrible’ said another. ‘If he is that bad at Super Rugby level’, wrote someone else, ‘how much of a weakness would he bring to the Wallabies at Test level!’ Others chose more colourful language.

The criticism is unwarranted. While I wouldn’t claim that Kurtley is a great defender, here I’ll suggest that he’s not bad. The argument turns on three points: first, the tackle ratio is a stupid measure in this case; secondly, in gauging the issue, we should heed the law of small numbers; thirdly, we should aim to compare like with like in context.

With these corrections, Beale still comes up short, but only slightly, such that no sound judgement can be made about his form without also weighing his attacking prowess.

The right ratio?
A tackle ratio is not a very relevant stat for evaluating an attacking player of the calibre of Kurtley, any more than it is for Israel Folau and others of their kind. No one has ever selected Kurtley because of his defence. Kurtley Beale is and has only ever been on the radars of the nation’s selectors because of his attacking ability.

What measure should be used? There are two ways to fix a low ratio; increase the upper bound or decrease the lower (or some combination). The first option is irrelevant because no-one really cares whether Beale tackles more frequently, particularly if this harms his attacking game.

No one wants Kurtley to start positioning himself for more tackles; we want him magnetised to the ball.

On the other hand, tackling is listed in every rugby player’s statement of duties, and we can all agree that we’d like to see fewer missed. Missed tackles are a real and present danger to a side. No matter how gifted an attacker, at some point any amount of talent can be outweighed by a propensity to miss tackles.

So, let’s put Beale’s 47 successful tackles and 62 per cent ratio aside and concentrate on what matters: the risk represented by the 29 missed tackles, which must be distributed over the games Beale played, i.e., 15. The relevant ratio, then, is 29/15, which is an average number of misses per game just short of 2, i.e., 1.9 misses per game.

The law of small numbers
The difficulty in evaluating an average of 1.9 is that we’re in the world of small numbers. The reason cricket averages are the most fabulous statistics in any sport I know is that the numbers are in the hundreds, or the thousands, or 15,921 runs over 329 innings (less 33 not out) = 53.78 if you’re Sachin Tendulkar.

Our problem is making sense of a paltry 29/14 =1.9; numbers so small that the player’s average could well be primarily determined by random variance due to the countless independent conditions that can potentially affect a tackle, otherwise known as sheer luck.

The law of small numbers – which is actually formally known as ‘the law of big numbers’, representing the same thing in reverse – gives us two useful axioms: (1) the smaller the denominator, the less informative the average (e.g., the tail-end batsman who flukes an average of 53.78 from his first Test won’t be hailed as the next Sachin); (2) the larger the numerator, the less informative the absolute number (e.g. if Michael Hooper ever misses more tackles than KB, this will only be because he makes many more; the more tackles that are attempted, the more the success ratio is indeed a relevant measure).

As we have both a small denominator (15) and a small numerator (29), the law says that absolute numbers will give us the more realistic picture in this instance.

Like with like
An even more inviolable law of fair comparison says that we can only measure like with like, of which two dimensions are obvious in our case.

Kurtley’s form should be compared with that of his team, which plays under the same coach in the same conditions against the same opponents; and with the form of other inside-centres playing under the closest similar conditions.

Starting with the Waratahs, forget comparisons with the forwards, whose jobs are obviously different.

We can also rule out the halves, for they live in heavy traffic, and the fullback, whose job is a special case.

This leaves the outside-centre and the two wingers; the three-quarters, who we will call the ‘running backs’. The job of each running back entails serious differences, but these three positions are as close as we can get to a fair comparison with the inside-centre; like with the nearest like.

Let’s take a quick glimpse at KB’s standing in this company, which includes two world-class tackling backs in Adam Ashley-Cooper and Rob Horne. The results are crude because we’re still averaging small samples with small numbers, but now at least we’re using a more appropriate ratio and a refined domain.

Ranking the defence from (the apparent) safest to the flakiest, Horne was the safest with 14 missed tackles over 14 games, the only running back to average one (1.0) miss per game. He was followed by Matt Carraro: 15/14 (1.1); Peter Betham: 11/10 (1.1); Taqele Naiyaravoro: 17/14 (1.2); and Ashley-Cooper: 16/12 (1.3).

In raw terms, part of Beale’s problem is immediately obvious: he played the most games. Even if everything else had been equal, he’d still have the highest number of missed tackles. To express this another way, if we cherry-picked KB’s 10 safest games, which happenstance can well do, he’d have the same ratio as Betham.

Something else also jumps out. The risk that we’re trying to evaluate turns out to be even smaller than we initially supposed. What’s approximately at stake here is not 1.9 misses per game, but the difference between 1.9 and a range of 1.0-1.3. Compared with his colleagues, the risk in KB’s ratio is actually smaller than one extra missed tackle per match: a risk of about an average 0.8 of one miss.

Let’s do the same quick and dirty with the other inside-centres. Here, the crudity is compounded by the variety of positions where the others played. We also have a worse small sample problem, and will adopt a Betham-standard 10-match cut-off.

As it happened, Robbie Coleman and Mitch Inman were the only inside-centres with better than an average of one missed tackle per match, with respective figures of 14/16 (0.9) and 15/16 (0.9). The other stats are: Matt Toomua: 12/11 (1.1); Luke Burton: 19/16 (1.2); Christian Lealiifano: 22/16 (1.4); and Samu Kerevi: 19/14 (1.4).

The average is virtually identical to the Waratahs’ backs, but the range is wider at 0.9-1.4. The interesting feature is that neither of the two safest defenders, Coleman and Inman, figure much in Wallaby selection debates.

Toomua’s sample is a soft 11 games, but next to the also fancied Lealiifano and Kerevi, KB only missed one extra tackle in every two games, shrinking the stakes another degree.

The whole box and dice
Let’s now get in very close and exacting by looking at the two dimensions together, using Beale’s absolute numbers, taking the real context of each game into account for the team stats, and comparing his form strictly with the other 12s that he played against and those who played against the same teams.

Within the Waratahs, Beale’s missed tackles have been measured against those of the (apparently) flakiest defender among the three other running backs in each game. The lower the figure the better. For example, if Kurtley missed two tackles and no other running back missed more than one, his stat would be: +1 missed tackle (2-1=+1); conversely, if he only missed one and, say, Betham missed two, KB’s stat would be -1 missed tackle (1-2=-1).

Note that the number of players who were safer or flakier in each game is irrelevant. The aim is to define the risk in Kurtley’s defence for the collective.

The result tells us the absolute number of extra tackles that Beale missed over the season compared with the flakiest of the nearest comparable other defenders in each game.

If this seems complicated, trust me, it’s not. It’s as straight-forward and transparent as can be. I’ve shown the full working in the first instance below, so you can crunch the rest (or any other sample) for yourself if you wish.

The other dimension is individual and compares Beale’s absolute number of missed tackles against the average of his peers under the nearest comparable conditions: the opposing inside-centre in each game and the inside-centres who played against the same team for either the Brumbies or the Reds at the closest point in the season. To maintain the same sample size throughout, where neither of these teams played against the same New Zealand and South Africa teams, I’ve substituted (in order of preference) the inside-centre for the Rebels or the Force.

In sum, the Waratah stats aim to measure Beale’s distinctive effect on the team’s risk envelope; the peer stats aim for an individual comparison.

In recounting the games, mention is also made of glaring misses by other comparable or near comparable players. The reason for this is that a missed tackle is such a grievous mortal sin in rugby that it’s embarrassing to single players out, yet the truth is that missed tackles are around six times more common than tries.

Even the greatest players will rarely play more than two matches in a row without missing a tackle. Backs in particular are bound to have a horror day sooner or later. To keep the analysis in perspective, I’ve thickened the background with a sense of the real incidence, although these mentions played no part in calculating Kurtley’s stats.

Conclusion: a minor issue
Contrary to the interpretations of SANZAR’s tackle ratio, Kurtley Beale couldn’t be classed as a liability to his team.

In total in 2015, he actually missed one tackle fewer than the (apparently) flakiest of the most comparable other defenders in his team. The crude averages for the Waratahs told us (above) that Beale is the flakiest of the running backs on an individual basis, albeit only by an average 0.8 of one missed tackle per game.

But when we gauge his form against the collective variance in the real context, it turns out that this has not usually translated into him being the weakest link on the day.

More precisely, KB was the flakiest defender in only six out of his 15 games, and this was by the thinnest possible margin with an extra six missed tackles. The six games were the first against the Reds, both games against the Rebels, both against the Brumbies, and the game against the Crusaders. As the Waratahs won every match, the misses plainly had no adverse effect on the team outcomes.

As an empirical measure, we could put this result over the 15 games and say that the collective variance reduced the risk in Beale’s defence to an average extra 0.4 of one missed tackle per game.

By the same gauge, Taqele Naiyaravoro was a riskier proposition, adding an average extra 0.5 of one miss per game, while Matt Carraro and Adam Ashley-Cooper added o.1 of one miss. Peter Betham and Rob Horne didn’t press the team’s risk envelope.

On the other hand, Kurtley was the safest or equal safest defender among the running backs in five games: in both games against the Force and the games against the Stormers, the Sharks and the Blues, when he missed, all up, two tackles. Ironically, these games also comprised three of the Waratahs’ only four losses in 2015. In the three actual losing games, Beale didn’t miss a single tackle, absolving his defence of any responsibility for the team outcomes.

Furthermore, the high correlation between Beale’s safest performances and the Waratahs’ losses raises the possibility that a more intense focus on defence might have detracted from his attack, with a greater loss to the team.

As mentioned, missed tackles are about six times more common than tries, which is to say that an instrumental role in scoring a try is worth at least six times the cost of a missed tackle to a team.

The possibility that a preoccupation with defence could have been a drag on Beale’s more valuable attack might also be implied by the game against the Super Rugby’s pacesetters, the Hurricanes.

In this match, Beale missed five tackles, his highest toll in any game over the season. This doesn’t necessarily mean that he was the weakest link on the day, since Horne and Betham also missed five each (and Ashley-Cooper missed three). Rather, the juicy point is that Kurtley’s personal horror day in defence correlates with the Waratahs’ greatest victory, the Hurricanes’ sole defeat!

The only game where the record even allows for Beale’s defence to have told on the team was the loss to the Highlanders in Dunedin, the Waratahs’ only other loss. Beale missed three tackles, but the issue is moot pending close scrutiny of the game, for Horne also missed three, Carraro missed two and Betham one. This was something of a horror day all-round for the running backs.

The results of the peer ratios are not so surprising in that KB missed 7.7 actual extra tackles in 2015, compared with the average for the inside-centres who played against him plus the two most similar others who played against the same teams. Yet notice that this reduces his crude individual ranking (see above) by about the same magnitude as the collective context. Against his peers, when measured with as much like with like as possible in the actual games, Beale’s defence represented an average extra 0.5 of one missed tackle per game, or only one extra miss every two games.

As the Waratahs’ results show, the chance of this risk crystallising as an adverse team outcome are remote. If we take the six-times miss-to-try frequency ratio, the selectors only need be satisfied that KB is likely to be instrumental in scoring at least one extra try every 12 games to cover the extra risk, although even this overstates the issue by a fair margin. While one try is scored on average for about every six missed tackles, six missed tackles don’t necessarily lead to the average try.

I’ve no real idea what proportion of tries are scored through missed tackles, as distinct from tries created by clean breaks, overlaps, kicks in play, push-over scrums, rolling mauls, intercepts, weird stuff-ups, or whatever, but let’s hazard a guess of 50 per cent for the sake of the argument.

If this were true, it would take on average some 12 missed tackles before one actually resulted in an opposition try being scored. Given that Beale brings the risk of an extra 0.5 of one missed tackle per game, he would only need to be instrumental in creating one extra try every 24 games to cover the issue!

No doubt that estimate could be refined. For example, some discount should be made for the fact that missed tackles by running backs are more likely to result in a try than a miss in any other position.

All the same, it ought to now be clear why it’s unlikely that Beale’s defence was more responsible than anyone else for the Waratahs’ four defeats. Hopefully, it will also now be clear why Kurtley Beale’s defence is, or should be, a minor issue in deciding whether he will continue to grace the Wallabies.
 

Kurtley Beale’s 2015 tackling record (full names only on first mention)

1-2. Force

Beale didn’t miss in either game. In the first, Taqele Naiyaravoro missed two and Adam Ashley-Cooper missed one (Rob Horne missed none). No running back missed in the second game.

Team: first game: KB = 0; TN = 2; KB’s team stat = 0-2 = -2; second game: 0

KB’s opposite, Solomoni Rasolea, missed four in the first and none in the second. In the Brumbies versus the Force, Christian Lealiifano missed 1 in the first game (Matt Toomua playing at 10 missed 4), and Robbie Colman missed one in the second. In the Reds versus Force, Samu Kerevi missed one in the first and none in the second.

Peers: first game: SR = 4 + CLL = 1 + SK = 1 = 6/3 = 2; KB’s peer stat = 0-2 = -2; second game: -0.3

3. Reds

KB missed 2 in the first game. Peter Bethan and RH missed 1 each (Matt Carraro missed none). KB didn’t play in the second game.

Team: +1

Anthony Fainga’a missed 2 (Chris Feauai-Sautia at 13 missed four Lachie Turner at 11 missed 5). versus Brumbies, CLL missed two versus Rebels (first), Mitch Inman missed one

Peers: +0.3

4-5. Rebels

KB missed two in both, ACC missed one in both, and RH missed one in the second (TN missed none in the first; PB missed none in the second).

Team: first game: +1; second game: +1

Mitch Inman missed 2 in the first game (Tamati Ellison at 13 missed 4), and none in the second. versus Reds, AF missed 2 in the first. SK missed none in the second (Karmichael Hunt at 13 missed 4). versus Brumbies, CLL missed none in the first (Tevita Kuridrani at 13 missed 3). Nigel Ah Wong missed 1 in the second.

Team: first game: +0.7; second game: +1.5

6-7. Brumbies

KB missed three in both. The other running backs missed one apiece in the first and Adam Ashley-Cooper missed two in the second (when RH and PB missed none).

Team: first game: +2; second game: +1

Christian Lealiifano missed 1 in the first (Joe Tomane at 11 missed 4, TK at 13 and Henry Speight at 14 missed 3 each). RC missed 1 in the second. versus Reds, SK missed 3 in the first and AF missed 3 in the second. versus Rebels, MI missed 1 in both (Tamati Ellison at 13 missed 3 in the first).

Peers: first game: +1.3 second game: +1.3

7. Stormers

KB missed none, TN missed 4, Matt Carraro missed 1.

Team: -4

Damian de Allende missed 2. versus Brumbies, RC missed none (CLL at 10 missed 3). versus Rebels, MI missed 1.

Peers: -1.0

8. Sharks

All the running backs missed 1.

Team: 0

Francois Steyn missed 3. versus Reds, SK missed 1 (KH at 10 and Chris Kurindrani at 12 missed 3 each). versus Rebels, MI missed none (Don Shipperley at 14 missed 3).

Peers: -0.3

9. Cheetahs

KB missed 1, Adam Ashley-Cooper missed 2, MC missed 1 (TN missed none).

Team: -1

Rayno Benjamin missed 3. versus Reds, SK missed none. versus Force, Luke Burton missed 1.

Peers: -0.3

10. Lions

KB missed 2, TN missed 3, RH and Adam Ashley-Cooper each missed 1.

Team: -1

Harold Vorster missed 2. versus Brumbies, Toomua missed 1 (TK at 13 and CLL at 10 missed 5 each). versus Reds, AF missed 1.

Peers: +0.5

11. Blues

KB missed 1, MC and TN each missed 2, RH missed 1.

Team: -1

Francis Saili missed 3. versus Brumbies, CLL missed 2. versus Rebels, MI missed 1.

Peers: -1.5

12. Highlanders

KB missed 3, as did RH. MC missed 2 and PB 1.

Team: 0

Shaun Treeby missed 2. versus Brumbies, RC missed none. versus Reds, AF missed 2.

Peers: +1.7

13. Crusaders

KB missed 4, Adam Ashley-Cooper missed 3 and TN missed 2 (RH missed none).

Team: +1

Dan Carter missed 1 (Nemani Nadolo at 11 missed 4). versus Reds, AF missed 4 (as did SK at 13). versus Brumbies, RC missed 3.

Peers: +1.3

14. Hurricanes

KB missed 5, as did both RH and PB, while Adam Ashley-Cooper missed 3.

Team: 0

Ma’a Nonu missed 2. versus Reds, AF missed 2. versus Rebels, MI missed 1.

Peers: +3.3

15. Tahs absent KB versus Reds

Although excluded, if anything, the running backs were less safe without KB, who missed 2 in the first game. MC at 12 missed 3, PB missed 2 and TN missed 1 (Adam Ashley-Cooper missed none). For the Reds, SK missed 2. In the Brumbies second game against the Reds, CLL missed 2. In the Rebels second game against the Reds, MI missed 2.

Results

Team total: number of extra tackles missed over the season compared with the flakiest of the three nearest comparable other defenders in each game: -1

Peer total: number of missed tackles compared with the average of the opposing inside-centre in each game plus the two nearest comparable inside-centres who played against the same team: +7.7

(All stats from the official SANZAR site)