Lies, damned lies, and statistics: How analysis and stats are moving ever further apart

Nicholas Bishop Columnist

By Nicholas Bishop, Nicholas Bishop is a Roar Expert

 , , , ,

128 Have your say

Popular article! 4,558 reads

    During the production of our book The Iron Curtain: My rugby journey from league to union, Phil Larder, England’s defence coach for the World Cup winning team in 2003, told me a very funny story.

    The story centred around England’s quarter-final match against Wales in that tournament.

    One of England’s plans for that game – the brainchild of kicking coach Dave Alred – was to have a big forward stay out wide so they could take advantage of the diminutive stature of Welsh winger Shane Williams with a high cross-field kick.

    Cometh the hour, cometh the man, in the shape of lock Ben Kay – all 6’6″ and 19 stone of him.

    Centre Mike Tindall put in the kick and Kay gamely pursued it down the right touch-line. So far, so good. But at this point, things began to go wrong. The kick was too long and it did not give Kay the chance to contest the ball in the air. Suddenly he found himself in a second row’s darkest nightmare – isolated against a wing with magical feet and no defender within 20 metres!

    Williams duly stepped Kay and set up a try at the other end of the field.

    The kicker came up in the review of the game afterwards. While doing his own presentation on defence, Larder credited Kay with a missed tackle at the start of the sequence.

    The big Leicester man immediately responded in typically irreverent Welford Road fashion, “How could it be a missed tackle? I didn’t get within five metres of him!”

    The meeting erupted in laughter, but it also exposed a deeper truth.

    Who was really responsible for the ‘missed tackle’ that led to the try? Was it Kay’s failure to get near enough to tackle Williams? Was it Tindall’s failure to place the cross-kick more accurately? Or was it the Alred plan, which saw the positives of the match-up between Kay and Williams but glossed over the negatives?

    Statistically, the error belonged to Kay, but Phil admitted to me that it made him seriously question the value of stats as a description of the true malfunctions involved in the play. Thereafter he took them with a pinch of salt and relied on his own sources of information.

    In the modern game, the real world of analysis has moved ever further away from raw statistical definitions. While stats can give a picture ‘at a glance’, it can often be a mistake to lend them any more credence than that.

    Nowhere is this truer than in real-life assessments of productive defence. Stats like ‘tackles missed’, and even ‘line-breaks conceded’ are categories which now have to be taken with Phil’s pinch of salt.

    As the Hurricanes proved with their defensive revolution in last year’s Super Rugby tournament, line-speed is the contemporary killer, and elite defensive coaches will do almost anything to trigger and sustain high line-speed. As Andy Farrell used to say with England, “our defence is not a tidy one.”

    Frequently ‘Faz’ would be prepared to accept a higher proportion of missed tackles and line-breaks in order to get what he really wanted – pressure on the ball-carrier and ultimately, turnovers. His defence looked horrible on paper but it worked where it counted, out on the field.

    The current Australian defence coach, Nathan Grey, has much the same cast of mind as Farrell. He wants ‘the nut’ back as quickly as possible and he is prepared to take a statistical hit in order to get it.

    I reviewed the Wallabies’ end-of-year games against Wales, Scotland and Ireland with this in mind. The two main points of focus in the Aussie defence are No.7 Michael Hooper in the forwards and No.13 Tevita Kuridrani in the backs, and they set the tone for the Wallabies stress on line-speed ‘come what may’.

    Aggressive Wallaby transition zones
    Last week I discussed the importance of ‘transition zones’ (between last forward and first back) on defence, in relation to England’s defence of their Six Nations title.

    Wallaby action in those transition zones tends to be very aggressive, and much of that aggression is linked to Hooper’s great upfield speed.

    In the first clip on the reel, he is the organiser next to Kuridrani, pointing out the threats, making the tackle on Luke Charteris and then getting up to disrupt the ball.

    In the second clip (65:16), his speed off the mark forces the Wales attack to become uncoordinated, with the ball-carrier, Dan Lydiate, running into his own teammate (Justin Tipuric) for an accidental obstruction call.

    Nothing epitomises the value of Hooper’s speed to the Wallabies better than the kick return by Scotland’s quickest and most dangerous back (No.15, Stuart Hogg) at 15:20. Hogg would normally accelerate around any opposition forward, but here Hooper (with Kuridrani’s assistance) cuts him down and creates the chance of another turnover.

    When is a missed tackle not a missed tackle (Part 1)?
    The first sequence from the Scotland game (11:10-11:24) gives a taste of how easily stats can replace real analysis. On two occasions, it appears Hooper is missing tackles, first on Scotland No.12 Alex Dunbar at 11:12, then running straight past No.4, Richie Gray, at 11:18.

    In fact, he is executing his key role in Nathan Grey’s defensive pattern, flying straight upfield and forcing the play back inside. On the second occasion, Gray is forced back into David Pocock’s clutches, and we all know how that scenario ends. So despite Scotland ‘trending’ fashionably on attack by making two offloads in the sequence, the end-product is a turnover to the Wallabies.

    I would also guess the initial break by Finn Russell (Scotland 44:21) would be a ‘miss’ debited to Rob Simmons’ account rather than Hooper, if it is debited to anyone. It is here that the untidiness of Grey’s defensive scheme appears in its truest light. It is the kind of break that Grey might even consider acceptable risk in the context of what he is trying to achieve – getting the ball back as quickly as possible by turnover.

    When is a missed tackle not a missed tackle (Part 2)?
    The central character in the clip from the Ireland game (18:24-18:40) is Pocock rather than Hooper. Ireland have a six-to-four overlap out to their right at the start of the sequence, and two of those Australian four are forwards (Pocock and Simmons), so things look promising for the Irish attack.

    Ten years ago, at 18:27, most defensive teams would settle for a drift with Pocock, Reece Hodge (12) and Henry Speight (11) sliding out to cover the far sideline. But that is not the modern way.

    Hodge continues to drive upfield hard onto No.13 Jared Payne and (knowingly) gives up the overlap outside. Just why the line-break is not considered to be the end of the world for the Wallabies only becomes clear after Ireland have made another 25 metres down the field!

    At that point, Bernard Foley has come up from the backfield, Will Genia has covered across from his sweeping position and Australia’s two forwards (Pocock and Simmons) are ahead of their Irish opposites (Rory Best and Devin Toner).

    The situation is reversed at 18:37 and Australia now have the numbers, with the priceless bonus of having Pocock attached to the tackle ball.

    Who missed Gary Ringrose for the Ireland try?
    The sequence at the end of the reel begs the question of who should be assigned a missed tackle on No.12, Gary Ringrose, as he slaloms his way between Ireland ‘poles’ to score their second try.

    Hooper does what he is asked to do, driving up past Ringrose’s outside shoulder and forcing play back inside, Rory Arnold is blocked out by Toner on one side of the ball-carrier, likewise Sekope Kepu by Jack McGrath on the other. Dean Mumm is required to make a tackle on an elusive inside back in space, only five metres from the Australian goal-line.

    Is any one individual responsible, is it a system breakdown, is it poor officiating or just sheer misfortune?

    The second reel looks at the same issues from a back-line standpoint:

    Straight-line speed and funnelling the attack back inside
    In the clips from Wales at 46:10, and Scotland at 27:23 and 31:07, Kuridrani is required to play a similar role to Hooper in the forwards. He drives straight upfield, and in all three instances he is the furthest defender upfield at the point of contact.

    He ‘misses’ Lydiate and Huw Jones in two (very similar) examples, and the second of those misses results in a try for Scotland – but he also forces one turnover indirectly by cutting off play to the outside and funnelling the ball carrier back into Pocock, and comes back to make the tackle on Tipuric in the Welsh example anyway.

    The risk-reward balance is indeed on a knife edge!

    Using the ‘hit and fold’ against overlaps
    As in the earlier sequence from the Ireland game (18:24), the Wallabies do not allow obvious overlap situations to inhibit their desire to drive upfield and make a hit. At 46:40, the Welsh have engineered a four-on-two overlap out to their right, but Dane Haylett-Petty still makes the hit on the inside man, Scott Williams, apparently freeing up the attackers out wide.

    But the situation has changed dramatically by 46:43. Kuridrani flattens Ross Moriarty after folding in behind Haylett-Petty’s hit and Hooper is closer to the ball than any other Wales support. When play reaches the touchline, George North has no option but to throw the ball tamely into Kuridrani’s hands for the turnover!

    At Scotland 82:32, Haylett-Petty again hits up in the knowledge that Hodge is folding in behind him towards touch. In the Ireland scrum attack at 21:00, Hodge drives up hard on to Jared Payne, giving op the width of the field to number ten Paddy Jackson on the wrap-around play. But Hooper is already folding in behind him, and by 21:08 although the ‘line break’ has been made officially, there are six Australians around the ball compared to only four Ireland supports.

    With Hooper and Pocock as two of those Wallabies, it is no surprise that the ball is turned over.

    The endgame of Grey’s ‘get the nut back as quickly as possible’ policy is very clear at Wales 75:15. There is a spare Welsh attacker wide, but Haylett-Petty breaks straight upfield on to the pass for the interception try.

    When is a missed tackle not a missed tackle (Part 3)?
    The issue of where to assign a ‘miss’ raises its head again at Wales 68:35 and Ireland at 9:25 (both from first-phase lineout). In the Wales case, it looks like a miss by Kuridrani, but in fact he is faced by a two-on-one and is already firmly behind the eight-ball at 68:36.

    Hodge probably should have shifted up on to the Wales second receiver Same Davies (in the black hat) and left Jamie Roberts to Hooper. In the Ireland sequence, Henry Speight loses his angle on the No.15 Rob Kearney, forcing Israel Folau to square up on the Irish fullback. In the event, both situations are saved by magnificent cover tackles by Haylett-Petty from the blind-side wing.

    The relationship of statistics to actual game analysis is very much in a process of dynamic flux. At their best, stats can supply a start-point for that analysis to begin, but just as often can be misleading – unless the intent and tactical doctrine of the teams involved are fully understood.

    With stats now regularly being used on TV analysis, this can easily result in a misperception of the abilities of individual players.

    The number of international defences (and defence coaches) who are prepared to look bad on paper in order to achieve real results out on the field is on the increase. No longer will the coach simply look at the figures in front of him and make a judgment solely on that basis. He will concede the higher percentage of missed tackles, and even the number of line-breaks given up on the official stats if he gets what he wants – turnovers and forced errors.

    He will look at how well the players are performing the roles required of them instead, and take the blame himself when the system itself is at fault.

    Step forward, Andy Farrell and Nathan Grey.

    Nicholas Bishop
    Nicholas Bishop

    Nick Bishop has worked as a rugby analyst and advisor to Graham Henry (1999-2003), Mike Ruddock (2004-2005) and most recently Stuart Lancaster (2011-2015). He also worked on the 2001 British & Irish Lions tour to Australia and produced his first rugby book with Graham Henry at the end of the tour. Three more rugby books have followed, all of which of have either been nominated for or won national sports book awards. Nick?s latest is a biography of Phil Larder, the first top Rugby League coach to successfully transfer over to Union, entitled ?The Iron Curtain?. He is currently writing articles for The Roar and The Rugby Site, and working as a strategy consultant to Stuart Lancaster and the Leinster coaching staff for their European matches.

    This video is trending right now! Submit your videos for the chance to win a share of $10,000!

    Have Your Say

    If not logged in, please enter your name and email before submitting your comment. Please review our comments policy before posting on the Roar.

    Oldest | Newest | Most Recent

    The Crowd Says (128)

    • February 1st 2017 @ 4:55am
      Hello Everybody. said | February 1st 2017 @ 4:55am | ! Report

      Statistics have always had a need for context. Its not new.

      It doesnt mean statistics are at odds with reality. It simply means they need context.
      A high line speed with no missed tackles is better than a high line speed defence with a lot of missed tackles generally.

      • Columnist

        February 1st 2017 @ 6:14am
        Nicholas Bishop said | February 1st 2017 @ 6:14am | ! Report

        A high line speed with no missed tackles

        This would be very nice HE, but it doesn’t often happen unless there is a clear mismatch between the two sides. One thing you learn is that you cannot have everything on defence. Either you have to be prepared to be more passive, give up the yards but prevent line-breaks, or you look to create turnover but grit your teeth for a higher proportion of missed tackles etc…

        If you come up hard on D the window to be in the right position to make a tackle inevitably shrinks 🙂

    • Roar Guru

      February 1st 2017 @ 5:27am
      Fionn said | February 1st 2017 @ 5:27am | ! Report

      Utterly fascinating, Nicholas, thank you.

      Out of interest, what do you think of the Wallabies’ defensive strategy? How much will it be be changed now that we are losing Pocock, or will it remain the same?

      • Columnist

        February 1st 2017 @ 6:07am
        Nicholas Bishop said | February 1st 2017 @ 6:07am | ! Report

        Hi Fionn.

        I think Grey’s policy still needs more refinement – and maybe some better personnel to improve it. It may need some tweaks without DP, as there’s no-one else in Oz with his ball-stealing ability…

        But again, I suppose I need to question the word ‘refinement’ in the context of the article! … Too much refinement and you prob end up playing a safer and less committal form of D!

        • Roar Guru

          February 1st 2017 @ 6:53am
          Fionn said | February 1st 2017 @ 6:53am | ! Report

          I feel like that if Grey and Cheika can get this high-risk hugh-reward strategy working it (and that is a big if) then it probably offers our best opportunity to win the Bledisloe given how much better then ABs currently are.

          However, I don’t know if such a risky strategy is the best strategy to take us to number 1 or 2 in the world, or win us a World Cup. Seems like it is bound not to work every few matches.

          That said, perhaps such a risk approach is needed to climb back onto the top of the world standings. I’m just worried we’ll become like Argentina, where we always look brilliant for patches of matches but it never really turns into anything.

          Still, great article, Nick.

          • Columnist

            February 1st 2017 @ 7:26am
            Nicholas Bishop said | February 1st 2017 @ 7:26am | ! Report

            As long as people like Hooper are in the run-on side, it makes sense. MH has such an unusual skill-set for a #7 that it is a logical course to pursue… If for some reason he wasn’t available for a lengthy period, they might have to adopt a more conservative policy.

    • February 1st 2017 @ 5:59am
      Ken Catchpole's Other Leg said | February 1st 2017 @ 5:59am | ! Report

      Its easy for you to make it look simple, isnt it, Nicholas?
      A very clear analysis in this piece. And it makes a mockery of so many stat based armchair arguments on this and other forums.
      I couldnt help but wonder- the accountability post mortem. Do teams look at “Missed tackle” credits for example, and look for where the real ‘credit’ belongs eg. tackler, kicker, passer to kicker, tactic (coach)?
      Dont know where such an analysis would end though,eg “well actually Chek it’s probably your fault for selecting me”

      • Columnist

        February 1st 2017 @ 6:11am
        Nicholas Bishop said | February 1st 2017 @ 6:11am | ! Report

        Much would depend on the time-frame for the prep for the next game Ken…

        A 6-day turnaround would not allow much time for deeper analysis, so maybe they would be more reliant on the stats pure and simple.

        Some coaches in the NFL are well-known killers when they are given that larger time-frame to do their prep work – Bill Belichick, Andy Reid and Tom Landry (in a previous generation) spring to mind. In our world the AB’s are particularly good in my experience at reversing things that did not go well for them in a previous game – witness their driving lineout D in the second game against Ireland.

        • February 1st 2017 @ 6:23pm
          Fin said | February 1st 2017 @ 6:23pm | ! Report

          Bill Belichick – He will have had 2 weeks to prepare for the Superbowl. Could that give him and his team the edge next week?

          • Columnist

            February 1st 2017 @ 6:28pm
            Nicholas Bishop said | February 1st 2017 @ 6:28pm | ! Report

            If I had to put money on anyone, it would be him Fin..

            • February 1st 2017 @ 6:37pm
              Fin said | February 1st 2017 @ 6:37pm | ! Report

              I just returned from a holiday in the US on the weekend. From what I could gather most pundits over there beileive it will be New England in a tight one – Great Defence, Great Coach, Great team culture. Falcons on the otherhand have a great offence so an entralling matchup looms. But it’s hard to go past that cliche in sport – Offence wins matches, Defence wins titles.

              • Columnist

                February 1st 2017 @ 6:56pm
                Nicholas Bishop said | February 1st 2017 @ 6:56pm | ! Report

                Yes it could be a very exciting game – contrast in styles. But NE have the experience in these kind of contests.

      • February 1st 2017 @ 12:58pm
        Celtic334 said | February 1st 2017 @ 12:58pm | ! Report

        “makes a mockery of so many stat based armchair arguments”

        Spot on!

    • February 1st 2017 @ 6:37am
      Ken Catchpole's Other Leg said | February 1st 2017 @ 6:37am | ! Report

      Thanks Nicholas,
      Understood re time frames but I am wondering about ‘range’- whether the range of accountability would extend to tactics (and coaching) in an open team forum.

      • Columnist

        February 1st 2017 @ 7:23am
        Nicholas Bishop said | February 1st 2017 @ 7:23am | ! Report

        I would think so, at least in an enlightened coaching group! Pretty sure the coaches own up to ideas that didn’t work out in the AB culture in the post-game review…

    • February 1st 2017 @ 7:50am
      Ailanthus said | February 1st 2017 @ 7:50am | ! Report

      • Columnist

        February 1st 2017 @ 8:31am
        Nicholas Bishop said | February 1st 2017 @ 8:31am | ! Report

        The ghost of Geoffrey from the other side (the right hand column!)…

        • Columnist

          February 1st 2017 @ 9:04am
          Geoff Parkes said | February 1st 2017 @ 9:04am | ! Report

          Yes, slight technology malfunction there…. 🙂

          • Roar Guru

            February 1st 2017 @ 9:14am
            Machooka said | February 1st 2017 @ 9:14am | ! Report

            … and thankfully it was a ‘technology malfunction’ as opposed to a ‘dress malfunction’ eh 🙂

            • Columnist

              February 1st 2017 @ 9:16am
              Nicholas Bishop said | February 1st 2017 @ 9:16am | ! Report

              With his current photo, Geoffrey could offer a slick alternative to Trump in the US Chook!

              • Columnist

                February 1st 2017 @ 9:42am
                Geoff Parkes said | February 1st 2017 @ 9:42am | ! Report

                He’s too PC/left wing for me. I’d have that Mexican wall built already.

              • Columnist

                February 1st 2017 @ 9:54am
                Nicholas Bishop said | February 1st 2017 @ 9:54am | ! Report

                Ooh – like the farming theme Geoff…

    • Columnist

      February 1st 2017 @ 7:54am
      Geoff Parkes said | February 1st 2017 @ 7:54am | ! Report

      Thanks Nick, enlightening as always!

      It’s interesting to me how defensive patterns come in and out of vogue; it’s almost like some new head coaches and defensive coaches need to change things just for the sake of making their own mark.

      South Africa always seem like a far more dangerous side to me when they play the fast rushing outside to in, backline defence. It has risks if a step or short pass at the line gets an attacker in behind it, but they trade off the attacking side getting on the outside of it by forcing them to do it so deeply that the overlap is usually covered. And how many tries did they get by the attack panicking under pressure and Habana et al picking off an intercept?

      It seems that as Meyer flirted with a more rounded attacking game and then Coetzee tried to establish himself they have lost that definable defensive style, and become far less effective as a result. There’s more to their decline than that of course, but whatever their current defensive strategy is, it doesn’t appear to be effective.

      • Columnist

        February 1st 2017 @ 8:36am
        Nicholas Bishop said | February 1st 2017 @ 8:36am | ! Report

        South Africa were the first top nation to introduce a heavy blitz/press defence when Les Kiss worked for them back around 2001-2002 (they tended to rush off 10)…

        They lost a lot defensively with Jean de Villiers’ retirement – he was a key to their plans – and since Coetzee took over, the D has been a shambles. They should have gotten hold of the guy now at Munster – Jacques Nienaber.

        • Columnist

          February 1st 2017 @ 9:10am
          Geoff Parkes said | February 1st 2017 @ 9:10am | ! Report

          That’s interesting how that style of defence is underpinned by the 10 and JDV yet what stands out to viewers the most is the winger rushing in to crowd the outside shoulder of the attacking centre.

          • Columnist

            February 1st 2017 @ 9:19am
            Nicholas Bishop said | February 1st 2017 @ 9:19am | ! Report

            Nah it started in its prototype with Jaco van der Westhuizen blitzing out of the 10 channel, the outside blitzes came a bit later on…

            Habana has always had that electric ability to break really late on the ball, but I suspect they let him do it on a ‘spot’ basis (in other words it was his decision). Likewise Tommy Bowe with Ireland, who didn’t tend to blitz on the opposite side of the D to him.

        • Roar Rookie

          February 1st 2017 @ 1:42pm
          Comrade Bear said | February 1st 2017 @ 1:42pm | ! Report

          He’s been doing very well for a long time – Les Kiss – good bloke aswell.

          • Columnist

            February 1st 2017 @ 5:22pm
            Nicholas Bishop said | February 1st 2017 @ 5:22pm | ! Report

            Yep, and prob will get Ulster to the Pro 12 playoffs this year too.

    , , , ,