Do league tables under the new conference system in Super Rugby accurately rank teams from strongest to weakest? Under the new system, the difficulty of a team’s schedule is influenced by the conference that it is in and the draw for intra-conference matches.
Comparing strength measures that account for schedule difficulty can be used to objectively evaluate the impact of the conference system on league standings.
One approach is to determine rating values for each team that maximise prediction accuracy when rating values are used to predict match results.
The procedure generates ratings that sum to zero, so a rating greater than zero indicates a stronger than average team and a rating less than zero a weaker than average team. Differences between ratings for any two teams provides an estimate of the score difference if the teams played at a neutral venue.
Ratings accounting for schedule difficulty are presented in the table below. The Crusaders, with 12.5 rating points, were the strongest followed by the Stormers (7.4) and the Reds (7.0).
The main driver of ranking changes relative to the league table is that, as discussed below, the New Zealand and South African conferences are stronger than the Australian conference. At the other end of the scale, the weakest teams were the Lions ( 6.1), the Brumbies (-8.3), and the Rebels (-17.5).
The Crusaders did not play either the Lions or the Rebels, further highlighting the difficulty of the Crusaders schedule.
Super Rugby rankings accounting for schedule difficultly:
Rank Team Rating Rank Team Rating
1 Crusaders 12.5
2 Stormers 7.4
3 Reds 7.0
4 Waratahs 5.5
5 Blues 4.6
6 Sharks 4.2
7 Bulls 3.4
8 Cheetahs 1.9
9 Chiefs -0.2
10 Highlanders -2.7
11 Force -5.8
12 Hurricanes -5.9
13 Lions -6.1
14 Brumbies -8.3
15 Rebels -17.5
Conferences can be compared by averaging ratings across teams in each conference. The South African conference (2.1 rating points) was marginal stronger than the New Zealand conference (1.7).
The numbers also confirm the common opinion that the Australian conference (-3.8) was by far by the weakest.
On average, a New Zealand or South African team is predicted to beat an Australian team by about six points. While these results confirm Jamie Joseph’s comments concerning the Australian conference, variations from the average are possible and predictions are sometimes incorrect (as most schoolboys know).
Although league standings under the conference system can result in weaker teams outranking stronger teams, this does not necessarily indicate that a conference system is undesirable.
Supporters enjoy seeing underdogs punching above their weight and “luck of the draw” variations under the conference system enhance this possibility.
Also, ratings that account for schedule difficulty can rank a team with less wins above a team with more wins, which may cause greater controversy than inequities resulting from the conference system.
However, if one conference persistently offers an easier road to the playoffs, this matter may need to be addressed.
Niven Winchester is a Environmental Economist at the Massachusetts Institute of Technology and a Senior Lecturer at the University of Otago. He has an unhealthy interest in sports gambling.
Sandy B
Guest
how
zhenry
Guest
Sandy B: It will change, get over it.
Niven
Guest
Okay, here is my attempt at a simple example. Unfortunately some maths is required. Suppose there are only two teams (A and B) and two games in our sample. A is at home in the first game and A wins by 10 points, and B is at home in the second game and A wins by 2 points. Defining net score as (points scored by the home team) - (points scored by the away team), the net score is 10 in match 1 and -2 in match 2. The predicted net score (PNSC) for each match is: Match 1 (A at home): PNSC = (home advantage) + (A's rating) - (B's rating) Match 2 (B at home): PNSC = (home advantage) + (B's rating) - (A's rating) Our task is to select values for (home advantage), (A's rating) and (B's rating) that give the best predictions. To keep keep things simple, suppose we know home advantage is equal to 3 points, so we only have to determine ratings for the two teams. One way to define "best predictions" is to minimise the sum of squared prediction errors. As a first try, we could say that A's rating is equal to 1 and B's rating is equal to -1. This would result in the following predicted net scores: Match 1(A at home): PNSC = 3 + 1 - (-1) = 5 (i.e., A is predicted to win by 5 points) Match 2 (B at home): PNSC = 3 + (-1) - 1 = 1 (i.e., B is predicted to win by 1 point) These predictions result in prediction errors of 10 - 5 = 5 in match 1 and -2 - 1 = -3 in match 2 [Recall that the prediction error is equal to the actual net score minus the predicted net score]. The sum of squared prediction errors is thus 5^2 + (-3)^2 = 34. To minimise the sum of squared prediction errors, we could repeat the calculations with different rating values and select the combination that yields the lowest sum of squared errors (e.g., we know that we can improve the predictions by increases A's rating). This is tedious and time consuming, especially with a large number of teams and matches, so it much easier to use a computer algorithm. In our example, setting A's rating equal to 3 and B's rating equal to -3 minimises the sum of squared prediction errors (at 2). My friends always start laughing when I launch into one of my "simple" examples, so I hope this helps.
Photon
Guest
Fan of the Tahs All I see is that you don't understand statistics, you cannot remove a variable that fundementally affects the outcome of a study, then expect to be able to utilse data that includes the said variable in order to draw conclusions about future outcomes, basically for all we know the Aussie conference might suck till Kingdom come because the Super 15 essentially a new competition!!
Fan of Tahs
Guest
I understand that it matters this year, but what I'm saying is that there are great fluctuations between an individual team's overall performance from year to year. Comparing all the teams in each conference, the Aussie one is the weakest, but theoretically the Kiwi conference would have been the weakest last season if the same conference system was in use. For much of the history of Super rugby from 1996 until perhaps 2007 it was the Saffa teams which were by far the worse in the competition. Next year the Aussie conference may be the weakest again, but history shows that it could very well be the top South African or top New Zealand teams which seem to get the easy ride as the year to year form of their peers' fluctuates. See what I mean?
Sandy B
Guest
Enough of this gibberish. Fact one - pretty well all the alternative structures suggested above give the same 6 participants in the finals - albeit in a slightly different order. So the teams that should be there are, and if they are good enough they should win the final they are in and move forward. Prove it on the feild this weekend. Fact two - the conference system was very successful for the Super Rugby organisation with good crowds for the local derbies - they are not going to change it - get over it
Altus
Guest
Yes, but this is the first year where the relative quailty of the conference matter. Until last year every team played every other team. It made no difference whether the South African or the Australian teams were the worst. This year with the imbalanced schedule, it matters a great deal.
Chris
Guest
Super 15 table if IRB method for international rankings is here: http://www.lassen.co.nz/s14tab.php#hrh Certainly both methods are more accurate than a table - but far too confusing to be used by the average fan!
Ooaahh
Guest
You could argue Aussie conference was quiet tough and not weak based on the injuries sustained. Conversely it is also easy to state this fact in reverse. The Aussie conference is weakest, just look at the injuries picked up cos they're not battle hardened playing soft conference games all the time. Either way surely now it is really a case of who wins the prize to prove the best team.
Jerry
Guest
You do know that Crusaders draw was a cancelled match, right? Seems a bit harsh.
Jerry
Guest
"They dont whine.." Sure they don't...
kovana
Guest
"The trip also disadvantages teams traveling to South Africa. " Really? Every Oz and NZ team only plays 2 MATCHES in SA... Compare to the saffas who have to spend a whole MONTH on the road on the other side of the globe? Again.. They dont whine.. They just continue to play whats in front of them... and its paying dividends.
Nicksa
Guest
IMHO U cant take away from the tahs and reds, they did what they needed to do to qualify. Yes, i do believe the aus conference is the weakest and obviously the tahs and reds benefited from it. The thing is in the next couple of years it will change were sa or nz conference will be the weakest and other teams will get the advantage. That is were i think there is a huge problem!! U will always have good teams in each conference but why give them a HUGE advantage because of the competition format?? The old format also gave a lil advantage to certain teams due to the fixtures, i mean certain teams u dont want to play away from home but the new format will litterally get certain teams in the finals. With all that said, i must admit this was one of the most exciting super rugby tournaments held.
Photon
Guest
Last year was fundementally different, there were no conferences last year and some of the Sotuh African sides had 5 week tours, if we had conferences last year the league would probably have looked totally different, for one, the Reds would probably have gotten a home-semi :-)
Fan of Tahs
Guest
This year the Aussie conference has been the weakest. However, the strengths and weaknesses of each conference varies considerably from season to season. Take last year for example. The bottom four last year was made up of the Lions (14th, 0 Ws), Force (13th, 4 Ws), Highlanders (12th, 3 Ws) and Chiefs (11th, 4 Ws and 1 D). The top six was made up of the Bulls (1st), then the Stormers, Waratahs, Crusaders, Reds, and Brumbies. Although I can't present the same sort of statistics that Niven has used, according to the final placings after every team had played each other once the NZ conference seems to be the weakest by far with two teams in the bottom four and only one team in the top six. It seems that some Kiwis have very short memories, not to name any names but in particular I'm referring to a guy with the initials Jamie.Joseph.
The Bush
Roar Guru
Niven, Unfortunately I chose a professional path far, far away from maths. Please explain what the "(home advantage)", "(home rating)" and "(away rating)" variables in your equation are? Perhaps you could put up an example and run through it? You say "I then chose ratings to minimise the sum of squared prediction errors across matches" - how did you choose these ratings? What were your choices based on? The net scores, the subtraction of the away teams score from the home teams score is fairly self explanatory, but the rest is not.
Ralph
Guest
" where they play doesnt matter really" Except I think I am right in suggesting the chances of winning away are less than at home.
Ralph
Guest
It's a bit rich to demand respect whilst showing none yourself to the writer. Instead of telling other people how things are sitting with themselves and putting words in others mouths you should think about how much respect that kind of behaviour deserves.
Wall-Nut
Guest
Kovana You said it! It's like reading comments from so called Einstiens. It's not sitting well with Kiwis and Saffas that the Reds are on top. It's a lack of respect that Aussies are good at rugby as well. -- Comment left via The Roar's iPhone app. Download The Roar's iPhone App in the App Store here.
Winston
Guest
The trip also disadvantages teams traveling to South Africa. There are 3 Aussie teams in the bottom four. Its a weaker conference. I do think the Reds deserve to be number 1 though. Its not their fault that they didn't play any good NZ teams in NZ and that thier conference is weak. They did beat the Stormers in SA. They are a great team but I reckon the Crusaders with Maitland, Kahn and SBW back will be very hard to beat even in Brisbane.