by RBR AdminThis two part piece may prove to be somewhat controversial. A. Lennart Julin is a long time key observer of athletics. We have met and spoke at many of the European and IAAF Championship meets. A. Lennart Julin was very concerned about how the new IAAF World Rankings would work for the sport. This is part 1.
Khalifa Stadium, Doha, Qatar, photo by Doha LOC
WHY THE PROPOSED IAAF WORLD RANKINGS NEVER CAN BE A FAIR TOOL FOR DETERMINING CHAMPIONSHIP ENTRIES
PART 1: THE THEORY
Last year the IAAF (international Association of Athletics Federations) announced that they – in co-operation with Elite Ltd – would introduce an official World Rankings system with the main objective that it would from 2019 onwards replace the the traditional top lists as the tool used to determine the qualification for the major championships.
This was announced – and commented upon in the media – as a major innovation but actually IAAF had the very same ranking system in operation between 2001 and 2006. But that ranking never caught on, neither within the sport itself, nor in the media (and thus not among the public). So rather discretely the IAAF removed its patronage and since 2007 – i.e. for over a decade – these rankings have been maintained by Elite Ltd themselves at their own All-athletics website.
Given this history it is slightly bewildering that the very same concept is about to be relaunched, although now with the focus on it being a preferred tool for determining who will be accepted to participate in global championships.
Comparisons have been drawn to other sports like tennis, golf and skiing. However, those sports are characterised by having only relative results. Thus they have no option but to create some kind of rankings based upon the individual placings in combination with the status of the competition.
Athletics, however, indeed has absolute results and the thickness of our rule book illustrates to how great lengths we have gone to standardise the conditions to make performances from all meets directly comparable in a meaningful way. If our absolute results are good enough for our highly prestigeous records it is somewhat paradoxical that they would be considered not good enough to be used for qualification purposes.
But of course we all know that some results are exceptional one-off occurancies – Beamon’s 8.90 Long Jump in Mexico 1968 the most well-known example – not reflecting the “normal” capacity of the athlete. So a more balanced tool for evaluation of the relative strengths of different athletes could indeed be useful.
The problem is to find a tool that is balanced and fair. Athletics with its absolute results creates the impression that it is “just mathematical facts” perfectly suited for being mathematical processing. Using intricate algorithms also lends an unwarranted aura of being “scientific” and “objectitive” to those initiatives.
However, it must be clearly stated that even though the actual calculations are “objective” the choice of algorithms and parameters is always a 100% subjective decision!
And that is not athletics! Since the beginning of our sport the very fundament has been that athletes are compared solely on objective measurements of physical entities (times, distances, heights). No “ifs” and “buts” or “opinions” are taken into consideration – just the plain numbers read from the measuring devices!
So introducing a “decisive” – in this case determining who will be allowed to participate in our top events – ranking that subjectively weighs in other factors is really a step away from the very core of our sport!
Because then you as an athlete might have a superior measured performance but still end up ranked behind due to an intricate mathematical process that takes into consideration various factors over which you as an athlete have no control or even influence!
And this is a fact regardless of how the actual ranking system is constructed! It is inherent to the very idea of taking more than just measured results into consideration!
The proposed ranking system
There are two fundamental differences between traditional top lists and the proposal:
+ Rather than considering just the very best performance of each athlete the rankings are based upon the top-5 performances of the athlete. The rationale being that the general level rather than one-off performances should be decisive.
+ Significant weight is put upon the formal status of the meet and the finishing position. The rationale here being that a performance is “worth more” if it is accomplished in a major meet with quality competition rather than in a local all-comers meet against “nobody”.
We can all in principle subscribe to those two rationales but the key issue is how to realise them in a way that is unequivocally fair. Unfortunately the proposal immediately fails that criteria and the key issue here is the attribution of the bonus points for status of meet and placings.
Much could be said about the proposed categorization of meets and about the levels of bonus points chosen for the various categories – we all know that the devil is always in the details – but they are just inevitable consequences of the key problem! A problem of such a profound nature that it never ever can be solved by tweaking the algorithms and/or the parameters:
There simply are too few participants in the events at all our (arena) meets!!
Because an absolute requisite for creating a reasonably fair ranking system to a given depth you need to have approximately (at least) as many participants in the scoring meets. Why? This simple example illustrates the crucial factor: If you are No 9 in the world but you always compete against the top-8 you will get no bonus points at all! As there are bonuses for placing high in lesser meets the consequence will be a ranking lower than your “true” position as No 9!
As we all know the standard field at an international one-day-meet is 8-12 competitors per event which thus is just a fraction (between one third and one fifth) of what is required for the 32-56 deep rankings needed for the championship qualification process. So although the very best (top-10?) might regularly go head-to-head in the Diamond League you don’t have to go further down to than to places 15-20 in the events to find athletes that never face their “peers” from other geographical areas!
Instead athletes on this level end up competing in separate – mainly national – “bubbles” not because they opted to avoid tough competition but because there is no room for them in the major meets! The situation is further enhanced by lack of economical resources needed to travel far and wide. They are worldclass but still not “professionals” in the sense that they earn sufficient money from their competitions to “live on their sport”.
The contrast to e.g. golf is stark: There were no less than 107 male players winning over 1 million dollars in prize money 2017. How much money do you think the No 107 Javelin Thrower, 400m Hurdler or Triple Jumper in the world earned in 2017?
Talking about golf: In major golf tournaments (as well as in major tennis tournaments) there are usually 100+ participants (e.g. British Open golf 156, Wimbledon tennis 128) which creates the necessary foundation for deep rankings. We have similar numbers of participants in our major international one-day-meets, however, we spread them out over some 15 different events!
Athletics is inherently simply a different kind of sport, as we can’t have 128 or 156 participants going head-to-head in e.g. a Pole Vault competition or in a 400m Hurdles race. Therefore we can’t ever construct a fair ranking of the depth needed based upon the finishing order.
It is simply impossible and this has nothing to do with which algorithms we try to devise – it is due to the inherent competitive format our events demand to be practicable.
In Part 2 (The Reality) this “theoretical” analysis is illustrated by looking at how the proposed ranking system would work on a specific event.
Want bonus content on athletics? Support the sport of athletics! Support RunBlogRun as we cover the sport of athletics nearly 24/7, and get bonus content, please go to: https://www.patreon.com/
