Welcome to Day One of the new TRC Global Horse Rankings, which is a different - and we believe fairer and more accurate - way of ranking racehorses than the traditional method in use elsewhere.
Under the usual system, handicappers assign ratings for every run a horse makes, then use the best one to establish its place in an overall classification. This has always had the potential to wildly exaggerate some runners’ position in the hierarchy and under-appreciate where others should be ranked. The annual Longines World’s Best Racehorse Rankings, for instance, could more accurately be described as the ‘World’s Best Racehorse Performances’ because they use one number per horse to infer a hierarchy - with no reference to other performances which may support the assessment.
Here James Willoughby explains the theories and mathematics behind our new Horse Rankings:
Classifying racehorses according to their best effort alone has to be wrong. It could only be valid if ratings were always calculated with no error term and races contained no random component, such as the pace or the run of the race. Both those concerns are real, so an alternative approach is justified.
The racing public already think this way. It’s obvious to them that it is better to weigh horses against one another by considering a horse’s array of performances. People have a clear preference for horses who have done it time and again over one-hit wonders like the Japanese ‘world champion’ Just A Way. (Just A Way never recorded a rating remotely close to his 2014 Dubai Turf mark before or after that race - indeed he was soundly beaten in his final three runs - yet he still topped the classifications at the end of the year because of that one outlying run back in March.)
TRC Global Rankings approach the problem of which horse should be preferred over another by answering the following question: whose array of ratings represents a greater achievement? Although we don't get too bogged down in traditional collateral form techniques, we might prefer a horse who has run to 128 twice over one who has run to 130 once, for example, or we might rank a horse whose best efforts are 128, 126 over another who has earned 130, 116.
---------------------------------
Click here to see the first TRC Global Horse Rankings
---------------------------------
Some handicappers seem to think their figures are set in stone. But the truth is that a horse’s rating is merely an estimate. In technical terms, a horse’s performance figure should be that which is at the notional centre of the probability distribution of all reasonable estimates of the horse’s true merit on the day.
Finding the solution to the handicapping puzzle across all horses and all races in the data is called a ‘maximum likelihood estimate’ in statistics; it is the set of rating with the highest joint probability of representing the true, unknown merit of all performances, given the information available.
Most handicappers who serve in an official capacity around the world are highly capable; the talent of the humans is not the problem. The problem likes in the existing methods that have general acceptance among the humans. The truth is that these methods simply would not stand up to scrutiny outside the friendly confines of horse racing: when asked to explain the axioms of handicapping, for example, may handicappers tend to fall back on pseudoscientific reasoning outside the mathematical realm.
The task of handicapping is to find this maximum likelihood solution to the set of outcomes described by the finishing order of races: correspondingly, handicappers should be awarding the set of ratings that best explains the set of results in their database in a principled fashion. This is achieved either when some accuracy measure governed by future results is used, or when the total variance of ratings across all horses is the lowest.
When a computer approaches this task on horse racing data, it finds there are thousands of permutations for a set of ratings close to the most optimal solution. Yet, many of these are very different from one another. In other words, races can be assigned quite different numbers and still these numbers form a cogent ranking and achieve similar acccuracy ‘out of sample’ - i.e. when applied to predict future results.
So, a corollary of this is that several ‘solutions’ exist to the rating of a race. But, this does not qualify handicapping as ‘a matter of opinion’ as some racing writers insist on calling it - it only seems like a matter of opinion if you don't know what handicapping itself is trying to achieve.
No, handicapping is no more and no less than mathematical optimization - the minimising of some accuracy measure by changing a set of parameters, in this case the merit ratings of races and horses. It does not require arcane understanding given to just a few, and it isn’t (at least to any significant degree) ‘a matter of opinion’.
One problem that arises in horseracing data is the the connections between groups of horses are relatively sparse compared with those between football teams, for example. If there are 20 teams in a league and they play a home-and-away round robin, there are 20 x 19 = 380 games – the number of games compared to different competitors is large and so the estimates of team merit are relatively easy to make.
Refined assessments
By contrast, there are tens of thousands of horses and far fewer races in horse racing around the world. Worse still, they tend to compete in clusters because races are restricted by class, age and sex. Even though handicappers can use other evidence to refine their assessment, such as race standards, speed figures or sectionals, not everyone agrees as to how to make safe inferences from this data, compared to the traditional use of the distances between horses at the finish and the weights carried and ages of the horses.
TRC Global Rankings starts with Racing Post Ratings (RPR), the collateral form system of the Racing Post as the ground zero for learning the relative merits of groups of horses across the world. This is before we let the computer refine those assessments, approaching handicapping in the manner described above as an optimisation exercise. We might award much larger differences in ratings than the distances between runners descrbe because its a justifiable precept of assessing races that the order of finish is the more important variable. To our system, a horse is best represented by the mean and variance of all the possible ratings that make sense, given the set of results of Group/Graded races around the world. Notice we do not use information from outside these races, which would be an advantage in accuracy but would depart from our ethos as a ranking system, rather than a prediction system.
Fortunately for conventional handicapping, ratings in one race do depend on ratings in others. So, a horse’s best rating does find some support from its other performances. But some races – most notably shock wins or wide-margin ones – cause handicappers to assign ratings to horses that tend to break away from its others. It’s here that the trouble starts because some ratings are assigned with a much wider variance than others, but this is never admitted.
Not that an elegant solution for rating or ranking exists.
A number of factors need to be taken into account when performing the task in any sports. Most notably, there is the problem of how highly to weight recent evidence over that further back in the past. In horse racing, a sequence of a horse’s performances shows much greater serial correlation than in football or basketball, for example, because the ability of a horse who is growing, developing and learning to race changes a lot more quickly than that of a squad of players whose outturn is a combination of one another.
So, it is all that a handicapper can do to state the method and assumptions of their model. We have already done that extensively at TRC Global Rankings. You can read a primer here and an extended explanation here. Our new ranking of horses follows exactly the same ethos as the existing ones for Sires, Jockeys, Owners and Trainers.
Interesting alternative
The only difference are the parameters – such as that which controls the weighting of recency – which are learned from the data by the system tuning its prediction methods on results which are already known.
Starting today, we will produce a ranking of the Top 500 horses in the world, according to our belief that horses should not be ranked solely based on their best effort. We have also generated past rankings going back to the start of the TRC Global Rankings window in 2014.
We hope you will find this an interesting alternative to the one-number ranking systems found elsewhere.
As with the other categories in the TRC Global Rankings, they will be updated every week, although there is one big difference – whereas all individuals in the Sires, Jockeys, Owners and Trainers standings who have had runners over the previous three years are eligible, only Horses that have run in the past 200 days can be included.
Tomorrow: Seven years of TRC Global Rankings – a look at how the best horses have compared since 2014