Pace/speed figure comparison

[andicap]

If you wanted to test two pace/speed figure products side by side, how would you go about it?

[Tom]

The way you use them in real life.
Handicap the same races using both sets.
Do you get totally different results?
Does one help you evaluate class, form better?

Do you play turf sprints? Be sure to include somecof those races.
Classic graded stakes? AW tracks?

[headhawg]

To find out anything useful, you would need lots of data. Comparing a week's or month's worth of figures tells you nothing. The more data available to analyze also gives the option to break the comparisons down into subcategories such as sprint vs route, turf vs dirt, or even by track/circuit. But as you add more filters, the sample sizes get smaller which makes the results potentially more volatile.

My feeling is that you won't find any (statistically) significant differences in the figures assuming that the products come from reputable sources, and not from Joe SpeedFig at tookyourmoney.com. So I would just suggest that you use the figs from the company with the best customer service or the figs that you feel do the best job. Don't discount the psychological aspect of handicapping. I never was a better handicapper than when CJ had his PaceFigures site. Were his numbers objectively the best? I have no idea, but I felt that they were so I stopped worrying about the numbers and could focus on other aspects of handicapping.

Good luck in your research, andicap.

[MJC922]

The best way to test accuracy of any figures like that is simply the horse with best figure in the most recent race. There's no way a less accurate figure is going to have a higher win rate based upon that metric over say 1200-1500 races, highly unlikely IMO but the more races the better. Best fig in last race is an extremely rigorous test of a number because it tests both distance and surface adjustments that many figure makers struggle with because they've failed to scale them properly.

The best way to test the accuracy of the early pace figure is what percentage of the horses with the best early pace figure have the lead after the first quarter or second quarter, whichever one of those the early figure is measuring. If you test my numbers which are very easy to test without spending a penny because they're free to download from Trackmaster the day after the races then you're going to want to test the early pace figure by what percentage of those ranked first on early pace actually made the lead at the first quarter.

I applaud anyone who takes the time to test these things, very few people put forth the effort so I do thank you for that. ROI can be tracked as well but is not a measure of accuracy. One or two bombs can skew ROI even in fairly large samples. If you have two sets of figures, one hits 22% on best figure last race but loses 17% over 1500 races and another figure hits 25% wins over 1500 races but loses 19% you'd have to be crazy to choose to go forward with the less accurate 22% win rate figures. It's like choosing a less accurate gun for hunting purposes.

It would be great if someone would test every set of numbers over a large sample. Ragozin, Thorograph, Beyer, Equibase, Timeform, and of course my own On-Target. I don't know who else is out there but feel free to step forward, Colt's Neck maybe? Let's get everyone in the pool so we can settle this. I'll pay someone to test it and make it public.

A problem I've seen in the past is how ties for top rank are handled in a test. Numbers are scaled differently such that some have many more ties for top ranking. If you're counting ties as wins that can skew things quite a bit. An extreme example we can roll dice to rate performances as a 1, 2 or a 3 and end up getting 33% wins if ties count as wins right?

[classhandicapper]

The major problem with testing figures is getting the data so you don’t have to do it manually. It’s a huge job manually. I test top figure last race.

[Dave Schwartz]

I test our own figures against themselves with the Speed Reliability Index (SRI).

Back in 1973 Dr. Fred Davis wrote his tiny paper, Percentages & Probabilities. In that publication, he stated that in sprint races, comparing each horse's Best-of-Last-2 speed ratings, resulted 62% of all winners coming from the top 3.

In the mid-80s, I hand-entered every such race in California for an entire year and found that it did, in fact, produce precisely 62% of all winners.

Eventually, when I got a database, I discovered the same phenomenon in route races if I used Best-2-of-Last-3.

This formed the basis for the SRI and led to a scoring system.

I called 62% "100" and scaled the number accordingly.

One standard deviation was +/- 9 points.

The first couple of years, I found that my pars produced - on average - 102.

But now that I had a benchmark to work against, I could take steps to improve my par-making techniques.

Now, a decade later, the average is almost 108!

With large samples such as those I have (currently 488,000 races), it is easy to see how it is working on a per-track basis.

BTW, an interesting point of this exercise is that the top 3 = 62% only changes by 1-2% as the field size gets larger or smaller.

The exception to that is when the field size is below 6 horses, at which point it increases substantially.

FWIW.

[andicap]

Originally Posted by Dave Schwartz

I test our own figures against themselves with the Speed Reliability Index (SRI).
.

Dave,
I presume that is best 2 out of last 3 regardless of distance, surface, recency?

[Dave Schwartz]

Originally Posted by andicap

Dave,
I presume that is best 2 out of last 3 regardless of distance, surface, recency?

You assume correctly.

BTW, I should have mentioned I was using Quirin Pars back in the 1980s.

[vegasone]

I have the capability of comparing systems or ratings but haven’t done it in a long time. I only tested for the top two, and just compared two different ratings at the time and found only a few percentage difference between them. I was just testing for ROI since picking winners isn’t the optimum result for ROI, at least for me.
My feeling is that if you stick with one system and speed numbers if you use them, then once you get comfortable with them, they would be just as good as any other. More of a practice and record keeping process.

Getting a good ROI with all the short fields is much more difficult nowadays of course. I keep putting off rerunning some tests to see how things are since not concentrating on horses at the moment.

[ranchwest]

Originally Posted by andicap

If you wanted to test two pace/speed figure products side by side, how would you go about it?

I would suggest testing:

1) 2 of 3
2) most recent 1
3) pace line
4) CSR ( last 4 with preference to the races in sequence)

In considering the pace line, does the software automatically select a pace line and how well does it do in selecting the pace line? How well does the pace line do in comparison with the other methods?

[MJC922]

Maybe I'm misunderstanding the question. We have two sets of numbers, is one more accurate than the other? Top fig most recent race, it's really as simple as that... and yes a 3% difference is actually very significant for this test assuming the sample size is large. It's actually giant tbh. Once that's out of the way, how to obtain the highest win rate from this more accurate set of numbers, that's another question. Now it steps beyond just the most recent race and into which race or recent races (or even back races) need to be considered to obtain a higher win rate. That's the optimization phase.

[ranchwest]

Originally Posted by MJC922

Maybe I'm misunderstanding the question. We have two sets of numbers, is one more accurate than the other? Top fig most recent race, it's really as simple as that... and yes a 3% difference is actually very significant for this test assuming the sample size is large. It's actually giant tbh. Once that's out of the way, how to obtain the highest win rate from this more accurate set of numbers, that's another question. Now it steps beyond just the most recent race and into which race or recent races (or even back races) need to be considered to obtain a higher win rate. That's the optimization phase.

Good points.

[classhandicapper]

Originally Posted by MJC922

Maybe I'm misunderstanding the question. We have two sets of numbers, is one more accurate than the other? Top fig most recent race, it's really as simple as that... and yes a 3% difference is actually very significant for this test assuming the sample size is large. It's actually giant tbh. Once that's out of the way, how to obtain the highest win rate from this more accurate set of numbers, that's another question. Now it steps beyond just the most recent race and into which race or recent races (or even back races) need to be considered to obtain a higher win rate. That's the optimization phase.

Exactly my thinking.