Tweaking Power Ratings and judging results

[chickenhead]

I'm working on creating a power rating, and looking for some thoughts on judging whether changes are improvements of not. I could take it all the way out to an oddsline and judge it on the basis of overlay performance of the contenders, say, but for reasons not worth getting into I don't want to do that right now. I'd rather try to gauge it on the rank and gap level performance of the power rating itself.

Sometimes it is clear (seemingly)...sometimes it is not. For instance, say I make a change and the win rate on the top ranked goes up slightly, the ROI goes down slightly, the win rate and ROI on the 2nd ranked go up more significantly. How much better of worse is that? Tough to say without a hard and fast metric. A lot depends on what I want it to do, ultimately, I know. I want the same thing everyone wants, overlaid contenders to be profitable, or as close to profitable as possible.

I'm not sure there is a metric to use at that level that will necessarily relate all that well to overlay performance, but if anyone has some methods they've used I'd like to hear about them. Maybe I've forgotten something basic that will make sense.

[sjk]

I always thought of a power rating as a means of predicting the speed rating for the upcoming race so I would see how well each proposed power rating correlates with the actual speed rating run.

[chickenhead]

bad form to reply to my own question, but about the only thing I've come up with is to treat the "contenders" as one group. Say based on field size, take all the horses (based on gap win %) who I give a greater than natural odds chance of winning and treating them as a betting group.

So I'd get a single I.V. and ROI for my contenders, and compare different power rating iterations based on that. Could combine the I.V. and ROI together with maybe I.V*(ROI/0.75) or soemthing like that for a single number.

That would be in the ballpark I think.

[chickenhead]

I always thought of a power rating as a means of predicting the speed rating for the upcoming race so I would see how well each proposed power rating correlates with the actual speed rating run.

This is a little different, it's not meant to correlate to a speed rating so there isn't any way to measure accuracy like that.

[singunner]

How big of a sample are you testing against?

[chickenhead]

one year

[BillW]

This is a little different, it's not meant to correlate to a speed rating so there isn't any way to measure accuracy like that.

What is the goal of this rating? A good start would be to define it before formulating a validation method.

BTW answering your own question is only bad if you value sanity as a defining attribute in your life. :lol:

[chickenhead]

What is the goal of this rating? A good start would be to define it before formulating a validation method.


The observed win % based on any particular relative strength (gap) is the basis, ultimately, for an odds line. So it differs a bit from predicting a speed figure, because as a power rating it doesn't predict anything. It just says this horse is this arbitrary "amount" better or worse than the rest of these horses. Based on past observance, that means he should/could/might :( win x% of the time.

If it's *good enough* (unlikely of course) I can just bet all the overlays it uncovers and retire to some tropical island. Otherwise, maybe it can be at least useful for contender selection, and I keep my day job (for now).

Breaking it all the way out to actually look at overlay performance on each iteration would be really cumbersome and time consuming...I'm looking for an 80/20 solution (my favorite kind).


BTW answering your own question is only bad if you value sanity as a defining attribute in your life. :lol:

I don't think that question is in doubt any longer :lol:

[singunner]

I just read your blog post. How many 3yo, non-maiden, dirt sprints are there in the year you tested?

Also, for your initial question, I have a little personal experience. I don't know how similar our systems are, but I was testing something about a year ago that would act similarly to what you've stated. If I went with horses that had a higher percentage chance of winning, the ROI dropped slightly. I figured that I'd reached the intersect of the public's predictions and my own. Meaning if I analyzed my highest win% horses, their ROI would be approaching the public's favorites (going down in ROI).

A redesign or two later, I found a better way. Now the ROI increases, but increases at a slower rate the higher the win%. My solution was found in the way I determined probable win%, if that helps you at all.

[chickenhead]

I just read your blog post. How many 3yo, non-maiden, dirt sprints are there in the year you tested?

14,901


Also, for your initial question, I have a little personal experience. I don't know how similar our systems are, but I was testing something about a year ago that would act similarly to what you've stated. If I went with horses that had a higher percentage chance of winning, the ROI dropped slightly. I figured that I'd reached the intersect of the public's predictions and my own. Meaning if I analyzed my highest win% horses, their ROI would be approaching the public's favorites (going down in ROI).

A redesign or two later, I found a better way. Now the ROI increases, but increases at a slower rate the higher the win%. My solution was found in the way I determined probable win%, if that helps you at all.

I'm not seeing that yet. I see that in some of the individual factors, i.e. too much of a good thing is a bad thing. I assume that is for two reasons, one, it's becomes overly obvious, and two, it means something out of the ordinary is going on (like a Hollywood MSW placing, vanned off, three year layoff, shows up in a Prairie Meadows $8K MCL). At least right now, my rating has a nice upward trending ROI curve that peaks out with the highest gap.

[loveracing]

I look at a horse in just 2 easy ways, is it going to run better or worse than its last race. Very rarely will it ever run the same exact race. Then try and figure fairly how many points it will increase or decrease.