Proper baseline for system performance

[podonne]

Greetings,

I've been tossing this question around for a bit about how to properly baseline a prospective system\angle\tout\etc. By that I mean something to compare against to tell if the system is performing "well".

In stock markets I would compare a prospective strategy against a "buy and hold" strategy, or against a standard nominal interest rate.

I'm wondering if such a "default strategy" exists in horse racing. For example, you might compare all prospective strategies against a "bet the favorite to show" strategy. Or you might compare against a standard track takeout (- 15%).

Ideally a commonly known, meaningful (as in not random), applies to most races, and performing better than the track takeout (but not profitable strategy).

Thoughts?
Podonne

[GameTheory]

Greetings,

I've been tossing this question around for a bit about how to properly baseline a prospective system\angle\tout\etc. By that I mean something to compare against to tell if the system is performing "well".

In stock markets I would compare a prospective strategy against a "buy and hold" strategy, or against a standard nominal interest rate.

I'm wondering if such a "default strategy" exists in horse racing. For example, you might compare all prospective strategies against a "bet the favorite to show" strategy. Or you might compare against a standard track takeout (- 15%).

Ideally a commonly known, meaningful (as in not random), applies to most races, and performing better than the track takeout (but not profitable strategy).

Thoughts?
PodonneThe takeout rate is a meaningful and real benchmark, and requires no calculation or simulation of a default system. Betting the favorite to win also is a good one -- a bit more work. Maybe people mistakenly think that betting randomly one will approach losses equal to the takeout rate, but that is untrue -- you lose MUCH more than the takeout rate betting randomly.

[HUSKER55]

Just a thought, you could buy the data from HDW and pit the system against a very large base and caluculate the results.

Use to be about $135 a month for 40 or so tracks and if memory serves you could go back 3 months, maybe more. that is a lot of data.

GL :)

[podonne]

The takeout rate is a meaningful and real benchmark, and requires no calculation or simulation of a default system. Betting the favorite to win also is a good one -- a bit more work. Maybe people mistakenly think that betting randomly one will approach losses equal to the takeout rate, but that is untrue -- you lose MUCH more than the takeout rate betting randomly.

Can you expand on the notion that betting randomly will result in losses of more than the track takeout? Have you seen this in practice?

If you bet on every horse in every race, your losses should precisely equal the track takeout, so it stands to reason that randomly choosing any horse in any race with equal probability will also result in a loss rate equal to the track takeout.

Even so, that might be the most appropriate benchmark. A system is good if it can achieve a better ROI than the weighted takeout rate at the tracks it is betting at (proportional to the number of races played at each track).

What's the general assumption for takeouts from the Win\Place\Show pools? 15%?

[Turkoman]

Can you expand on the notion that betting randomly will result in losses of more than the track takeout? Have you seen this in practice?

If you bet on every horse in every race, your losses should precisely equal the track takeout, so it stands to reason that randomly choosing any horse in any race with equal probability will also result in a loss rate equal to the track takeout.

Even so, that might be the most appropriate benchmark. A system is good if it can achieve a better ROI than the weighted takeout rate at the tracks it is betting at (proportional to the number of races played at each track).

What's the general assumption for takeouts from the Win\Place\Show pools? 15%?

That is simply not true. The reality is that the higher the odds, the lower the return on investment. In the book The Power Of Early Speed, author Steve Klein provides a huge sample of races, proving this point. To give you an idea, the biggest favorites returned as much as $1.74 for every $2.00 wager. Horses in the 10-10.99 to 1 range returned $1.65, and the ones in the 20-29.99 to 1 range returned as little as $1.52. I hope this information helps.

Turkoman

[Dave Schwartz]

Turkoman,

I am assuming that Mr. Klein's references were to horses that qualified as "early speed' because otherwise, the numbers are much, much worse.

As an example, horses at like 30/1 and above lose about 45% of each dollar wagered.

Regards,
Dave Schwartz

[GameTheory]

Can you expand on the notion that betting randomly will result in losses of more than the track takeout? Have you seen this in practice?Sure. It is easy to demonstrate. Just bet randomly.

If you bet on every horse in every race, your losses should precisely equal the track takeout, ...No they won't -- not if you bet equal amounts on each one.

...so it stands to reason that randomly choosing any horse in any race with equal probability will also result in a loss rate equal to the track takeout.
Again, no. Because when you bet randomly, by which we explicitly mean "uniformly random" you have an equal chance of picking each horse in the race. But the prices you'll get are determined by the betting odds, which are not random at all. So you end up favoring longshots when betting randomly, and longshots lose much more than the track take.

So getting back to betting every horse in every race to have our losses equal the track takeout, in order to do that you need to bet amounts in proportion to the existing bets in the pool. Let's say you get to make the last bets before the race goes off and you've got $100 to spend. If horse A has 20% of the win pool, then you should bet $20 on horse A, and if horse B has 10% of the pool, you bet $10 on B, etc on every horse. You'll win every race as before, but now you'll have more money on the lower odds horses and in fact your bets will not have affected the odds at all because you mirrored the pool when making them. And so from your perspective, the result is the same as if you were the only bettor and your losses will equal the track take. But if you bet equal amounts (and you are not the only bettor), again you favor the longshots and your losses will be much greater. And so it follows if you only bet one horse and choose him randomly (uniformly randomly) then you will also lose greater than track take (betting all horses with same amount or betting one horse randomly are equivalent over time).

So if you still want to be "random", but only lose track take, you need to choose your random horse according to the distribution of the existing money in the pool -- stochastically. Which can not be done in practice precisely, of course.


So, once you realize that you can NOT bet randomly and only lose the track take -- the floor is much lower than that -- you realize just how tough the game is.

[Turkoman]

Turkoman,

I am assuming that Mr. Klein's references were to horses that qualified as "early speed' because otherwise, the numbers are much, much worse.

As an example, horses at like 30/1 and above lose about 45% of each dollar wagered.

Regards,
Dave Schwartz

Dave, this information is from the very first chart in the book, and it simply shows the return on win wagers for horses at different prices. This chart has nothing to do with early speed at all. Like you said, though, maybe these longshots do even worse now. The book is from 2005, so I'm pretty sure the data goes back to the 1990's.

[Turkoman]

Turkoman,

I am assuming that Mr. Klein's references were to horses that qualified as "early speed' because otherwise, the numbers are much, much worse.

As an example, horses at like 30/1 and above lose about 45% of each dollar wagered.

Regards,
Dave Schwartz

Well, I just went back and took a look now. Horses in the 30-39.99 to 1 range returned just $1.26 for every $2.00 bet. That's a 37% loss, so I guess this is much closer to the number you mentioned.

[podonne]

Sure. It is easy to demonstrate. Just bet randomly.

No they won't -- not if you bet equal amounts on each one.

Again, no. Because when you bet randomly, by which we explicitly mean "uniformly random" you have an equal chance of picking each horse in the race. But the prices you'll get are determined by the betting odds, which are not random at all. So you end up favoring longshots when betting randomly, and longshots lose much more than the track take.

So getting back to betting every horse in every race to have our losses equal the track takeout, in order to do that you need to bet amounts in proportion to the existing bets in the pool. Let's say you get to make the last bets before the race goes off and you've got $100 to spend. If horse A has 20% of the win pool, then you should bet $20 on horse A, and if horse B has 10% of the pool, you bet $10 on B, etc on every horse. You'll win every race as before, but now you'll have more money on the lower odds horses and in fact your bets will not have affected the odds at all because you mirrored the pool when making them. And so from your perspective, the result is the same as if you were the only bettor and your losses will equal the track take. But if you bet equal amounts (and you are not the only bettor), again you favor the longshots and your losses will be much greater. And so it follows if you only bet one horse and choose him randomly (uniformly randomly) then you will also lose greater than track take (betting all horses with same amount or betting one horse randomly are equivalent over time).

So if you still want to be "random", but only lose track take, you need to choose your random horse according to the distribution of the existing money in the pool -- stochastically. Which can not be done in practice precisely, of course.


So, once you realize that you can NOT bet randomly and only lose the track take -- the floor is much lower than that -- you realize just how tough the game is.

This was interesting, thank you! I think I can see your point.

Would you expect that if someone did structure the bets so they were proportional to the share of each horse, would that result in an ROI equal to the track takeout? Or are you saying that late money will likely come in on lower priced horses, therefore skewing the proportions in a non-random way so that your proportions will still skew towards longshots?

Still, it re-raises the question that I thought had an answer, if there is no simple and practical way to easily have an ROI that equals the track takeout, what strategy would provide a comparison? It has to be one that can be applied to (nearly) any race...

[GameTheory]

This was interesting, thank you! I think I can see your point.

Would you expect that if someone did structure the bets so they were proportional to the share of each horse, would that result in an ROI equal to the track takeout? Or are you saying that late money will likely come in on lower priced horses, therefore skewing the proportions in a non-random way so that your proportions will still skew towards longshots?Well, I don't know where the late money will go, but it naturally won't be in *exact* proportion to existing money anyway.

Still, it re-raises the question that I thought had an answer, if there is no simple and practical way to easily have an ROI that equals the track takeout, what strategy would provide a comparison? It has to be one that can be applied to (nearly) any race...Bet the favorite, or if you need something where practice doesn't get in the way of theory (because of late money) then bet the ML favorite.

It really doesn't matter -- the only benchmark that truly matters is profit/loss. But there is that zone of "losing but still beating the crowd" between breaking even and loss equal to track takeout where you want to see improvement. Even if there is no automatic strategy that can be implemented in practice to EXACTLY match the track takeout with your losses, what does it matter? You aren't going to actually implement it anyway. The track takeout is still therefore a useful benchmark. As is betting the favorite, which usually beats the takeout slightly. But you actually have to calculate that one. There obviously is no "risk-free" default strategy in horse-racing, but if you insist on a actual strategy as a default (rather than just a number, e.g. the takeout), then I'd go with betting the favorite.

BTW, charting the performance of favorites is a useful thing to do in any case. If you aren't a low odds bettor and you on a losing streak, you'll often find the favorites are on a tear, maybe even flat bet profitable. And if you are a low odds bettor and you are making a killing, but again find that favorites have just been out of this world lately you should bank that money before reality asserts itself again...

[therussmeister]

Can you expand on the notion that betting randomly will result in losses of more than the track takeout? Have you seen this in practice?

If you bet on every horse in every race, your losses should precisely equal the track takeout, so it stands to reason that randomly choosing any horse in any race with equal probability will also result in a loss rate equal to the track takeout.

Even so, that might be the most appropriate benchmark. A system is good if it can achieve a better ROI than the weighted takeout rate at the tracks it is betting at (proportional to the number of races played at each track).

What's the general assumption for takeouts from the Win\Place\Show pools? 15%?

I did a study on roi of random selections three years ago. Using data from multiple tracks returned an roi of a bit more than 0.80. The sample size was not as large as I would have liked, I quit due to boredom. Also, flat bets on every horse in every race returned almost exactly the same roi. I am sure with a large enough sample it would have been exactly the same. This was actually the reason for the study; to prove that random bets give the same roi as bets an all runners.

In my opinion, the reason random bets lose more than the takeout is because winning players have the effect of increasing takeout on everybody else.

[Robert Goren]

The average takeout rate is probably 18% for win pools . Then maybe another 1% for breakage. Rounding to 20% would get you pretty close. The bench mark I use is breaking even. Although I tend give some thing that loses around 10% a closer look to see if I can make it profitable. I have recently quit "handicapping" and began looking at/designing systems. For me at least, it appears to be the way to go. I figure thing a different when I am looking to see how it does. I take the sum of 1/(odds+1) and compare that number to the number of winners. I think that gives a better picture of what is happening. Good luck in your endeavors.

[Dave Schwartz]

The average $net is actually around $1.50. That represents a 25% loss if you wager completely randomly.

However, if you have odds restrictions, you can, of course, lower that.

[GameTheory]

I did a study on roi of random selections three years ago. Using data from multiple tracks returned an roi of a bit more than 0.80. The sample size was not as large as I would have liked, I quit due to boredom. Also, flat bets on every horse in every race returned almost exactly the same roi. I am sure with a large enough sample it would have been exactly the same. This was actually the reason for the study; to prove that random bets give the same roi as bets an all runners.That sounds better than I would think, and have seen on similar tests.


The average $net is actually around $1.50. That represents a 25% loss if you wager completely randomly.

However, if you have odds restrictions, you can, of course, lower that.That sounds closer to the mark.

[Dave Schwartz]

One more comment about "random play."

There are some interesting papers written about "prediction markets." There were even some early works done (early as in 1990s) where random stock market systems/agents were tested. Eventually, the random agents were completely abandoned as total failures.

However, a high number of agents using "random factors" can, in fact, be effective. This was shown in version 2 of the above-mentioned stock market approach.)

The theory (as it would be applied to horse racing) is that if every horse was rated on (say) hundreds of random factors, the best horses would just naturally come to the top.

My experience (and I believe GT's as well) would have some agreement with this.


Dave

[therussmeister]

The average takeout rate is probably 18% for win pools . Then maybe another 1% for breakage. Rounding to 20% would get you pretty close.
Average takeout for win is less than 18% HANA lists only 9 tracks with takeout more than 18% and 40 tracks with takeout less than 18%. In my study average takeout was 16.83% I used AP, Bel, Crc, CD, PRX. also breakage would be about 0.7% based on the average $2 win price of $11.82 in my study. This 0.7% does not take into account Belmont's varying breakage, I assumed $0.10 breakage for all tracks.

[pondman]

The reality is that the higher the odds, the lower the return on investment.

Turkoman

Only when randomly playing races or being required to play all races, without regards to conditions. If you are required to play all races randomly, then your expectations will be similiar to a slot machine. And of course your expectations for hitting the progressive, will result in a huge loss.

I wouldn't agree with the market analogies. Most personal wealth (or losses) in the U.S. markets was created with a few large historical moves, due to something unexpected. You could boil it down to less than 30 days in the last 75 years. The market trades sideways 85% of the time.

[Turkoman]

Only when randomly playing races or being required to play all races, without regards to conditions. If you are required to play all races randomly, then your expectations will be similiar to a slot machine. And of course your expectations for hitting the progressive, will result in a huge loss.

I wouldn't agree with the market analogies. Most personal wealth (or losses) in the U.S. markets was created with a few large historical moves, due to something unexpected. You could boil it down to less than 30 days in the last 75 years. The market trades sideways 85% of the time.

If you take the time to go back and read post #5, you will see that it was a direct reply to that; playing at random.

[Dave Schwartz]

Only when randomly playing races or being required to play all races, without regards to conditions. If you are required to play all races randomly, then your expectations will be similiar to a slot machine. And of course your expectations for hitting the progressive, will result in a huge loss.

How can a guy who claims to only bet 50 races per year even begin to discuss random play?

[podonne]

One more comment about "random play."

There are some interesting papers written about "prediction markets." There were even some early works done (early as in 1990s) where random stock market systems/agents were tested. Eventually, the random agents were completely abandoned as total failures.

However, a high number of agents using "random factors" can, in fact, be effective. This was shown in version 2 of the above-mentioned stock market approach.)

The theory (as it would be applied to horse racing) is that if every horse was rated on (say) hundreds of random factors, the best horses would just naturally come to the top.

My experience (and I believe GT's as well) would have some agreement with this.


Dave

Are you thinking of a paper in particular? I've read about adaptive agents that start with random factors but learn to prefer one strategy over another based on success. The random walk people would say that any truely random approach would average out to the market return (minus commissions). That doesn't seem to apply to horse racing, per earlier posts in this thread.

In my own research, I have found that random agents can outperform a system when using the exact same factor set, but only because the random agent would occasionally not bet versus the system that bets every time, and since every system is negative ROI, just from not betting the random agents do better.

[Dave Schwartz]

In my own research, I have found that random agents can outperform a system when using the exact same factor set, but only because the random agent would occasionally not bet versus the system that bets every time, and since every system is negative ROI, just from not betting the random agents do better.

Only in a small sample, I would assume. In the long run, random means, well,... random.


I will look for those papers. I rarely lose anything.

[podonne]

Only in a small sample, I would assume. In the long run, random means, well,... random.


I will look for those papers. I rarely lose anything.

I was thinking of it in the context of a system (making a series of bets) and an agent that decides when to use the system and when to ignore it, essentially if the agent agrees with the system, it bets as well.

In this context, any system has a negative ROI, so any agent that decides even once not to bet along with the system will have a higher ROI (typically). Not necessarily positive ROI, just not as bad.

Thanks for looking.

[Dave Schwartz]

Ah... a better explanation. And a highly sophisticated approach as well.

:ThmbUp:

[Robert Goren]

Average takeout for win is less than 18% HANA lists only 9 tracks with takeout more than 18% and 40 tracks with takeout less than 18%. In my study average takeout was 16.83% I used AP, Bel, Crc, CD, PRX. also breakage would be about 0.7% based on the average $2 win price of $11.82 in my study. This 0.7% does not take into account Belmont's varying breakage, I assumed $0.10 breakage for all tracks.You are right. according to Hana the median for all tracks is 17%. If you used strickly handle to get your numbers it would be a bit lower because the most of the big handle tracks are below 17%. At one time, charts had return to the public numbers. It would make it a lot easier if they still did.

[podonne]

Ah... a better explanation. And a highly sophisticated approach as well.

:ThmbUp:

Thanks. Its not working yet, but I'm trying to create an agent that bets into and out of a system, but the agent learns over time, so I need a baseline to measure "improvement". When the agent can do better just by not betting, it tends to learn to just not bet, which is not what I'm looking for. Hence the need for a different baseline besides the underlying system.

Thinking maybe measuring a learning agent vs a random agent might be more accurate. The random agent should be better than the system (per the above), so if the learning agent can be better than the random agent, there's something there... Thus my interest in the papers you mentioned, to properly construct the random agent to measure against (its rarely as simple as just, well, being random).

[podonne]

You are right. according to Hana the median for all tracks is 17%. If you used strickly handle to get your numbers it would be a bit lower because the most of the big handle tracks are below 17%. At one time, charts had return to the public numbers. It would make it a lot easier if they still did.

I don't suppose there's an easy list of takeout by track? A baseline based on takeout would have to be weighted based on an arbitrary underlying track-bet mix. Probably a good thing to know either way...

[podonne]

I don't suppose there's an easy list of takeout by track? A baseline based on takeout would have to be weighted based on an arbitrary underlying track-bet mix. Probably a good thing to know either way...

Whoops didn't see the link on HANA

http://www.horseplayersassociation.org/hanatrackratingsbyoverallscore2012.html

[Magister Ludi]

Greetings,

I've been tossing this question around for a bit about how to properly baseline a prospective system\angle\tout\etc. By that I mean something to compare against to tell if the system is performing "well".

In stock markets I would compare a prospective strategy against a "buy and hold" strategy, or against a standard nominal interest rate.

I'm wondering if such a "default strategy" exists in horse racing. For example, you might compare all prospective strategies against a "bet the favorite to show" strategy. Or you might compare against a standard track takeout (- 15%).

Ideally a commonly known, meaningful (as in not random), applies to most races, and performing better than the track takeout (but not profitable strategy).

Thoughts?
Podonne

I would vote for "bet the favorite to show" as your benchmark. It has, by far, the highest return of any naive extrapolation of which I know in racetrack betting markets.

[raybo]

I don't know the exact hit rate for all horses that went off at 1/1 and paid $3, but if that hit rate is 50%, then after only 2 races you have lost 25% (bet $4, received $3 = $1 loss, -$1/$4 = -.25, -25%, 0.75 ROI), lower than the track takeout.

The bench mark, IMO, should be long term net profit, if you're testing a system with set play and wagering rules (flat bet on all qualified plays).

[flatstats]

I think you need to scrutinise the method first before you baseline it. Doing so will prevent you from taking a biased view against the baseline.

Is the method sound? Did you create the method by hacking away at data to make it look good? Are all the rules justifiable?

Only after you have answered that can you move on to the comparing against a baseline stage.

---

Point about horses 30/1 to 40/1 range compared to favourites. There is only one favourite in the race whilst there can be 2/3/4... 30/1 to 40/1 horses in a race so that is not a comparable test.

[PICSIX]

Good point about only one favorite vs. several long shots in a given race. Dave, how bad is the ROI on the longest priced horse in the race vs. the favorite?

Thanks,

Mike

[HUSKER55]

maybe I don't understand the situation but here goes.

If you are going to compare longshots, wouldn't a better standard be how many horses does the public miss and set the favorites at a max of (for example) 8:1? Maybe I am wrong but aren't you trying to tell how well your system picks the winners when the public is wrong.

just a question...

[Magister Ludi]

Good point about only one favorite vs. several long shots in a given race. Dave, how bad is the ROI on the longest priced horse in the race vs. the favorite?

Thanks,

Mike


Favorite Position.....Win ROI
1............................. -16.43%
9-12........................ -24.19%


M. Gramm and D. H. Owens. "INEFFICIENCIES IN PARIMUTUEL BETTING MARKETS ACROSS WAGERING POOLS IN THE SIMULCAST ERA".

[raybo]

maybe I don't understand the situation but here goes.

If you are going to compare longshots, wouldn't a better standard be how many horses does the public miss and set the favorites at a max of (for example) 8:1? Maybe I am wrong but aren't you trying to tell how well your system picks the winners when the public is wrong.

just a question...

I think (but things may have changed recently) about 80% of all winners come from the top 4 public choices. That means that the public, given 4 chances, loses only 20% of the time. To me, that means that, if your top choice isn't in the top 4 public choices, a majority of the time, then you're not going to do well. However, that only means that your percentage of winners will not do well, that doesn't mean that you cannot be profitable.

Your ability to pick enough winners, at a high enough average payout, will determine whether you win or lose money, long term.

How does that relate to a "baseline" against which to compare your system's performance? That depends on whether you just want to hit winners, or if you want to make money. They aren't the same.

Again, IMO, the baseline should be long term profit/loss, and this can only be determined by keeping thorough records, and staying consistent in your methods, both handicapping and wagering.

[HUSKER55]

THANKS RAY!

[PICSIX]

Favorite Position.....Win ROI
1............................. -16.43%
9-12........................ -24.19%


M. Gramm and D. H. Owens. "INEFFICIENCIES IN PARIMUTUEL BETTING MARKETS ACROSS WAGERING POOLS IN THE SIMULCAST ERA".


Thanks :ThmbUp:

[bob60566]

]I think (but things may have changed recently) about 80% of all winners come from the top 4 public choices. That means that the public, given 4 chances, loses only 20% of the time. To me, that means that, if your top choice isn't in the top 4 public choices, a majority of the time, then you're not going to do well.[/B] However, that only means that your percentage of winners will not do well, that doesn't mean that you cannot be profitable.

Your ability to pick enough winners, at a high enough average payout, will determine whether you win or lose money, long term.

How does that relate to a "baseline" against which to compare your system's performance? That depends on whether you just want to hit winners, or if you want to make money. They aren't the same.

Again, IMO, the baseline should be long term profit/loss, and this can only be determined by keeping thorough records, and staying consistent in your methods, both handicapping and wagering.
Ray
Very intresting stat to base your selections.
Txs

Bob :)

[raybo]

Ray
Very intresting stat to base your selections.
Txs

Bob :)

Bob,

I hope you didn't misunderstand what I meant. I'm not recommending the 80% winners in the top 4 public choices as a measurement or a baseline, unless you're only concerned with hit rate (number of winners versus number of bets).

I cannot stress enough that hit rate is only 1 part of making profit, long term, the other is average payout of your hits. So if you're hit rate is high but your average payout on those hits is low, then you really haven't improved your game, if you're looking for long term profit.

[bob60566]

Bob,

I hope you didn't misunderstand what I meant. I'm not recommending the 80% winners in the top 4 public choices as a measurement or a baseline, unless you're only concerned with hit rate (number of winners versus number of bets).
I cannot stress enough that hit rate is only 1 part of making profit, long term, the other is average payout of your hits. So if you're hit rate is high but your average payout on those hits is low, then you really haven't improved your game, if you're looking for long term profit.

The latter.

Bob :)

[davew]

I do not understand what you mean 'baseline'?

Blindly betting on every favorite to show has got to be one of the worst bets you can make ROI wise - besides the track take-out, you have to deal with breakage and what the track minimum pay-out is. ($2.39's become $2.20, andmost places $2.19 becomes $2.10 - plus the bridgejumpers popping in every once in awhile).

Hit frequency is probably good, but could be made better with some selection and pulling out vulnerable favorites.

So what are you trying to predict? factors that increase hit rate or factors that increase ROI? as they are not necessarily the same.

[Red Knave]

I do not understand what you mean 'baseline'?
I think the OP wants to have something that he can compare to the results of his own method or system. The baseline system needs to be something that can easily be created or recreated from any data set. The OP's own system is in development and he needs to have something that will tell him whether or not any change was positive or negative when compared to the baseline. If betting the favorite to show in a certain set of data returns, say, $1.75 he needs to know that if a change improves his method from, say, $1.65 to $1.70 he's still probably on the wrong track.
I think he understands about breakage and hit rate etc.

[barn32]

I think you need to scrutinise the method first before you baseline it. Doing so will prevent you from taking a biased view against the baseline.

Is the method sound? Did you create the method by hacking away at data to make it look good? Are all the rules justifiable?

Only after you have answered that can you move on to the comparing against a baseline stage.

---

Point about horses 30/1 to 40/1 range compared to favourites. There is only one favourite in the race whilst there can be 2/3/4... 30/1 to 40/1 horses in a race so that is not a comparable test.21 posts in ten years! This is a perfect example of someone having something very sound to say before saying it.