Backfitting Deluxe for the Kentucky Derby

[Helles]

I decided to look at the Derby races since 2005 (the extent of my database archive) and see if I could find any interesting factors that were significant in pointing to winners. Interesting as in "arcane".

I started with 900 factors and ran them against races from 2005 to 2014. As I progressed, I threw out factors that had little predictive effect and started over running the remaining factors again over the same 10 races. My program also plugs in different weights for the individual factors and will also run multiple factors together as an analyst. As a factor or analyst goes broke because the factor/analyst is not effective, it is replaced by a new factor/analyst with a new bankroll and a new set of factors and weights.

Eventually I whittled the 900 factors down to 48 that seemed the most effective. I really didn't know what to expect when I then checked my 48-factor weighted analyst against each individual Derby over the sample.

I thought perhaps it would hit the lower-priced winners as the analyst's top choice as we saw last year and the year before, but miss some higher-priced winners completely.

This is where the surprise came in. I unexpectedly found that the analyst had every winner save one in the top two choices. This included $102.60 winner Giacomo in 2005. The only winner that was not in the top two choices was actually the third choice; Mine That Bird who paid $103.20 in 2007.

Here are the results of the analyst against each individual race:
2014 1st choice wins paying 7.00
2013 1st choice wins paying 12.80
2012 2nd choice wins paying 32.60
2011 2nd choice wins paying 43.80
2010 1st choice wins paying 18.00
2009 3rd choice wins paying 103.20
2008 1st choice wins paying 6.80
2007 1st choice wins paying 11.80
2006 1st choice wins paying 14.20
2005 2nd choice wins paying 102.60

Perhaps somebody can explain or theorize how this analyst, using the same factors, weighted the same way, can do so well picking horses with such disparate final odds. I fully understand I am backfitting and backfitting to a VERY small sample. But one would not expect to see what works for a $7.00 horse also working for a $102.60 horse.

The idea of this exercise was to post some of the factors that seemed to point to Derby contenders. As I ran my study, I forced my program to make only one bet per race per factor/analyst. Running the study this way actually didn't do a very good job of picking contenders. (Perhaps I will rerun the study and force three or four bets per race) If one had boxed the top three choices for instance in an exacta, only one exacta would have been hit. However, wheeling the top three choices against the top 12 choices would have hit every exacta in the sample.

The next surprise came when I decided to look at this year's field. No big surprises until I glanced down the list and saw Stanford. I realized I had to scratch him and put in Frammento. I re-analyzed the race and this is how the analyst's top five shook out:

American Pharaoh
Firing Line
Frammento
Dortmund
Ocho Ocho Ocho
(Carpe Diem, Materiality, War Story, International Star and Frosted round out the top 10)

It will be interesting, to me at least, to see if the winner is in the top three this year. It would be no surprise if AP or FL won, but Frammento certainly would surprise.

Good luck to all.

Doug

[kevb]

Thanks for sharing this.

[castaway01]

I like Firing Line to win it, so you're probably safe taking him out of your wagers.

[mikesal57]

if you use this going forward and get winners in the top 2-3 ..you'll be changing the title of this post :)
and going to the bank!!
mike

[DeltaLover]

As you say yourself, what you describe here is nothing else that a huge back-fit that simply memorizes the results and behaves accordingly; simply a through away model :)

[acorn54]

the derby will always remain a mystery imo for a variety of significant reasons.
1-distance untested by ALL the contestants
2- with 20 horses, the element of racing luck much greater than normal, too much can go wrong during the 2 minutes of the race.
3-confidence level of model not high, because of uniqueness of race, unable to get a large enough sample for testing and validation.

[Magister Ludi]

Perhaps somebody can explain or theorize how this analyst, using the same factors, weighted the same way, can do so well picking horses with such disparate final odds.

If you can afford the Purina Monkey Chow, you may eventually develop a model where you can predict the winner of every race and not just the Kentucky Derby.

[Helles]

It sounds like DeltaLover and Master Luidi are saying that this is the perfect combination of factors, 48 of them, weighted just right, to be able to get 9 out of 10 winners in the top 2. I had not considered that I would be able to do that using so many factors in my final model. But that actually makes sense. Using fewer factors, say 20, would probably not backfit so well and fewer winners would have been hit.

Thank you for the input.

[Helles]

the derby will always remain a mystery imo for a variety of significant reasons.
1-distance untested by ALL the contestants
2- with 20 horses, the element of racing luck much greater than normal, too much can go wrong during the 2 minutes of the race.
3-confidence level of model not high, because of uniqueness of race, unable to get a large enough sample for testing and validation.

And this is probably exactly why the same model that backfit so well to winners was not useful in identifying contenders.

[whodoyoulike]

...
This is where the surprise came in. I unexpectedly found that the analyst had every winner save one in the top two choices. This included $102.60 winner Giacomo in 2005. The only winner that was not in the top two choices was actually the third choice; Mine That Bird who paid $103.20 in 2007.

Here are the results of the analyst against each individual race:
2014 1st choice wins paying 7.00
2013 1st choice wins paying 12.80
2012 2nd choice wins paying 32.60
2011 2nd choice wins paying 43.80
2010 1st choice wins paying 18.00
2009 3rd choice wins paying 103.20
2008 1st choice wins paying 6.80
2007 1st choice wins paying 11.80
2006 1st choice wins paying 14.20
2005 2nd choice wins paying 102.60

Perhaps somebody can explain or theorize how this analyst, using the same factors, weighted the same way, can do so well picking horses with such disparate final odds. I fully understand I am backfitting and backfitting to a VERY small sample. But one would not expect to see what works for a $7.00 horse also working for a $102.60 horse...

The next surprise came when I decided to look at this year's field. No big surprises until I glanced down the list and saw Stanford. I realized I had to scratch him and put in Frammento. I re-analyzed the race and this is how the analyst's top five shook out:

American Pharaoh
Firing Line
Frammento
Dortmund
Ocho Ocho Ocho
(Carpe Diem, Materiality, War Story, International Star and Frosted round out the top 10)

It will be interesting, to me at least, to see if the winner is in the top three this year. It would be no surprise if AP or FL won, but Frammento certainly would surprise.

Good luck to all.

Doug


I don't know the reason for your results but, I hope you provide an update after the Derby is run (referencing this post #1). Let us know whether you made a wager and the type of wager(s) using your contenders.

I'm just curious, would it be an easy task to show the fractional call times for each race and the fractional positions of each of the winning horses in the prior Derby races?

I'm a believer in understanding a race's pace scenario.

I'd copyright the word "PaceAdvantage" but, there might be a problem trying this.

[Tom]

http://www.anddownthestretchtheycome.com/2012/4/28/2983836/kentucky-derby-pace-1980-2011

Splits are here, for horse positions, you can get the charts free at EB, BRIS.

[whodoyoulike]

Thanks Tom,

Appears the early and 6f fractions were very fast for 2005 - 2011. Now, where were the winners at each fractional call?

[Pensacola Pete]

They did the same thing with the Dosage Index, backfitting the Derby results by adjusting the indices. They finally gave up when too many odd results made it impossible.

[Dexter C. Hinton]

It is interesting how many individuals can only respond with:

a) Back Fitting (bah):

b) No statistical substance: (not enough data points, etc.)

For (a), the methodology you utilized of course was to determine, to your best ability, the correlated factors resulting from your analysis. It is easy to be critical and scream 'back fit'. I would rather give the benefit of the doubt, and say 'utilization of predictable elements'.

For (b), there will always be individuals who also scream, 'No statistical substance' because of limited sample size, etc. From a pure statistician's view that will always be true because we live is a world of limited data points (i.e., perhaps if you had detailed data back a hundred or so years, for this particular race, they may give you a break). I doubt it however, they would most likely tell you a 100+ data points is not enough for statistical relevance.

The interesting point of your post is that studying a specific track at a specific distance with a consistent level of participants (i.e., three (3) year olds, etc.), under ALL types of weather conditions, with what one can be said to be to many horses, utilizing a type of 'simple AI', you were able to legitimately test your algorithm against back races.

For the record, I do not put a lot of faith in 'black box handicapping'. However as old as I am, one thing I have learned is the game has changed. 'Dumb money' has become pretty much non-existent and wagers have to utilize all the resources in their toolbox. What you have articulated is one of those 'tools'.

This should not be criticized for what was done, but only questioned for its applicability to be useful in the future (in conjunction with the individual handicapping resources each one has).

As I am sure you are aware, it is much easier to criticize, combined with offering no legitimate ideas or alternatives.

Even if your findings do apply to this year, maybe next year also, and beyond, you can be rest assured that one year it will fail and the response you will get will be "What do you expect of back fitting, it had no statistical significance to begin with".

dch
05/02
10:37.31..31

[traynor]

It is interesting how many individuals can only respond with:

a) Back Fitting (bah):

b) No statistical substance: (not enough data points, etc.)

For (a), the methodology you utilized of course was to determine, to your best ability, the correlated factors resulting from your analysis. It is easy to be critical and scream 'back fit'. I would rather give the benefit of the doubt, and say 'utilization of predictable elements'.

For (b), there will always be individuals who also scream, 'No statistical substance' because of limited sample size, etc. From a pure statistician's view that will always be true because we live is a world of limited data points (i.e., perhaps if you had detailed data back a hundred or so years, for this particular race, they may give you a break). I doubt it however, they would most likely tell you a 100+ data points is not enough for statistical relevance.

The interesting point of your post is that studying a specific track at a specific distance with a consistent level of participants (i.e., three (3) year olds, etc.), under ALL types of weather conditions, with what one can be said to be to many horses, utilizing a type of 'simple AI', you were able to legitimately test your algorithm against back races.

For the record, I do not put a lot of faith in 'black box handicapping'. However as old as I am, one thing I have learned is the game has changed. 'Dumb money' has become pretty much non-existent and wagers have to utilize all the resources in their toolbox. What you have articulated is one of those 'tools'.

This should not be criticized for what was done, but only questioned for its applicability to be useful in the future (in conjunction with the individual handicapping resources each one has).

As I am sure you are aware, it is much easier to criticize, combined with offering no legitimate ideas or alternatives.

Even if your findings do apply to this year, maybe next year also, and beyond, you can be rest assured that one year it will fail and the response you will get will be "What do you expect of back fitting, it had no statistical significance to begin with".

dch
05/02
10:37.31..31

I disagree. The point is not criticism of the OPs intent. It is criticism of a process used by many to create "betting models" that don't work in the real world. Especially if one bets on them.

[Helles]

I am not waving this algorithm declaring I have found the one true way. I am still trying to wrap my mind around the fact that it successfully ranked low-priced as well as longshot winners in the top 2 or 3. That's the reason my post took the direction it did. I was originally looking to make a contribution by pointing out a few factors that some of you might not have considered that winners had in common.

Out of curiosity, I used this analyst to handicap the Oaks yesterday and this was the output:
8 I'M A CHATTERBOX
9 MONEY'SONCHARLOTTE
7 LOVELY MARIA
Result was 7,2,8

I know this means nothing in the big picture, but the result was interesting nonetheless.

[DeltaLover]

It is interesting how many individuals can only respond with:

a) Back Fitting (bah):

b) No statistical substance: (not enough data points, etc.)

For (a), the methodology you utilized of course was to determine, to your best ability, the correlated factors resulting from your analysis. It is easy to be critical and scream 'back fit'. I would rather give the benefit of the doubt, and say 'utilization of predictable elements'.

For (b), there will always be individuals who also scream, 'No statistical substance' because of limited sample size, etc. From a pure statistician's view that will always be true because we live is a world of limited data points (i.e., perhaps if you had detailed data back a hundred or so years, for this particular race, they may give you a break). I doubt it however, they would most likely tell you a 100+ data points is not enough for statistical relevance.

The interesting point of your post is that studying a specific track at a specific distance with a consistent level of participants (i.e., three (3) year olds, etc.), under ALL types of weather conditions, with what one can be said to be to many horses, utilizing a type of 'simple AI', you were able to legitimately test your algorithm against back races.

For the record, I do not put a lot of faith in 'black box handicapping'. However as old as I am, one thing I have learned is the game has changed. 'Dumb money' has become pretty much non-existent and wagers have to utilize all the resources in their toolbox. What you have articulated is one of those 'tools'.

This should not be criticized for what was done, but only questioned for its applicability to be useful in the future (in conjunction with the individual handicapping resources each one has).

As I am sure you are aware, it is much easier to criticize, combined with offering no legitimate ideas or alternatives.

Even if your findings do apply to this year, maybe next year also, and beyond, you can be rest assured that one year it will fail and the response you will get will be "What do you expect of back fitting, it had no statistical significance to begin with".

dch
05/02
10:37.31..31

:ThmbDown: :ThmbDown:

Sorry, but what you are saying here is complete wrong and creates the wrong impression to people who try to handicap using a computer...

[Saratoga_Mike]

:ThmbDown: :ThmbDown:

Sorry, but what you are saying here is complete wrong and creates the wrong impression to people who try to handicap using a computer...

What did you find most objectionable DL? I don't use a database, so I always like to hear from people who do - I think you do?

[DeltaLover]

What did you find most objectionable DL? I don't use a database, so I always like to hear from people who do - I think you do?

Back-fitting is the number one reason for erroneous conclusions, especially when combined with small data

[garyoz]

Back-fitting is the number one reason for erroneous conclusions, especially when combined with small data

and highly correlated variables.

[Helles]

... I hope you provide an update after the Derby is run (referencing this post #1). Let us know whether you made a wager and the type of wager(s) using your contenders.


Buoyed by the success of the analyst picking the Oaks, I did use the output to make some Derby wagers.

Here is that output again:
18 American Pharoah 39|
10 Firing Line 25|
21 Frammento 11|
8 Dortmund 8|

Result 18,10,8,15

I made win wagers on the top three in varied amounts. I made exacta bets with varying base amounts by boxing the top two, boxing the top three and wheeling the top three against the next nine highest ranked horses and finally a $1 exacta with Frammento/All, "just in case".

[TexasDolly]

Buoyed by the success of the analyst picking the Oaks, I did use the output to make some Derby wagers.

Here is that output again:
18 American Pharoah 39|
10 Firing Line 25|
21 Frammento 11|
8 Dortmund 8|

Result 18,10,8,15

I made win wagers on the top three in varied amounts. I made exacta bets with varying base amounts by boxing the top two, boxing the top three and wheeling the top three against the next nine highest ranked horses and finally a $1 exacta with Frammento/All, "just in case".

I was glad to see your success with the analysis you made. I am growing weary of the backfitting arguments. I never have enough data that has any degree of consistency
to do any thing but backfit. Raybo has made the point a dozen times about his program which uses around 30 days of current data and he maintains that it works profitably and there are others I imagine. At any rate, I would be very pleased to be able to process 900 factors and find 48 that seemed to fit
various horses such as you describe.
Congratulations to you on your work.
TD

[Saratoga_Mike]

Buoyed by the success of the analyst picking the Oaks, I did use the output to make some Derby wagers.

Here is that output again:
18 American Pharoah 39|
10 Firing Line 25|
21 Frammento 11|
8 Dortmund 8|

Result 18,10,8,15

I made win wagers on the top three in varied amounts. I made exacta bets with varying base amounts by boxing the top two, boxing the top three and wheeling the top three against the next nine highest ranked horses and finally a $1 exacta with Frammento/All, "just in case".

Nice work

[flatstats]

The Titanic Will Never Sink
Backfitting is a bigger thief of time than procrastination x 10. It is one of the worst things a handicapper can do because not only does it give false results, it gives a dangerous false sense of security and it also wastes many hours, weeks and months of a handicappers time.

The reason for this is because a handicapper will never give up on his pet project. He will look at the results, refine it (backfit it), whittle away at it putting in so many hours such that he conditions himself to believe it can never fail.

The reality is though that it will fail because the handicapper is so oblivious to the truth that his method is so faulty to start with.

I am Robin Hood
Backfitting is akin to shooting an arrow at a tree and painting a target around it. You can shoot dozens of arrows at a nice row of Elms and when one hits the centre of such a tree you run up to it, get out your red, blue, and gold paint and paint concentric circles around the arrow. Marvel in the delight in showing off to your friends that you hit BULLS EYE. Yeee ha.

Now try the same shot again when everyone is looking at you; when you have your max stake on it. It's not going to happen.

Cluedo Handicapping
One of the most widely used type of backfitting is what I call Cluedo Handicapping. This is where a handicapper thinks something will happen again because he is convinced it is Mrs White in the Conservatory with the Lead Pipe.

The racing angle with this is on the lines of "4 out of the past 17 winners of the Epsom Derby had previously run at Royal Ascot as a 2yo", or something like "8 out of 11 winners of the St Leger had run over 12f last time out", or "25 grey horses have run in the York Ebor but none have won"

The problem with those stats is that they do not:

a) take into consideration the number of winners and starters in that group
b) take into consideration the prices of those runners

The other classic mistakes you can probably spot are the selective use of figures. Why use the past 17 winners for the Epsom Derby but only 11 for the St Leger? What was the timescale of those 25 grey horse in the Ebor? Were they all 2008 to 2015 or 1908 to 2015?

How To Avoid Backfitting
There is a simple statistical techinque you can use to prevent backfitting and that is to use sample sizes, chi square and sandboxing.

This involves looking at the results you have observed and firstly determining if the sample size is sufficient, secondly determining if the results are likely to be due to randomness. Thirdly you then wait from the date you observe results to when the 'live' sample size is sufficient.

But if you want to keep it simple just try this:

"If you can not justify why a rule or filter should be used then don't use it."

If you find that colts have a poor record in August and you can not work out why that is then don't rely on that fact. If you find that A Jockey is worse at B course but not C and you don't know why that is then don't use it.

Summary
Backfitting is pure evil. It is the biggest cause of failure for handicappers without question.

Handicappers would rather tap their bank out a dozen times because "just in case" their system comes good. They would rather be seen to be a mug that is losing, rather than a mug who didn't back a 25/1 shot because they gave up.

[thaskalos]

The fact that Helles's system picked this Derby's exacta cold is an encouraging development...but one's excitement is largely diminished by the disturbing fact that the system also selected Frammento as a better winning possibility than Dortmund.

All in all, more work on this method needs to be done, in my opinion.

[whodoyoulike]

Buoyed by the success of the analyst picking the Oaks, I did use the output to make some Derby wagers.

Here is that output again:
18 American Pharoah 39|
10 Firing Line 25|
21 Frammento 11|
8 Dortmund 8|

Result 18,10,8,15

I made win wagers on the top three in varied amounts. I made exacta bets with varying base amounts by boxing the top two, boxing the top three and wheeling the top three against the next nine highest ranked horses and finally a $1 exacta with Frammento/All, "just in case".


Congrats, nice call and thanks for providing an update.

[Helles]

Thank you for your kind words TD.

[Helles]

Thank you, Saratoga Mike. I enjoy your posts.

[Helles]

A well-reasoned and well-written argument. If you read my posts, it should be clear to you I understand the problems with this approach as well as the miniscule size of the available data.

I thought the personal jab in your summary was wholly unnecessary. It made you come across as petty and angry.

[Helles]

Gus,

That is exactly what spurred me to start this thread. In 2009 how could Mine That Bird be ranked higher than Pioneer of the Nile or Chocolate Candy? In 2005 how was Giacomo rated higher than Afleet Alex?

Well, we know it's because Mine that Bird and Giacomo won those races and the algorithm was backfit to find those winners. What I still don't understand is how the same algorithm that found 50/1 horses also found 5/2 horses. Well, the 5/2 horses won too it could be argued. The algorithm was backfit to find them too. But how much do the 50/1 horses and the 5/2 horses have in common over the 48 factors used? It would appear they have plenty in common, but that is counter intuitive. This makes me ask the question, did Frammento have more in common with the winner that made Frammento an overlay? Albeit a losing one?

Does it mean at this highest level of racing there isn't much difference between the two extremes? They are all champions after a fashion and have plenty of heart?

I don't know the answers and I don't defend the methodology. I am simply puzzled by the results.

Of course it could all be a coincidence too.

[Helles]

Congrats, nice call and thanks for providing an update.

Thank you. I don't have the pressure of doing this for a living like Thaskalos so I could afford to have some fun with it. And it WAS fun.

This is not my regular methodology. I have actually been working on something that was inspired by a thread DeltaLover started a few months back. It involves no backfitting whatsoever, just testing hypotheses against samples and live testing.

I will publish my results when my sample size is a bit larger. Larger than the Derby sample size in any case.

[Helles]

Hmmm, I see I must have hit the wrong button. Instead of clicking on quote I simply responded.

The "well-reasoned" post was directed at FlatStats.
The one with the greeting "Gus" is of course directed at Thaskalos.

Sorry for any confusion caused.

Doug

[thaskalos]

Gus,

That is exactly what spurred me to start this thread. In 2009 how could Mine That Bird be ranked higher than Pioneer of the Nile or Chocolate Candy? In 2005 how was Giacomo rated higher than Afleet Alex?

Well, we know it's because Mine that Bird and Giacomo won those races and the algorithm was backfit to find those winners. What I still don't understand is how the same algorithm that found 50/1 horses also found 5/2 horses. Well, the 5/2 horses won too it could be argued. The algorithm was backfit to find them too. But how much do the 50/1 horses and the 5/2 horses have in common over the 48 factors used? It would appear they have plenty in common, but that is counter intuitive. This makes me ask the question, did Frammento have more in common with the winner that made Frammento an overlay? Albeit a losing one?

Does it mean at this highest level of racing there isn't much difference between the two extremes? They are all champions after a fashion and have plenty of heart?

I don't know the answers and I don't defend the methodology. I am simply puzzled by the results.

Of course it could all be a coincidence too.
These are all legitimate questions, Doug...and the answers are not clear-cut. It isn't that back-fitting is such a bad thing, because all handicapping can be considered "back-fitting" to an extent. We all start out as handicappers by identifying certain winning and losing characteristics, with the benefit of hindsight. But, even as you focus your search on the Derby winners, and you are trying to find what made them victorious, you should still be wary of totally illogical predictions...such as Mind That Bird over Pioneer Of The Nile...Giacomo over Afleet Alex...or Frammento over Dortmund. These illogical predictions may have been proven right within the narrow scope of your experiment...but they can hardly be considered accurate.

This is a funny game, and it can lead you down a thousand blind alleys...if you are not careful. :)

[flatstats]

I am just trying to save you time and money Helles.

I can assure you I have seen hundreds of people do it and ultimately they all fail. This would be rookie handicappers to pundits to high rollers. They all think they have the golden goose but it's an illusion.

I have data on thousands of systems that were created by various people. Where backfitting took place a typical pattern emerges.

1. The initial results are bland.
2. The rules are tweaked to make the results look better
3. The system is run and bets are placed. Some win straight away, some don't, some give up at this stage (this is a whole new subject on Paper Trading)
4. After a few days the results are checked and the rules tweaked to try and keep it afloat
5. Go back to 4 a few times
6. Ultimately give up

What those handicappers failed to do is to use the basic techniques of checking the sample size and chi square figure. They then failed to sandbox the systems and check the live results after a period of time. If they did that they would have not wasted their time on illogical handicapping. They would have been more focused on what works and what would be more likely to work in the future.

I can not emphasise enough the point that backfitters will do all they can to protect their puppy. They spend so much time on it that they will never believe it will fail. They will do all they can to keep it afloat, whilst all around them they are sinking.

[Hoofless_Wonder]

I am just trying to save you time and money Helles.

I can assure you I have seen hundreds of people do it and ultimately they all fail. This would be rookie handicappers to pundits to high rollers. They all think they have the golden goose but it's an illusion.

I have data on thousands of systems that were created by various people. Where backfitting took place a typical pattern emerges.

1. The initial results are bland.
2. The rules are tweaked to make the results look better
3. The system is run and bets are placed. Some win straight away, some don't, some give up at this stage (this is a whole new subject on Paper Trading)
4. After a few days the results are checked and the rules tweaked to try and keep it afloat
5. Go back to 4 a few times
6. Ultimately give up

What those handicappers failed to do is to use the basic techniques of checking the sample size and chi square figure. They then failed to sandbox the systems and check the live results after a period of time. If they did that they would have not wasted their time on illogical handicapping. They would have been more focused on what works and what would be more likely to work in the future.

I can not emphasise enough the point that backfitters will do all they can to protect their puppy. They spend so much time on it that they will never believe it will fail. They will do all they can to keep it afloat, whilst all around them they are sinking.

You are raining on the parade, flatstats. If Helles had heeded your advice, he might not have cashed a ticket yesterday. You didn't save him money.

Nobody who knows statistics will argue with the merits of your points. The problem is that horse racing doesn't fit in the box of stats. Sample sizes alone are a hurdle that can't be overcome - since most of the field in yesterday's race will never attempt 10 furlongs on dirt again. If horseplayers demanded that their handicapping process meet the stringent requirements of statistics, nobody would wager on the Derby.

Rather than expend energy commenting on the evils of backfitting, you'd be better off accepting that some methods cross over in the "art" of handicapping, and leave the math far behind. Perhaps 20 years from now the approach from Helles will no longer be valid, but my gut feeling tells me his 2016 Derby selections will be worth more in risking an investment on the Derby than anything the "stats" guys come up with.....

[flatstats]

It's not about stats. It's about basic principles of forecasting.

Backfitting techniques tend to cause illusory correlation, confirmation bias, and belief perseverance. Those traits make forecasting correctly impossible.

[Hoofless_Wonder]

It's not about stats. It's about basic principles of forecasting.

Backfitting techniques tend to cause illusory correlation, confirmation bias, and belief perseverance. Those traits make forecasting correctly impossible.

You can't have it both ways, flatstats. Your comment of the failure to "the basic techniques of checking the sample size and chi square figure" is all about the basic principle of statistics, not forecasting.

You've been critical of backfitting techniques, yet the line is blurry in the sense that many handicapping/forecast techniques employ backfitting. And what handicapper doesn't suffer from "belief perseverance"? And if Helles' process generates "illusory correlations" like yesterday, I dare say more handicappers would benefit from that level of success....regardless of how the results were obtained.

[whodoyoulike]

I am just trying to save you time and money Helles..

I have data on thousands of systems that were created by various people. Where backfitting took place a typical pattern emerges...

They will do all they can to keep it afloat, whilst all around them they are sinking.

I think the reason people have the tendency to see patterns is because that's the way the brain works. It usually attempts to make sense of things at least for most people IMO.

[whodoyoulike]

Thank you. I don't have the pressure of doing this for a living like Thaskalos so I could afford to have some fun with it. And it WAS fun...


I think I understand where the other people advice to you is coming which is not to rain on your good results. Until you can figure out why your program is giving you these results from a money management POV, it's crucial to your BK to not get carried away in your wagering scheme.

[mikesal57]

if you use this going forward and get winners in the top 2-3 ..you'll be changing the title of this post :)
and going to the bank!!
mike


on your way!!

good going

[Greyfox]

all handicapping can be considered "back-fitting" to an extent.

Well said.:ThmbUp:
Past performance may not be a perfect predictor, but it is the best predictor of future performance if the horse is fit.

[Helles]

AP08

Prg Name m Pct|
1 Mr Lexis 23|
4 Love You Mon 23|
7 Poker Player 20|
3 Top Gato 19|

[Helles]

I wanted to post that Arlington race before it went off.

I was catching up on this thread when I decided to look at some other algorithms I had built for specific tracks, surfaces and distances. Most of these have many classes thrown in together to get a decent sample size.

I am going to post the picks and results from AP today. Small sample. Let's see how it worked.

[Helles]

Okay, I'm an idiot. I edited my race post AFTER the race was run. I did this to remove unnecessary garbage text above the output.

It also occurs to me that I could post ANYTHING. I wouldn't do that because that is also idiotic, but you don't know me. Just for kicks, I'll post some upcoming races that I have algorithms for. I know there is a Picks section on the forum, but I think they fit in this discussion. How about 10 races? 20? 30?

[Helles]

LS 08

Prg Name m Pct|
9 T A's Bay 38|
1 Humor Me Lolly 31|
4 Espera 16|
3 Boss Me 10|

Lonestar 08

[Helles]

Prg Name m Pct|
2 Fast Track 25|
6 Indian Nate 20|
7 Ahasuerus 10|
3 Sizzlin' Joe 9|
9 Ezio 9|

SA06

[Helles]

LS 08

Prg Name m Pct|
9 T A's Bay 38|
1 Humor Me Lolly 31|
4 Espera 16|
3 Boss Me 10|

Lonestar 08

Result 9,1,2,3

[Helles]

Prg Name m Pct|
2 Fast Track 25|
6 Indian Nate 20|
7 Ahasuerus 10|
3 Sizzlin' Joe 9|
9 Ezio 9|

SA06

Result 2,3,1,5

[Helles]

HST04 Analyst-SA. Object:sw)HST6065

Prg Name m Pct|
3 Finality's Charmer 51|
1 Splendid Scat 20|
5 Ambleside Park 12|
4 Quatre Cat 11|

[Helles]

HST04 Analyst-SA. Object:sw)HST6065

Prg Name m Pct|
3 Finality's Charmer 51|
1 Splendid Scat 20|
5 Ambleside Park 12|
4 Quatre Cat 11|

Results 4,6,1,5

[Helles]

SA 07 Analyst

Prg Name m Pct|
2 Grandiose Tactics 37|
4 My Monet 35|
5 Honey Ride 16|
1 Valiant Emilia (PE 8|

[Helles]

HST05 Analyst

Prg Name m Pct|
9 Lookout Dubai 29|
8 London 17|
6 Idontrollonshabbos 16|
2 Proxyman 13|

[Helles]

SA 07 Analyst

Prg Name m Pct|
2 Grandiose Tactics 37|
4 My Monet 35|
5 Honey Ride 16|
1 Valiant Emilia (PE 8|

Result 4,2,5,3

[Helles]

HST05 Analyst

Prg Name m Pct|
9 Lookout Dubai 29|
8 London 17|
6 Idontrollonshabbos 16|
2 Proxyman 13|

Result 8,4,9,3

[Secondbest]

I would interested to see The preakness picks

[raybo]

As you say yourself, what you describe here is nothing else that a huge back-fit that simply memorizes the results and behaves accordingly; simply a through away model :)

Yeah, just throw it all away, it means nothing at all, everyone had the winner and place horses in their top 2, and the trifecta in their top 4. :lol:

I certainly didn't have AP as my top pick (but did have Firing Line as my 2nd pick).

All I can say to the OP is, try it again next year! (Might want to do the same study for the Preakness and Belmont, also. If you do, don't post them here until the the horses are in the gate - LOL!)

[raybo]

I was glad to see your success with the analysis you made. I am growing weary of the backfitting arguments. I never have enough data that has any degree of consistency
to do any thing but backfit. Raybo has made the point a dozen times about his program which uses around 30 days of current data and he maintains that it works profitably and there are others I imagine. At any rate, I would be very pleased to be able to process 900 factors and find 48 that seemed to fit
various horses such as you describe.
Congratulations to you on your work.
TD

I agree with your post!

Regarding my own "backfitting"/"small database" work (and I also agree that all, or almost all, of us use historical data in our analysis, which could be considered "backfitting", to an extent), I database pace pressure and the number of E horses, only. I do not use the database for anything else. And the results of that database "query"/"lookup" is used for eliminations from win contention, only. I use this process to help integrate field dynamics into my Black Box approach, for one of the 11 rankings methods, by eliminating those horses that are not advantaged by the pace dynamics of the race. Other eliminations are also used in conjunction with the pace pressure and number of E horses, the program also looks for pace "matchups" (velocity ranges) in both the 1st fraction and the 2nd fraction, if the same horses are "matched up" in both of those fractions they are eliminated also. Lone E horses that pass the matchup test are passed through automatically, not eliminated because of the pace pressure test. The horses that pass these tests are then ranked by a total velocity rating to get the win contenders. Note I said "win" contenders, this method has nothing to do with the other finish positions at all, although sometimes the rankings get the exacta, tri, super, also. But that is not the goal of the program, it is a win program, specifically, however, there are 10 other rankings methods (and an additional 6 optional/variable rankings methods that can also be auto-recorded during track testing activities) in the program so there is plenty of data that can be used for the verticals if the user wishes.

My "small" databases are track specific, and only recent cards and results are in it, older cards and results are deleted as new ones are added. So, the database stays at the same number of cards all the time (unless you change it manually of course).

[dilanesp]

They did the same thing with the Dosage Index, backfitting the Derby results by adjusting the indices. They finally gave up when too many odd results made it impossible.

That's the way most gambling "systems" perform. They work... until they don't.

[dilanesp]

Nobody who knows statistics will argue with the merits of your points. The problem is that horse racing doesn't fit in the box of stats.

This is both true and false.

It is true that sample sizes of a lot of esoteric handicapping angles-- especially ones involving a single unique annual race like the Derby-- are never large enough to make good statistical conclusions.

But it is false in the sense that the principles of statistics still govern horse racing. Just because we don't have a device to accurately predict the future doesn't mean you should go to an oracle or look into a crystal ball. It just means that some bets in horse racing are basically close to pure gambling, like the lottery, because of gross insufficiencies in necessary information.

What is really going on here is this, and it has nothing to do with statistics and everything to do with psychology and compulsive gambling:

The Kentucky Derby is an ego race. We WANT to hit it. It's the only race non-horse racing fans know anything about. If people in your life know you bet the races, they will ask you who you like in this race, or ask you if you had the winner after it. It's the most famous race in America. It also draws a big field and big betting pools.

So we WANT to hit it. And therefore we WANT to believe there are statistical angles that will run us a profit. And therefore, we will do whatever we can to try and create them. Even though in the end, there's no way to know if ANY of these angles are real-- there's just too few Derbies and too much variance.

If you think about it for half a second, the way to make a reliable profit on the races is to invest one's intellectual energies into coming up with reliable handicapping angles for races that are run over and over throughout the year. Because with those races, we have a large enough sample size to know that the angles are real, and we also have enough betting opportunities to outrun variance.

In contrast, a Derby "angle" will never have a sufficient sample size, and will also never be playable enough times to outrun variance anyway. You will always just be gambling.

The Derby? The rational course for a smart and disciplined bettor would probably be to pass the race completely, because if one is to be intellectually honest, one doesn't have any assurance that any handicapping angles are going to be sufficiently reliable to profit at above the level of the takeout.

We bet the Derby because we are indisciplined, because we have egos, because we are gamblers. And this whole conversation is the window dressing we put on it.

[Hoofless_Wonder]

Helles, I don't think you named this thread very accurately.

What you appear to have done is taken the results of 10 Derbies and correctly identified the 48 most important factors in predicting the horses with the best chances to run well. I don't know what you call that process, but it appears at first glance to be something other than simply backfitting.

I don't believe anyone has explained how it identifies these runners so accurately, and at such wildly varying odds. If we assumed it just picks the best horses, then the odds are less relevant. But what really blows my mind is how these horses got such good trips.....

[raybo]

This is both true and false.

It is true that sample sizes of a lot of esoteric handicapping angles-- especially ones involving a single unique annual race like the Derby-- are never large enough to make good statistical conclusions.

But it is false in the sense that the principles of statistics still govern horse racing. Just because we don't have a device to accurately predict the future doesn't mean you should go to an oracle or look into a crystal ball. It just means that some bets in horse racing are basically close to pure gambling, like the lottery, because of gross insufficiencies in necessary information.

What is really going on here is this, and it has nothing to do with statistics and everything to do with psychology and compulsive gambling:

The Kentucky Derby is an ego race. We WANT to hit it. It's the only race non-horse racing fans know anything about. If people in your life know you bet the races, they will ask you who you like in this race, or ask you if you had the winner after it. It's the most famous race in America. It also draws a big field and big betting pools.

So we WANT to hit it. And therefore we WANT to believe there are statistical angles that will run us a profit. And therefore, we will do whatever we can to try and create them. Even though in the end, there's no way to know if ANY of these angles are real-- there's just too few Derbies and too much variance.

If you think about it for half a second, the way to make a reliable profit on the races is to invest one's intellectual energies into coming up with reliable handicapping angles for races that are run over and over throughout the year. Because with those races, we have a large enough sample size to know that the angles are real, and we also have enough betting opportunities to outrun variance.

In contrast, a Derby "angle" will never have a sufficient sample size, and will also never be playable enough times to outrun variance anyway. You will always just be gambling.

The Derby? The rational course for a smart and disciplined bettor would probably be to pass the race completely, because if one is to be intellectually honest, one doesn't have any assurance that any handicapping angles are going to be sufficiently reliable to profit at above the level of the takeout.

We bet the Derby because we are indisciplined, because we have egos, because we are gamblers. And this whole conversation is the window dressing we put on it.

I don't know about your "we", but I bet the TC races and the BC races because there is extreme profit potential, life changing potentials. These races may represent very small sample sizes, but that doesn't mean they are unplayable, or should be avoided by handicappers, regardless of their edge or skill. I don't bet the Derby because it's the "Derby", I bet it because many years they are chaos races, which is right up my alley, and the pools are huge, which as a superfecta player is extremely important. Your hit percentage in pools that size doesn't have to be as high as everyday races, because the frequent large payouts more than make up for the lower hit rate. But, many years I pass some of these big races because I can't find the value I am looking for.

[Helles]

I would interested to see The preakness picks

I'll run through the exercise for the Preakness. The Belmont too if I have time. I am out of town until the day before the Belmont.

Thank you Mikesal57, Whodoyoulike and Raybo for your kind words. Thanks again also to Hoofless Wonder, Texas Dolly and Saratoga Mike.

Thank you also to Flatstats, Deltalover, Thaskalos, dilanesp, et al for your interesting contributions. I continue to marvel at the intellect displayed by so many of the "heavy hitters" on this site.

One more question: Where was Dave Schwartz in all of this? lol I do use HSH after all!

[jk3521]

I'll run through the exercise for the Preakness. The Belmont too if I have time. I am out of town until the day before the Belmont.

Thank you Mikesal57, Whodoyoulike and Raybo for your kind words. Thanks again also to Hoofless Wonder, Texas Dolly and Saratoga Mike.

Thank you also to Flatstats, Deltalover, Thaskalos, dilanesp, et al for your interesting contributions. I continue to marvel at the intellect displayed by so many of the "heavy hitters" on this site.

One more question: Where was Dave Schwartz in all of this? lol I do use HSH after all!
Dave could be mining your results with his larger databases.

[porkchop]

I am just trying to save you time and money Helles.

I can assure you I have seen hundreds of people do it and ultimately they all fail. This would be rookie handicappers to pundits to high rollers. They all think they have the golden goose but it's an illusion.

I have data on thousands of systems that were created by various people. Where backfitting took place a typical pattern emerges.

1. The initial results are bland.
2. The rules are tweaked to make the results look better
3. The system is run and bets are placed. Some win straight away, some don't, some give up at this stage (this is a whole new subject on Paper Trading)
4. After a few days the results are checked and the rules tweaked to try and keep it afloat
5. Go back to 4 a few times
6. Ultimately give up

What those handicappers failed to do is to use the basic techniques of checking the sample size and chi square figure. They then failed to sandbox the systems and check the live results after a period of time. If they did that they would have not wasted their time on illogical handicapping. They would have been more focused on what works and what would be more likely to work in the future.

I can not emphasise enough the point that backfitters will do all they can to protect their puppy. They spend so much time on it that they will never believe it will fail. They will do all they can to keep it afloat, whilst all around them they are sinking.

save us all time and money and keep your methods and negativity to your self I think it was great that he put time and effort into his post and guess what, winner / exata / triple lets hope he moves on from here and doesn't listen to the genius of flatstats blah blah blah

[castaway01]

I wanted to post that Arlington race before it went off.

I was catching up on this thread when I decided to look at some other algorithms I had built for specific tracks, surfaces and distances. Most of these have many classes thrown in together to get a decent sample size.

I am going to post the picks and results from AP today. Small sample. Let's see how it worked.

Since you're posting picks for other races now, are these the same 48 factors you used for the Derby; different sets of factors each time; one set of factors for each track, or something else?

[thaskalos]

I'll run through the exercise for the Preakness. The Belmont too if I have time. I am out of town until the day before the Belmont.

Thank you Mikesal57, Whodoyoulike and Raybo for your kind words. Thanks again also to Hoofless Wonder, Texas Dolly and Saratoga Mike.

Thank you also to Flatstats, Deltalover, Thaskalos, dilanesp, et al for your interesting contributions. I continue to marvel at the intellect displayed by so many of the "heavy hitters" on this site.

One more question: Where was Dave Schwartz in all of this? lol I do use HSH after all!
I couldn't help but notice that you have included my name among the more critical, more "interesting", of the "heavy-hitting intellectuals" here. :) I hope that you have taken no offense at anything I have said...

[dilanesp]

I don't know about your "we", but I bet the TC races and the BC races because there is extreme profit potential, life changing potentials. These races may represent very small sample sizes, but that doesn't mean they are unplayable, or should be avoided by handicappers, regardless of their edge or skill. I don't bet the Derby because it's the "Derby", I bet it because many years they are chaos races, which is right up my alley, and the pools are huge, which as a superfecta player is extremely important. Your hit percentage in pools that size doesn't have to be as high as everyday races, because the frequent large payouts more than make up for the lower hit rate. But, many years I pass some of these big races because I can't find the value I am looking for.

There is extreme life changing profit potential in the lottery too!

And the term "unplayable" is vague. You can play anything if you want gambling action.

But if your definition is "making bets I can demonstrate generate sufficient EV to exceed takeout", the Derby doesn't meet that.

[dilanesp]

save us all time and money and keep your methods and negativity to your self I think it was great that he put time and effort into his post and guess what, winner / exata / triple lets hope he moves on from here and doesn't listen to the genius of flatstats blah blah blah

BTW, I do agree it's great he does the work and posts it. Don't confuse skepticism with disrespect.

[flatstats]

save us all time and money and keep your methods and negativity to your self I think it was great that he put time and effort into his post and guess what, winner / exata / triple lets hope he moves on from here and doesn't listen to the genius of flatstats blah blah blah

Do you think I have not put time and effort into this?

I can assure you those methods do not work. I did them in the 80s, saw others do it in the 90s, and it is exasperating to see handicappers still using these flawed techniques today.

[porkchop]

Do you think I have not put time and effort into this?

I can assure you those methods do not work. I did them in the 80s, saw others do it in the 90s, and it is exasperating to see handicappers still using these flawed techniques today.
so if you can't get the method to work no one can,don't think maybe the 48 items he used might have been overlooked in the flatstat paradigm that has become a legend in the industry

[raybo]

Do you think I have not put time and effort into this?

I can assure you those methods do not work. I did them in the 80s, saw others do it in the 90s, and it is exasperating to see handicappers still using these flawed techniques today.

I doubt you did exactly what the OP is doing. To "dis" a method without knowing exactly what is being done is both wrong, and unintelligent (as well as being extremely biased).

[flatstats]

If someone says they are backfitting then they are backfitting and IMO backfitting is not how you should handicap.

I don't know what the OP did to create his method. I was just pointing out the futility and sheer waste of time backfitting is. It is a thief of time, it does make someone protective of their method, it does ultimately fail.

[flatstats]

so if you can't get the method to work no one can,don't think maybe the 48 items he used might have been overlooked in the flatstat paradigm that has become a legend in the industry

No that is not what I am saying. There are many methods that work. What does not work is backfitting and in the sense of backfitting being massaging data to make the results good and expecting the results to continue in the future.

Certainly examine past results and use dozens of variables. But only use those variables if you can justify why they should be used.

[jk3521]

If someone says they are backfitting then they are backfitting and IMO backfitting is not how you should handicap.

I don't know what the OP did to create his method. I was just pointing out the futility and sheer waste of time backfitting is. It is a thief of time, it does make someone protective of their method, it does ultimately fail.

Ferchristsakes ! Let the guy have his fun, cash a bunch of tickets and stop preaching to him that he ultimately will become a failure if he keeps on the same path.. Jeez!!!

[Helles]

Since you're posting picks for other races now, are these the same 48 factors you used for the Derby; different sets of factors each time; one set of factors for each track, or something else?

This is something I played around with last year about this time. The factors for each track are different. Different factors and even different number of factors. The way I build those analysts has evolved since last year, but I really haven't done anything with them since last year. Until I decided to take a shot at the Derby that is.

[Helles]

I couldn't help but notice that you have included my name among the more critical, more "interesting", of the "heavy-hitting intellectuals" here. :) I hope that you have taken no offense at anything I have said...

I didn't take offense at anything you said. I met you in Chicago a few years back at AP. I respect your opinions and welcomed your well-meant comments.

[raybo]

Now that we know the OP is using the HSH program, maybe those who are hollering 'backfit', including the OP, might want to back off that terminology a bit. I don't have the HSH program, but I have read about it and Dave Schwartz can explain what "analysts" are in his program and how they work. I doubt their use could be put in the traditional category of 'backfitting' results.

[Helles]

Now that we know the OP is using the HSH program, maybe those who are hollering 'backfit', including the OP, might want to back off that terminology a bit. I don't have the HSH program, but I have read about it and Dave Schwartz can explain what "analysts" are in his program and how they work. I doubt their use could be put in the traditional category of 'backfitting' results.

I started with the data from the last 10 Derbys. I took a library of factors and played them back against those races. Paring out the factors that clearly weren't predictive of THOSE 10 RACES, and starting over again. Whittling down until I got to 48 factors from the 900 I started with. 48 factors weighted in a way to find the winners of those 10 races.

Compare this to something else one can do with HSH, New Pace for example. One starts with the New Pace idea that the field is divided into Early and Late horses. Different sets of factors are used to identify the Early and the Late horses, usually two of each. The user then uses other sets of factors to decide which ones to bet.

It seems the former is more akin to backfitting than the latter. Both possible with HSH. Among others.

[whodoyoulike]

I don't know the reason for your results but, I hope you provide an update after the Derby is run (referencing this post #1). Let us know whether you made a wager and the type of wager(s) using your contenders....

One of the biggest problems I have with "backfitting" is usually after the fact, people would say I would've bet the exacta straight or wheeled the top with the next three contenders etc. Until you have a good understanding of why and how the contender selections are made, you should be cautious with betting. I think this is what the others are attempting to warn you about regarding "backfitting".

Which is the reason I asked how you bet the race because people think differently whenever they don't have any "skin" in the game. It's the money anxiety or adrenaline rush which affects one's game but, there are those where these things don't influence them at all (these people IMO don't care about going bankrupt).

[whodoyoulike]

I vividly remember years ago, I got lost and was wandering for a long time near the Salton Sea in the nearby desert. I saw the lodge on the other shore and I needed to find some way to cross over.

I found a narrow and shallow part and I walked on large boulders which were just under the water until I was able to reach the other side.

There was a father and his young son walking on the other side and the son said to me, “it looked like you were walking on water”.

I said bless you my son and then went to the lodge to eat something because it had to be at least 4 – 5 hours since breakfast.

What I’m trying to say is that “backfitting” can sometimes just be an illusion.

[ReplayRandall]

You're walking a thin-line there, Whodo. I would highly suggest you use a different metaphor in the future, lest the MAN upstairs "backfits" you permanently........Then again, I'm merely being facetious.

[raybo]

I started with the data from the last 10 Derbys. I took a library of factors and played them back against those races. Paring out the factors that clearly weren't predictive of THOSE 10 RACES, and starting over again. Whittling down until I got to 48 factors from the 900 I started with. 48 factors weighted in a way to find the winners of those 10 races.

Compare this to something else one can do with HSH, New Pace for example. One starts with the New Pace idea that the field is divided into Early and Late horses. Different sets of factors are used to identify the Early and the Late horses, usually two of each. The user then uses other sets of factors to decide which ones to bet.

It seems the former is more akin to backfitting than the latter. Both possible with HSH. Among others.

Most handicappers attempt to find out what factors are most predictive in specific race types and at specific tracks, me included. But, that doesn't necessarily mean that they are 'backfitting' results in the traditional sense. It merely means finding what factors are important and what factors are not. In a race like the Derby, those factors can be very important as it is a specific race type that occurs rarely, but always at the same track. This kind of 'backfitting' could very well work for the Derby, at least for the near future, unless something dramatic happens, like changing the track's surface, or similar. How you get there isn't as important, in such cases, as getting there. And, it appears that you got there this year, in fine fashion!

[Dave Schwartz]

I've read a couple of pages of this but not all.

It never ceases to amaze me. Again and again. A guy tells people what he is doing... and whether it works or not, he gets criticized.

Even more amazing is how many people who are (probably) not doing well at the window will tell someone else all the things wrong with their approach.

Is his approach working? That is a good question, but not really too important to this conversation.


What Helles did was utilize a built-in A.I. feature in our software, HSH (http://store.thehorsehandicappingauthority.com/products/The-HorseStreet-Handicapper.html). That feature is called "Swarms," my version of what Swarm Technology should be. Unlike the computer industry's definition for "swarm intelligence," ours is a true "hive-based" approach.

It works like this:

1. You select races from your database to use.

2. You select factors from our factor list. (Roughly about 3,000 available factors for each horse.)

3. You set some training parameters and turn them loose.

Before going on, the recommended approach is to use a "substantial" number of races, as opposed to 10. These guys ("ants" as I called them) are smart enough to really optimize and backfit to a small sample.

For my on-going example, let's assume that you had a 6,000-race database of somewhat similar races. Logically, you would split them into three samples:

*Training Sample
*Testing Sample
*Final Testing Sample


Everyone is accepting of the fact that the public does an excellent job of handicapping the races. The do this using a "Wisdom of the Crowd" approach. That is, horse players represent the ultimate prediction market.

What we have accomplished is to create an artificial prediction market.

We create the "ants," give them a bankroll, and let them bet into their own pool. After some amount of training, the pool stablizes (i.e. the ant hill or "hive" has matured). At this point it is safe to assume that the percentage of pool wagered on a particular horse is roughly equivalent to its probability of winning.


[B]The Training Process
There are several strategies but the basic one goes like this:

1) Generate 500 ants at random.
Each ant
*is a small system, using from 1 to 6 factors, weighted.
*starts with a $1,000 bankroll.

2) Draw (say) 200 races from the training database at random for a "tournament."

3) Each ant handicaps and plays each race, betting according to his projected hit rate RANK on each of the horses. Thus, he will wager more on #1 than #2, more on #2 than #3, etc. He will play as deep as you tell him to play (i.e. 3-deep, 6-deep, 9-deep, etc.)

4) After each race is handicapped, the winner of the race is determined using a simple but robust (a computer word) algorithm that uses a randomness component based upon lengths behind. Thus, if a horse won by 10 lengths, he will always win the race, but if he only won by a nose, he will share the wins with the 2nd horse.

5) Once the winner of the race is determined, the pool (comprised of money wagered by all the ants) is analyzed and the proper payoff is computed. There is no takeout or breakage (both are programmable).

6) All the winning tickets are paid and winning ant bankrolls are debited with winnings.

7) When an ant runs out of money he no longer bets.

8) This process repeats itself until the tournament of (200) races is over.

9) At the end of a tournament, ants without money are killed and replaced by new random ants.
(Note: There are other training strategies.)

10) Repeat steps #2 thru #9 until some goal has been reached.



The goal (step 10) could one of several:

A) To train a certain number of passes. The goal is for the ants to stablize. As the average mutuel price goes lower, the ants are agreeing more on who the best horses are.

B) Do not replace the ants that run out of money, but rather get it down to a small, finite number. Normally in this option one would begin with a very large number of ants Like 10,000 (or 50,000) and train down to a few hundred.

There are other goals/strategies, but these are primary.

There are some other bells and whistles that are pretty snazzy.

What Helles did was to periodically poll the ants looking at the factors used, how they were weighted and the relative success of each ant. These factors were sorted and he culled out the ones that were not being used very much. Then, with this new, smaller list of factors, he began the entire process again.

Of course, with a small number of races the process probably did not take very long.


Does this process work? Like with all handicapping processes, they work for some people. The approach has a lot of potential. Probably needs a version 2.0 to realize its real potential.



Regards,
Dave Schwartz

[raybo]

I've read a couple of pages of this but not all.

It never ceases to amaze me. Again and again. A guy tells people what he is doing... and whether it works or not, he gets criticized.

Even more amazing is how many people who are (probably) not doing well at the window will tell someone else all the things wrong with their approach.

Is his approach working? That is a good question, but not really too important to this conversation.


What Helles did was utilize a built-in A.I. feature in our software, HSH (http://store.thehorsehandicappingauthority.com/products/The-HorseStreet-Handicapper.html). That feature is called "Swarms," my version of what Swarm Technology should be. Unlike the computer industry's definition for "swarm intelligence," ours is a true "hive-based" approach.

It works like this:

1. You select races from your database to use.

2. You select factors from our factor list. (Roughly about 3,000 available factors for each horse.)

3. You set some training parameters and turn them loose.

Before going on, the recommended approach is to use a "substantial" number of races, as opposed to 10. These guys ("ants" as I called them) are smart enough to really optimize and backfit to a small sample.

For my on-going example, let's assume that you had a 6,000-race database of somewhat similar races. Logically, you would split them into three samples:

*Training Sample
*Testing Sample
*Final Testing Sample


Everyone is accepting of the fact that the public does an excellent job of handicapping the races. The do this using a "Wisdom of the Crowd" approach. That is, horse players represent the ultimate prediction market.

What we have accomplished is to create an artificial prediction market.

We create the "ants," give them a bankroll, and let them bet into their own pool. After some amount of training, the pool stablizes (i.e. the ant hill or "hive" has matured). At this point it is safe to assume that the percentage of pool wagered on a particular horse is roughly equivalent to its probability of winning.


[B]The Training Process
There are several strategies but the basic one goes like this:

1) Generate 500 ants at random.
Each ant
*is a small system, using from 1 to 6 factors, weighted.
*starts with a $1,000 bankroll.

2) Draw (say) 200 races from the training database at random for a "tournament."

3) Each ant handicaps and plays each race, betting according to his projected hit rate RANK on each of the horses. Thus, he will wager more on #1 than #2, more on #2 than #3, etc. He will play as deep as you tell him to play (i.e. 3-deep, 6-deep, 9-deep, etc.)

4) After each race is handicapped, the winner of the race is determined using a simple but robust (a computer word) algorithm that uses a randomness component based upon lengths behind. Thus, if a horse won by 10 lengths, he will always win the race, but if he only won by a nose, he will share the wins with the 2nd horse.

5) Once the winner of the race is determined, the pool (comprised of money wagered by all the ants) is analyzed and the proper payoff is computed. There is no takeout or breakage (both are programmable).

6) All the winning tickets are paid and winning ant bankrolls are debited with winnings.

7) When an ant runs out of money he no longer bets.

8) This process repeats itself until the tournament of (200) races is over.

9) At the end of a tournament, ants without money are killed and replaced by new random ants.
(Note: There are other training strategies.)

10) Repeat steps #2 thru #9 until some goal has been reached.



The goal (step 10) could one of several:

A) To train a certain number of passes. The goal is for the ants to stablize. As the average mutuel price goes lower, the ants are agreeing more on who the best horses are.

B) Do not replace the ants that run out of money, but rather get it down to a small, finite number. Normally in this option one would begin with a very large number of ants Like 10,000 (or 50,000) and train down to a few hundred.

There are other goals/strategies, but these are primary.

There are some other bells and whistles that are pretty snazzy.

What Helles did was to periodically poll the ants looking at the factors used, how they were weighted and the relative success of each ant. These factors were sorted and he culled out the ones that were not being used very much. Then, with this new, smaller list of factors, he began the entire process again.

Of course, with a small number of races the process probably did not take very long.


Does this process work? Like with all handicapping processes, they work for some people. The approach has a lot of potential. Probably needs a version 2.0 to realize its real potential.



Regards,
Dave Schwartz

Dave,

I'm glad you jumped in here and explained the "swarm"/"ants" process. I knew the basics from reading about your program, but not the specifics. While it does "backfit" results, generally, it still gets at the crux of the matter, that being 'what is important in these types of races?'. The Derby, because it is probably the most unique race we as handicappers, and the horses, jockeys, and trainers, will ever face, necessarily will force the use of small samples. But, because it is so unique, this approach is probably as good as most, if not any other approach we as handicappers might utilize.

Thanks for the entertaining and informative explanation, I'm sure it went a long way in clarifying the OP's method, and somewhat explains the method's success in the last several Derbies. Will it continue? Possibly for the very near future, but by continually retraining the 'ants', after adding new Derbies to the sample, it would not be such a large leap to expect that it might do well, as well or better than most other methods handicappers use, in future Derbies. :ThmbUp: :ThmbUp:

ps: Am I correct that you use GAs rather than NNs in your AI process?

[flatstats]

That's a great explanation Dave.

I did not know what process the OP was using. To me I read it as "backfitting" and "whilttling", which IMO is so wrong - even if just for fun.

Of course there is still the question of how 900 factors are reduced to 48.

If using something like R-Stats then you can see the figures, which indicate if what is observed is reliable to use or not. It is then upto to the individual to take these results and choose to include the rules. The sensible approach would be to listen to what the data is telling the user, rather than trying to make the data fit to what the user wants it to be, which is the classic backfitting mistake.

[Helles]

That's a great explanation Dave.

I did not know what process the OP was using. To me I read it as "backfitting" and "whilttling", which IMO is so wrong - even if just for fun.

Of course there is still the question of how 900 factors are reduced to 48.

If using something like R-Stats then you can see the figures, which indicate if what is observed is reliable to use or not. It is then upto to the individual to take these results and choose to include the rules. The sensible approach would be to listen to what the data is telling the user, rather than trying to make the data fit to what the user wants it to be, which is the classic backfitting mistake.

I can interrupt the process at any time and see to what effect each individual factor is being used. Then I commence ta whittlin'. Dave explained that in his post.

[Dave Schwartz]

ps: Am I correct that you use GAs rather than NNs in your AI process?

No, the engine is just as I defined it: an Artificial Prediction Market (http://en.wikipedia.org/wiki/Prediction_market).


Of course there is still the question of how 900 factors are reduced to 48.

To truly understand what this prediction market does, considering a real world example.

Imagine we want to build a team of handicappers. So, we put an ad in the DRF and get thousands of applicants. Now, understand, we are not looking for a single good handicapper, but rather a cross-section of handicappers that, unbeknownst to them, are working together.

That is, they work independently, each with their own system, totally ignoring the input of others. Then, their picks are processed together for conclusions to be drawn.

So, we have this (literally) endless line of handicappers at our door. We usher in a few hundred of them and turn them loose on a few hundred races, and track the results, i.e. how their bankrolls grow.

As some of them are (effectively) dominated by their competitors and run out of money, we let a few new ones in the front door for future tournaments.

(BTW, there is a natural bias towards the guys at the front of the line because, simply by having survived longer they would probably have more money. Therefore, the bankrolls are actually a 20-tournament moving average to give the newer guys a chance to catch up. The last step in the process is generally to reset the bankrolls to equal and run a single pass against a large sample of races to make the weightings "fair.")


At some point we call a halt and examine the "value" of each factor.

Starting with the handicapper who has the most money, we look at the factors he is using and tally the weights of each factor times his bankroll.

We do this with each subsequent handicapper and his factors, eventually building a scoresheet with totals used for each factor. By sorting that scoresheet by the totals, we can tell which factors are being used the most and which are relatively insignificant.

The insignificant ones are removed and the entire process is repeated.



If using something like R-Stats then you can see the figures, which indicate if what is observed is reliable to use or not. It is then upto to the individual to take these results and choose to include the rules. The sensible approach would be to listen to what the data is telling the user, rather than trying to make the data fit to what the user wants it to be, which is the classic backfitting mistake.

In our example, the scoresheet would represent the underlined portion.

Of course, in our prediction market approach, there is no need for the user to do any of that.


There is a natural tendency for the user to want to use the "king" ant. This is not a good idea.

You see, the best ant is not the best because he has the BEST system. He is the best because he has a GOOD and his system is unique enough amongst his competitors that he does not have to split the pools too many ways.

IOW, the top ant is using somewhat uncommon factors!

It is the synergy of the "collective" (as in, "Welcome to the Borg") that makes the hill profitable against the real world.


In its simplicity, the collective is actually a powerful A.I. engine. It is somewhat similar to an neural network, but without the "perfect" backfitting issues natural to neural nets.

Please note that this is 100% my own invention.

[dilanesp]

Part of the problem is this is a discussion between some very specific hard workers (who again, I admire even if I disagree with them) who are doing things that in their mind should give rise to profitable betting opportunities and some people who are aware of some very general truths about gambling, and quasi-gambling enterprises such as the stock market.

And I don't think there's any middle ground here.

Statistical principles say you can't be sure what you are doing is actually working, and that some set of "systems" will "work" for a period of time purely do the effects of random variance, but that as statistical significance thresholds are approached, the results will regress back to the mean.

And there are a history of such systems. Probably not so many with respect to the Derby (although there have been a few), but certainly a ton of them with respect to the stock market, for instance.

I know with respect to the Derby ones like Dosage which worked for awhile and then regressed to the mean, you guys look at them and say "well mine is much more sophisticated", which is clearly true, but that's not relevant to the statistical problem.. If there's too much variance and you have an insignificant sample size, it really doesn't matter if your modeling is sophisticated and computerized or not. You simply don't have enough results to test your hypotheses, whether the hypotheses are simplistic or sophisticated.

And when you get away from the Derby and look at the stock market, well, the "technical school" of stock market investment strategies has generated all sorts of HIGHLY sophisticated models. And many of them have done the same thing-- worked for awhile, and then regressed to the mean. So what I am saying about sophistication of the model has been borne out in other circumstances.

Two other points here:

1. The long run is very long. The reality is because there's only one Derby a year, a system could actually succeed for a long, long time without actually being +EV. I suspect this is very hard for people to wrap their heads around. But even, say, profits over 35 years might not be a statistically significant result. The sample size is just tiny.

2. The fact that you are cashing tickets is, of course, great. But it isn't really proof of anything. One of the amazing things about gambling is that someone who is completely right can lose money and someone who is completely wrong can make a killing. I actually know people who were throwing out non-contenders on Dosage in the 1980's and making money on horses like Ferdinand. It didn't matter to their short term bankrolls at all that Dosage was a completely bogus system. It was working and they were making money. Gambling is funny in that respect.

[DeltaLover]

I've read a couple of pages of this but not all.

It never ceases to amaze me. Again and again. A guy tells people what he is doing... and whether it works or not, he gets criticized.

Even more amazing is how many people who are (probably) not doing well at the window will tell someone else all the things wrong with their approach.

Is his approach working? That is a good question, but not really too important to this conversation.


What Helles did was utilize a built-in A.I. feature in our software, HSH (http://store.thehorsehandicappingauthority.com/products/The-HorseStreet-Handicapper.html). That feature is called "Swarms," my version of what Swarm Technology should be. Unlike the computer industry's definition for "swarm intelligence," ours is a true "hive-based" approach.

It works like this:

1. You select races from your database to use.

2. You select factors from our factor list. (Roughly about 3,000 available factors for each horse.)

3. You set some training parameters and turn them loose.

Before going on, the recommended approach is to use a "substantial" number of races, as opposed to 10. These guys ("ants" as I called them) are smart enough to really optimize and backfit to a small sample.

For my on-going example, let's assume that you had a 6,000-race database of somewhat similar races. Logically, you would split them into three samples:

*Training Sample
*Testing Sample
*Final Testing Sample


Everyone is accepting of the fact that the public does an excellent job of handicapping the races. The do this using a "Wisdom of the Crowd" approach. That is, horse players represent the ultimate prediction market.

What we have accomplished is to create an artificial prediction market.

We create the "ants," give them a bankroll, and let them bet into their own pool. After some amount of training, the pool stablizes (i.e. the ant hill or "hive" has matured). At this point it is safe to assume that the percentage of pool wagered on a particular horse is roughly equivalent to its probability of winning.


[B]The Training Process
There are several strategies but the basic one goes like this:

1) Generate 500 ants at random.
Each ant
*is a small system, using from 1 to 6 factors, weighted.
*starts with a $1,000 bankroll.

2) Draw (say) 200 races from the training database at random for a "tournament."

3) Each ant handicaps and plays each race, betting according to his projected hit rate RANK on each of the horses. Thus, he will wager more on #1 than #2, more on #2 than #3, etc. He will play as deep as you tell him to play (i.e. 3-deep, 6-deep, 9-deep, etc.)

4) After each race is handicapped, the winner of the race is determined using a simple but robust (a computer word) algorithm that uses a randomness component based upon lengths behind. Thus, if a horse won by 10 lengths, he will always win the race, but if he only won by a nose, he will share the wins with the 2nd horse.

5) Once the winner of the race is determined, the pool (comprised of money wagered by all the ants) is analyzed and the proper payoff is computed. There is no takeout or breakage (both are programmable).

6) All the winning tickets are paid and winning ant bankrolls are debited with winnings.

7) When an ant runs out of money he no longer bets.

8) This process repeats itself until the tournament of (200) races is over.

9) At the end of a tournament, ants without money are killed and replaced by new random ants.
(Note: There are other training strategies.)

10) Repeat steps #2 thru #9 until some goal has been reached.



The goal (step 10) could one of several:

A) To train a certain number of passes. The goal is for the ants to stablize. As the average mutuel price goes lower, the ants are agreeing more on who the best horses are.

B) Do not replace the ants that run out of money, but rather get it down to a small, finite number. Normally in this option one would begin with a very large number of ants Like 10,000 (or 50,000) and train down to a few hundred.

There are other goals/strategies, but these are primary.

There are some other bells and whistles that are pretty snazzy.

What Helles did was to periodically poll the ants looking at the factors used, how they were weighted and the relative success of each ant. These factors were sorted and he culled out the ones that were not being used very much. Then, with this new, smaller list of factors, he began the entire process again.

Of course, with a small number of races the process probably did not take very long.


Does this process work? Like with all handicapping processes, they work for some people. The approach has a lot of potential. Probably needs a version 2.0 to realize its real potential.



Regards,
Dave Schwartz

:ThmbUp:

One of the best postings since very long time.

Clearly this is methodology is a very interesting exercise that certainly will improve your understanding of the game..

To simplify things, I think that what Dave is saying here, is that he is using genetic programming to detect the optimal finite state machine, expressed as a decision tree of binary nodes, each of them representing a specific handicapping factor having leafs of BET / PASS decision.

I have implemented several such systems in the past, the best of them was using a hand crafted version of a LISP descendant, that was serving a DSL, specializing in handling nominal, scalar and binary values, using various fitness functions (based in PNL, ROI, IV or winning percentage)...

I have to admit that none of my approaches towards this direction, was able to discover a long term viable solution optimized for profitability..

More than this though, I was able to generate state machines that were mimicking (or even beating in some cases) the crowd's consensus very well.

Today, I am using a completely different approach though... Still applying AI, but not GP or NN any more..

[raybo]

:ThmbUp:

More than this though, I was able to generate state machines that were mimicking (or even beating in some cases) the crowd's consensus very well.



This alone, would be valuable, IMO. If we can accurately predict the live wagering, we have less concern as to how the late money will be bet. That means that we can more accurately assign underlays and overlays, meaning our "value" seeking will be more accurate, and thus, more profitable long term.

[DeltaLover]

]That means that we can more accurately assign underlays and overlays, meaning our "value" seeking will be more accurate, and thus, more profitable long term[/SIZE].

:ThmbUp:

Excellent comment! The most successful systems I have ever created are based on a very similar concept...

[raybo]

:ThmbUp:

Excellent comment! The most successful systems I have ever created are based on a very similar concept...

If we consider that the final odds are a composite of all the players in the pool, then a system like Dave's (and the OP's), that uses a composite of all the "artificial" players in the pool, would seem to mimic the public's actions, and probably could be used successfully, for that reason alone.

[DeltaLover]

If we consider that the final odds are a composite of all the players in the pool, then a system like Dave's (and the OP's), that uses a composite of all the "artificial" players in the pool, would seem to mimic the public's actions, and probably could be used successfully, for that reason alone.

In my opinion, this is very close to the solution of the puzzle! Everything else, like differential equations, elementary physics and wind measurements are just insignificant details and nothing more!

[Dave Schwartz]

Delta,

Not genetic programming.

Although I can understand why you would see that as similar, it is really not even close.


Dave

[thaskalos]

:ThmbUp:

Excellent comment! The most successful systems I have ever created are based on a very similar concept...

These "most successful systems"...do you have any for sale? For a friend, I mean... :)

[DeltaLover]

These "most successful systems"...do you have any for sale? For a friend, I mean... :)

Not for sale, but i can give you access in a second if u want to learn more about it... just let me know.

[Hoofless_Wonder]

Part of the problem is this is a discussion between some very specific hard workers (who again, I admire even if I disagree with them) who are doing things that in their mind should give rise to profitable betting opportunities and some people who are aware of some very general truths about gambling, and quasi-gambling enterprises such as the stock market.

And I don't think there's any middle ground here.

.........

2. The fact that you are cashing tickets is, of course, great. But it isn't really proof of anything. One of the amazing things about gambling is that someone who is completely right can lose money and someone who is completely wrong can make a killing. I actually know people who were throwing out non-contenders on Dosage in the 1980's and making money on horses like Ferdinand. It didn't matter to their short term bankrolls at all that Dosage was a completely bogus system. It was working and they were making money. Gambling is funny in that respect.

So what I don't get is that the "stats" guys are always quick to point out the flaws of a successful approach to predicting future events with respect to statistical principles, but rarely offer useful alternatives. An assumption is made that the basis of the system must be flawed, and cannot be validated because the sample size is still within the bounds of random variance. The problem we have is that by the time valid numbers are available, the human species is extinct.

This is what I meant by horse racing not fitting neatly into the box of laws governing statistics. Statistics cannot be applied to everything, at least pragmatically, in the universe. An example might be the models that predict hurricane tracks. While getting more accurate over time, there looks to be far fewer hurricane events than what's needed to validate any of the models. That doesn't mean some aren't useful - it just means that there are limitations to applying models, and large statistical models, to predicting storm tracks. Unless you believe that eventually we'll have perfect hurricane modeling.

Contrary to your viewpoint, cashing tickets is proof of everything in this game. Maybe an approach is valid, or it's just plain lucky, but as long as a method is showing profits, it's worth sticking to - statistical confirmation be damned. What I believe Helles has come up with is not backfitting, but more of defining a model.

And I don't believe dosage was ever shown to be a bogus system. Granted, it was backfitting to quite a degree, but the reasoning behind the "system" was sound, with of course the exception of not having Alydar on the chef's list until after Strike The Gold won the Derby. What I believe happened was that the decline of the usefulness of dosage predicting the Derby winner happened to coincide with the rise of drug use in the industry. Breeding stats, as related to distance limitations in general have declined in worth in North America, yet continue to be quite relevant overseas where race day medication is banned.

[raybo]

About the only stats I care about are: cost of playing versus total payouts, and starting bank versus ending bank. Both of those quarterly, for entire individual track meets, and yearly. Hit rates are indicative of consistency but not overly important except in relation to average payouts. All the other stats mean very little to me.

Net profit in the bank is prime, regardless of how it is achieved.

[dilanesp]

So what I don't get is that the "stats" guys are always quick to point out the flaws of a successful approach to predicting future events with respect to statistical principles, but rarely offer useful alternatives. An assumption is made that the basis of the system must be flawed, and cannot be validated because the sample size is still within the bounds of random variance. The problem we have is that by the time valid numbers are available, the human species is extinct.

This is what I meant by horse racing not fitting neatly into the box of laws governing statistics. Statistics cannot be applied to everything, at least pragmatically, in the universe. An example might be the models that predict hurricane tracks. While getting more accurate over time, there looks to be far fewer hurricane events than what's needed to validate any of the models. That doesn't mean some aren't useful - it just means that there are limitations to applying models, and large statistical models, to predicting storm tracks. Unless you believe that eventually we'll have perfect hurricane modeling.

Contrary to your viewpoint, cashing tickets is proof of everything in this game. Maybe an approach is valid, or it's just plain lucky, but as long as a method is showing profits, it's worth sticking to - statistical confirmation be damned. What I believe Helles has come up with is not backfitting, but more of defining a model.

And I don't believe dosage was ever shown to be a bogus system. Granted, it was backfitting to quite a degree, but the reasoning behind the "system" was sound, with of course the exception of not having Alydar on the chef's list until after Strike The Gold won the Derby. What I believe happened was that the decline of the usefulness of dosage predicting the Derby winner happened to coincide with the rise of drug use in the industry. Breeding stats, as related to distance limitations in general have declined in worth in North America, yet continue to be quite relevant overseas where race day medication is banned.

For the record, my answer is I think people shouldn't bet for the purpose of making a long term profit in situations where they cannot establish via statistically valid means that their bets are +EV.

Now, that doesn't mean they cannot gamble, so long as they are honest with themselves that this is what they are doing and that the purpose is fun and the hope of luck, not positive expectation.

So a race like the Derby, which is a one-off with too small a sample size, the answer is if you want to put some bets on it for fun or to try and make a lottery-style score, go ahead. All in good fun. But don't have any pretensions that you can know that what you are doing is +EV.

But if you are trying to make a long term profit in the game, the Derby should be passed.

Your post implies (though you never quite say) that if there is no statistically justifiable reason for a bet, one should bet a race anyway. But it's precisely the opposite-- if you can't establish that what you are doing is +EV, you should pass and wait for a situation that can be established to be +EV. Betting in situations where one cannot establish positive expectations can cut your overall winrate and is at best a form of pure gambling.

[raybo]

For the record, my answer is I think people shouldn't bet for the purpose of making a long term profit in situations where they cannot establish via statistically valid means that their bets are +EV.

Now, that doesn't mean they cannot gamble, so long as they are honest with themselves that this is what they are doing and that the purpose is fun and the hope of luck, not positive expectation.

So a race like the Derby, which is a one-off with too small a sample size, the answer is if you want to put some bets on it for fun or to try and make a lottery-style score, go ahead. All in good fun. But don't have any pretensions that you can know that what you are doing is +EV.

But if you are trying to make a long term profit in the game, the Derby should be passed.

Your post implies (though you never quite say) that if there is no statistically justifiable reason for a bet, one should bet a race anyway. But it's precisely the opposite-- if you can't establish that what you are doing is +EV, you should pass and wait for a situation that can be established to be +EV. Betting in situations where one cannot establish positive expectations can cut your overall winrate and is at best a form of pure gambling.

With all the vast variance in racing, I'd be very surprised if any hard core "statistician" here can prove, in real world bets, that they have a +EV over a significant number of wagers, and no, 'significant' does not mean over their entire lifetime either, like many of these stats guys seem to think is the only way one can be considered profitable). Why don't you give us an example of a 'for sure' +EV bet? And don't give me that "Why should I give away my secrets" crap either. Can't, can you? Put up the proof or give us a break and stop running everyone down that has a method that doesn't meet your personal, school taught or conventional business understanding of long term viability. This stuff sure gets old and aggravating when your bankroll and betting records show years of profitable play, without all the stats stuff even being considered.

Profitability boils down to cost of racing versus average payouts, period. If the latter is higher than the former then you are profitable, and when that is the case for several years in a row, you are long term profitable, in my book, regardless of the put downs and negativity the stats guys incessantly try to feed us. Do you really think that someone with several years of profitable play is suddenly going to get dumb and turn back into a novice? Do you really think that those players will not evolve with the game? Do you really think that "luck" got them to that point? On second thought, forget the "do you really think" stuff because you probably really do think that stuff, instead, substitute "can you really prove".

Give us some real world horse racing proof of +EV bets, not textbook lectures, or non-racing correlations. This is horse racing, not black jack or craps, or even the stock market for that matter (although some of the analysis processes are very similar).

[MJC922]

I've read a couple of pages of this but not all.

It never ceases to amaze me. Again and again. A guy tells people what he is doing... and whether it works or not, he gets criticized.

Even more amazing is how many people who are (probably) not doing well at the window will tell someone else all the things wrong with their approach.

Is his approach working? That is a good question, but not really too important to this conversation.


What Helles did was utilize a built-in A.I. feature in our software, HSH (http://store.thehorsehandicappingauthority.com/products/The-HorseStreet-Handicapper.html). That feature is called "Swarms," my version of what Swarm Technology should be. Unlike the computer industry's definition for "swarm intelligence," ours is a true "hive-based" approach.

It works like this:

1. You select races from your database to use.

2. You select factors from our factor list. (Roughly about 3,000 available factors for each horse.)

3. You set some training parameters and turn them loose.

Before going on, the recommended approach is to use a "substantial" number of races, as opposed to 10. These guys ("ants" as I called them) are smart enough to really optimize and backfit to a small sample.

For my on-going example, let's assume that you had a 6,000-race database of somewhat similar races. Logically, you would split them into three samples:

*Training Sample
*Testing Sample
*Final Testing Sample


Everyone is accepting of the fact that the public does an excellent job of handicapping the races. The do this using a "Wisdom of the Crowd" approach. That is, horse players represent the ultimate prediction market.

What we have accomplished is to create an artificial prediction market.

We create the "ants," give them a bankroll, and let them bet into their own pool. After some amount of training, the pool stablizes (i.e. the ant hill or "hive" has matured). At this point it is safe to assume that the percentage of pool wagered on a particular horse is roughly equivalent to its probability of winning.


[B]The Training Process
There are several strategies but the basic one goes like this:

1) Generate 500 ants at random.
Each ant
*is a small system, using from 1 to 6 factors, weighted.
*starts with a $1,000 bankroll.

2) Draw (say) 200 races from the training database at random for a "tournament."

3) Each ant handicaps and plays each race, betting according to his projected hit rate RANK on each of the horses. Thus, he will wager more on #1 than #2, more on #2 than #3, etc. He will play as deep as you tell him to play (i.e. 3-deep, 6-deep, 9-deep, etc.)

4) After each race is handicapped, the winner of the race is determined using a simple but robust (a computer word) algorithm that uses a randomness component based upon lengths behind. Thus, if a horse won by 10 lengths, he will always win the race, but if he only won by a nose, he will share the wins with the 2nd horse.

5) Once the winner of the race is determined, the pool (comprised of money wagered by all the ants) is analyzed and the proper payoff is computed. There is no takeout or breakage (both are programmable).

6) All the winning tickets are paid and winning ant bankrolls are debited with winnings.

7) When an ant runs out of money he no longer bets.

8) This process repeats itself until the tournament of (200) races is over.

9) At the end of a tournament, ants without money are killed and replaced by new random ants.
(Note: There are other training strategies.)

10) Repeat steps #2 thru #9 until some goal has been reached.



The goal (step 10) could one of several:

A) To train a certain number of passes. The goal is for the ants to stablize. As the average mutuel price goes lower, the ants are agreeing more on who the best horses are.

B) Do not replace the ants that run out of money, but rather get it down to a small, finite number. Normally in this option one would begin with a very large number of ants Like 10,000 (or 50,000) and train down to a few hundred.

There are other goals/strategies, but these are primary.

There are some other bells and whistles that are pretty snazzy.

What Helles did was to periodically poll the ants looking at the factors used, how they were weighted and the relative success of each ant. These factors were sorted and he culled out the ones that were not being used very much. Then, with this new, smaller list of factors, he began the entire process again.

Of course, with a small number of races the process probably did not take very long.


Does this process work? Like with all handicapping processes, they work for some people. The approach has a lot of potential. Probably needs a version 2.0 to realize its real potential.



Regards,
Dave Schwartz

Everything starts as a backfit, that's unavoidable and should be expected because it's step one. Step two, your indication of using a 2000 race sample to forward test against will save many people their sanity. I agree with that Rx 100%. There is absolutely nothing more valuable in this process than the completely isolated and unblemished forward test sample. It should never be lumped together into the sample where the back test was done, once someone does that the process breaks down immediately and a new forward test sample would have to be obtained, IMO this has to be an unbreakable rule. Step three, personally I don't use a third 'paper' sample, instead the actual logistics of placing $2 wagers through the ADW dealing with real odds over a period of months is my third sample, after that we know what we have.

[dilanesp]

Why don't you give us an example of a 'for sure' +EV bet?

"For sure" is a slippery meaning. A +EV bet is not "for sure"; it is any bet where there's a statistical basis for making it based on a large enough sample size.

To state one that doesn't give away any secrets whatsoever: in the 1970's, before speed figures became universally available, there were a number of tracks in which a win bet on the horse with the top speed figure showed a flat bet profit over sample sizes involving months and even years worth of races. There is extensive documentation of this.

There are bets like that now, where reams of data establish that a certain situation is profitable. They are usually calculated by computers, and they are of course far more intricate than the speed figure angle was in the 1970's.

Do you really think that someone with several years of profitable play is suddenly going to get dumb and turn back into a novice?

No, I think it's possible to have several years of profitable play without actually making long term +EV plays. And that one thing that happens over time is that people regress back towards the mean (which in horse racing is minus the takeout percentage).

Do you really think that those players will not evolve with the game?

No, I think the fact the game "evolves" (which is true) is part of the variance which makes it that much harder to know if one is a winning player or whether one has simply been fortuitous and has run into a temporary condition. Further, as the game evolve, a particular player's adjustments might turn out to be incorrect.

Do you really think that "luck" got them to that point?

That fundamentally misunderstands statistical significance. One can no more say definitively that positive results were as a result of luck as one can say they were a result of skill.

It is always POSSIBLE that a positive short term result IS a result of skill and will hold up over the long term. We just don't know it yet because we don't have a sample size that will warrant that conclusion.

What I "think" is that people should not conclude even from somewhat long-term favorable results that they are in fact winning players as opposed to getting lucky, unless they have done the statistical spadework.

And that with respect to specific handicapping angles, one should not be quick to assume that they reflect reality as opposed to random variance when the sample sizes are small.

On second thought, forget the "do you really think" stuff because you probably really do think that stuff, instead, substitute "can you really prove".

Everything I have said is basically Statistics 101.

I come from the world of poker. In poker, basically all the long-term winning players understand the statistical principles backwards and forwards; many short-term winners regress to the mean over time.

And as I have said in earlier posts, people who play or predict the stock market have demonstrated similar tendencies to regress to the mean over time.

There isn't much to prove here on my side. Statistical principles which have been demonstrated over and over say that you need a certain quantum of information to conclude that a play is a winning one. Horseplayers sometimes are unable to come up with the information, but still want to believe the play is a winning one so they declare that the laws of statistics do not apply to them. I know which side of the argument persuades me; you do not, however, have to be persuaded.

[raybo]

dilanesp, Isn't everything you stated about +EV, based on the past? Regardless of the sample size, isn't it still based on the past? If so, then aren't all those statements based on "backfitting" past events? If so, then what absolute value does a +EV determination, made on at least a large portion of distant past results, have going forward? You already stated that the Beyer phenomenon stopped working over time, because racing and players evolved to the point that that phenomenon no longer had any positive value.

What time period are we really looking at for a +EV bet being successful going forward? Days, weeks, months, years, forever? Isn't it just as likely to start failing tomorrow as it is to start failing weeks, months, or years from now? My point is that if one waits for a method to remain successful long enough to have "statistical" significance, haven't we already missed the profit making boat? And, if we have based that method on a very large sample of distance past results to a significant extent, spanning years of constantly evolving events, how likely is that method to represent future reality? Aren't we forced to base our method on smaller, more recent results, in order to more likely be successful in leveraging that method before it loses it's value?

The fallacy of very large sample sizes is that they are diluted by time and evolution, and today's reality, as well as the near future reality, probably won't be representative of that composite large diluted sample of results.

The beauty of smaller more recent samples is that they more accurately reflect the current state of events. Methods based on those smaller, more recent results, is inherently evolutionary, not bogged down in a now non-representative record of distant past results.