Genetic algorithms

[markgoldie]

Wondering if any of the forum members have any information on the state of the art in genetic algorithms as they apply to race handicapping.

For those unfamiliar with the term, a genetic algorithm is essentially one that has the capacity to learn on its own and to apply new strategies to solving difficult problems. For example, the genetic algorithm will keep trying to find the relationships between data that cause a certain outcome. Many of these relationships are dead ends, but some are not. To guard against simply identifying shorter-term trends, you would want a long-term data base. With enough examples, a "trend" can turn into a "law".

At any rate, wondering if anyone is up to date on their use in racing.

Thanks. Mark

[Bruddah]

I am not, but wish I were smart enough to find and understand these avenues of information. They have always peaked my curiousity and interest. I came along to late in the Technology revolution to be able to fully understand computers and software. Sometimes I wished I had been born 40-50 years later. :(

[fmolf]

Wondering if any of the forum members have any information on the state of the art in genetic algorithms as they apply to race handicapping.

For those unfamiliar with the term, a genetic algorithm is essentially one that has the capacity to learn on its own and to apply new strategies to solving difficult problems. For example, the genetic algorithm will keep trying to find the relationships between data that cause a certain outcome. Many of these relationships are dead ends, but some are not. To guard against simply identifying shorter-term trends, you would want a long-term data base. With enough examples, a "trend" can turn into a "law".

At any rate, wondering if anyone is up to date on their use in racing.

Thanks. Mark
Mark may i ask you a question?...Do you believe that the outcome of a horserace can be predicted strictly by a computer and numbers? Without regard to what a race appears to be on paper to the naked eye.By this i mean certain races I just seem to have a feel for what is going to unfold and I am sure all handicappers have this feeling about certain races too.this to me is the "art of handicapping" and i believe that if i reduced it to sheer numbers and computer programs it would cease to be enjoyable.

[Overlay]

Mark may i ask you a question?...Do you believe that the outcome of a horserace can be predicted strictly by a computer and numbers? Without regard to what a race appears to be on paper to the naked eye.By this i mean certain races I just seem to have a feel for what is going to unfold and I am sure all handicappers have this feeling about certain races too.this to me is the "art of handicapping" and i believe that if i reduced it to sheer numbers and computer programs it would cease to be enjoyable.

I know that you directed your question to Mark, but I hope that I can chime in. I understand your perspective completely, and have great respect for the intuitive side of the game. But I personally find it just as enjoyable to try to outsmart the public by being able to detect instances where they have misjudged the chances of an individual horse or exotic-wager combination. I've found that, based on my own personal preferences and tastes, the use of a quantitative orientation allows me to achieve a level of orderliness, accuracy, and replicability from race to race that I don't think that I personally could match with a strictly intuitive approach, especially with regard to having enough confidence in my estimate of a horse's fair odds to base wagering decisions on it. William Quirin (among others) demonstrated the feasibility of this type of handicapping to my satisfaction through the selection and odds-prediction formulas that he presented in Winning at the Races.

[markgoldie]

Mark may i ask you a question?...Do you believe that the outcome of a horserace can be predicted strictly by a computer and numbers? Without regard to what a race appears to be on paper to the naked eye.By this i mean certain races I just seem to have a feel for what is going to unfold and I am sure all handicappers have this feeling about certain races too.this to me is the "art of handicapping" and i believe that if i reduced it to sheer numbers and computer programs it would cease to be enjoyable.

This is a good, serious question and it deserves a like answer. I remember clearly when the world chess champion Garry Kasparov played the IBM computer "Deep Blue" the second time for the world championship of "Man vs. Machine." Being a life-long chess player and enthuiast, I followed with baited breath. Kasparov, the strongest player who ever lived had said on numerous occasions that chess playing is more art than science and that he believed that no computer program would beat the world's best player, because the game simply could not be boiled down to computer language. Machines do not understand the grace, elegance, and beauty of some positions and possibilities. Furthermore, a human relies on his intuition, getting (as you put it) a "feeling" for a certain position or situation. This is the "art" part in chess playing.

As it turns out, Kasparov was wrong, and while I value my intuition and feeling in approaching a problem, the fact is that such feelings are subconscious vestiges of past games (or races), long forgotten to the conscious brain, but nonetheless, floating around in the dark shadows forever.

I once had a rather spirited conversation with my brother, who took exception to my trying to put numbers on too many shadings of angles and principles. For example, can you put a number on a rider switch from the sixth-leading rider to the top jock on the grounds? Well, it's hard. But my argument was that your brain is trying to calculate these shades of circumstance that horses undergo anyway. Is it better to just try and digest all the information and come up with a "feel" for a race?

I don't think so, primarily because our minds are programmed to work with numbers. As a species we were doing a whole lot of counting tens of thousands of years before computers were invented. We understand the world through counting and a computer is just a fancy counting machine.

So my answer is yes. I believe sophisticated counting is as good a way to beat the races as there is. I also think some of the handicapping "artists" might do even better if they broke their art into the component numbers. If nothing else, at least they would be guaranteed of consistency.

Mark

[dutchboy]

If you google "horse racing genetic algorithm" you will find numerous articles on the subject of neural networks.

The Neurax software from Bris has an explanation on how neural networks work and how a computer can create an odds line based on the trained neural network. The software is free to download but the data files are expensive.

[fmolf]

This is a good, serious question and it deserves a like answer. I remember clearly when the world chess champion Garry Kasparov played the IBM computer "Deep Blue" the second time for the world championship of "Man vs. Machine." Being a life-long chess player and enthuiast, I followed with baited breath. Kasparov, the strongest player who ever lived had said on numerous occasions that chess playing is more art than science and that he believed that no computer program would beat the world's best player, because the game simply could not be boiled down to computer language. Machines do not understand the grace, elegance, and beauty of some positions and possibilities. Furthermore, a human relies on his intuition, getting (as you put it) a "feeling" for a certain position or situation. This is the "art" part in chess playing.

As it turns out, Kasparov was wrong, and while I value my intuition and feeling in approaching a problem, the fact is that such feelings are subconscious vestiges of past games (or races), long forgotten to the conscious brain, but nonetheless, floating around in the dark shadows forever.

I once had a rather spirited conversation with my brother, who took exception to my trying to put numbers on too many shadings of angles and principles. For example, can you put a number on a rider switch from the sixth-leading rider to the top jock on the grounds? Well, it's hard. But my argument was that your brain is trying to calculate these shades of circumstance that horses undergo anyway. Is it better to just try and digest all the information and come up with a "feel" for a race?

I don't think so, primarily because our minds are programmed to work with numbers. As a species we were doing a whole lot of counting tens of thousands of years before computers were invented. We understand the world through counting and a computer is just a fancy counting machine.

So my answer is yes. I believe sophisticated counting is as good a way to beat the races as there is. I also think some of the handicapping "artists" might do even better if they broke their art into the component numbers. If nothing else, at least they would be guaranteed of consistency.

Marki understand totally that computers have their place in modern day handicapping....the esoteric approach still has its place as well.Paddock inspection will never be performed by a computer....first off the claim equiptment changes is another area where detailed notetaking is still a fruitful endeavor...shoe type changes...tongue ties added....I think i should investigate computers in my handicapping. I am just a bit nervous about what it will do to my bottomline.this may be off topic being i am a neophyte computer user(just started downloading bris 3 mos ago)does anyone have a good program for me to start with?something easy to use and not to difficult to understand the premise behind why the program selects the horses it does.Any suggestations will be greatly appreciated as i try to bring my handicapping into the 21st century!

[off'nclear]

Numbers drive me nuts...however, I do believe that a Spook takes care of me.

There are times when I am driving, and for some unknown reason, i take my foot off the pedal...and the car that would have crashed into me as it changes lanes doesn't.

When I was diligently eliminating horses on my old printout, I've written the WRONG information on the horse. I didn't know that till later, but the Spook had given me the CORRECT horse and eliminated the one that I would have chosen.

Until the black box does as well as gut feeling or spook, ...just lost my train of thought. :>)

[proximity]

Do you believe that the outcome of a horserace can be predicted strictly by a computer .......

a follow up question i'd ask is are the best speed figures produced strictly by a computer?

well the answer to that question is no.

a simple two step process (speed ratings and a track variant) and yet the best numbers remain those with at least some degree of human intervention involved in their creation.

in the end, knowing your game and the program you're using..... what it can and can't do for you.... is paramount.

[Dave Schwartz]

What do you want to know about GAs? I have written 38 of them.


Regards,
Dave Schwartz

[CBedo]

I've played a little with some GAs to look at factor weighting.

For anyone interested in this, I'd highly recommend reading Dave's article about ants.

[markgoldie]

What do you want to know about GAs? I have written 38 of them.


Regards,
Dave Schwartz


Dave;

My understanding of GA's is very limited. Just to what I have read. My understanding is that they can be used as a sophisticated data mining tool. As such, if you have a large data base and want to solve a particular problem involving that data base, the GA will look for ways to solve the problem by exploring relationships among the data. In many cases, they are said to be helpful when the user doesn't even have a clue as how to begin approaching a solution to the problem.

Certainly, in horse racing we have more than enough ideas as to how to solve the problem (that of producing consistently positive ROI's). The difficulty, of course, is that the overwhelming majority of ideas do not achieve the goal.

I would have thought (and I am probably wrong) that a GA could explore the relationships inherent in the data in new and creative ways that may not be intuitive to human analysts. And if it were able to do so, it might come up with hidden relationships among data factors that solve the problem.

Were this the case, feeding the GA the best possible PP's, along with full race charts including payoffs over a long period of time, might produce some interesting results. But since you say you have written 38 different GA's, although you did not say they were all devoted to race handicapping, it would seem to indicate that GA's must be specialized in some way since a single GA would not achieve the goal. Am I wrong about that?

Thanks. Mark

[Cratos]

Wondering if any of the forum members have any information on the state of the art in genetic algorithms as they apply to race handicapping.

For those unfamiliar with the term, a genetic algorithm is essentially one that has the capacity to learn on its own and to apply new strategies to solving difficult problems. For example, the genetic algorithm will keep trying to find the relationships between data that cause a certain outcome. Many of these relationships are dead ends, but some are not. To guard against simply identifying shorter-term trends, you would want a long-term data base. With enough examples, a "trend" can turn into a "law".

At any rate, wondering if anyone is up to date on their use in racing.

Thanks. Mark

I don’t believe genetic algorithms are applicable to race horse handicapping because I think it is very difficult or nearly impossible to find inherited or mutated characteristics of either a horse or a race that will consistently optimize a winning solution. Why not try a derivative algorithm?

[formula_2002]

Wondering if any of the forum members have any information on the state of the art in genetic algorithms as they apply to race handicapping.

For those unfamiliar with the term, a genetic algorithm is essentially one that has the capacity to learn on its own and to apply new strategies to solving difficult problems. For example, the genetic algorithm will keep trying to find the relationships between data that cause a certain outcome. Many of these relationships are dead ends, but some are not. To guard against simply identifying shorter-term trends, you would want a long-term data base. With enough examples, a "trend" can turn into a "law".

At any rate, wondering if anyone is up to date on their use in racing.

Thanks. Mark

I am a genetic algorithm!! :)

http://www.paceadvantage.com/forum/showthread.php?t=58822

[Dave Schwartz]

Mark,

First, understand what a generic algorithm really is.

It is a tool for discovering a very good solution to a problem from a large search space.

Image that you have a large set of factors and you want to give them a weight - much like a regression analysis system where the result is a formula like:

ProbForHorse= BaseNum + (Factor1 * Weight1) + (Factor2 * Weight2) ... + (Factor? * Weight?)

This is a perfect use for a GA because it will learn very quickly, zeroing in on the best wieghts. It will do so much faster than an iterative process such as some form of multiple linear regression.


If you really work at it, you can codify any kind of analysis into a GA - even a rule-based one. Example:

I have written GAs that use if-then statements to make rules. Imagine a rule that looks like this:

(In sentence form)
If Factor #17 is less than 4 then multiply by 2.35.

Consider that what we really have is:

if (Factor#) is (sign) (value) then take this (action) with a (actionVal)

[b]So, suppose you have a list with 200 factors
(Factor# is 000 to 200)

Signs are <,<=,=,>=,>
(signs are codified as 1 to 5)

Suppose values are all scaled 0-100
(values are codified from 000 to 100)

You have a list of actions.
Actions can be:
1. Eliminate horse
2. Make horse contender
3. Add some points (i.e. ActionVal)
4. Subtract some points (i.e. ActionVal)
5. Multiply by ActionVal
6. Divide by ActionVal

(Actions are 1 to 6)


Finally, you have an ActionValue
(ActionVals are from 000.00 to 999.99)


So here is a string:
If Factor #17 is [b]less than 4 then multiply by 2.35.

017|1|004|5|00235

Of course, we remove the "|" and we have:
0171004500235


So, with the proper programming you can do quite a bit.

I actually give classes on this stuff. I have helped set up several of Asian/Australian "teams." It is, of course, fee-based and the fee is not cheap (but affordable for anyone desring to set up such an operation).


Dave Schwartz

[Greyfox]

I am a genetic algorithm!! :)


Exactly! :ThmbUp::ThmbUp:

[Handiman]

I have read this thread with great interest. I have had several heated discussions with friends and foes alike, and the bottom line is mathematics lie at the core of all things.

I know Art(feelings) is held out as a major part of horse handicapping. But to clarify, both visual art and music are both dependent on mathematics. Music is associated with measured chunks of time. Such as quarter time and such. Visual Art to be pleasing must have mathematical relationships in concert with each other, both in space and color.

Beautiful people have faces that have mathematical ratios that must match up between the right side and left side of the face. If the correlation between the two sides are off, then the less pleasing to the eye the face becomes.

This is all a very shallow explanation, but it buttress' my point concerning horse handicapping being dependent on numbers. Those numbers and their relationships may change from day to day and from location to location, but horse racing can still be boiled down to a set of numbers.

The question is can you successfully take numbers and cook up some soup that tastes like success. Not many can....cause if they could, there wouldn't be 95% losing horse players.

Handiman :)

[fmolf]

I have read this thread with great interest. I have had several heated discussions with friends and foes alike, and the bottom line is mathematics lie at the core of all things.

I know Art(feelings) is held out as a major part of horse handicapping. But to clarify, both visual art and music are both dependent on mathematics. Music is associated with measured chunks of time. Such as quarter time and such. Visual Art to be pleasing must have mathematical relationships in concert with each other, both in space and color.

Beautiful people have faces that have mathematical ratios that must match up between the right side and left side of the face. If the correlation between the two sides are off, then the less pleasing to the eye the face becomes.

This is all a very shallow explanation, but it buttress' my point concerning horse handicapping being dependent on numbers. Those numbers and their relationships may change from day to day and from location to location, but horse racing can still be boiled down to a set of numbers.

The question is can you successfully take numbers and cook up some soup that tastes like success. Not many can....cause if they could, there wouldn't be 95% losing horse players.

Handiman :)obviously the numbers are all we have to analyze a race,unless you do not use pp's but paddock inspection only.The question here is in the interpreting of the numbers and can it be done in a cold calculating way by computers or is it better to use the artsy esoteric way through human interpretation.data mining is a great way to come up with useful statistics which may or may not translate into useful wagers,this would take all of the fun out of the game for me.i thoroughly enjoy trying to decide is this horse entered to win or entered to get in racing form for a race two weeks from now?will this runner fire first off the layoff?will this front runners improving form be able to carry him all the way to the wire while he jumps up one class off a strong finish in previous race?...

[ryesteve]

The question here is in the interpreting of the numbers and can it be done in a cold calculating way by computers or is it better to use the artsy esoteric wayIt still seems like you're carrying some heavy misiconceptions about what it's like to use computers in handicapping. Re-read some of Dave's comments if you still don't think writing and using software can be "artsy" and "esoteric"

[fmolf]

It still seems like you're carrying some heavy misiconceptions about what it's like to use computers in handicapping. Re-read some of Dave's comments if you still don't think writing and using software can be "artsy" and "esoteric"
possibly i will never be comfortable in my own handicapping using computers. I admire people who can, it just seems so antiseptic to me and seems like it is taking some of the fun out of the game....a person still has to assign values to the different handicapping factors so the computer can rate them ....am i wrong in this assumption?can a computer decide which horses are overlaid?....again if it can be programmed to do that, a human still has to input the data and the key factors for the computer to use in its analysis?do not computer programmers always say"garbage in garbage out"?....i understand their use in datamining. They must cut the amount of paperwork and time needed to accumulate data immensely. As far as making selections i prefer to do this the old fashioned way with paper and pencil and the pp's before me

[ryesteve]

a person still has to assign values to the different handicapping factors so the computer can rate them ....am i wrong in this assumption?Kind of... I mean, that is one approach you can take, but that's just one of many.

Have you ever read any of Cramer's stuff, like Kinky 'capping? That kind of stuff is totally artistic and creative, but every time he'd come up with a proposition, he'd retire to a stack of yellow racing forms to see if it held up. The only difference between that, and someone with a good database, is that the guy with the database can do the same thing in a fraction of the time. And if you find some angles you like, the guy with the racing form has to spend an hour or two scanning the PPs to look for qualifiers, whereas the guy with the software can just download the PP files and get a day's worth of qualifiers instantly.

For this type of approach, there's absolutely no difference in how "human" the selection process is... you're just offloading the grunt work to software that can do it faster and more accurately than you can.

[W2G]

As one who has managed predictive modeling projects in both the commercial and government realms, I have to say that I look at rosy claims about handicapping and data mining with a jaundiced eye. Never once in my experience did I think that the predictive techniques widely used elsewhere with success could be adapted to predict the outcomes of horse races with a high degree of accuracy. But as with anything, the customer decides if a product has value.

IMO, the best genetic algorithms in handicapping are living, breathing, careful-thinking, sharp horse players; and the best neural networks are their brains.

[fmolf]

Kind of... I mean, that is one approach you can take, but that's just one of many.

Have you ever read any of Cramer's stuff, like Kinky 'capping? That kind of stuff is totally artistic and creative, but every time he'd come up with a proposition, he'd retire to a stack of yellow racing forms to see if it held up. The only difference between that, and someone with a good database, is that the guy with the database can do the same thing in a fraction of the time. And if you find some angles you like, the guy with the racing form has to spend an hour or two scanning the PPs to look for qualifiers, whereas the guy with the software can just download the PP files and get a day's worth of qualifiers instantly.

For this type of approach, there's absolutely no difference in how "human" the selection process is... you're just offloading the grunt work to software that can do it faster and more accurately than you can.
yes i understan the database work is much much easier i wish i knew how!when i retire i am going to buy myself a good fast notebook and plunge into it.....but i will never get rid of my pp's and my pencil paper and calculator!cramer is one of my favorite handicapping authors if not the favorite...!

[fmolf]

As one who has managed predictive modeling projects in both the commercial and government realms, I have to say that I look at rosy claims about handicapping and data mining with a jaundiced eye. Never once in my experience did I think that the predictive techniques widely used elsewhere with success could be adapted to predict the outcomes of horse races with a high degree of accuracy. But as with anything, the customer decides if a product has value.

IMO, the best genetic algorithms in handicapping are living, breathing, careful-thinking, sharp horse players; and the best neural networks are their brains.
I agree but i definitely see the other side of the equation too, the researchers side...but as for making selections and getting value on these selections. I prefer the computer between my ears!

[ryesteve]

As one who has managed predictive modeling projects in both the commercial and government realms, I have to say that I look at rosy claims about handicapping and data mining with a jaundiced eye. Never once in my experience did I think that the predictive techniques widely used elsewhere with success could be adapted to predict the outcomes of horse races with a high degree of accuracy.
Well, as one who actually executes predictive modeling in the commercial realm, I will say that although there are many dangerous dead ends and red herrings to be avoided, you can find some success.

Look at F2002's posts in the Selections forum... he's spent a career trying to prove that you can't win betting on horses, and yet the latest version of his automated model has been showing a profit.

[46zilzal]

Only problem with an algorithm is that it simply does NOT represent the reality of randomness. Horses have 64 chromosomes, with each gamate getting 32, but WHICH 32 and along that string, just how many and where has recombination taken place? We do not KNOW what the multiple various combinations and permutations of genotype compare to the phenotypes we are looking for. EACH breeding ultimately is a crap shoot.

Like our vet is oft stated: "people try to apply population genetics to INDIVIDUALS and it is patently absurd."

[Dave Schwartz]

FM,


yes i understan the database work is much much easier i wish i knew how!when i retire i am going to buy myself a good fast notebook and plunge into it.....but i will never get rid of my pp's and my pencil paper and calculator!cramer is one of my favorite handicapping authors if not the favorite...!

The biggest problem the "between-the-ears" computer has is coming up with a single number to represent a horse's value.

As you asked in Which Handicapping Method... thread:

[i guess i did not phrase the question correctly.. i am was looking to find out what is the most important factor in your handicappingsuch as ....you have field narrowed down to three contenders..one owns the best final speed figures at the distance...two owns the best early pace figs...the other is a first off the claim horse for a trainer successful with this move...all three horse are overlaid because a class horse is dropping severely and is odds on false favorite...which horse would your handicapping lead you too?

So, obviously, you, too have a problem accomplishing the "make a number for each horse" problem.

The reason for this is that you are forced to re-invent the wheel in every race. While I am sure that you have a process for your evaluation, it does not result in a statement that ends with something like, "I will bet this horse at 7/2 or higher."

Even if you did have such a procedure (that ends with a number) you have no way to tweak it to check for improvement.

You don't have the ability to look back at (say) your last 100 (or 1000) races and say, "What would have been my result if I prefer the recently claimed horse just a little more?"

This is why most horse players have a difficult time improving their play.

Understand that I am not on the attack here. I have no need to make you believe what I believe; see it the way I see it. I am simply stating fact.


In contrast, the "database player," as they have come to be called around here, has the ability (as Steve said) to say, "What if I turn up the weight of the recently claimed animal by 13%?" and get an almost immediate answer.

Steve said it really well when he said that database players are not without artfulness. It is just that we choose to apply the artfulness up front instead of at 5 minutes to post. I can tell you that it is alot less stressful that way.


I must be honest here and tell you that I actually do not use GAs in my own current approach. We have a couple of them in HSH - as well as a neural net approach - but they are actually a couple of versions old. (Still there, just not part of our mainstream.) Instead, we have moved to "prediction models" and concentrated on other directions to face the challenges of the whales. For us, those approaches are working.


Good fortune to you.


Regards,
Dave Schwartz

[Dave Schwartz]

LOL - So, I am watching this thread and getting an email when someone replies.

46, you have the distinction of being one of just two on my ignore list but the thread watcher sent me your post anyway.


Thanks for confirming that you will repsond to any topic in any category, no matter how ignorant you are on the subject just to hear yourself speak.


Your response, indicates that you think this topic is about the genteics of the horse.
:lol: :lol: :lol:


You know, I would read your posts just for the occasional laugh if there weren't so many of them.


Dave Schwartz

[Cadillakin]

As one who has managed predictive modeling projects in both the commercial and government realms, I have to say that I look at rosy claims about handicapping and data mining with a jaundiced eye. Never once in my experience did I think that the predictive techniques widely used elsewhere with success could be adapted to predict the outcomes of horse races with a high degree of accuracy. But as with anything, the customer decides if a product has value.

IMO, the best genetic algorithms in handicapping are living, breathing, careful-thinking, sharp horse players; and the best neural networks are their brains.

Every horse is a unique study... Horse races are one-time occurrences in time. Never again will they be reproduced with the same variables. No exceptions.

We can study Affirmed and Alydar. And we can plug in reams of data into our computers about their matchups.. the times, the trainers, the surfaces, the preps, their Beyer figures, etc.. And then we can do it with the other Triple Crown contenders from other years...We can feed the data-in till the cows come home.. But that's not going to give us any insight on whether we should bet Sunday Silence or Easy Goer in the Belmont Stakes.

Is it?

[46zilzal]

LOL - So, I am watching this thread and getting an email when someone replies.

46, you have the distinction of being one of just two on my ignore list but the thread watcher sent me your post anyway.


Thanks for confirming that you will repsond to any topic in any category, no matter how ignorant you are on the subject just to hear yourself speak.



Only problem is that MOST go on wishful thinking and my responses are based upon HARD SCIENCE. So be it

[PaceAdvantage]

Only problem is that MOST go on wishful thinking and my responses are based upon HARD SCIENCE. So be itMan, how can you continue to be this clueless?

[Cratos]

As one who has managed predictive modeling projects in both the commercial and government realms, I have to say that I look at rosy claims about handicapping and data mining with a jaundiced eye. Never once in my experience did I think that the predictive techniques widely used elsewhere with success could be adapted to predict the outcomes of horse races with a high degree of accuracy. But as with anything, the customer decides if a product has value.

IMO, the best genetic algorithms in handicapping are living, breathing, careful-thinking, sharp horse players; and the best neural networks are their brains.

You make very valid points, but in predictive modeling doesn't it depends on the assumptions in the model.

[bcgreg]

Dave,

Can GA's be written in VBA that would function within Access? Thanks in advance.

bcgreg

[LottaKash]

Mark Cramer....."When I die, I hope my survivors will heed my request for the following inscription on my gravestone: HE SPENT HIS LIFE SEARCHING FOR THE AUTOMATIC BET."

best,

[46zilzal]

Man, how can you continue to be this clueless?
Books on the subject, years of experience and research, many many people IN THE BUSINESS who know just how random it is,,,,,SCIENCE explains it not grandstanders with computers.

FIRST: NO ONE KNOWS just what genes produce speed, stamina, solid bone, good wind etc etc.

SECOND: EVEN IF YOU DID, there is currently NO way to isolated these genes in the random dispersal of genes in gamates of a breeding.

Third: Controller genes can turn on and off the expression and one does not know WHICH controller genes are in the offspring's genetic makeup

Fourth: in the new science of Epigenetics, we cannot correlate just which environmental aspects can influence the partial or fill expression of the genetic complement.

Fifth: some genes though inherieted are not expressed via partial penetrance



and On an On and On and On

[Dave Schwartz]

Can GA's be written in VBA that would function within Access?

Of course, but they would be very slow. Too slow.

[ryesteve]

Books on the subject, years of experience and research, many many people IN THE BUSINESS who know just how random it is,,,,,
Kinda like your responses to a given thread?

Seriously, how could you have read this thread (as opposed to just skimming the subject header) and concluded it had anything to do with horse genetics?

[46zilzal]

Kinda like your responses to a given thread?

Seriously, how could you have read this thread (as opposed to just skimming the subject header) and concluded it had anything to do with horse genetics?
I stand corrected

[Wizard of Odds]

the GA fitness function which is NOT maximizing number of winners


just my 2 cents

[Dave Schwartz]

the GA fitness function which is NOT maximizing number of winners

Wiz,

It can be whatever you design it to be.

Personally, I have used a weighted scoring system, where hit rate counts for something but profitability counts more.


Dave

[PaceAdvantage]

I stand correctedThis is just like a MasterCard commerical....PRICELESS!

[formula_2002]

Well, as one who actually executes predictive modeling in the commercial realm, I will say that although there are many dangerous dead ends and red herrings to be avoided, you can find some success.

Look at F2002's posts in the Selections forum... he's spent a career trying to prove that you can't win betting on horses, and yet the latest version of his automated model has been showing a profit.

It only seems that way because that's how it always turns. :)

[CBedo]

obviously the numbers are all we have to analyze a race,unless you do not use pp's but paddock inspection only.The question here is in the interpreting of the numbers and can it be done in a cold calculating way by computers or is it better to use the artsy esoteric way through human interpretationPersonally, I have no doubt that, currently, comparing my personal handicapping with that of the computer program I am working on, that in handicapping ONE track, my results will over time crush the results of the computer program, but could I have handicapped TEN or TWENTY tracks as effectively? Not likely in a timely manner.

In any business, yoiur return on assets is equal to your margin multiplied by your asset turnover. The key for me is balancing maximizing my edge (non-computer) with getting a larger number of bets (computer).

Also, with regards to taking the fun out of it by using a computer, I get huge amounts of intellectual enjoyment out of finding new nonlinear relationships and new ways of learning how to be better at this. And of course, I have WAY more fun when I'm winning MORE money!

[CBedo]

Only problem with an algorithm is that it simply does NOT represent the reality of randomness. Don't confuse random with chaotic. And last time I checked, a chaotic system is still deterministic.

[fmolf]

Personally, I have no doubt that, currently, comparing my personal handicapping with that of the computer program I am working on, that in handicapping ONE track, my results will over time crush the results of the computer program, but could I have handicapped TEN or TWENTY tracks as effectively? Not likely in a timely manner.

In any business, yoiur return on assets is equal to your margin multiplied by your asset turnover. The key for me is balancing maximizing my edge (non-computer) with getting a larger number of bets (computer).

Also, with regards to taking the fun out of it by using a computer, I get huge amounts of intellectual enjoyment out of finding new nonlinear relationships and new ways of learning how to be better at this. And of course, I have WAY more fun when I'm winning MORE money!
:D the winning part i understand!.... you hit my nail on the head i only concentrate on two tracks at any one time.The nyra track and usually one other.....monmouth when it is running...i would love to use computers and hope to learn when i have more time.i understand how valuable they can be in organizing and collecting data.for now with my limited time to play this game i will continue what i have been doing...to all you computer handicappers and dataminers keep presssing on so it will be even easier for me in a few years when i plunge into computers.. and continued good luck

[fmolf]

Books on the subject, years of experience and research, many many people IN THE BUSINESS who know just how random it is,,,,,SCIENCE explains it not grandstanders with computers.

FIRST: NO ONE KNOWS just what genes produce speed, stamina, solid bone, good wind etc etc.

SECOND: EVEN IF YOU DID, there is currently NO way to isolated these genes in the random dispersal of genes in gamates of a breeding.

Third: Controller genes can turn on and off the expression and one does not know WHICH controller genes are in the offspring's genetic makeup

Fourth: in the new science of Epigenetics, we cannot correlate just which environmental aspects can influence the partial or fill expression of the genetic complement.

Fifth: some genes though inherieted are not expressed via partial penetrance



and On an On and On and On
was Sartin a geneticist too?

[Dave Schwartz]

was Sartin a geneticist too?

Oh, he had a high IQ, but I don't know if he was a geneticist. :rolleyes:

(Do you actually have to know what these words mean to use them?)

[Bruddah]

I love reading these intelligencia threads. As an average handicapper, I compare myself to a swimmer jumping into the deep end of a pool. I know it will be over my head, but I jump in hoping I can remain afloat. Most of the time, I end up swallowing water and trying to hold onto the edge.

You guys really amaze me with the sophistication of your intelligence and the depth of your knowledge. I am envious of your abilities.

Now that I have given you the good, let me drop the other shoe. Sometimes you all are like kitttens chasing your tails. Going in circles until you tire of the sport. Playful, full of energy and very entertaining.

Thanks from an old man which appreciates your minds but not fully able to comprehend the thinking within. :D

[fmolf]

I love reading these intelligencia threads. As an average handicapper, I compare myself to a swimmer jumping into the deep end of a pool. I know it will be over my head, but I jump in hoping I can remain afloat. Most of the time, I end up swallowing water and trying to hold onto the edge.

You guys really amaze me with the sophistication of your intelligence and the depth of your knowledge. I am envious of your abilities.

Now that I have given you the good, let me drop the other shoe. Sometimes you all are like kitttens chasing your tails. Going in circles until you tire of the sport. Playful, full of energy and very entertaining.

Thanks from an old man which appreciates your minds but not fully able to comprehend the thinking within. :Di enjoy these forums also its great entertainment ,provides some laughs and i generally do get some good ideas from other posters!

[Cratos]

I am still waiting for the developers of genetic algorithms to illustrate by using finite steps of inherited attributes of the thoroughbred that a unknown consistent winning methodology will be produced.

[formula_2002]

I am still waiting for the developers of genetic algorithms to illustrate by using finite steps of inherited attributes of the thoroughbred that a unknown consistent winning methodology will be produced.
how about they just post a few hundred picks in the selections forum.


publish, publish, publish, else all is BS

[fmolf]

how about they just post a few hundred picks in the selections forum.


publish, publish, publish, else all is BSi am not a computer guy but i think that their are different programs written for databasing .....for selecting, and again for culling certain winning characteristics from the pp's in order to use these for future selections?...computer men am i correct in these assumptions?

[Dave Schwartz]

I am still waiting for the developers of genetic algorithms to illustrate by using finite steps of inherited attributes of the thoroughbred that a unknown consistent winning methodology will be produced.

What do "inherited attributes" have to do with this conversation?


Personally, I use them to solve pieces of the handicapping puzzle. That is, I have used them to develop high-level factors, weights etc.


Regards,
Dave Schwartz

[Cratos]

What do "inherited attributes" have to do with this conversation?


Personally, I use them to solve pieces of the handicapping puzzle. That is, I have used them to develop high-level factors, weights etc.


Regards,
Dave Schwartz

Aren’t you solving an optimization problem and don’t you need “inherited” factors for the values for which the different variables are being optimized?

[Dave Schwartz]

I guess I am not familiar with what you mean by the word "inherited."

It sure sounds a lot like something to do with the genetics of the horse. If that is not what you meant please explain.

If it is what you meant, please go have a beer with 46Z. (Put it on his tab.)


Dave

[Hosshead]

Kind of... I mean, that is one approach you can take, but that's just one of many.

Have you ever read any of Cramer's stuff, like Kinky 'capping? That kind of stuff is totally artistic and creative, but every time he'd come up with a proposition, he'd retire to a stack of yellow racing forms to see if it held up. The only difference between that, and someone with a good database, is that the guy with the database can do the same thing in a fraction of the time. ...

Agree Steve. However, as far as finding (and tweaking) what you want to research and develop, there is one advantage I've noticed with looking up "systems" by hand. (or at least a sample of that system's picks)

That is, when you are checking by eye, with downloaded pp's or even (like Cramer) the DRF, to find instances when a certain system has picked a horse, you can sometimes "see" a certain "situation" that surrounds or has an impact on why that system's horse did or did not win. This may be a factor or combination of factors that you never would have thought of, (to run through a computer), UNTIL you caught it with your eye, while checking out a system.
You can then add this new information to your "recipe" and THEN use the computer to run a new search.
But the eye and the intuition of the brain gave you the idea to re-calculate or add new factors/weights before going back to data crunching.

-- Hoss

[formula_2002]

i am not a computer guy but i think that their are different programs written for databasing .....for selecting, and again for culling certain winning characteristics from the pp's in order to use these for future selections?...computer men am i correct in these assumptions?
Obviously, first generation computerized data evaluation methods and programs are designed by a mind.
The interpreted value of that first generation is then the basis for the next generation and so on and so on.
Some where along the way, something of value may occur, either by design, luck or other circumstance.Some examples of “luck or other circumstance” that have always stayed with me.
From wikipedia;

Chasos theory
“An early pioneer of the theory was Edward Lorenz whose interest in chaos came about accidentally through his work on weather prediction in 1961”

Vulcanization
“Most textbooks point out that Charles Goodyear (1800–1860) invented vulcanization of rubber as used today by the addition of sulfur in high heat. Depending on what you read, the Goodyear story is one of either pure luck or careful research. Goodyear insisted that it was the latter, though many contemporaneous accounts indicate the former”

When it come to horseracing, the proof of a systems ROI value is quite simple.
Just publish your picks.

Demonstrate the success first, then tell me about your programs, theory etc.

To judge the roi value for the published picks see
Horse Sense: A Rigorous Application of Mathematical Methods to Successful Betting at the Track
Joe M
.

[fmolf]

Obviously, first generation computerized data evaluation methods and programs are designed by a mind.
The interpreted value of that first generation is then the basis for the next generation and so on and so on.
Some where along the way, something of value may occur, either by design, luck or other circumstance.Some examples of “luck or other circumstance” that have always stayed with me.
From wikipedia;

Chasos theory
“An early pioneer of the theory was Edward Lorenz whose interest in chaos came about accidentally through his work on weather prediction in 1961”

Vulcanization
“Most textbooks point out that Charles Goodyear (1800–1860) invented vulcanization of rubber as used today by the addition of sulfur in high heat. Depending on what you read, the Goodyear story is one of either pure luck or careful research. Goodyear insisted that it was the latter, though many contemporaneous accounts indicate the former”

When it come to horseracing, the proof of a systems ROI value is quite simple.
Just publish your picks.

Demonstrate the success first, then tell me about your programs, theory etc.

To judge the roi value for the published picks see
Horse Sense: A Rigorous Application of Mathematical Methods to Successful Betting at the Track
Joe M
.
i tried to read that book...took it out of my local library but it was to dry for me.I doubt i will ever have enough knowledge to ever fully use computers to their fullest capacity ...the proof is in the pudding eh?.....I have fun.....i stay slightly ahead of the game most months(very slightly ahead)some years i am a slight loser ..but this is how i choose to spend my entertainment dollar.when i have time i will investigate computer handicapping....does anyone have a good basic program to start with?easy to use and not too complicated?

[formula_2002]

i tried to read that book...took it out of my local library but it was to dry for me.I doubt i will ever have enough knowledge to ever fully use computers to their fullest capacity ...the proof is in the pudding eh?.....I have fun.....i stay slightly ahead of the game most months(very slightly ahead)some years i am a slight loser ..but this is how i choose to spend my entertainment dollar.when i have time i will investigate computer handicapping....does anyone have a good basic program to start with?easy to use and not too complicated?


if slight is say a 3% loss, those are great stats.if you can get a 5% rebate, your a winner. :)
it may be a bit involved, but Jcapper is a very worth while program (thats what use with my own program) and the guy that created and sells it willl help you with it.
check out jcapper.com

[DeanT]

Formula/Dave,

Let's say I am a breakeven player, and have been so for some time. Say I have $5M of handle on win bets and got back $5M over the years, with thousands upon thousands of tickets.

If I gave you all those numbers, and the way I bet them, would you be able to plug them into a computer program, get rid of some of the bad bets that I make, or bet size me optimally so I could be a winner? Would I gain 2% or 3% or what?

Would it be mathematically possible and sound, or would the statistical noise of the data and the changing game make that impossible?

Perhaps an easier question: Let's say you followed public picks, available at 8AM each day, and this person was dead break even. Would you be able to take those picks, plug them into something, and come out a winner?

Just wondering. It is something that interests me.

[Cratos]

I guess I am not familiar with what you mean by the word "inherited."

It sure sounds a lot like something to do with the genetics of the horse. If that is not what you meant please explain.

If it is what you meant, please go have a beer with 46Z. (Put it on his tab.)

Dave

No, Dave I am not talking about "something to do with the genetics of the horse" and I am not astute enough to understand what 46Z was talking about.

I am talking about using historical data to optimize winning patterns.

[Dave Schwartz]

Cratos,

Sorry for my apparent ignorance. Could you please explain the word "inherited" in its application here? I searched for any definition that applied to statsitics and could not find one.


What does that mean?


Regards,
Dave Schwartz

[formula_2002]

Formula/Dave,

Let's say I am a breakeven player, and have been so for some time. Say I have $5M of handle on win bets and got back $5M over the years, with thousands upon thousands of tickets.

If I gave you all those numbers, and the way I bet them, would you be able to plug them into a computer program, get rid of some of the bad bets that I make, or bet size me optimally so I could be a winner? Would I gain 2% or 3% or what?

Would it be mathematically possible and sound, or would the statistical noise of the data and the changing game make that impossible?

Perhaps an easier question: Let's say you followed public picks, available at 8AM each day, and this person was dead break even. Would you be able to take those picks, plug them into something, and come out a winner?

Just wondering. It is something that interests me.


Dean T
My approach requires many factors (i'm currently use 50 factors) with numerical values such as a bris speed figure and of course they have to be in a format I could use in a data base, such as comma delimited.
If you have generated these factors your self or are using someone else’s
I’d be interested in analyzing the data using my program.
I would need at least 15000 races to be able to make some sort of projection on races outside of the 15000 races.
If you are breaking even with that sort of data, I would HOPE, I could do as well and perhaps better.

[markgoldie]

Looks like Dave has a question posed to him and sooner or later, I know he'll get around to answering it. So, since he'll be back on the thread, let me pose another one.

I understand that his handicapping program is highly advanced in what it can do, but it doesn't seem to solve the problem, which is a reliable method of producing positive roi. Instead, it appears to be a tool that can be customized in a hundred different ways at the whim of the user. As such it seems like a tool to help the individual bettor solve the problem. That is, it is a great research aid, or possibly a shortcut to application of the winning formula once the problem has been solved.

My belief was that the intent of genetic algorithms is to solve problems. Can my reading of that be wrong? If that is the case, why then can we not make the following statement: GA's either can or cannot solve the problem.

It would seem to me that a world-class GA should operate pretty much the same whether you are asking it about solving a question of positive roi in horse racing or some industrial construction problem. It should take all the known data and relentlessly test it, combining them in many different ways (as if producing new generations, hence the term "genetic"), testing each new factor for fitness (again, "genetic" as if subjected to Darwinian evolution), and even concocting random mutations of factors and testing those factors for fitness in eventually solving the problem (again, such mutation would be like genes).

The bottom line is that at the end, we should either have an answer or not. One or the other. Either it is solvable or insolvable. Since we know that there are winning bettors out there, albeit few, we would suspect strongly that the problem is solvable.

If the problem is solvable, why then do we need programs to assist us in our handicapping and/or research? Once the job is done, it should be done. At least until the data base shifts to the degree that it is no longer profitable to use the same methodology recommended by the GA. When that day comes, we would need to reload the updated data base and run the GA over again.

I'm hoping this make some logical sense, because I am not a computer person, just someone who tries his best to think clearly and with logic.

And now for the $64,000 question to Dave. Hopefully he will not be offended, since his website touts him as a man who makes few friends due to his bluntness of speech and failure to "pull punches."

So the question is this, and it's a time-honored one. It goes something like this: If you actually knew how to beat the races by consistently producing positive roi from a handicapping strategy, why in the world would you want or need to sell products to other players in what is clearly a zero-sum game?

I have heard every conceivable explanation from touts of all nature on this question. None of them make any sense at all. The balance of profits in this scenario are not comparable in any way.

And so this would lead to the final question. The 38 GA's that you have produced are apparently all flawed in one major way. They don't solve the problem. Is that correct or incorrect? Fair statement or unfair statement?

Mark

[Cratos]

So the question is this, and it's a time-honored one. It goes something like this: If you actually knew how to beat the races by consistently producing positive roi from a handicapping strategy, why in the world would you want or need to sell products to other players in what is clearly a zero-sum game?

I have heard every conceivable explanation from touts of all nature on this question. None of them make any sense at all. The balance of profits in this scenario are not comparable in any way.

And so this would lead to the final question. The 38 GA's that you have produced are apparently all flawed in one major way. They don't solve the problem. Is that correct or incorrect? Fair statement or unfair statement?

Mark

Well stated and a very good question because in pari-mutuel wagering bettors do not want to reveal anything that would affect the odds.

However you can make an argument that selling a winning “handicapping tool” is more profitable or requires less work than handicapping and that becomes the rationale for selling it as oppose to using it.

[Dave Schwartz]

And so this would lead to the final question. The 38 GA's that you have produced are apparently all flawed in one major way. They don't solve the problem. Is that correct or incorrect? Fair statement or unfair statement?

The last GA I wrote was about 6 years ago.

With every GA I wrote, the results got better. However, I must agree with you that GAs did not take me where I wanted to go.

With the knowledge I have gained in the past year or so, I believe I could create a GA today (okay - maybe in 100 hours of programming) that would "solve the problem" to my satisfaction.

In the interim, however, I moved to another form of AI. It is working - I use the output of it every day in my handicapping.


One more thing about the GAs... in my software there is a GA. And at least one of our professional clients use that approach. Of course, they don't see it like a GA. And it doesn't look like a GA - it actually has the structure of a neural net - but the training process is pure GA.


BTW, you referenced "another question." What is that question?

Regards,
Dave Schwartz

[fmolf]

The last GA I wrote was about 6 years ago.

With every GA I wrote, the results got better. However, I must agree with you that GAs did not take me where I wanted to go.

With the knowledge I have gained in the past year or so, I believe I could create a GA today (okay - maybe in 100 hours of programming) that would "solve the problem" to my satisfaction.

In the interim, however, I moved to another form of AI. It is working - I use the output of it every day in my handicapping.


One more thing about the GAs... in my software there is a GA. And at least one of our professional clients use that approach. Of course, they don't see it like a GA. And it doesn't look like a GA - it actually has the structure of a neural net - but the training process is pure GA.


BTW, you referenced "another question." What is that question?

Regards,
Dave SchwartzMy question in strictly laymans terms is this...1)-do we or do we not have a handicapping program that can do better than a person like me using bris ultimate and pencil and paper as a worksheet? 2)...do we have a computer program that will strictly tell me when my horse is an overlay and when it is not?...some people reported positive roi's with mattels analyzer....doesn't formulator put itself forward as a computer datamining tool?...

[Dave Schwartz]

However you can make an argument that selling a winning “handicapping tool” is more profitable or requires less work than handicapping and that becomes the rationale for selling it as oppose to using it.

Well, I can tell you that is simply not true at my place.

In the last 8 weeks I have some made huge discoveries. I made these discoveries by handicapping almost 25,000 races one at a time and charting the results.

I have brought this together into a series of strategies and new tools and am releasing it to our clients.

Have I kept a coupole of secrets for myself? Sure. But 95% of what I have learned and built is being shared. Is there a cost? Yes, but it is not huge. Just enough to break me even for the time it took to build the training tools.

What makes this workable is that no two people will use these tools the same way. Some will choose to continue doing what they are doing because it is working for them. They may not use these tools at all.

Will it have an effect on my prices. Yes, but not a huge one. And certainly nothing in comparison to the other impacts of the day (whales and poly track are a much bigger concern).


That whole "nobody would share anything that worked" thing is just not what I have experienced. One of the biggest strides I made in the last couple of months came from a member of one of the "whale" teams when he explained how to do something that they are doing that I didn't know how to do."


Regards,
Dave Schwartz

[Dave Schwartz]

My question in strictly laymans terms is this...1)-do we or do we not have a handicapping program that can do better than a person like me using bris ultimate and pencil and paper as a worksheet? 2)...do we have a computer program that will strictly tell me when my horse is an overlay and when it is not?...some people reported positive roi's with mattels analyzer....doesn't formulator put itself forward as a computer datamining tool?...

We absolutely do.

And so do a whole bunch of other people.


regards,
Dave Schwartz

PS: I still have a working Mattel Analyzer.

[markgoldie]

Well stated and a very good question because in pari-mutuel wagering bettors do not want to reveal anything that would affect the odds.

However you can make an argument that selling a winning “handicapping tool” is more profitable or requires less work than handicapping and that becomes the rationale for selling it as oppose to using it.


Yes, that's correct and having re-read what I wrote (courtesy of your quoting me), I realize that I should point out clearly that the producer and seller of a handicapping tool is not to be confused with a tout. The former may be selling an aid to success, while the latter is outright claiming success (just follow me). There is a fine line here, but a line nonetheless. The seller of a tool to success may imply that the tool will lead to success with its proper use. But unless the seller says outright the he himself uses the tool to produce consistent winnings for his own account or somehow guarantees success if properly used, then in my opinion the tool seller is off the hook as a tout.

But that wasn't my larger intent anyway. My question was about the goal of using a GA as applied to a particular problem. Can the GA solve the problem or not? That's my main question. If it can't, then I suppose there's nothing wrong with selling a tool. As to using the terminology "GA" as a component of the tool that you are selling, even though said GA component cannot solve the problem, I'm less sure.

While I was writing this, Dave responded in another post. Let's analyze what he says.

[markgoldie]

The last GA I wrote was about 6 years ago.

With every GA I wrote, the results got better. However, I must agree with you that GAs did not take me where I wanted to go.

With the knowledge I have gained in the past year or so, I believe I could create a GA today (okay - maybe in 100 hours of programming) that would "solve the problem" to my satisfaction.

In the interim, however, I moved to another form of AI. It is working - I use the output of it every day in my handicapping.


One more thing about the GAs... in my software there is a GA. And at least one of our professional clients use that approach. Of course, they don't see it like a GA. And it doesn't look like a GA - it actually has the structure of a neural net - but the training process is pure GA.


BTW, you referenced "another question." What is that question?

Regards,
Dave Schwartz

The questions essentially were: If the purpose of a GA is to solve a particular problem, then can we not say that either the GA can or cannot solve the problem?

I believe you answered this question. The GA's you wrote were unable to solve the problem. But, as you say, possibly a better GA might.

The other question was a corollary. If you were able to write a GA that solved the problem of finding consistent positive roi (through a handicapping technique using weigthed factors, etc.), why then, in a zero-sum game would you want to sell the solution for a minor amount of money as compared to the financial rewards of keeping it to yourself and using it to wager?

I believe you also answered this question. You have not found such solution, so therefore you sell the software to mitigate expenses for the so-far failed endeavor. Fair enough. Perfectly legitimate.

But here's a fresh question based on your response. You admit to keeping some things to yourself. Is it then ethical to continue to sell something to the public that you describe as "state of the art" or as the absolute best available handicapping program?" Or is the loophole in the word "available" so as to mean buyable by the general public?

Mark

[Dave Schwartz]

My question was about the goal of using a GA as applied to a particular problem. Can the GA solve the problem or not?

In my opinion, a GA can solve many problems, "who to bet" being a likely candidate.

However, the structuring of the data - that is, how the GA views the real world - is critical.

Whether it be GAs, neural nets or some other esoteric form of analisys, the idea that one can just throw the factors into the pot and the AI will produce an eatable stew is just not realistic.

I think that is the problem with most people who purchase an AI product - they simply do not have the expertise to use the product. I ran into the same problem when I attempted to apply statistics - especially on a high level. I simply do not have the knowledge to do it properly. I am still working on getting the knowledge necessary. (I am not talking about any simple form of regression analysis here.)


So, to make this answer perfectly clear, "Yes, a GA can solve the problem provided one knows how to use the tool."


Regards,
Dave Schwartz

[fmolf]

In my opinion, a GA can solve many problems, "who to bet" being a likely candidate.

However, the structuring of the data - that is, how the GA views the real world - is critical.

Whether it be GAs, neural nets or some other esoteric form of analisys, the idea that one can just throw the factors into the pot and the AI will produce an eatable stew is just not realistic.

I think that is the problem with most people who purchase an AI product - they simply do not have the expertise to use the product. I ran into the same problem when I attempted to apply statistics - especially on a high level. I simply do not have the knowledge to do it properly. I am still working on getting the knowledge necessary. (I am not talking about any simple form of regression analysis here.)


So, to make this answer perfectly clear, "Yes, a GA can solve the problem provided one knows how to use the tool."


Regards,
Dave Schwartzdave....thanks for answering my questions.... it would be interesting to check the analyzer with its basic data input against your more powerful algorythm methods .....just a thought!continued good luck in your quest.....what you are selling is it similar to what formulator does?....i have found formulator to be very time consuming,because i still need to decide what factors i want to research with it

[Dave Schwartz]

I believe you answered this question. The GA's you wrote were unable to solve the problem. But, as you say, possibly a better GA might.

Actually, I said "... to my satisfaction."

The other question was a corollary. If you were able to write a GA that solved the problem of finding consistent positive roi (through a handicapping technique using weigthed factors, etc.), why then, in a zero-sum game would you want to sell the solution for a minor amount of money as compared to the financial rewards of keeping it to yourself and using it to wager?

I believe you also answered this question. You have not found such solution, so therefore you sell the software to mitigate expenses for the so-far failed endeavor. Fair enough. Perfectly legitimate.


A. Again, I use the phrase, "My satisfaction."

B. Why would I share it? Because I owe the people who have supported me a significant amout of my discoveries.


But here's a fresh question based on your response. You admit to keeping some things to yourself. Is it then ethical to continue to sell something to the public that you describe as "state of the art" or as the absolute best available handicapping program?" Or is the loophole in the word "available" so as to mean buyable by the general public?


Please... If I build an engine that is capable of generating (say) $500k per year in profit, do you really expect that I owe you that for a measly $600?

If so, then your reasoning is faulty.


Once, years ago, on this forum, I asked the question:

"If I produced a software product that would provide you with $20,000 per month of profit, how much would you be willing to pay?"

The biggest answer I got was $1,000. Not $1,000 per month, but $1,000 forever. That is an unrealistic answer.

Anyone hwo thinks that I (or anyone else) is selling the holy grail of handicapping for $600 is unrealistic. What I sell for $600 is a tool so that you may discover your own personal grail.

As I see it, making the result of extensive research available at no cost or "reasonable" cost is a courtesy owed to my clients, just like upgrades.


I am a direct guy - I tell people what's what. I don't hustle. I don't "spin a tale" to sell product. I'd probably make more money if I did but I like sleeping at night.

My ethics do not say that I owe someone a lifetime income for $600.


Regards,
Dave Schwartz

[Dave Schwartz]

what you are selling is it similar to what formulator does?....i have found formulator to be very time consuming,because i still need to decide what factors i want to research with it

First, I make it a point to not use this forum for sales pitches. I live here just like everyone else. So, I will answer this question briefly, and if you have further interest, please contact me privately and we can spend time discussing it further.


I am sure there are some similarities to Formulator but probably not enough to call them "similar."

Our software, with its 4,000 internal factors, can be very time consuming to refine to a professional solution on your own. That is why I try to collaborate with the clients. That is why I share my findings.


Regards,
Dave Schwartz

[fmolf]

First, I make it a point to not use this forum for sales pitches. I live here just like everyone else. So, I will answer this question briefly, and if you have further interest, please contact me privately and we can spend time discussing it further.


I am sure there are some similarities to Formulator but probably not enough to call them "similar."

Our software, with its 4,000 internal factors, can be very time consuming to refine to a professional solution on your own. That is why I try to collaborate with the clients. That is why I share my findings.


Regards,
Dave Schwartzthank you Dave i will not ask you any more product info questions....i admire your stance on this matter and feel foolish now for asking.....hopefully in a few years when i delve into computer handicapping you will still be here ....thanks again....4,000!....wow!

[Jake]

The questions essentially were: If the purpose of a GA is to solve a particular problem, then can we not say that either the GA can or cannot solve the problem?

I believe you answered this question. The GA's you wrote were unable to solve the problem. But, as you say, possibly a better GA might.

Mark

I have worked with GA programming over 15 years. I use to privately discuss expert systems, neural nets, and GA's with Dave Schwartz many years ago, so he has probably been working with them that long as well. He has given you some very straight forward answers, and why he has moved on.

In my experience, GA's can be used successfully to solve most of the "solvable " parts of the handicapping problem. The key here is breaking everything down into subgoals and "the structuring of the data". No one who has done this successfully is going to share their framework with you for doing this, unless it's already embedded into a commercial product, because it's involves too many hours of hard work, a tough learning curve, and the expertise to understand where the chokepoints are. All critical stuff for for getting the right answers. So, for most people, not the easiest way to get there.

If you are really interested, I'll be glad to give you a selected reading list from Amazon that will get you started. Just sent me a pm.

Jake

[Greyfox]

Anyone hwo thinks that I (or anyone else) is selling the holy grail of handicapping for $600 is unrealistic. What I sell for $600 is a tool so that you may discover your own personal grail.


That's a fair stance. :ThmbUp: :ThmbUp:
The fact is that even if you gave different people the winning horse in every race for a week, you'd find at the end of the week they would have all bet quite differently. Perhaps even a sizeable subset would have lost money. Another subset would have been sceptical and bet against your picks. Different strokes for different folks still applies here.

One can only imagine the huge number of permutations that different people would take with a handicapping program of some type.

[fmolf]

That's a fair stance. :ThmbUp: :ThmbUp:
The fact is that even if you gave different people the winning horse in every race for a week, you'd find at the end of the week they would have all bet quite differently. Perhaps even a sizeable subset would have lost money. Another subset would have been sceptical and bet against your picks. Different strokes for different folks still applies here.

One can only imagine the huge number of permutations that different people would take with a handicapping program of some type.
precisely why i stick with what i know and am comfortable with...

[Bison]

GA's or not, 2 questions that cannot be answered with 100% accuracy:

How does my horse feel today?
How will my jockey ride today?

[markgoldie]

"Please... If I build an engine that is capable of generating (say) $500k per year in profit, do you really expect that I owe you that for a measly $600?

If so, then your reasoning is faulty."



The question was not whether your program is fairly priced. I am by no means an expert on such things, so let's say that it is.

The question was if an better product exists and you know that there exists such a product because you are in possession of same, is it then ethical to say that the inferior product is the best there is?

I just reviewed you homepage and see that you make absolutely no claims at all about the strength of your program versus anything else on the market, or anything else in existence for that matter. I must have had you confused with something else. My apologies.

So the question does not apply vis-a-vis your product. However, as a general principle, I do not believe that a low selling price should give a blank check to all outrageous claims involving a product based on the logic that any reasonable person should not believe that a product at such a low price could possibly do what the claims say it will.

Again, your sales literature does not do so. Congrats.

[JeremyJet]

This whole conversation reminds me of a Dimention X episode I listened to the other night. Unfortunatly, nobody asked Joe what the fair odds were for the feature at Belmont Park. ;)

DIMENTION X
A Logic Named Joe
07/01/50

Tale of computers that tend to most of mans needs. Suddenly, they overcome their electronic censors and start attending to all their needs.

[Dave Schwartz]

How does my horse feel today?
How will my jockey ride today?

Who cares about this race?

I am interested in the day or the week, definitely the month.



The question was if an better product exists and you know that there exists such a product because you are in possession of same, is it then ethical to say that the inferior product is the best there is?

So, the product has now turned into the "inferior" product? :lol:

I think we are all grown up enough to recognize that there is an inference to the fact that a product so labeled is the best of those available.


Good fortune to you.

Dave

[Dave Schwartz]

PS: Late breaking news - I left this part out of the last post.


The product is precisely the same. It is the use of the product that might be different. Bringing every user to my level of understanding with the product (or handicapping in general) is outside the scope of my responsibility.


Regards,
Dave Schwartz

[fmolf]

PS: Late breaking news - I left this part out of the last post.


The product is precisely the same. It is the use of the product that might be different. Bringing every user to my level of understanding with the product (or handicapping in general) is outside the scope of my responsibility.


Regards,
Dave Schwartz
precisely why i use the old fashioned method :lol: ....good luck Dave and continued good fortune!

[Cratos]

Cratos,

Sorry for my apparent ignorance. Could you please explain the word "inherited" in its application here? I searched for any definition that applied to statsitics and could not find one.


What does that mean?

Regards,
Dave Schwartz

Dave, I found the following three definitions of GAs and you appear to be the most knowledgeable on the forum about GAs and am I wrong to say as you make your GA “learn” you are optimizing the evolved data through “inherent” solutions?

(1) A genetic algorithm (GA) is a search technique used in computing to find exact or approximate solutions to optimization and search problems. Genetic algorithms are categorized as global search heuristics. Genetic algorithms are a particular class of evolutionary algorithms(EA) that use techniques inspired by evolutionary biology such as inheritance, mutation, selection, and crossover (also called recombination

(2) Genetic algorithms are based on a biological metaphor: They view learning as a competition among a population of evolving candidate problem solutions. A 'fitness' function evaluates each solution to decide whether it will contribute to the next generation of solutions. Then, through operations analogous to gene transfer in sexual reproduction, the algorithm creates a new population of candidate solutions."

(3) Genetic Algorithms (GAs) are adaptive heuristic search algorithm premised on the evolutionary ideas of natural selection and genetic. The basic concept of GAs is designed to simulate processes in natural system necessary for evolution, specifically those that follow the principles first laid down by Charles Darwin of survival of the fittest. As such they represent an intelligent exploitation of a random search within a defined search space to solve a problem.

Thus this statement that I found might sum up GAs: "the advantage of the GA approach is the ease with which it can handle arbitrary kinds of constraints and objectives; all such things can be handled as weighted components of the fitness function."

[Dave Schwartz]

I am sorry to be difficult here but now I have a problem with "inherent." That sounds like permanent characteristics of the horse.

Honest - I am really not trying to be difficult.

Look, what I am doing is to take several hundred of our factors and let the GA (or "Ant Hill" as I call it) have their way with them. Their goal is to produce the best possible solution in a reasonable amount of time.

Understand, that I have no real hope of finding "optimum." I am quite happy to find "good enough."


Remember, that I (personally) use a different form of AI.


Regards,
Dave Schwartz

[Cratos]

I am sorry to be difficult here but now I have a problem with "inherent." That sounds like permanent characteristics of the horse.

Honest - I am really not trying to be difficult.

Look, what I am doing is to take several hundred of our factors and let the GA (or "Ant Hill" as I call it) have their way with them. Their goal is to produce the best possible solution in a reasonable amount of time.

Understand, that I have no real hope of finding "optimum." I am quite happy to find "good enough."


Remember, that I (personally) use a different form of AI.


Regards,
Dave Schwartz

Not really, I am not speaking of permenant characteristics of the horse because if you are optimizing a “rail path” could be an inherent factor because it is the shortest distance around the track or “front-running style” could be another factor since front-runners win about 60% of all races.

[Dave Schwartz]

See, there you go, complicating things.

How, exactly, would you determine which "path" a horse was going to take in a race?

I'd really be interested in how you would build a factor for that.

[Cratos]

See, there you go, complicating things.

How, exactly, would you determine which "path" a horse was going to take in a race?

I'd really be interested in how you would build a factor for that.

Isn't it the complexities that make handicapping difficult? If it was easy, all horses might go off at 1-9 odds.

[Dave Schwartz]

Obviously you want to play a semantics game.

This is where I check out.

Good fortune to you.

[LottaKash]

Obviously you want to play a semantics game.

This is where I check out.

Good fortune to you.


:cool: :cool: :cool:

best.

[CBedo]

Just a note to all you aspiring (not expiring, hehe) GA users. There is open source software that you can use to build these models if you want to play around, but the learning curve is steep, and as Dave alluded to earlier, you can't just throw "stuff" against the wall and see what sticks. Data preparation and planning are integral parts of successful models, whether in horse racing or in the financial arena (where I first encountered some of this).

[formula_2002]

Dave, I found the following three definitions of GAs and you appear to be the most knowledgeable on the forum about GAs and am I wrong to say as you make your GA “learn” you are optimizing the evolved data through “inherent” solutions?

(1) A genetic algorithm (GA) is a search technique used in computing to find exact or approximate solutions to optimization and search problems. Genetic algorithms are categorized as global search heuristics. Genetic algorithms are a particular class of evolutionary algorithms(EA) that use techniques inspired by evolutionary biology such as inheritance, mutation, selection, and crossover (also called recombination

(2) Genetic algorithms are based on a biological metaphor: They view learning as a competition among a population of evolving candidate problem solutions. A 'fitness' function evaluates each solution to decide whether it will contribute to the next generation of solutions. Then, through operations analogous to gene transfer in sexual reproduction, the algorithm creates a new population of candidate solutions."

(3) Genetic Algorithms (GAs) are adaptive heuristic search algorithm premised on the evolutionary ideas of natural selection and genetic. The basic concept of GAs is designed to simulate processes in natural system necessary for evolution, specifically those that follow the principles first laid down by Charles Darwin of survival of the fittest. As such they represent an intelligent exploitation of a random search within a defined search space to solve a problem.

Thus this statement that I found might sum up GAs: "the advantage of the GA approach is the ease with which it can handle arbitrary kinds of constraints and objectives; all such things can be handled as weighted components of the fitness function."


I’m struck by some of the terms here. Mutation, Evolve, Selection, Evolution.
I doubt that horse racing roi’s have changed significantly since the 1930’s.
I pick the 30’s because that was the publication era of a book I read some time ago that mentioned favorites win about 30 to 33 % of the time.
Pretty much the same win percentage today.
The point is, I think roi’s are more closely related to the dynamics of coin flipping than evolution.
As far as horse racing roi’s go I think what’s here is here and it does not change.
There are those people that are in the .80 roi, .90 roi, .95 roi and perhaps the .99 roi groups.
Can you “Evolve “ from a .80 roi to a .99 roi? I imagine so, buts its more a mater of better understanding what is there and the “there” does not evolve, its there already.

[fmolf]

I’m struck by some of the terms here. Mutation, Evolve, Selection, Evolution.
I doubt that horse racing roi’s have changed significantly since the 1930’s.
I pick the 30’s because that was the publication era of a book I read some time ago that mentioned favorites win about 30 to 33 % of the time.
Pretty much the same win percentage today.
The point is, I think roi’s are more closely related to the dynamics of coin flipping than evolution.
As far as horse racing roi’s go I think what’s here is here and it does not change.
There are those people that are in the .80 roi, .90 roi, .95 roi and perhaps the .99 roi groups.
Can you “Evolve “ from a .80 roi to a .99 roi? I imagine so, buts its more a mater of better understanding what is there and the “there” does not evolve, its there already.
if its there already then the computer models should be able to ferret out what it is and make recommendations either on selections, or value or most optimum way to construct wagers....

[formula_2002]

if its there already then the computer models should be able to ferret out what it is and make recommendations either on selections, or value or most optimum way to construct wagers....


yes. It's not going to build its way to a new state of things, it will just discover what is already there, and what is there will not change so long as the enviorment remains constant (as it seems to have done in the straight pools for about 70 years).

[Cratos]

Obviously you want to play a semantics game.

This is where I check out.

Good fortune to you.


No Dave this wasn’t a game of being probative through semantics, but it was an attempt to understand why you think that GAs could prove to be a useful tool in the handicapping of racehorses.

However I wish you success with your GA methodology.

[ryesteve]

No Dave this wasn’t a game of being probative through semantics, but it was an attempt to understand why you think that GAs could prove to be a useful tool in the handicapping of racehorses
Why would pointing out factors that can't be known invalidate GAs any more than any other method of handicapping?

[markgoldie]

I’m struck by some of the terms here. Mutation, Evolve, Selection, Evolution.
I doubt that horse racing roi’s have changed significantly since the 1930’s.
I pick the 30’s because that was the publication era of a book I read some time ago that mentioned favorites win about 30 to 33 % of the time.
Pretty much the same win percentage today.
The point is, I think roi’s are more closely related to the dynamics of coin flipping than evolution.
As far as horse racing roi’s go I think what’s here is here and it does not change.
There are those people that are in the .80 roi, .90 roi, .95 roi and perhaps the .99 roi groups.
Can you “Evolve “ from a .80 roi to a .99 roi? I imagine so, buts its more a mater of better understanding what is there and the “there” does not evolve, its there already.


I think it's possible you may have a basic misunderstanding of the way GA's use "evolutionary" tools to data mine. Your statement: "what is 'there' and the there does not evolve, it's there already." is correct. But GA's are not attempting to prove that the game of race handicapping is evolving or not evolving. Ga's are designed to answer questions about what's already there (as you say).

What GA's do do is search for relationships in the data that you provide it which may not be intuitively understood by the human beings that have examined and provided the data.

In order to do that, the GA's look at the fitness of particular data to answer the question posed to it. Like what? Well, let's say we downloaded five years of Brisnet Ultimate PP's of all North American tracks, along with the resultant race charts for each race. Certainly we'd have a lot of data here. Now, let's say we asked the GA to sort through all the pre-race data and tell us how the use of the factors available in this data might enable us to construct wagers that produced a positive roi over the 5-year period.

Okay. Now the GA starts sorting through the data in an attempt to find which of the data seem to have a bearing on achieving the final goal. In this quest, it will test each particular data category by itself. When this fails to solve the problem, as we all know it will, it begins to test groups of data together to see if grouping certain factors works any better. In the initial testing of such groups, failure is assured. However, the GA doesn't stop here. It wants to move on to another, more sophisticated, step to try to solve the problem. But in order to do so, it adopts (what we humans decide to term for want of a better phrase) a "fitness" test of what it has learned so far. This "fitness" is nothing more than another way of saying, "let's stick to the stuff that looks promising." For example, the time of day that each race is run is part of the data. But for argument sake, let's say that in initial testing and in testing grouped with other data factors, this "time of day" factor, if you will, doesn't seem to add anything useful to solving the problem. So why continue exploring "time of day" angles? This factor lacks fitness, so let's omit it from our continuing search.

Before going on, we can see that the concept of fitness has a corresponding meaning in Darwinian evolutionary theory. You know, the survival of the fittest Darwinian discovery. For this reason (and one other coming up shortly) people have taken this analogy to describe what the GA is doing. That's all it is. An ANALOGY. And we could easily write a book on how we humans love analogies. Suffice it to say, analogies are the preferred way in which we understand new concepts. "You see, what it's like is... It's like this..."

Now. Our hard-working GA keeps slicing and dicing the data and eliminating the stuff that doesn't seem promising. It creates data made up of percentages of the raw data: 15% this, 10% that, 12% of the other. Here again, the ANALOGY comes in handy. It sort of like the GA is making new generations of data, killing off the old generations and allowing the new generations to live due to their apparent fitness. Like if your great, great grandfather was Asian, what percent Asian are you? So far, all of this could be done by human intuition quite easily. But now for the kicker.

Since we humans, for the most part, think in only a linear mathematical way about data that describes a phenomenon, we might lack the imagination to see non-linear mathematical relationships even though they are right there in front of our noses, so to speak. In fact, Mother Nature may face the same problem in dreaming up solutions to make species more adaptable to their environment. So maybe we could do what Mother Nature does to create new and dynamic changes that may increase the fitness of the species. What does Mother nature do? She throws in something called mutations. This is Mother Nature's way of finding new and brilliant solutions to a problem. Scramble the factors in some insane and crazy way, and while 99.99% of these will turn out to be failures, it's that miniscule proportion that might, through random luck, produce the miracle we're looking for- a giant leap ahead in fitness!

So why not have our little GA use the Mother Nature approach? Every now and then, while we're examining the new generations of data for fitness, let's throw in something totally illogical, because maybe, just maybe, that illogical something might reveal a hidden relationship in the data which was there all along but which we didn't have the creativity or intuition to investigate.

Like Mother Nature, we will fail 99.99% of the time. But if we connect, we might well solve the problem.

This second factor, the random-mutation factor of GA's, cements the ANALOGY for us. This is now, truly like genetics.

That's where this talk of genetics comes from in the use of GA's. It has nothing to do with the evolution of the racehorse, the genetics of horses, the evolution of wagering, winning favorites' percentages, changing roi's or anything else of the sort. It is an ANALOGY that is applied to a certain form of algorithm. That's all.

Hope this illuminates some of the misconceptions.

Mark

[Dave Schwartz]

Mark,

Sounds like you know a lot about GAs.

The more cutting edge GAs are not so dependant upon mutation as they are "crossover." Just like the real world.

The guys at the Santa Fe institute rather debunked Darwinian evolution some years ago, but being the sensitive topic that it has become, they stopped talking about it. What the "discovered" was ultimately to become misunderstood as "intelligent design."

Specifically, they showed that mutations could not possibly drive evolution as we know it. Instead, it fueld by crossover and latency.

My GAs use a lot of crossover and latency (about 30% of all the genes are unused).

Probably the best book on GAs was written by Mark Goldberg back in 1986, simply titled Genetic Algorithms.

Most of my GAs are of the "LS-1" variety.


Regards,
Dave Schwartz

PS: For anyone interested in a very simple explanation of GAs, here is ana rticle I wrote years ago titled The Concept of Ant Hills.

http://www.horsestreet.com/freestuff/articles/ants/index.html

[Greyfox]

Mark,

Sounds like you know a lot about GAs.

Probably the best book on GAs was written by Mark Goldberg back in 1986, simply titled Genetic Algorithms.



Hmmm? Is Markgoldie = Mark Goldberg? Hmm?

[Dave Schwartz]

Funny, I never thought of that.

If that is true this is the guy that I learned GAs from. That book was just tremendous. I learned far more than from John Holland's books.


What say you, Mark?

[markgoldie]

Yes and no. I am Mark Goldberg, but not that Mark Goldberg.

[fmolf]

Funny, I never thought of that.

If that is true this is the guy that I learned GAs from. That book was just tremendous. I learned far more than from John Holland's books.


What say you, Mark?
horserace handicapping is easy compared to following this thread....its beginning to get a little "Twilight Zonish"....no?

[raybo]

horserace handicapping is easy compared to following this thread....its beginning to get a little "Twilight Zonish"....no?

I understand the theory of neural nets and GAs pretty well, having read quite a bit about both. The application part, the algorithms themselves, how to implement them, etc. is keeping me from further exploration. I'm pretty damn good at Excel design, organizations, formulations, macros, etc., but anything closer to programming, than writing some rather basic macros, and higher mathematics, above college algebra, and my brain starts shutting down.

[formula_2002]

I think it's possible you may have a basic misunderstanding of the way GA's use "evolutionary" tools to data mine......
Mark

I'm sure it would help some people more than others.
Then again, someone who has the ability to understand it, may not need it to solve the problem of picking winners :)
I dont think the game itself is that complicated.
Rather it's peoples expectations that complicate things.

Ofcourse if some GA guys want to post a few hunndred picks, I'll follow along..

[CBedo]

I understand the theory of neural nets and GAs pretty well, having read quite a bit about both. The application part, the algorithms themselves, how to implement them, etc. is keeping me from further exploration. I'm pretty damn good at Excel design, organizations, formulations, macros, etc., but anything closer to programming, than writing some rather basic macros, and higher mathematics, above college algebra, and my brain starts shutting down.There are data mining packages that have all the pieces you need, so you really don't need programming experience. The catch is that setting them up and getting the data in the right formats does require some understanding of the models you are trying to use.

[fmolf]

I understand the theory of neural nets and GAs pretty well, having read quite a bit about both. The application part, the algorithms themselves, how to implement them, etc. is keeping me from further exploration. I'm pretty damn good at Excel design, organizations, formulations, macros, etc., but anything closer to programming, than writing some rather basic macros, and higher mathematics, above college algebra, and my brain starts shutting down.
so he's Mark Goldberg but he's not Mark Goldberg!..."

Norton I'm going bowling, but i'm not going bowling"

[W2G]

Every horse is a unique study... Horse races are one-time occurrences in time. Never again will they be reproduced with the same variables. No exceptions.

We can study Affirmed and Alydar. And we can plug in reams of data into our computers about their matchups.. the times, the trainers, the surfaces, the preps, their Beyer figures, etc.. And then we can do it with the other Triple Crown contenders from other years...We can feed the data-in till the cows come home.. But that's not going to give us any insight on whether we should bet Sunday Silence or Easy Goer in the Belmont Stakes.

Is it?

That about sums up my viewpoint as well.

The major problem I see in the application of AI to handicapping is not in the chosen technique (GAs are the purported topic of this thread) but in the modeled phenomenon itself. Horse races simply do not readily lend themselves to mathematical description and prediction. A researcher might as well try to model and predict the NCAA Basketball Tournament -- and I'm sure many have tried and failed to do just that.

The "engine" driving the model is really beside the point. Take your pick -- linear regression, logistic regression, neural nets, GAs -- any "lift" that one approach provides over another is bound to be incremental. The big breakthrough lies in the successful quantification of heretofore unquantified dimensions of the reality under study. This is where art meets science head on with statistical modeling. And I'm sure that Dave and others spend at least as much of their development time focusing on modeling new factors as they do exploring new statistical techniques or even combining techniques.

But it's all relevant. If for some reason the litmus test was handicapping, say 20 different cards over a 2 day period, I'd give the nod to an AI approach. Any competent, careful horseplayer would suffer significant fatigue in that task. On the other hand, if the litmus test was a much more realistic and practical 1 or 2 cards, my bet would always be on the skilled human horseplayer over the machine learning algorithm.

Again though, if certain players are simply using these AI models as a tool, albeit a primary tool, in their overall handicapping approach, and finding this tool to be of value -- that's pretty much the end of the story.

[CBedo]

On the other hand, if the litmus test was a much more realistic and practical 1 or 2 cards, my bet would always be on the skilled human horseplayer over the machine learning algorithm. Why is 1 or 2 cards more practical and realistic?

I know that I can get a better margin on 1 or 2 cards than the programs I use and am developing, but a higher edge doesn't mean more dollars. Return on assets is your margin multiplied by your asset turnover, and there's no way I can compete on the turnover side with a program.

The reality for me is that I fall in between. While I continue to strive for the automatic bet, I use computer (some AI, some not) based tools to help streamline the process, and they allow me to make more bets on more tracks per day thus generating more total return.

[fmolf]

Why is 1 or 2 cards more practical and realistic?

I know that I can get a better margin on 1 or 2 cards than the programs I use and am developing, but a higher edge doesn't mean more dollars. Return on assets is your margin multiplied by your asset turnover, and there's no way I can compete on the turnover side with a program.

The reality for me is that I fall in between. While I continue to strive for the automatic bet, I use computer (some AI, some not) based tools to help streamline the process, and they allow me to make more bets on more tracks per day thus generating more total return.
Why don't you just concentrate on one or two tracks and increase the amount you put thru the windows?... or just concentrate on what type races you are most successful with?

[CBedo]

Why don't you just concentrate on one or two tracks and increase the amount you put thru the windows?... or just concentrate on what type races you are most successful with?It's a personal choice, but I do have reasons. I'll do my best to explain, but I've just finished a marathon poker cash game session at the Rio, and I'm not sure my brain is functioning properly, haha.

1) If I were betting into Belmont size pools only, then I could up my bet size without even being a rounding error, but I tend to bet mostly into smaller size pools where it doesn't take much to have some pool impact. If I bet two equal odds horses instead of putting it all on one, then assuming they have equal win probabilities, my returns will be higher. (edit: now that I read this, I'm not even sure it makes sense to me, lol. I'll have to play with in in a spreadsheet).

2) Making more smaller bets helps smooth the volatility of returns, hopefully reducing the luck factor somewhat.

3) Psychologically, it doesn't bother me to pass races for value purposes since i know there will be lots more to come. If I had just handicapped one or two tracks, personally, I'd have harder time passing race after race.

I hope that makes sense. Your question is definitely a valid one.

[formula_2002]

I am a genetic algorithm!! :)

http://www.paceadvantage.com/forum/showthread.php?t=58822
PUT IT TO WORK
http://www.paceadvantage.com/forum/showthread.php?t=58919

[ryesteve]

Why don't you just concentrate on one or two tracks and increase the amount you put thru the windows?Because no matter what your edge is, your results will have far less volatility if you bet $100 each on 20 races, than if you bet $1000 on two.

[fmolf]

It's a personal choice, but I do have reasons. I'll do my best to explain, but I've just finished a marathon poker cash game session at the Rio, and I'm not sure my brain is functioning properly, haha.

1) If I were betting into Belmont size pools only, then I could up my bet size without even being a rounding error, but I tend to bet mostly into smaller size pools where it doesn't take much to have some pool impact. If I bet two equal odds horses instead of putting it all on one, then assuming they have equal win probabilities, my returns will be higher. (edit: now that I read this, I'm not even sure it makes sense to me, lol. I'll have to play with in in a spreadsheet).

2) Making more smaller bets helps smooth the volatility of returns, hopefully reducing the luck factor somewhat.

3) Psychologically, it doesn't bother me to pass races for value purposes since i know there will be lots more to come. If I had just handicapped one or two tracks, personally, I'd have harder time passing race after race.

I hope that makes sense. Your question is definitely a valid one.
that is the way professional cardcounting blackjack players operate ..thousands of hands at a small edge...increasing bet size as their perceived edge is greater.... in horse racing it seems to me the more races you play the more chance you have of having a negative roi.....when i perceive my horse to be a 10% overlay i bet x....when i perceive a 20% overlay i bet (x+y).....perhaps i am just ingrained with old school win only pre simulcast betting habits....although i do concentrate on 2 sometimes three tracks at once but go no further than betting a few exactas....i think i understand your points...i do about depressing the odds in the smaller pools.

[Dave Schwartz]

in horse racing it seems to me the more races you play the more chance you have of having a negative roi.....when i perceive my horse to be a 10% overlay i bet x....when i perceive a 20% overlay i bet (x+y).....

The number of races you play does not change your ROI. Your ROI is dependant upon your expectancy. A negative expectancy, that is a real negative expectancy (rather than perceived) will have to produce a negative result.

Positive produces positive. Negative produces negative.

BTW:

Everyone seems to accept that you cannot turn a losing expectancy into profit with a betting strategy but many seem to struggle with the fact that you cannot turn a winner into a loser either.

If you could, then the house's advantage at craps or roulette could be turned into a disadvantage with a betting strategy.

While you can certainly change (exagerate or minimize) your result with a betting strategy, you cannot "cross over" from negative to positive or positive to negative.


Regards,
Dave Schwartz

PS: I fully expect that THIS will cause a whole new area of disagreement. Should keep us busy most of the weekend and possibly into next week. <G>

[fmolf]

The number of races you play does not change your ROI. Your ROI is dependant upon your expectancy. A negative expectancy, that is a real negative expectancy (rather than perceived) will have to produce a negative result.

Positive produces positive. Negative produces negative.

BTW:

Everyone seems to accept that you cannot turn a losing expectancy into profit with a betting strategy but many seem to struggle with the fact that you cannot turn a winner into a loser either.

If you could, then the house's advantage at craps or roulette could be turned into a disadvantage with a betting strategy.

While you can certainly change (exagerate or minimize) your result with a betting strategy, you cannot "cross over" from negative to positive or positive to negative.


Regards,
Dave Schwartz

PS: I fully expect that THIS will cause a whole new area of disagreement. Should keep us busy most of the weekend and possibly into next week. <G>
i am sure it will.....what i think i meant before is that when i personally bet too many races my strike rate goes down and hence my roi follows.That is why i do better sticking to the same two tracks.

[Dave Schwartz]

In other words, when you play more races you lower your standards for plays and these lower-standard plays produce a lower result?

[fmolf]

In other words, when you play more races you lower your standards for plays and these lower-standard plays produce a lower result?
yes because i usually get four to six plays from 18 races(2 tracks)if i handicap another track or two i am not as thorough and my strike rate goes down....i only have a limited amt of time to use for handicapping so i only handicap two tracks

[Dave Schwartz]

So, the reduced ROI is a function of lowering your standards rather than just more plays.

Yes?

[fmolf]

So, the reduced ROI is a function of lowering your standards rather than just more plays.

Yes?
not really the more races i handicap the lesstime i spend on each one i guess you could say i lower my standards its all semantics really...bottom line the more races i handicap the less i seem to win...lol

[formula_2002]

I'm sure it would help some people more than others.
Then again, someone who has the ability to understand it, may not need it to solve the problem of picking winners :)
I dont think the game itself is that complicated.
Rather it's peoples expectations that complicate things.

Ofcourse if some GA guys want to post a few hunndred picks, I'll follow along..

as i've said, this is an easy game (as long as you have great data)http://www.paceadvantage.com/forum/showthread.php?p=704513#post704513

[Dave Schwartz]

What we need now is for someone with the tenacity of the "old Formula 2002" to come in and insist that it is impossible to win.

Then you could use even bigger font sizes - and maybe colors - to scream your frustration at him (like we tried to do with you when you instisted that the game was not beatable).

Now, if we can just get you to understand that there are many ways to beat the game. Admittedly some are better than others.


Joe, I truly congratulate you for your persistence and your success.


Regards,
Dave Schwartz

[formula_2002]

What we need now is for someone with the tenacity of the "old Formula 2002" to come in and insist that it is impossible to win.

Then you could use even bigger font sizes - and maybe colors - to scream your frustration at him (like we tried to do with you when you instisted that the game was not beatable).

Now, if we can just get you to understand that there are many ways to beat the game. Admittedly some are better than others.


Joe, I truly congratulate you for your persistence and your success.


Regards,
Dave Schwartz

Thanks Dave, but I really have not proven anything thing yet other than some people win, some lose and not always the same people :)

[ryesteve]

Thanks Dave, but I really have not proven anything thing yet other than some people win, some lose and not always the same people :)Given where you started, I THINK this is a step in the right direction :)

[Dave Schwartz]

Thanks Dave, but I really have not proven anything thing yet other than some people win, some lose and not always the same people

I am not totally sure I have grasped your meaning. In fact, I think we may be in disagreement here.


I have found that winners win and losers lose.

I have found that the winners group will be the same bunch of guys from year to year, as will the losers. When one crosses over from loser-to-winner, they generally remain as winners for years to come.


Regards,
Dave Schwartz

[fmolf]

I am not totally sure I have grasped your meaning. In fact, I think we may be in disagreement here.


I have found that winners win and losers lose.

I have found that the winners group will be the same bunch of guys from year to year, as will the losers. When one crosses over from loser-to-winner, they generally remain as winners for years to come.


Regards,
Dave Schwartzi call myself a recreational/serious handicapper ...Someone who does not bet every day but maybe i get to play once sometimes twice a week....i do not gamble from home and i prefer to be at the track.I have had some winning years and some losing years.I would not classify myself as either.the years that i did lose the most i was experimenting with different ways to play ..tri's, supers, p3's an 4's...now i have settled on win betting and exactas only.I still have losing weeks and months but now i win more often than i lose.

[Dave Schwartz]

I am referring to every day players.

Or, I suppose I could say after some number of wagers... say 4,000, I would expect your results would be consistent with the last and next block of such races.

The more longshots you play the more volatile your results would be.

If one chooses to embrace this type of belief - I do because it makes sense to me - then one realizes that there are weekend players that only make that many bets over the course of a couple of years (or longer).

This means they might not really know for several years whether they are really winning players.

[Mike A]

I am not totally sure I have grasped your meaning. In fact, I think we may be in disagreement here.


I have found that winners win and losers lose.

I have found that the winners group will be the same bunch of guys from year to year, as will the losers. When one crosses over from loser-to-winner, they generally remain as winners for years to come.


Regards,
Dave Schwartz

Yeah Dave, but he needs scientific proof. Oh well, back to the beginning ahhaahahahhaha.....

(Formula, don't take this the wrong way...you're a good guy, and a bright guy, and I admire your persistence and spirit of objective inquiry.)

[GameTheory]

I am resurrecting this thread that I missed, mainly because it doesn't seem right for me NOT to post in a thread titled "genetic algorithms". I've made at least 38 of them (why are you counting them, Dave?) -- hundreds. I only perused the earlier posts, and I see there was lots of clarification, but still plenty of major misconceptions about genetic algorithms.

So I'll boil it down to utter simplicity.

Genetic algorithms find solutions to problems. What's that mean?

That means you've got a bunch of "slots" you need to fill up with numbers:

slot1 slot2 slot3 slot4... etc

Why would you have slots you need to fill up with numbers? Because that's the solution to your problem. For instance, to use the classic example of factor weights, you've got four handicapping factors you'd like to combine some how into a final number. Nothing fancy, you just want to combine them. But you suspect some of the factors are more important than the others, so you wonder what the "optimal" mix might be? Well, now you've defined your problem as 4 slots that need filling up with numbers:

final number = (factor1 * weight1) + (factor2 * weight2) + (factor3 * weight3) + (factor4 * weight4)

So, we need weights, but we have no idea what they should be. Couldn't we just try every possible combination down to 3 decimal places (so the weights add up to 1)? YES. But what if you got 100 weights instead of 4? What if it takes a couple of seconds of computer time to check each possible solution? That would take a long time. So we might use a genetic algorithm to help narrow in on a good solution quicker. (Although GAs can be slow as well, but they don't get slower at the same pace as the problem complexity gets bigger. So they are most appropriate for hard problems where direct solutions are not practical, even if they are possible in theory.)

Anyway, so we start by generating a bunch of random solutions (vectors of 4 weights each) like so:

.02 .43 .23 .87
.56 .32 .11 .32
etc.

(Our weights won't necessarily add up to 1, so just assume we'll normalize them in the evaluation phase -- that's no problem.)

That's our "population" -- it can be big or small depending on how we code our GA. A vanilla implementation would use a fairly large population -- 100s or 1000s or potential solutions. Then we check how good they did by plugging in the weights of each potential solution into our equation and checking how that performs on a sample of horses. We have to choose a goal here, which means we set some criterion by which we can say one solution is better than another -- we need to give each one a score. This is completely up to the user. Do you want to optimize to pick winners or to maximize value? To pick the leader at the first call? Whatever you like. That's our goal -- that's what we are "evolving" toward.

So we go through and give each solution a score -- that is its "fitness". Then, guess what happens? The fittest of the lot are the ones that survive. We keep the better solutions, dump the worse solutions; and then move to the next generation.

The next generation is created primarily through crossover (usually). Crossover is just like it sounds, like a DNA sequence. We pick two "parents" from our good solutions and their "genes" survive. (Whether or not exact copies of the parents also survive into the next generation is up to you -- usually only the few very best are copied unchanged to the next generation -- the rest are new "children".)

So:

parent1: .02 .43 .23 .87
parent2: .56 .32 .11 .32

reproduce to create

child1: .02 .32. 11 .32
child2: .56 .43 .23 .87

See how they crossed-over between gene 1 & 2?

And we also throw in a bit of mutation here and there whereby we just randomly change some gene to something else.

And thus is the new generation born. We score those as before, rinse, repeat. Until when? Until we're satisfied with the best solution so far. With minimal mutation, the genetic pool becomes smaller each generation, and the solutions become more and more alike. We are "converging" upon a good solution. If there are multiple good solutions that are completely different, it will evolve in a few different directions for a while, and eventually (if we don't mutate) -- they will all end up the same. Does this mean we've found the very best solution there is? No -- we could run it again and it might break another way and come up with something completely different but with also good results. (And keep in mind you're just fitting to the training sample and probably over-fitting at that, but that's another discussion. You might be trying to solve a problem that has one definite solution and you'll likely find it.)


Q: Uhh... aren't their a gazillion other ways to come up with such solutions? Regression, neural networks, hill-climbing algorithms, particle swarm optimization, ant colony optimization (not what Dave's "ants" do, since he just said they are a GA), classifier systems, decision trees, Bayesian learning, thinking? What's special about GAs?

A: Nothing. They are just one technique to use to fill up those slots with numbers. They are good for large search spaces where you have no idea what the right answer might be. But so are a lot of other techniques.


Q: What if my problem can't be defined as slots that need to be filled up with numbers?

A: It probably actually can be, and you need to think about it some more.


See how easy it is? Just a way to generate a set of numbers that are a good solution to some user-defined problem. That's it.

See all the other things that it has nothing to do with?

[formula_2002]

Genetic algorithms find solutions to problems.
GT, have you used GA to make a long term profit, in say 2000, bets?
Of course I'm refering to pari-mutual, horse race betting.

Thanks

[GameTheory]

GT, have you used GA to make a long term profit, in say 2000, bets?
Of course I'm refering to pari-mutual, horse race betting.

ThanksNo, I don't think so. GAs are actually what got me into horse racing, but I rarely use them for anything anymore. I prefer other some of the "gazillion other methods".

Like I said, there is nothing special about them. Whether or not you make money using GAs really doesn't have anything to do with GAs because you could arrive at the same answers a lot of other ways. It is the factors you are using, the problem you are trying to solve, the way you bet that makes you money; not the method by which you arrive at an answer to a problem.

That is one of the misconceptions I'm trying to clear up -- that GAs are somehow a handicapping methodology unto themselves. They aren't -- it is just a way to fill up those slots with numbers.

[markgoldie]

GT;

I was the originator of the thread. I understand what GA's are and how they are used. The main question I have is a very simple one, but I don't believe it has been answered to my satisfaction.

As we know, a GA is nothing more than a tool which finds answers to problems. Fine. So, for example, if I used 5 years worth of Brisnet pps at all North American tracks and the corresponding race charts with payoffs as a data base, is there any reason why I couldn't simply ask the GA to find a wagering methodology that produces, say, a 10% positive ROI, without restriction to wager type? That would be the problem we want solved.

After turning the GA loose on the data base and problem, it would seem that only two things are possible: Either it succeeds or fails in providing the answer. If it fails, we might run it again and to help insure success, lower the parameter of positive ROI required.

Now for the part I DON'T get. Why is it necessary to "build" so many different GA's, unless each successive GA represents an improvement of some sort over the preceeding model? OR, is a GA somehow "problem-specific" in that it is never a general tool, but always must be re-written to attack the specific type of problem given to it? (Naturally, I realize that even as a general tool, some sort of customization would be necessary so as to "explain" the rules of the game in question.)

But I hope you can see what's at the heart of my confusion. If we have a tool that functions as you and other say they do, why so many necessary versions? And why wouldn't a competent GA, relentlessly doing its job of slicing, dicing, combining, and twisting (mutating) data be as good as any other possible GA in achieving it's task? Accepted that re-runs may be necessary. But (without addressing the issue of speed), why would one version of a GA be better than another?

And finally, I understand that what worked five years ago might not work today because of new bettors, whales, etc., and that this might necessitate a more recent time-parameter update for current profitability. But that being said, why don't we simply have a yes or no answer to the question?

Thanks.

Mark

[GameTheory]

As we know, a GA is nothing more than a tool which finds answers to problems. Fine. So, for example, if I used 5 years worth of Brisnet pps at all North American tracks and the corresponding race charts with payoffs as a data base, is there any reason why I couldn't simply ask the GA to find a wagering methodology that produces, say, a 10% positive ROI, without restriction to wager type? That would be the problem we want solvedLet's take it one step at a time, and I think it will become clear. You wonder above why can't you simply ask the GA to find a wagering methodology... stop right there.

How do you ask a GA to find a wagering methodology? How do you ask a GA to find anything? You've got to create a representation of the problem, i.e. the form of your question to the GA is a bunch of slots that need numbers in them. How are you going to represent a wagering methodology as a bunch of slots that need numbers? Let's switch to the more technical word "parameters" at this point -- GAs optimize parameters. What are the parameters? Show me how you're going to represent a possible wagering methodology (that could be checked against your db) as a set of parameters such that altering the value of any parameter represents a different methodology?

Once you can answer that, we can move on.

[Dave Schwartz]

As we know, a GA is nothing more than a tool which finds answers to problems. Fine. So, for example, if I used 5 years worth of Brisnet pps at all North American tracks and the corresponding race charts with payoffs as a data base, is there any reason why I couldn't simply ask the GA to find a wagering methodology that produces, say, a 10% positive ROI, without restriction to wager type? That would be the problem we want solved.

I'll take a swing here...

It all depends upon how good you are at structuring the data. You are not just going to throw in (say) 1,000 pieces of raw data and get a good answer. Instead, you must build meaningful factors.

What GT has been saying is that a GA is, in effect, a weight-making engine, just like different forms of regression analysis. Of course most forms of regression are deterministic in nature. Those generally fail as there are just too many factors to deal with and the issue of correllation vs. causation is difficult to overcome. (That is, there are big inter-dependance issues between the factors.)

That leaves statistical approaches of an iterative nature. Of those, GAs are one, as are neural nets, and other forms of machine learning. The ultimate output is a set of weights in some form.

The "form" of those weights are part of the "structuring" of the approach. As an example, consider the standard R^2 regression that everyone with any statistics background learns first... that form is:

BaseNumber + (factor1 * weight1) + (factor2 * weight 2) ... +(factorN * wegihtN).

So, you have variables:

BaseNumber= ? (i.e how many points does each horse start with?)
Factor#1 = ? (i.e. in your factor list, which factor is in this slot?)
weight#1 = ? (i.e. what is the multiplier for this factor?)

Factor#2 = ? (i.e. in your factor list, which factor is in this slot?)
weight#2 = ? (i.e. what is the multiplier for this factor?)

...
Factor#N = ? (i.e. in your factor list, which factor is in this slot?)
weight#N = ? (i.e. what is the multiplier for this factor?)


So, you see, ultimately, it becomes a table of blanks to be determined by the analysis method you have chosen.


In summary, there are three areas of concern:

1. How are the factors built?
2. What is the form of the variable output?
3. How is success measured?


#3 is not as cut and dried as you might think. Resist the tempation to say, "I want a system that is profitable." Think of the entire training process much like the process that a person goes through in order to learn.

In order to become a winning player you must try something and benchmark that result. Then you try something slightly different different and compare the new result with the previous best - the benchmark. If the new result is an improvement the new approach becomes your new benchmark system, if not then it is discarded.

How you measure improvement is critical!

Suppose you base everything on $Net. It is very likely that you will wind up with a system that is profitable (or at least has been profitable historically) but did so with a very low hit rate. In other words, if hit rate is not part of the goal then you will wind up playing for $60 horses.

So, a logical approach might be to weight $net more heavily than hit rate but not to ignore hit rate.



All this is simplistic, but should help your understanding.


Dave

PS: Someone asked if I (or maybe it was GT) had ever built something that was profitable over say 2,000 races. The answer is "yes."

[TrifectaMike]

GAs optimize parameters.





As in all surface response methodlogies, optimization of an objective function is the goal. In the case of a GA the objective function is a fitness function. Quite often a fitness function is very hard or nearly impossible to define.

Mike

[GameTheory]

As in all surface response methodlogies, optimization of an objective function is the goal. In the case of a GA the objective function is a fitness function. Quite often a fitness function is very hard or nearly impossible to define.Yes, that's right but "surface response?" "objective function?" "fitness function?" We're trying to make things clearer!

[Dave Schwartz]

GT,

Saw your tagline: "Over 7 1/2 years online!"

And just for the record, I would like to point out that I have been online at PA since March, 2001... Almost 8 1/2 years. :D



Dave

[sjk]

When I started downloading charts to make a database I made use of something called Tymnet. I could choose between 1200 and 2400 baud.

Things are much changed since 1993.