Custom Expression Builder

[nucsub]

I have been writing handicapping software for about 4 years. Initially, for my own pleasure and then I made it available to the public.

It wasn't until TomC asked my to write software for him that I began writing custom handicapping programs and some "greyhounds", too. Some of you may have remembered my thread called "Handicapping Chickens"...

Anyway, During the last few years many people have contacted me about writing them their own handicapping program. I would review the scope, and provide them with a quote. Too often they would see the cost of programming and then decide against the project.

Well, I have worked on something for about 6 months that I think is the next best thing. I am curious, has anyone used or written a program where you can actually develop expressions on the fly. No programming needed... Pick from a list of hundred or more variables, create and add constants, utilize mathematically operators. Select sampling criteria like average last 3 races, last 5 races, etc. Also added some elimination criteria like eliminate lowest value. high and low, etc.

I don't write too often at this site, I guess I'm too busy either working (nuclear power plant - Maintenance Mgr), Programming, or Handicapping. Although, I have read some nice comments about my software at this site from time to time.

So for those who would like to build there own expressions without programming and at a fraction of the traditional cost, you might want to check this out.

www.jzambuto.com

[Dick Schmidt]

Nucsub,

Yes, this has been done before. Dave Schwartz developed a similar program called the Handicapper's Notebook, and you could manipulate hundreds of different factors in a great variety of ways. It was fun to play with, but almost impossible to use.

The problem is that if you have 100 variables, plus a number of different ways to use them, you have created a problem that is not on a human scale. No human can possibly keep track of the literally billions of possible combinations. Dave tried to have the computer check combinations, but even that could take years. If presented with 100 variables, the first logical question is "which is the most important?" Answering that simple question is an enormous research project. Of course, the next question is "which two factors work together best?" I know that this seems a tempting approach, and some may well find it fun to tinker with, but I think it remains a problem far beyond human (or current computer) scale.

Dick

[nucsub]

Dick,
Thanks for the reply. I no exactly what you are saying. That's not what I am really talking about. Instead of manipulating hundreds of variables, I am talking about selecting what ever parameter or # of paramters, then building your own algorithm to see exactly how it performs.

The handicapper can build up to 4 of these algorithms. No different that what currently exists in virtually all handicapping software programs. Take a look at some of the screen dumps at my website and see if we are talking about the same thing.

Thanks again

www.jzambuto.com

[GameTheory]

The latest version of my charts parser (which at the moment is only available directly from me until I get around to finishing the docs) can do this (sort of), although only to generate your own custom numbers from the raw material in a single race (since we're dealing with charts) and it is not terribly graceful to use.

Is it not particularly hard to allow the user to enter a mathematical expression specifying what the various variables are to be and then computing that. So to make a tool that could take a comma-delimited pp file as input and run custom formulas on it and produce various figures would be no big deal, and I was actually thinking of doing that at some point. Making it easy to use (some sort of visual gui where you could see all the factors, check them off, etc.) would be the real effort, not to mention connecting them to a results database to see how they perform.

But to make a simple version wherein the program would take *any* comma-delimited file as input, and having the user specify: Ok, take fields #82, #83, #101, & #232 and run them through such-and-such set of formulas provided by me to produce this new set of figures as output wouldn't be hard. Basically the user would be doing his own programming as far as the math was concerned, but just having the input/output details of the files taken care of for him...

[ranchwest]

I find that often the most significant benefit is from conditional evaluations and it seems to me that complex conditionals combined with arithmetic formulae would be difficult to make available as a wide use tool, but I've never really tried to program something of sophistication in that regard.

[GameTheory]

Conditional evaluations based on numbers & stats? Or on human judgement?

What's an example of a "complex conditional"?

[ranchwest]

GT,

Let's suppose someone needs Quirin Speed Points and the calculated value is not available. I would think that providing the ability to calculate the points would be rather difficult because there are so many conditionals.

I was referring to conditionals based on numbers and stats, not human judgement.

When these posts refer to providing expressions, I'm envisioning something along the lines of "add 4 points to my total points if the speed index of the last race is better than the par rating of today's race", utilizing field names and mathematical expressions, of course.

[GameTheory]

Nah...

That kind of stuff isn't too hard. You can in fact do that with my charts parser now through the use of boolean expressions to create conditionals. (I completely misunderstood what you meant by conditionals -- I was think of the condition evaluations of the horse -- i.e. physical condition. I should have remembered we were talking about math.) You couldn't make speed points because you need data from more than this race, but using the exact same tools I've provided in the parser you could make speed points if applied to a pp file.

Now, using only mathematical expressions and not structures like IF THEN ELSE statements it is a bit more confusing, but can be done. For instance, in my parser, if you want to create a custom field that calculates some number and needs to use the 4f time in sprints or the 6f time in routes, you would do something like:

(v's are variables)

v1 = fraction:4f
v2 = fraction:6f
v3 = distance in furlongs
v4 = ((v3 < 8)*v1) + ((v3 >= 8)*v2)


Variable v4 now contains either the 4f time or the 6f time depending on whether the distance of the race is less than 8 furlongs. How? Because of the boolean expressions, which evaluate to 1 if TRUE, and 0 if FALSE:

If the distance is 6f (v3) , then

((v3 < 8)*v1) + ((v3 >= 8)*v2)

becomes

( 1 * v1) + ( 0 * v2)

becomes

v1 + 0

equals v1.

If the distance was 8f or more, the result would have been v2. So you have an if statement within a mathematical expression.

Now you can see how this would get kind of complicated, but basically you just chain things together like that...

[ranchwest]

Sounds pretty nifty.

[rrbauer]

Generally, as is being pointed out in the posts here, the problem with doing large scale statistical analysis is not with the computers. Today's desktop computers do millions of calculations per minute and never take a deep breath.

The problems are with properly classifying your variables, designing your study model and not getting drowned in the data (meaning doing some data reduction, and coding, and tossing out details that aren't contributing to what's being studied). The latter exercise falls in the realm of data mining.

If you have some interest in this stuff it's possible to get a student version of SPSS for under $100 and many of the leading college web sites have tutorials and "how to" pages for using SPSS (I think it stands for Statistical Procedures for Social Sciences!)

I think that nucsub and game theory have good ideas and I'm glad to see them getting some air in this thread.

[GameTheory]

There are a number of open-source / free / etc. packages available to get you started, including clones of the the expensive packages. Take a look at:


http://www.statpages.net


You really need to take a statistics course & be a programmer to understand much of it, however. Most of this advanced statistics stuff is beyond me...

[ranchwest]

I'm a programmer, but my knowledge of statistics is fairly rudimentary.

[hurrikane]

yeah, I'm a statistically rude programmer myself. Interesting ideas though. Will check into it...

[mgroves]

NUCSUB,

QuickHorse includes a "Method Builder" which allows the user to create up to 8 algorithms per "Method" to describe the horse based on the data in the database.

The generated Method can then be backtested and "supertuned" to determine which columns are more important in picking winners.

Our problem has been in getting feedback regarding the interface which is designed so that NO ERRORS are made on the part of the person building the software. We would like to create custom modules which work on the data. At present, average, max, and min are available in addition to +, -, *, and division.

You're certainly welcome to download the QuickHorse Software at -

http://business.gorge.net/jai/horses.htm

It has a 30 day free subscription so that a person can determine if it helps them or not, no obligation, and its $49.95 a year on a subscription basis.

Mike Groves
Manager - QuickHorse Software Development
JONSOG associates, Inc.