PDA

View Full Version : software comparison


green80
08-19-2015, 12:05 PM
Has anyone ever done a comparison of the many handicapping software packages on the market today? I know this would have to be the black box type as the ones with user input would have as many different results as it does users. Maybe what software works best out of the box would be the better question.

NorCalGreg
08-21-2015, 02:00 AM
Has anyone ever done a comparison of the many handicapping software packages on the market today? I know this would have to be the black box type as the ones with user input would have as many different results as it does users. Maybe what software works best out of the box would be the better question.

Pretty sure an honest evaluation has never been done. If a well-respected person, consortium, firm, group, whatever, got together and asked for software submittals for just such a test...they'd get MAYBE half dozen, you think? I heard more than one program writer blame the users--we didn't use the program properly. Well whose fault is that? Do a better job of explaining the sumbitch.
Anyway G-80...someone has a poll going at this time about which 'capping software people are using--some surprising results. Later-NCG

TonyMLake
08-25-2015, 08:48 PM
Has anyone ever done a comparison of the many handicapping software packages on the market today? I know this would have to be the black box type as the ones with user input would have as many different results as it does users. Maybe what software works best out of the box would be the better question.


We tried that. There were only three or four software developers who were willing to expose their apps for the comparison; Raybo and I were the two I remember, but I think a couple of others were willing as well.

We never really reached a good set of operational definitions - for example, as you alluded to above, what even constitutes "Black Box"?

I'd be interested in a "contest" with any of the software out there using the Handicapper Plus Algorithm at the default settings as long as my software only bets on the races HandicapperPlus calls "High Confidence" races.

Raybo wanted a different race selection algorithm for obvious and legitimate reasons, but that constituted another small barrier. I don't see why we couldn't just "pick 500 races from a season" or something to solve that issue.

How soon should the bets be made before the race? Obviously, we need to post the bets ahead of time.

These are small issues that came up. I think it was Raybo(?) who pointed out that by the time we reached a statistically significant reflection of a real ROI, well, let's just say the contest would take too long for most people to maintain interest... but If anyone is willing to work together to hammer out the rules, HandicapperPlus will play ON IT'S OWN, AS A BLACK BOX.

atlasaxis
08-25-2015, 09:59 PM
http://www.paceadvantage.com/forum/showthread.php?t=66274&highlight=black+contest

TonyMLake
08-26-2015, 12:01 AM
http://www.paceadvantage.com/forum/showthread.php?t=66274&highlight=black+contest

Didn't know about that one. We were talking about it a couple months ago as well.

By the way, what happened after the first day five years ago?

raybo
08-26-2015, 03:14 AM
Didn't know about that one. We were talking about it a couple months ago as well.

By the way, what happened after the first day five years ago?

Handifast won the "true" black box category (1.26 ROI ?), AllData PPs (the free program, not my current Black Box) finished 2nd (1.24 ROI - went "0 fer" at Aqueduct :bang: )

pandy
08-26-2015, 08:57 AM
Has anyone ever done a comparison of the many handicapping software packages on the market today? I know this would have to be the black box type as the ones with user input would have as many different results as it does users. Maybe what software works best out of the box would be the better question.


I've been posting the top ranked pick for all races (except races with several first time starters) from my system (Diamond System) for Del Mar and Saratoga all this meet at http://www.handicappingwinners.com/free_picks.htm

You have to scroll down the page because I've also been testing a new method which is near the top of the page.

Both meets have a profitable ROI. What's interesting, too, a couple of weeks ago I started posting alternative picks, usually either the second or third ranked horse, whichever has the longer odds, and they are showing a profit, too. I've also been posting top two early and late speeds from the consensus box feature.

DeltaLover
08-26-2015, 09:58 AM
I've been posting the top ranked pick for all races (except races with several first time starters) from my system (Diamond System) for Del Mar and Saratoga all this meet at http://www.handicappingwinners.com/free_picks.htm

You have to scroll down the page because I've also been testing a new method which is near the top of the page.

Both meets have a profitable ROI. What's interesting, too, a couple of weeks ago I started posting alternative picks, usually either the second or third ranked horse, whichever has the longer odds, and they are showing a profit, too. I've also been posting top two early and late speeds from the consensus box feature.

So, where we can see your picks for today and the rest of the week?

pandy
08-26-2015, 10:08 AM
So, where we can see your picks for today and the rest of the week?


The picks are posted, you have to scroll down to get past the test for the new method. It's nothing fancy, just today's date and a number for each pick but the picks are there.

http://www.handicappingwinners.com/free_picks.htm

DeltaLover
08-26-2015, 10:30 AM
The picks are posted, you have to scroll down to get past the test for the new method. It's nothing fancy, just today's date and a number for each pick but the picks are there.

http://www.handicappingwinners.com/free_picks.htm

ok

punteray
08-26-2015, 01:21 PM
How does the program determine a "high confidence " race?

TonyMLake
08-26-2015, 01:28 PM
Handifast won the "true" black box category (1.26 ROI ?), AllData PPs (the free program, not my current Black Box) finished 2nd (1.24 ROI - went "0 fer" at Aqueduct :bang: )

Incredible returns, there. I'm impressed. How many trials?

pandy
08-26-2015, 01:41 PM
How does the program determine a "high confidence " race?


If you're talking about my program, it would depend on a couple of factors, the point spread advantage being one, but there are a couple of other ways to run the system to quantify the strength of the top pick. For instance, these picks are based on best of last 3 which means that it could have used the third line back. If I run option Q it gives me each horse's last two races. If the horse is also highly ranked based on its last two races, that increases the likelihood of the horse winning the race. That's just one of the way to determine how strong the top pick is. Of course, when it picks a $60 winner on top that horse rarely looks like a high confidence play.

raybo
08-26-2015, 02:05 PM
Incredible returns, there. I'm impressed. How many trials?

Just 6 specified tracks, 1 track on each of 6 consecutive days, bet all races on the card, 1 pick per race (and maybe one alternate should your pick scratch?), $2 flat bet to win, best ROI wins.

raybo
08-26-2015, 02:26 PM
Just 6 specified tracks, 1 track on each of 6 consecutive days, bet all races on the card, 1 pick per race (and maybe one alternate should your pick scratch?), $2 flat bet to win, best ROI wins.

I posted some of the individual day's results here, starting with post #11: http://www.paceadvantage.com/forum/showthread.php?t=65749

FocusWiz
08-26-2015, 02:52 PM
Just 6 specified tracks, 1 track on each of 6 consecutive days, bet all races on the card, 1 pick per race (and maybe one alternate should your pick scratch?), $2 flat bet to win, best ROI wins.Ray,

Other than bragging rights, do you truly believe that this provides useful information that can be used to evaluate which is the "best" package? Is your package designed to play every race? Some are, and some are most useful if they likewise decide how to spread the wagers across multiple runners. Others can do that but need very late odds to decide how (or if) a wager should be made.

I realize that this would cause other problems, but in my opinion, I would want to see how a package performs when it is recommending a play rather than when some arbitrary play was dictated. Especially if I were to quantify something as "Black Box" I would not want to be overriding it by telling it what to do and how.

It may just be me, but I have a lot of trouble with understanding how this provides more than bragging rights. Dictating how much to play and how many to play could be thwarting the underlying design of the software.

raybo
08-26-2015, 05:59 PM
I'm quoting your comments and questions below, and answering, to the best of my ability, in red font.

Ray,

Other than bragging rights, do you truly believe that this provides useful information that can be used to evaluate which is the "best" package? Is your package designed to play every race? Some are, and some are most useful if they likewise decide how to spread the wagers across multiple runners. Others can do that but need very late odds to decide how (or if) a wager should be made.

No, the "Challenge" proved very little about the viability of any program or method, because it was way too short in duration, and it only included 6 tracks, of one card each. The only thing the contest accomplished was what each program or method produced for win play, flat $2 betting for those 6 cards, nothing else.

My program is not designed to play every race. Most of the rankings methods use fractional velocities for their ratings, so any race where more than one or two horses have no races at all, or don't have any distance and surface qualified races to use for their calculations, are pass notifications in the program. Basically, if 20% or more of the field have no qualified races then the pass notification is issued, meaning the race shouldn't be played.

We "test" tracks, using data and results files from previous periods of time, which populates each rankings method's own record keeping sheet with the program's picks (top 3 ranked horses) and their results. We have 11 primary rankings methods, so we also have 11 record keeping sheets, one for each of those rankings methods. Each record keeping sheet includes a summary of the top 3 picks, individually, as well as all 3 of those picks combined, showing number of plays, dollars bet, dollars paid, hit rate, and $2 ROI, and total dollars won or lost, for each of the 3 picks and also for all 3 picks combined. We also have a minimum odds report on each record keeping sheet showing the same stats for each of several minimum odds levels: .01/1 or higher, 1/1 or higher, 2/1 or higher, 5/2 or higher, etc., through 11/1 or higher. of course each record keeping sheet also displays each race played showing track, date, race number, each of the 3 top picks' win, place, and show amounts won, number of bets, total amount wagered, $2 profit/loss, $2 ROI, surface condition, rankings method used, race distance, surface type, day of the week, and class of race.

The "testing" of each track allows us to see which of the 11 rankings methods did best and at what minimum odds level, for all 3 picks, both individually and in combination for multiple horse win betting, all according to what each user's goals and priorities are. Some are ok with playing fewer races and making a better ROI, some want lots of plays at a lesser ROI, and some are somewhere in between those. So, we choose the method we want to use for a particular track, which of the 3 picks we want to play, and at what minimum odds as the horses enter the gate. We can choose to play 1 pick, 2 picks, or all 3 picks, based on the results of our testing.

I realize that this would cause other problems, but in my opinion, I would want to see how a package performs when it is recommending a play rather than when some arbitrary play was dictated. Especially if I were to quantify something as "Black Box" I would not want to be overriding it by telling it what to do and how.

The program produces picks for all 11 rankings methods, in a single display/view, all at the same time, our "testing" tells us which one to use and at what minimum odds, and which of the 3 picks to play, according to our goals and priorities. The pass notifications tells us when to play and when to pass the race completely. Of course the user is free to do whatever he/she wants, they can play races that the program says to pass, they can play any rankings method they want to play for any race type they choose, they choose to play at any odds they want, it's all up to them. So, they can choose to play the way the track testing tells them is best, or they can decide to override that testing and play the way they choose instead. But, the track testing routine plays only program notified "play" races and ignores all "pass" notification races, for each of the 11 primary rankings methods.

It may just be me, but I have a lot of trouble with understanding how this provides more than bragging rights. Dictating how much to play and how many to play could be thwarting the underlying design of the software.

Our track testing routine uses only $2 flat bet wagering for its results. However, the user can choose to use the automated bet sizing app inside the program for track testing and/or daily play, by entering starting bankroll, percentage of current bankroll to wager, and percentage of profit to trigger automated bet size increases. This app is a variation of percentage of bankroll wagering, with stop/loss notifications displayed when bet size increase is warranted or bet size decrease when the bankroll declines below specific levels. These stop/loss notifications can either be acted on, or ignored, again it's up to the user.

No bragging rights were expected, nor expressed, before or after the contest. It was what it was, very short in duration and very specific to a single card at each of those 6 tracks, nothing more and nothing less.

FocusWiz
08-26-2015, 07:02 PM
I'm quoting your comments and questions below, and answering, to the best of my ability, in red font.Ray,

Thank you for taking the time to respond.

I tend to think such an activity may sway individuals to think that one package is better than another simply based on performance in a "competition" like this, but I do not think it would demonstrate which package is best.

On the other hand, I have not fully conceptualized how I would want to compare packages either for a software comparison.

raybo
08-26-2015, 07:22 PM
Forgot to mention that the original black box I used in the "Challenge" is not the same one I currently offer, and use. That original one was created on the spur of the moment, specifically for the "Challenge", and included only 2 algorithms, one for dirt and one for turf, and took a total of 2 hours to create. It was a very "generic" selection process that would hopefully attack all 6 tracks equally well (Aqueduct inner was/is obviously unlike the other 5 we played). My current black box program is designed to attack a specific track, not all tracks, in a specific way/rankings method. We have to create a separate Excel workbook for each track we play or test, and test each track in its own workbook. So, if we're playing 3 tracks that are running at the same time, then we will have 3 workbooks open at the same time, one for each track, some or all of them using different rankings methods and/or different minimum odds, either playing one pick, two picks, or all 3 picks. Totally different, and miles more complex and refined than the original black box used in the "Challenge".

raybo
08-26-2015, 07:28 PM
Ray,

Thank you for taking the time to respond.

I tend to think such an activity may sway individuals to think that one package is better than another simply based on performance in a "competition" like this, but I do not think it would demonstrate which package is best.

On the other hand, I have not fully conceptualized how I would want to compare packages either for a software comparison.

Yeah, as I said, that challenge/contest was strictly for fun and entertainment, and to prove that there are black boxes out there that may be viable at those 6 tracks, but certainly couldn't be construed as any of them being viable in a longer contest, and/or at other tracks.

A true comparison of black boxes would need a much longer period of time, and any evaluations as to which is best would depend on each users goals and priorities regarding tracks, number of races played, hit rates, ROIs, and profit/loss. Everyone is different, and have different goals/expectations from such a program. What is best for you might not be best for others.

The design goal of my program was/is to make long term positive ROI, however, every user can play it using whatever priorities they consider most important of all the results stats available.

TonyMLake
08-26-2015, 07:41 PM
Forgot to mention that the original black box I used in the "Challenge" is not the same one I currently offer, and use. That original one was created on the spur of the moment, specifically for the "Challenge", and included only 2 algorithms, one for dirt and one for turf, and took a total of 2 hours to create. It was a very "generic" selection process that would hopefully attack all 6 tracks equally well (Aqueduct inner was/is obviously unlike the other 5 we played). My current black box program is designed to attack a specific track, not all tracks, in a specific way/rankings method. We have to create a separate Excel workbook for each track we play or test, and test each track in its own workbook. So, if we're playing 3 tracks that are running at the same time, then we will have 3 workbooks open at the same time, one for each track, some or all of them using different rankings methods and/or different minimum odds, either playing one pick, two picks, or all 3 picks. Totally different, and miles more complex and refined than the original black box used in the "Challenge".

Hehe... I was going to say... last time we thought about doing this, you were (rightfully) wanting to be superlatively precise about black box definitions (at least in terms of program settings), so I KNEW that had to be the case when you all did it a few years ago.

I get it I get it about the sample size and goals, but honestly, I am impressed. On guts alone if nothing else, haha! Great job folks!

raybo
08-26-2015, 07:51 PM
Hehe... I was going to say... last time we thought about doing this, you were (rightfully) wanting to be superlatively precise about black box definitions (at least in terms of program settings), so I KNEW that had to be the case when you all did it a few years ago.

I get it I get it about the sample size and goals, but honestly, I am impressed. On guts alone if nothing else, haha! Great job folks!

No guts involved, at least from me, I went into it knowing the BB might perform terribly, but it was just something to do, and to try to knock down much of the negativity projected by most about a good black box program being impossible.

pandy
08-26-2015, 08:04 PM
No guts involved, at least from me, I went into it knowing the BB might perform terribly, but it was just something to do, and to try to knock down much of the negativity projected by most about a good black box program being impossible.


You deserve credit for putting some picks out there in a contest, or any format. There are a lot of people who sell stuff who would never provide any sort of online testing of their product, which I do all the time, by the way. As you say, over a short period of time it could bomb, so maybe that's not smart marketing, but I've found that if you have something that is of value you have nothing to worry about.

TonyMLake
08-26-2015, 11:49 PM
No guts involved, at least from me, I went into it knowing the BB might perform terribly, but it was just something to do, and to try to knock down much of the negativity projected by most about a good black box program being impossible.


Ok, so how did you break down "Black Box" back then? Did you just publish which setting your app would use?

Thanks

raybo
08-27-2015, 02:27 AM
Ok, so how did you break down "Black Box" back then? Did you just publish which setting your app would use?

Thanks

I stated earlier that the BB I used then (February 2010) is not the same one I offer or use now. That one back then was created strictly for the "Challenge", it used 2 algorithms, one for dirt and one for turf, and produced one set of ratings, the top ranked rating was the win pick. My current BB is a completely new program, from the ground up, not even remotely like that old one.