What Rules Would You Create For a Black Box Software Handicapping Contest?

[TonyMLake]

The following rules seem to be fair rules for a "black box" handicapping contest. Bullets with a "*" next to them are probably going to be debatable, so, please be respectful and use reasoning to support your arguments. Not by any means do I expect this to be a settled list of rules... only a starting point. Here's mine:

1. Anyone confident enough to enter their software into the contest deserves to be treated with respect. Since our sample size won't be infinite, some of us may do very well, some may do very poorly, but the fact that the author or user has the confidence to share pre-race deserves some credit.

2. Most software vendors "cherry pick" data post-hoc (after the fact) to make their software look good. We'll post our picks BEFORE the race, always in the same place, same forum, same thread, so that post hoc analyses and interpretations are impossible.

* 3. Your black box must be actual software, not just talk about what your software would or could or should do if it existed. (Possible point of contention: "Do Excel spreadsheets count?" (I say some do, some don't. What do you think?)

* 4. If your pick is conditional on something that can't be known far in advance you must post your condition in advance. You may not choose to bet differently under two race conditions - but you may choose one "bet" condition and any number of "don't bet" conditions. For example, "I will bet $2 to show on the 4, unless the race moves to the main track (in which case I will not bet) or if it rains (in which case I will not bet)."

* 5. You may customize your software to work in a certain way, but you can't alter the controls during the contest EXCEPT as pre-described by you, the author (or user), pre-contest. For example... if you consider Tomlinson ratings when it's wet, that's fine... but you must describe the change you will make to the settings pre-contest, and still must post your pick pre-race. Eg., "On races where it rains, I will use a customization to my algorithm". You must still post your pick pre-race and may still only have one bet condition.

6. You need not post your actual algorithm, nor post your contingency customization.

7. We need a large enough set of races to make the "contest" result meaningful, but not so many that most of us won't be able to play. I think (as a start to this debate), the contest should include 100 races of your choice, to be completed between October 18, 2014, and May 5th, 2015. You must make EXACTLY 100 picks. No bets after the first 100 will count and may not be posted in the contest thread. If you get less than 100 bets - you're out.

**** 8. $2 Straight win, place, and show bets only. Why? A bettor can lose or win so much on one bet that the other 99 won't matter. We are trying to get an actual baseline on black box software here (let the calm, respectful, well reasoned debate begin).

* 9. I personally don't care if the contest is public or private. But we should decide PRE contest and if we decide it's public, every communication should be public. If we decide it's private, then we should announce only that the contest is taking place, and absolutely nothing else (or perhaps just who won).

* 10. No contest without at least 5 different contenders.



SOOOOoooo, what do you think?

[GameTheory]

I think that if there were a such a thing as a negative probability of something occurring, it would be such a contest. Maybe you could get some users of different software together and agree to post selections based on some preferred mechanical usage of each, but that's the best you could hope for. (And even that would be a miracle if you could pull it off.) Certainly you're not going to get commercial software developers involved in such a contest, and this idea has been floated many times before. (I don't think that should be taken as a criticism, there is simply nothing in it for them.)

If you acquired all this software yourself at your own expense, you could have your own contest. I suppose in a way many individuals that have gone through several programs have already done that...

[DLigett]

Basically, I like the idea. But if the software creator is picking the 100 best bets... maybe we're really introducing a huge human element -- and not really testing the software? The sw developer might be doing a tremendous amount of handicapping by hand, and then only showing the sw picks that agree with him. Which means ordinary users could not match the results.

[TonyMLake]

GameTheory: Right - I don't think many commercial vendors would participate in this. Post-hoc analysis is too valuable to ruin it by actually making picks BEFORE the race... hehe...

DLigget: All picks would be posted in the thread BEFORE the races went off.

... but you're right... the handicappers themselves would be able to pick the conditions under which a bet would be made; however, even those conditions can be stated pre-contest. Thus, it still reflects a duplicate-able "mechanical" algorithm.

For me, the bet would come from one of 26 tracks, only fast and firm, not even a drop of rain, over 4 furlongs, etc... but all conditions are "quantifiable" and postable PRE-CONTEST. All contenders would have to post their conditions and the other contenders agree to allow the contender to play by those betting conditions before the season started.

[DLigett]

My software might generate 1,000 picks over the span of the contest. Do I get to choose the 100 I like best -- or do we use the first 100 -- or what?

[GameTheory]

GameTheory: Right - I don't think many commercial vendors would participate in this. Post-hoc analysis is too valuable to ruin it by actually making picks BEFORE the race... hehe...Again, that wasn't meant as a criticism. It is easy to say, "of course they wouldn't participate, because it would reveal their software as a fraud" or some such, but even if the software is great there isn't much good that can come of it.

[TonyMLake]

Sorry gametheory - my mis- characterization.

Dligget, yes... You could choose any races during the season assuming your software settings and conditions aren't violated, and post them anytime up to post time for the race.

[raybo]

We had a BB contest here a few years ago (the "Pace Advantage Black Box Challenge"). 6 different tracks on consecutive days, 1 track per day, every race on the card, win only, one pick with an alternate (I believe) in case of a scratch of the 1st pick. $2 bets only, ROI was the determinant, although some of us tracked hit rate too. We had 2 divisions, one was for "true" BB programs (load the race and the selections automatically appear), and one allowed modifications like watching the tote, etc..

Overall I think the contest was interesting and a couple of us BB users even showed a positive ROI for the 6 days, one of the non-BB programs also produced a positive ROI, if memory serves. Small sample, I know, but it made the contest easy to pull off, and generated quite a bit of interest. Any contest that spans a lengthy period of time, IMO, will not succeed, as there is just too much time involved, and people who get a bad start seem to drop out before completing the contest. Maybe you could run a short contest first, and then maybe run another one of longer length later.

Too many rules, rules out too many people, too few allows too much room for deception and manipulation. Win, place, and show wagers, while interesting, do not reflect the way people actually use a BB, so win only would be the way to go, IMO. While my program is designed to be used with multiple bets (more than one horse bet to win) I could of course use only the top pick by itself.

Track both ROI and hit rate, and declare winners for both. Why hit rate? Because some people use BBs to play vertical exotics (I am one of those), where hit rate is very important, much more important than in win betting. If you get the winner in Tris and/or Supers often enough, and have adequate coverage below, then you will most likely be profitable long term. So, BBs need not be restricted to win or picks play.

[PaceAdvantage]

I've been running this website for 15 years...and not once has a software purveyor come on here and dazzled the masses with selections.

Yes, there have been a handful who have posted a series of picks that have turned out to have a positive ROI.

But nobody has ever come on here and BLOWN US AWAY.

That should tell you something...15 years is a lot of time...and not one vendor thought his product was good enough to promote in this fashion.

I can't think of a BETTER way to promote your software....

Lots of people are convinced that there are people who have created sophisticated computer programs that do some serious winning...so how come in 15 years nobody has come on here and dazzled us with an incredible series of money-winning selections?

Even people who aren't selling their warez...private individuals with private software...they wouldn't be risking their method by simply posting selections...but nobody ever does it...or maybe I missed it... :lol:

[vegasone]

I wouldn't limit it to any specific color box :lol:

[TonyMLake]

Raybo: "6 different tracks on consecutive days, 1 track per day, every race on the card" would put me out!

I rarely find more than one or sometimes three races at a track on a given day... most of my betting is the latter half of the day, in the latter half of the season, and at the best tracks. I have to buy PPs for 5 or 6 tracks to get 5-10 races on a day, if that.

How could the contest be constructed to allow for a case like mine? Do you think at least five contenders would stick it out for one season or until all of us had 100 races?

[GameTheory]

I've been running this website for 15 years...and not once has a software purveyor come on here and dazzled the masses with selections.

Yes, there have been a handful who have posted a series of picks that have turned out to have a positive ROI.

But nobody has ever come on here and BLOWN US AWAY.

That should tell you something...15 years is a lot of time...and not one vendor thought his product was good enough to promote in this fashion.

I can't think of a BETTER way to promote your software....

Lots of people are convinced that there are people who have created sophisticated computer programs that do some serious winning...so how come in 15 years nobody has come on here and dazzled us with an incredible series of money-winning selections?

Even people who aren't selling their warez...private individuals with private software...they wouldn't be risking their method by simply posting selections...but nobody ever does it...or maybe I missed it... :lol:

Well one reason would be if someone was selling a "black box" and it did really well then that would be the end of that pretty quickly as the selections were bet down, and so no one sells black boxes and no one wants to buy them either unless they are the only customer (or else it will soon be worthless). People want programs with lots of features and factors so they can make their own black boxes that only have themselves as the one user. And if the software developer comes on here and posts a bunch of picks and they do really well and then says, "Yeah, ok, I used my software to do it but I'm not going to say how" then he's just asking for all sorts of trouble. (And if he does say how, that will be the end of that.) I recall Jeff having to actually *remove* the most powerful single factor from JCapper because it was getting bet down too much.

As far as posting picks, there was that one guy that made pencil and paper systems (I wanna say Steve Wolson? I have some of his stuff around here somewhere) that posted 500 races of picks (I think) with a healthy positive ROI -- that was impressive...

[raybo]

Raybo: "6 different tracks on consecutive days, 1 track per day, every race on the card" would put me out!

I rarely find more than one or sometimes three races at a track on a given day... most of my betting is the latter half of the day, in the latter half of the season, and at the best tracks. I have to buy PPs for 5 or 6 tracks to get 5-10 races on a day, if that.

How could the contest be constructed to allow for a case like mine? Do you think at least five contenders would stick it out for one season or until all of us had 100 races?

The purpose of the "Challenge" was to see how software handled races, as a whole, not just cherry picked ones. Had that not been the case, I am pretty sure most everyone would have done better than they did. It was a tough test, for sure, but still a couple handled it well (I think Handifast won the BB section with about 1.26ROI and I finished 2nd with about 1.24ROI, one of the non-BB guys beat both of us, if my memory is correct). I remember one of the tracks was Aqueduct ("inner" I believe) which I had never played before, I went 0-fer that day - LOL. Of course, the BB I was using for the Challenge was thrown together in a couple of hours specifically to participate in that contest, so it was necessarily pretty "well-rounded" because I knew I would be playing 6 different tracks, rather than just ones that I knew well.

Just another reason to track both ROI and hit rate, because one of those should be an indication of how well, or how badly, a BB handles racing as a whole. Yes, there will be some tracks, and some days, that BBs (or non-BBs) will not handle well, that is to be expected, and should be taken into account by anyone tracking the contest.

A long contest has many obstacles, and I find it doubtful that more than one or two people would complete a long contest.

But then, if one has a decent BB, and there are 30+ tracks running, it shouldn't be too hard to find 100 plays in a week or so. The problem with that, as you mentioned, is that one would be forced to play tracks that one normally wouldn't, so the cost for data would be a problem for some, unless they have an unlimited data plan already.

[raybo]

I've been running this website for 15 years...and not once has a software purveyor come on here and dazzled the masses with selections.

Yes, there have been a handful who have posted a series of picks that have turned out to have a positive ROI.

But nobody has ever come on here and BLOWN US AWAY.

That should tell you something...15 years is a lot of time...and not one vendor thought his product was good enough to promote in this fashion.

I can't think of a BETTER way to promote your software....

Lots of people are convinced that there are people who have created sophisticated computer programs that do some serious winning...so how come in 15 years nobody has come on here and dazzled us with an incredible series of money-winning selections?

Even people who aren't selling their warez...private individuals with private software...they wouldn't be risking their method by simply posting selections...but nobody ever does it...or maybe I missed it... :lol:

I agree, and as a result of that previous "Challenge" the free AllData Project received quite a spike in interest, and as a result, I put that "BB" method in the workbook because of that interest.

Participating in a contest, against other well known products, is an excellent way to promote ones software, assuming of course that one does well in the contest - LOL. Unfortunately there weren't many "well known" products represented in the "Challenge", I think Dave and his HSH software was about the only commercial product represented, but the overall winner (also a non-BB method) may have been using a known product as well, it's been too long for me to remember what he was using.

[raybo]

Well one reason would be if someone was selling a "black box" and it did really well then that would be the end of that pretty quickly as the selections were bet down, and so no one sells black boxes and no one wants to buy them either unless they are the only customer (or else it will soon be worthless). People want programs with lots of features and factors so they can make their own black boxes that only have themselves as the one user. And if the software developer comes on here and posts a bunch of picks and they do really well and then says, "Yeah, ok, I used my software to do it but I'm not going to say how" then he's just asking for all sorts of trouble. (And if he does say how, that will be the end of that.) I recall Jeff having to actually *remove* the most powerful single factor from JCapper because it was getting bet down too much.

As far as posting picks, there was that one guy that made pencil and paper systems (I wanna say Steve Wolson? I have some of his stuff around here somewhere) that posted 500 races of picks (I think) with a healthy positive ROI -- that was impressive...

That is a concern, for sure, and one that had initially worried me with my program. I was afraid that too many users would deflate the prices we received, but as things evolved, and more methods were added, that pretty much became a non-issue, because not all users were playing the same tracks, nor the same method, nor the same races, nor at the same odds ranges, so we weren't playing all the same horses. It really depends on how a BB is designed, if it's a "single kick" thing, then yes, it's selections would get bet down and the value lost. Limiting the number of users, however, is probably a good idea for any BB method.

Steve Wolson sounds right to me too, I think he worked with "Hcap" later, if I'm thinking about the same person, and Harry developed an Excel method for him to automate his method. I could be thinking of someone else though.

[headhawg]

I believe it was Powerline software by Steve Wolson; had a positive ROI after 500 picks, I think. Not to be confused with Steve Wolfson.

And I agree with PA and GT. Why do a contest? If someone wanted to promote their software's black box picks to stir sales wouldn't they just post them in the Selections section? With no competitors you can't really "lose", and if the picks show a high hit rate or decent ROI that should generate some software sales.

[sjk]

The rules look good to me. If you actually do get enough people I might join.

[TonyMLake]

I just thought it would be fun.

Raybo, what would your exact scoring algorithm be? Bear in mind we are trying to factor OUT the effect of betting strategies (that's another "contest").

Might it be something like X points if your favorite wins, Y if it places, Z if it shows.. Then X1 points if your second favorite wins... X2 points if your third favorite wins ... Etc...

Or perhaps just count the payout only on your top pick to w+p+s?

Or what?

[TonyMLake]

Also Raybo, I'm starting to see the light but I'm not quite there with you yet on the duration and framing of this contest.

Let's look for some extremes. One day is too short. The entire season too long. 10 races too few, 200 races too expensive.

Perhaps each contender could pick any 100 races from any track with published data files during the same pre-scheduled 6 week period?

[JJMartin]

I think predetermining the 100 races and keep the same for everyone is better

[TonyMLake]

Since there are so few races I'd bet on, that would almost certainly rule me out, or at least give me a "challenging" ROI. If the conditions under which a contender intends to bet can be stated pre-contest, why would it be better that everyone bet in the same races?

How would you prearrange which races?

[JJMartin]

Since there are so few races I'd bet on, that would almost certainly rule me out, or at least give me a "challenging" ROI. If the conditions under which a contender intends to bet can be stated pre-contest, why would it be better that everyone bet in the same races?

How would you prearrange which races?

Because that would provide a more objective summary of each BB. As you know all tracks do not behave the same way even when you apply many conditions. It just wouldn't seem like a true BB comparison without all the conditions being the same for everyone. The races could be decided and agreed upon before hand such as claiming only, or dirt only. Not saying there should be too many restrictions but some parameters should be present.

[TonyMLake]

What you're saying, as unattractive as it is to me, does seem well reasoned and difficult to argue.

There would have to the following stipulations at bare minimum:

1) the race would have to be importable into all software apps contending, at least as is reasonable

2) absolutely no scores of "0", "-0" , or "-" in any major data file format (formulator, brisnet) for any horse in any race selected (this is going to be rough)

3) every horse in every race selected must have at least 3 PPs in their data line... Not workouts- races- ... And the three most recent data lines must be race data

There are many other stipulations I would make about my own bb bets before the contest. They're all important to my ROI, and to me stipulating them pre-contest seems fair enough, but based on what you say, I'd at least like to see a "no rain" clause. And I don't mean "fast and firm". I've seen take-hungry tracks call slop fast and firm during a deluge to keep the rubes lined up. It'd be unfair for bb's that don't automatically compensate.

[TonyMLake]

Hey, Where'd everybody go? I thought we almost had a set of parameters here... hehe...

[sjk]

I have been following the board for 10 years and I have very few others who seemed to be operating (and betting for real) using a black box over sustained period.

[DLigett]

I started to lose interest when the idea of 100 predetermined races came to the fore.

One of the most important attributes of my software is helping you find the 90% of the races you DON'T want to bet on.

[raybo]

I have been following the board for 10 years and I have very few others who seemed to be operating (and betting for real) using a black box over sustained period.

Truthfully, there are probably very few (maybe even only a couple) here who actually have a true, viable, black box program. Cherry picking races is the first sign that one doesn't have a black box, unless the races have no horses with any past races, or any horses that have run similar distances and surfaces. My program would not produce any picks in such a case, however, it might still be playable in my FTS method if the right trainer criteria is present on at least one horse. In other words, my program would probably allow me to play 70-80% of all races, at all tracks. Now, that process might not produce long term profit, but I doubt any other programs out there would either.

When you allow cherry picking tracks and races then you bring almost anyone into the contest, not just black box programs, and they might only play a couple of tracks, or maybe even a single track, depending on the number of races required in the contest. IMO, that is not a true black box program, because the user is inputting too much human subjective judgement into the mix. A true black box only requires importing or entering the data, and then it gives you a pass or play indication, and the ranked selection(s).

That being said, someone set the rules, I'll either play or not.

[DLigett]

I think we're working with at least two definitions of "black box" in this thread.

1) Automatic picking, with no inputs.
2) Automatic picking, with user inputs, but with no insight into the sw internals.

Wikipedia thinks in terms of the second definition:

http://en.wikipedia.org/wiki/Black_box

[sjk]

I started to lose interest when the idea of 100 predetermined races came to the fore.

One of the most important attributes of my software is helping you find the 90% of the races you DON'T want to bet on.

I would not be interested in anything with predetermined races either. For one I don't play turf races or races with lots of FTS and layoff horses.

If we are allowed to post any races that the software picks it will take 1-2 minutes a day until 100 races are posted which should not take more than a month or two. I am not interested in anything that would add to the time required.

[TonyMLake]

Maybe for now we can agree about how such a contest would be scored. Raybo seemed to have the most experience with that.

I have no idea but this came to mind. I probably overlooked something glaringly stupid about this idea, but I need to run and still wanted to put this out there:

Scoring would be

1st Favorite Horse (Win $ + Place $ + Show $)
+ 2nd Favorite Horse (Place $ + Show $)
+ 3rd Favorite Horse (Show $)

That still gives a HINT as to how the picks would do in the exotics, it "sort of" counts the hits, and gives a good idea about the ROI.

OR, maybe even just 1st Favorite Horse (Win $ + Place $ + Show $) for brevity.

Anybody have a better idea?

[raybo]

I think we're working with at least two definitions of "black box" in this thread.

1) Automatic picking, with no inputs.
2) Automatic picking, with user inputs, but with no insight into the sw internals.

Wikipedia thinks in terms of the second definition:

http://en.wikipedia.org/wiki/Black_box

In horse racing, which is what we are talking about here, a black box has a set of inputs and produces an output, that output being the black box's ranked selections, and in better black boxes a pass/play indication and/or an odds requirement. Anything else allows the user to have input into the decision process, which makes it a non-black box approach. if the user becomes part of the decision process then it is just software, not a black box, because the users would not necessarily arrive at the same selections.

Yes, some black boxes can also be used "subjectively", but if you do that then you are just using it as a piece of analysis software, not as a black box. In order to test black boxes against one another, each one must be operated as a black box, not as just another piece of analysis software. And when the user decides that a race is not playable, instead of the software making that decision, then it's not a black box.

What i'm getting at is that the user, and his/her subjective input, must be removed from the equation, completely.

My 2 cents worth on that subject!

[sjk]

Maybe for now we can agree about how such a contest would be scored. Raybo seemed to have the most experience with that.

I have no idea but this came to mind. I probably overlooked something glaringly stupid about this idea, but I need to run and still wanted to put this out there:

Scoring would be

1st Favorite Horse (Win $ + Place $ + Show $)
+ 2nd Favorite Horse (Place $ + Show $)
+ 3rd Favorite Horse (Show $)

That still gives a HINT as to how the picks would do in the exotics, it "sort of" counts the hits, and gives a good idea about the ROI.

OR, maybe even just 1st Favorite Horse (Win $ + Place $ + Show $) for brevity.

Anybody have a better idea?

I don't bet a lot of favorites. I imagined that we would just pick horses that we would bet.

[raybo]

As I said earlier, there are very few true black boxes, and few users that would use them that way. Even mine "should not" be used as a true black box, but it can be, with absolutely no user input at all. The user should use the program to test tracks, and the user should then make a "subjective" judgement as to which of the 11 (actually more than 11) rankings methods meet his needs, regarding hit rate, ROI, number of plays, and total profit/loss ("loss" in the case of rebate players, or whether or not to play a particular track at all). So, subjective decisions should be used in order to enhance success, in the real world. But, for the purposes of this contest, only the software should be examined, not the user.

[raybo]

I don't bet a lot of favorites. I imagined that we would just pick horses that we would bet.

I hear you, but in the case of a black box contest, those decisions should be made by the software, not the user, so everyone should have to play the same races, let the chips fall where they will, and the best software is the winner, not the best player. Agreed, there will probably be few, if any, that produce a profit, but that should be understood by everyone beforehand. Chances are that we will all lose money in the contest, but the one that loses the least should still be the best piece of "black box" software.

Basically, if you're afraid of being embarrassed, then you shouldn't enter. It's not about losing or winning, it's about how the software handles varying situations.

If you have designed your software well, then you should beat the public's ROI, whether that be their top pick for win only, or their top pick's wps, or the top 2 picks, or top 3, whatever.

[raybo]

Maybe for now we can agree about how such a contest would be scored. Raybo seemed to have the most experience with that.

I have no idea but this came to mind. I probably overlooked something glaringly stupid about this idea, but I need to run and still wanted to put this out there:

Scoring would be

1st Favorite Horse (Win $ + Place $ + Show $)
+ 2nd Favorite Horse (Place $ + Show $)
+ 3rd Favorite Horse (Show $)

That still gives a HINT as to how the picks would do in the exotics, it "sort of" counts the hits, and gives a good idea about the ROI.

OR, maybe even just 1st Favorite Horse (Win $ + Place $ + Show $) for brevity.

Anybody have a better idea?

Sounds good to me, I don't care personally. Doubtful that everyone will like everything anyway. Just do it or don't, participate or don't.

[JimG]

I don't bet a lot of favorites. I imagined that we would just pick horses that we would bet.

Hi sjk,

I think when Tony said "favorite", he meant "pick"...such as top pick, 2nd pick, etc.

Hope you are doing well.

Jim

[sjk]

Hi sjk,

I think when Tony said "favorite", he meant "pick"...such as top pick, 2nd pick, etc.

Hope you are doing well.

Jim

Hi Jim,

Hope you are well. I might not bet a horse under 10-1 in the entire contest and would not care to be held to horses that I think are the most likely winners.

I would set the program to give me a short list of horses that appear to be overlays based on morning line. Of course, in practice I use near post time odds. My plan would be to set it up as a black box where I copy and paste a short list of names and races to bet in 1-2 minutes per day.

It does not sound like this is going in a direction that would work for me so I wish the others the best.

[TonyMLake]

Sjk- we're all on the same page about that now in terms of our personal ROI... But for the purposes of this measurement process I am now in complete agreement with Raybo... We're not competing personally - just measuring efficacy of black box software objectively.

I do have the caveat that the last three PPs should be data lines not workouts and not empty... Let's at least give the programs some data to work with.

personally think rain should be avoided but I can forego even that if its an issue.

[dartman51]

What about races that are full of FTS? Some programs will handle them, while others don't.

[TonyMLake]

Mine will likely fail, at least with DRF data. It might load from Brisnet data, but honestly, we are not expecting these apps to do ALL the work... Just guide us to a quick "reasonable" set of picks.

I'd like to properly request that the three most recent past performance data lines be race data - therefore no races with FTSers

In my experience many handicappers would call handicapping on three races only is too "top heavy" so I'm hoping this isn't considered a "concession."

Its just technically a time and effort saver. Will someone second that?

[FocusWiz]

If you have designed your software well, then you should beat the public's ROI, whether that be their top pick for win only, or their top pick's wps, or the top 2 picks, or top 3, whatever.I have only been sparsely watching this topic since I have been trying to train myself to identify races that should be skipped, but this comment makes a lot of sense.

I think if you are going to have a black box contest, you should measure not only how they perform against each other, but also against the public selections. This will not only tell us which black box is best, but whether or not it is better than just playing the favorites.

This is similar to how weather forecasting models were evaluated. They used to measure whether a forecasting model was better than just assuming that tomorrow's weather would be similar to todays (within certain tolerances). Few weather models actually were better than that (probably even fewer are now).

[TonyMLake]

It's fine with me if we track the top three contenders on the tote at post time as well... but is that final tick of the tote published somewhere? Or is somebody other than me going to be doing extra work to record that?

:^)

EDIT: Also, I don't care how we score it. I'm okay with whatever Raybo says or everyone else agrees on. I do have the issue that I want actual data on at least the last 3 PP lines, to harp on a theme.

[FocusWiz]

Realistically, no one can actually play based on the final tick of the tote, so my suggestion may ultimately be pointless. However, while I understand the rationale for picking every race, I would prefer a black box that informed me to skip a race that is unplayable. Even skipping races full of first time starters for this competition may not be as reasonable as it sounds. A black box that focuses on wagering may be able to identify a value play that other black boxes focusing solely on past performance data could not.

While it would be interesting and fun to see what your results are, I am unclear if it will ultimately prove or disprove which one is "best" if that is a goal, especially if races are selected based on what some tools need to make their selections and if other tools must turn off their ability to skip races. If that isn't a goal, then I think including races with little or no past performance data and comparing the results to what the masses select would be, for me, equally interesting.

[JJMartin]

I think including races with little or no past performance data and comparing the results to what the masses select would be, for me, equally interesting.

How can the program make a determination and selection if there is no data?

[FocusWiz]

You can look for inconsistencies among the wagering in various pools to identify a horse that is played heavier in some but not all pools.

For example, my spreadsheets evaluate my selections for which ones offer a better value based on my selection order. If I have not made any selections in a race, they focus on wagering inconsistencies relative to exacta wagering and the morning line (not useful at some tracks). I have found that this is a reasonable fallback for certain maiden races.

My point is that if a black box is designed to skip races, you are not using its decision making capabilities if you force it to play every race. However, if you create rules eliminating certain races, you essentially favor those tools which would have skipped those races. If there is a tool which uses bloodlines or trainer or jockey statistics to find value plays in such races, eliminating those races will not highlight this feature.

While I cannot imagine how you would run a competition if you allowed the black boxes to be selective in their races, I do not think a competition like this will evaluate these tools so much as provide interesting bragging rights.

[TonyMLake]

EXACTLY, JJMartin.

A human might make a subjective call, or include non-standard data (The horse has kidney sweat on his ears and the jockey smells funny).

Black box software might say "don't bet" or it might say "my best guess is post position order" but it won't SEEK subjective data like you or I might, if we were too drunk to stay out of the race.

Either way, including races where there aren't some races to base picks on is not conducive to this test, in my opinion, and will yield a result neither meaningful nor useful for our mutual desire to see how the field (of black box wares) is doing.

SO - do we agree to leave out races with no useful (for this purpose) PP data?

[raybo]

How can the program make a determination and selection if there is no data?

trainer , breeding, works, etc..

[raybo]

I think some have a different definition of what a black box is, and that presents problems when trying to compare them. IMO, a black box does everything you would do, without one, except place the wager (and I'm sure some even do that). It is not just software that speeds things up for the handicapper, it does all the handicapping. If yours doesn't do all that, then it's not a black box. I do not care to enter a black box comparison that compares black boxes to players. Doing so is just another handicapping contest, been there done that, and it's boring and full of chest slapping, and ego boosting nothingness.

[TonyMLake]

I'm not much of a chest thumper, hehe...I just wanted to do this for fun and to see what the apps can do. My program is not a general purpose robot that I can just send to the track to get me some picks and bring home a case of beer, unfortunately.

Perhaps my program is not as full featured as yours. But there should be some amount of compromise on that, as I'm not asking to choose the races- just that some race data exists for my program to use.

[Flysofree]

I think it could still be considered a black box, if the only requirement was to have 3 full races and be on dirt for each entrant. But I see what some are saying.

[TonyMLake]

I can live with any surface or distance at this point. I'm just hoping for race data... Hehe

[FocusWiz]

Please forgive my posts, I am not trying to intrude on your planning nor add work to your recordkeeping. I think whatever rules you guys agree on would be fine and it would be interesting whatever way you choose.

However, since each black box would be expected to have different strengths and weaknesses, for me as an observer, I would think it would be useful to see how each black box does without any limitations as to which races are used (including races where only bloodlines, trainers, jockeys, ML Odds, workouts, and Post Time Odds are available for identifying a wager). Separately, it would be good to know how each black box would have done if it were allowed to exclude races that either the developer did not build it for or for which the logic within the product identifies it as a race to skip (including races with all first time starters, if the product is not built to select for those). Thus I would want to know if the product did well with the races it was built for.

Black Box A may finish 10th overall, but may have the highest ROI of any product if you only bet the wagers it suggested. Black Box B may be lousy with first time starters, but may have the greatest overall win percentage on the wagers it was built to evaluate. Black Box C may be the winner overall with the smallest negative ROI, but it may not help you decide which races to play and which ones to skip. I do not see why that sort of information would not be worthwhile compiling (though I understand how much work would be entailed).

Again, I am just offering an opinion that I am fine with folks rejecting. This is hardly anything I feel strongly about, but if this is to accomplish anything more than provide interesting bragging rights, it would be best to showcase what each black box does which the others may not. I will likely not post any more in this thread, but if I were to let my mind wander enough, I would also then suggest a breakdown by race type (some black boxes may be best suited for cheap claimers while others may be good for stakes races) and maybe even track type (dirt versus turf) or track conditions. I just offer this as food for thought.

Thanks for tolerating.

[TonyMLake]

You're not intruding. We're discussing what rules everybody thinks would make it fair.

But what you're suggesting does sound like two different contests, or at least, a lot of work for one.

In this contest, we are trying to find a way to test how Black boxes do in general. There are some bumps in the road - mainly due to varying degrees of ability to the boxes themselves, but eventually we'll have ourselves a set of mutually agreed upon parameters ...and away they go.