Small Sample Size?

[Actor]

Originally Posted by thaskalos

And it still backfired.

Did it?

What is your source for that? It might explain why I couldn't make it work.

[Actor]

Originally Posted by DeltaLover

CP/M rocked

Way better that MS-DOS who eventually became the market's standard

CP/M begat UNIX.

UNIX begat MS-DOS

MS-DOS begat Windows

Frustration begat Linux

[Ray2000]

The number of races (wagers) needed before you can have any confidence that they mean anything depends on whether you're looking at hit rate or ROI. For Hit Rates, the statistical calculations are easy but must be stated in the proper way:

How many wagers do I need in my test to be (?)% sure that my HitRate is true within a certain Margin of Error (X).

The formula is :
Number of Races needed = HitRate times (1 minus HitRate) times "Z" all divided by ("X"^2)
where
"Z" = 2.71 to be 90% sure
....= 3.84 to be 95% sure
....= 6.63 to be 99% sure

"X" = the margin of error you'll accept, say ±5%

Example: for an observed HitRate of 35% and you want to be 95% confident that the HitRate is significant within ±5%

You'll need ...

(0.35 x (1 - 0.35) x 3.84) / (0.05 x 0.05) = 350 races

reducing the margin of error to a tighter ±3%----> needs 970 races



For the number of wagers needed when testing ROI the calculations get more involved. The outcome is not binomial (hit/miss, yes/no) but includes hit rate and average return when successful. Also the distribution of returns is not normal (bell shaped curve) and outliers mess things up so usual statistic formulas are "iffy". For ROI the best way to evaluate your system is too calculate the P value by using Z scores or T test as mentioned by whodoyoulike. This will give you an indication of the results being attributed to just LUCK.

[coljesep]

What would be a satsifactory ROI over say 100 races? Anything in the plus, but would small negative be considered realistically satisfactory?

[Capper Al]

Originally Posted by Dave Schwartz

At least 30 winners in each category.

Thus, if you were looking at odds, and had broken the horses into (say) 5 classes and the upper class was 30/1 and above, you would want 30 winners in that group, at 30/1 or higher.

Dave,

This isn't how I remember you on this topic. Wasn't a 1000 races too small before?

[Capper Al]

I just bought 12000 race results from BRIS. There are areas that I still need more races to cover. For instance, not many handicap races to choose from.

[hcap]

A word of caution. Often an increase in r.o.i or win % over many races, can be
artificially obtained by "back fitting". Confidence in a method and the odds of a method or system going forward should also be verified by taking what appears to be a dynamite system and testing it on a set of races not used to devise that system in the first place.

[traynor]

I would suggest paying a LOT of attention to Dave's suggestion. I would also suggest adding a ratio component. The higher the ratio of winners to sample size, the more likely it is that the scenario could be predictive.

30 winners in a 300 race sample is one thing. 30 winners in a 3000 race sample is quite another. Some claim to make exacta wagers on the basis of 1% probability. That may make sense, if one has a database of 100,000 or so races to generate that figure. Wagering on something with a frequency of a few percentage points in anything less than several thousand races--AND that has not been thoroughly tested to assure normal distribution on other sets of races--is probably not a good idea.

Small(ish) samples can be useful, if the strike rate/sample size is decent. 10% is low end, more is better. Again, the initial sample should be replicated on other groups of races before giving it serious (as in betting real money) consideration.

[traynor]

Originally Posted by hcap

A word of caution. Often an increase in r.o.i or win % over many races, can be
artificially obtained by "back fitting". Confidence in a method and the odds of a method or system going forward should also be verified by taking what appears to be a dynamite system and testing it on a set of races not used to devise that system in the first place.

Absolutely. If a scenario only works on a specific chunk of races and has to be tweaked frequently "because racing changes," one should be extremely skeptical. "Normal distribution" implies that if something happens 10 times in 100 races, testing it on another (randomly selected) sample of 100 races should produce (approximately) the same number of hits. If it doesn't it is likely only applicable to the specific sample of races "studied."

[hcap]

Originally Posted by traynor

Absolutely. If a scenario only works on a specific chunk of races and has to be tweaked frequently "because racing changes," one should be extremely skeptical. "Normal distribution" implies that if something happens 10 times in 100 races, testing it on another (randomly selected) sample of 100 races should produce (approximately) the same number of hits. If it doesn't it is likely only applicable to the specific sample of races "studied." Otherwise known as "back fitting."

Many "systems" sold, if not just plain hype, suffer from the system creator/seller starting with a few across the board rules and as the testing continues the system seller/creator will see that perhaps "just" a slight modification to the basic rules would have gotten a winner missed by the ordinal qualifications. (Usually a $30 payoff )

Also a minor twist would be.......

"if I only eliminated that horse i would have avoided another loser" and a new elimination rule is born

A few dozen rounds of this maybe good intentioned fudge, and Voilà an artificially naive manipulation of the overall bottom line gets done, and often may encourage the creator to daydream about retiring early

Unfortunately the paper and pencil sellers are not the only ones. Fast computers sometimes provide sophisticated variations on "if I only zigged instead of zagged" at speeds 1000's of times faster than manual back fitting of our pencil and paper fudging system developer.

RPM a long time system mill, seems to follow this practice

[thaskalos]

Originally Posted by hcap

Many "systems" sold, if not just plain hype, suffer from the system creator/seller starting with a few across the board rules and as the testing continues the system seller/creator will see that perhaps "just" a slight modification to the basic rules would have gotten a winner missed by the ordinal qualifications. (Usually a $30 payoff )

Also a minor twist would be.......

"if I only eliminated that horse i would have avoided another loser" and a new elimination rule is born

A few dozen rounds of this maybe good intentioned fudge, and Voilà an artificially naive manipulation of the overall bottom line gets done, and often may encourage the creator to daydream about retiring early

Unfortunately the paper and pencil sellers are not the only ones. Fast computers sometimes provide sophisticated variations on "if I only zigged instead of zagged" at speeds 1000's of times faster than manual back fitting of our pencil and paper fudging system developer.

RPM a long time system mill, seems to follow this practice

Not to mention the PHILLIPS RACING NEWSLETTER.

This rag has supposedly researched and tested HUNDREDS of handicapping systems...and they've identified over a hundred that they've deemed to be "highly profitable".

And about 98% of the players STILL lose in this game.

[traynor]

Originally Posted by thaskalos

Not to mention the PHILLIPS RACING NEWSLETTER.

This rag has supposedly researched and tested HUNDREDS of handicapping systems...and they've identified over a hundred that they've deemed to be "highly profitable".

And about 98% of the players STILL lose in this game.

I agree completely about PRN "systems." However, they are no worse than many CALDs (computer-assisted losing devices). Many seem to believe that numbers generated by a computer are somehow of a higher order of reality than everyday numbers. That is, the ordinary constraints that would apply to numbers generated on any other topic are believed to be held in abeyance when crunching horse race numbers.

As in, "Never mind all that statistics nonsense, this is how it works in the real world." Amazing.

[thaskalos]

Originally Posted by traynor

I agree completely about PRN "systems." However, they are no worse than many CALDs (computer-assisted losing devices). Many seem to believe that numbers generated by a computer are somehow of a higher order of reality than everyday numbers. That is, the ordinary constraints that would apply to numbers generated on any other topic are believed to be held in abeyance when crunching horse race numbers.

As in, "Never mind all that statistics nonsense, this is how it works in the real world." Amazing.

The only difference that I can see between the computer-assisted systems and the pencil-and-paper ones is that the computer-assisted systems are more expensive. I guess the vendors of these systems think that the higher price-tag gives them added credibility.

[Tom]

This rag has supposedly researched and tested HUNDREDS of handicapping systems...and they've identified over a hundred that they've deemed to be "highly profitable".

Profitable tot eh guy using it or the guy selling it?

BUT, I do have to give them credit for one thing - I first read in PN about this guy who was winning lots of races and had some computer program he was selling. It was Doc Sartin.

[thaskalos]

Originally Posted by Tom

Profitable tot eh guy using it or the guy selling it?

BUT, I do have to give them credit for one thing - I first read in PN about this guy who was winning lots of races and had some computer program he was selling. It was Doc Sartin.

Well then...I guess I owe the rag an apology. 1 out of 500 ain't bad.