lies, darned lies, and statistics

[dilanesp]

I just bought the print edition of the Racing Form, and I'm shocked the number of writers who are throwing out the "statistic" that Exaggerator cannot win because no 2nd place Derby finisher has won the Derby since Prairie Bayou in 1993, and only three have done so since 1960.

This is just an absurd example of the tendency of handicappers to create "statistics" based on tiny Triple Crown sample sizes that are clearly due to variance.

First of all, there's nothing special about running second in the Derby. Some horses who did so were excellent horses (Easy Goer, Summer Squall). Other horses who did so were nobodies (Commanding Curve, Golden Soul). Evaluating the statistic, rather than the individual horses, is ridiculous. Obviously, Easy Goer and Summer Squall had a much higher chance of winning the Preakness than Commanding Curve or Golden Soul did.

Second, 56 Preaknesses (starting in 1960) sounds like a lot, but really is a tiny sample sizes. When Sheldon Kovitz and Andrew Beyer first developed their speed figures, they used the data set from a year's worth of races in New York to develop the par times-- in those days that would have been about 180 race days with nine race cards, or 1620 races! If they had simply looked at 56 races, do you think their calculations would have been accurate? This is a problem with ALL Triple Crown stats, and is a reason why we keep seeing things that NEVER used to happen (favorites winning the Derby, horses with bad dosage winning the Derby, horses without good 2 year old form winning the Derby, 2 year old champions winning the Derby, Breeders' Cup Juvenile winners winning the Derby, horses without sufficient prep races at 3 winning the Derby) happen with regularity.

Third, let's look at some of the horses who lost the Preakness after finishing 2nd in the Derby. Easy Goer, in 1989, lost by a nose in a head-bob. Yet that loss counts equally in the stat to Firing Line running up the track two years ago. Similarly, Captain Bodget lost a three horse photo in 1997 to Silver Charm and Free House.

A number of the second place finishers finished second to the Derby winner again in the Preakness, which actually DOES NOT necesssarily suggest they were bad bets, because if something had happened to the Derby winner, they would have won. These include Bodemeister losing to I'll Have Another, Menifee losing to Charismatic, Victory Gallop losing to Real Quiet, Easy Goer, of course, losing to Sunday Silence, Bet Twice losing to Alysheba, Alydar losing to Affirmed, Sham losing to Secretariat, and Arts and Letters losing to Majestic Prince.

So it sounds bad. Like it never happens. The second place finisher in the Derby never wins the Preakness. But this is perfectly clear that this is just a product of RANDOM STATISTICAL VARIANCE.

The lesson: stop overemphasizing small-sample stats on big races like the Triple Crown and Breeders' Cup in your handicapping. They really are just noise most of the time. Handicap based on fundamentals, not that something can "never" happen.

[Tom]

+1

[clocker7]

I wholeheartedly agree that the smaller sample size creates a larger margin of error. But rather than using paragraphs of anecdotal data, what is it?

[dilanesp]

Originally Posted by clocker7

I wholeheartedly agree that the smaller sample size creates a larger margin of error. But rather than using paragraphs of anecdotal data, what is it?

It's actually impossible to know the "real" margin of error, although it is possible to calculate a p value of some sort.

The reason it's impossible to know the "real" margin of error is because the Preakness isn't even a consistent trial. It's different every year. The field size is different, the track bias is different, the track condition is different, the strength of the 3 year old crop is different, the pace is different, the medication rules have changed, etc. The Preakness of 1960, when these people started their calculation, is not really even the same race as the Preakness of 2015.

Calculations that treat the Preakness won by Spectacular Bid over 4 other horses as equivalent to the Preakness won by Rachel Alexandra over 12 other horses are grouping apples and oranges.

So I don't think a significance value based on all 56 runnings is really going to be very helpful here. At the very least, I'd want to throw out some Preaknesses that were clearly unrepresentative before running any sort of statistical analysis.

[RXB]

Originally Posted by dilanesp

I just bought the print edition of the Racing Form, and I'm shocked the number of writers who are throwing out the "statistic" that Exaggerator cannot win

Please show us all of these writers who have said that "Exaggerator cannot win." I mean, if it's a shocking number there must be a whole whack of them in the DRF who have stated that it is flatly impossible for Exaggerator to win.

[dilanesp]

Originally Posted by RXB

Please show us all of these writers who have said that "Exaggerator cannot win." I mean, if it's a shocking number there must be a whole whack of them in the DRF who have stated that it is flatly impossible for Exaggerator to win.

Brad Free:

"Exaggerator faces a historical challenge. Over the last 20 years, Derby runners-up are 11-0-3-3 in the Preakness." [He picks him 3rd, behind Stradivari]

Marty McGee:

"Exagerrator will be trying to become just the third Derby runner-up in the last 55 years to win the Preakness... It's an extraordinary span that encompasses 40 starters... including 10 beaten favorites and an overall ROI of a mere $0.33 for every $2 bet." [followed by a response to Keith Desormeaux to a question about whether his colt could win given that]

Plus a feature box, "How Kentucky Derby Runners-Up Have Fared in the Preakness", listing all of them since 1960.

I mean, given this is a completely bogus, phony, non-sensical statistical angle with no validity, that's way over the top, isn't it?

[RXB]

Again, where did anybody, never mind "a shocking number" of DRF writers say that "Exaggerator cannot win" as you claimed? They didn't, of course. You start a post about "lies" and "damned lies" with your very own willful distortion of facts.

Agree or disagree as you will with their reasoning but don't distort their words. These guys have professional reputations to upkeep.

[Tom]

I wouldn't call most of theses article professional by any means.
Filler would be a better word.

I don't think he was calling anyone a liar, but using an age old phrase that correctly describe the alleged statistic being used to fill column space.

[dilanesp]

Originally Posted by RXB

Again, where did anybody, never mind "a shocking number" of DRF writers say that "Exaggerator cannot win" as you claimed? They didn't, of course. You start a post about "lies" and "damned lies" with your very own willful distortion of facts.

Agree or disagree as you will with their reasoning but don't distort their words. These guys have professional reputations to upkeep.

You apparently have never heard the phrase "lies, damned lies, and statistics". It's an expression.

Where did I call any drf writer a liar?

As for their professional reputations, I'd worry more about what they published and less about the title of my post.

[AndyC]

Originally Posted by dilanesp

You apparently have never heard the phrase "lies, damned lies, and statistics". It's an expression.

Where did I call any drf writer a liar?

As for their professional reputations, I'd worry more about what they published and less about the title of my post.

I think it is an interesting fact but not a significant fact for handicapping. I don't know that any of the writers claimed it to be the latter.

[whodoyoulike]

Originally Posted by dilanesp

I just bought the print edition of the Racing Form, and I'm shocked the number of writers who are throwing out the "statistic" that Exaggerator cannot win because no 2nd place Derby finisher has won the Derby since Prairie Bayou in 1993, and only three have done so since 1960.

This is just an absurd example of the tendency of handicappers to create "statistics" based on tiny Triple Crown sample sizes that are clearly due to variance.

First of all, there's nothing special about running second in the Derby. ...

Exactly! I always wonder when people write or say that there is a statistical significance when there really isn't. And, I'm not referring to the sample size.

[RXB]

Originally Posted by dilanesp

Where did I call any drf writer a liar?

I didn't say that you called a DRF writer a liar. I just asked you a specific question that you haven't answered: why did you claim that many DRF writers said that "Exaggerator cannot win" when apparently none did any such thing? They merely quoted historical statistics that the Derby runner-up has not landed in the Preakness winners' circle very often over the decades. I'd be very surprised if Exaggerator is not the consensus second choice among them.

You stated in an earlier thread that you're backing Exaggerator in part because of his big win on a sloppy surface in the Santa Anita Derby which you indicated is proof that he moves up on a sloppy track. Isn't one the smallest and therefore least reliable sample size possible? And wasn't that at a different track with very different competition-- specifically, no Nyquist? And yet you seemed to try to downplay Nyquist's four wins in four races over Exaggerator-- with Exaggerator never finishing within a length of Nyquist. So, I'm confused as to where you truly stand regarding data, statistics, sample sizes, what constitutes valid information vs non-valid information, etc.

[dilanesp]

Originally Posted by RXB

I didn't say that you called a DRF writer a liar. I just asked you a specific question that you haven't answered: why did you claim that many DRF writers said that "Exaggerator cannot win" when apparently none did any such thing? They merely quoted historical statistics that the Derby runner-up has not landed in the Preakness winners' circle very often over the decades. I'd be very surprised if Exaggerator is not the consensus second choice among them.

You stated in an earlier thread that you're backing Exaggerator in part because of his big win on a sloppy surface in the Santa Anita Derby which you indicated is proof that he moves up on a sloppy track. Isn't one the smallest and therefore least reliable sample size possible? And wasn't that at a different track with very different competition-- specifically, no Nyquist? And yet you seemed to try to downplay Nyquist's four wins in four races over Exaggerator-- with Exaggerator never finishing within a length of Nyquist. So, I'm confused as to where you truly stand regarding data, statistics, sample sizes, what constitutes valid information vs non-valid information, etc.

RXB, you are trying to manufacture a controversy. Turn down your outrage meter a bit. This is the Internet. People say things you disagree with.

If you think I'm wrong about Exaggerator in the Preakness, feel free to bet what you think will be the likely outcome. I sincerely wish you luck.

As for the rest, note that several regulars here agreed with my point about the DRF's statistic and what it says about Triple Crown statistics. Which suggests that rather than me being wrong, you grossly misread my post (which you did).

[garyscpa]

I never use the Kovitz's to bet the triple crown races.

[burnsy]

That goofy, quirky so called "stat" is the least of his worries. There's the pace, not giving up ground, and a horse named Nyquist, that's done whipped him 4 times. Stat be damned, winning is the most important one, and now Nyquist will most likely sit a garden trip. Good luck closing on that.

Will use a little but if Nyquist loses, I think it will be a dark horse. I don't know if this horse can.