Database Question

[eagleeye]

New to the forum and hope I'm putting this into the right place. My wife, a former programmer, would like to do some analysis on the ponies. Is there a place where we can get, or purchase, a copy of a database that would show most of the major fields such as the DRF? Some things that we'd like to see would be track, race number, Beyer ratings, Track speed and variant, calls on the track, etc..

Any help with this would be greatly appreciated.....

Thanks guys,
Eagleeye

[Tom]

You can export data from Formulator - that would be the only place you are likely to find the Beyers. All the fileds in the PPs are in the exports.
You would need to buy files fro each track/day and then exports the data for use in a spreadsheet of DB.

[njcurveball]

Is there a place where we can get, or purchase, a copy of a database that would show most of the major fields such as the DRF? Some things that we'd like to see would be track, race number, Beyer ratings, Track speed and variant, calls on the track, etc..



HDW has the best service in the business and many vendors support them. Most of them charge $100 to $140 a month for the data.

If you want a great export, look into HTR which has fields that are not in the DRF and everything you asked for except Beyer Speed Ratings. They carry a much better (IMO) speed rating influenced more by actual performance than speed figure guys trying to make their own figures fit into a huge database.

HTR is also free for download and if you sign up for a month you can download the previous 45 days. So if you start their service on September 1, you can download all of August and the last half of July to have in your database.

HSH is also a great program with a great export but the start-up and maintenance costs are much higher.

[eagleeye]

First, thanks guys. Your imput is very helpful. I'll start with the DRF first because I'm more familar with the data. After downloading the data, how do I access the data? The file extension is something like .drf2 (guess). How do I open this data?


Thanks in advance,
Eagleeye

[BillW]

Eagleeye,

I believe you need to download the Formulator program

http://www.drf.com/formulator/formulator_ad.html

This will allow you to open the files and then export the data into comma delimited files for import into a spreadsheet or database.

Bill

[highnote]

New to the forum and hope I'm putting this into the right place. My wife, a former programmer, would like to do some analysis on the ponies. Is there a place where we can get, or purchase, a copy of a database that would show most of the major fields such as the DRF? Some things that we'd like to see would be track, race number, Beyer ratings, Track speed and variant, calls on the track, etc..

Any help with this would be greatly appreciated.....

Thanks guys,
Eagleeye

You can buy a year's worth of archived comma-delimited charts from Daily Racing Form for something like $800 per year. For $1,600 you'd could create a heluva lot of past performances for analysis.

[garyoz]

Before you plunge in, remember this is well tread ground. You aren't going to discover fire.

[highnote]

Before you plunge in, remember this is well tread ground. You aren't going to discover fire.


But you might end up burning the midnight oil and your candle from both ends! It's a lovely light.

[JustRalph]

Before you plunge in, remember this is well tread ground. You aren't going to discover fire.

There is some excellent stuff already out there.............fyi

[singunner]

This being "well-tread ground" is open to debate. The analyses I've done up to now have taken thousands of processing hours on the latest, top-quality computers. Five years ago, you couldn't have done the things I'm doing today on a PC. Hell, even on the computers I'm using, it has taken me an inordinant amount of time.

Just because a lot of people have been thinking of how to approach this game for a very long time, it doesn't mean it's been properly researched and played out. There are very few people who are first scientists, then racing enthusiasts.

[sjk]

As long as the people who have done research use it for personal wagering only there is room for others near the fire.

[Kelso]

You can buy a year's worth of archived comma-delimited charts from Daily Racing Form for something like $800 per year.


Swetyejohn,
Does the $800 cover one track for one year, or is that the price for several/all tracks for a year? Do you know if the data is available on a calendar-year-only basis ... such that 2007 data would be the most recent available at that price?

Thanks very much.

[highnote]

Swetyejohn,
Does the $800 cover one track for one year, or is that the price for several/all tracks for a year? Do you know if the data is available on a calendar-year-only basis ... such that 2007 data would be the most recent available at that price?

Thanks very much.


I believe it is every result chart for every track for the entire year. You'll need to confirm this by looking at the price schedule and other info on DRF's website.

[ryesteve]

Before you plunge in, remember this is well tread ground. You aren't going to discover fire.
You mean everything good has already been invented?

[SMOO]

You mean everything good has already been invented?
Yes, the last good invention has already been invented. We can all go home now.

[midnight]

BRIS/TSN has specifications of what their comma-delimited data files contain, by field. That would probably be the starting point.

If you're datamining, you probably won't find anything useful that will produce standalone positive expectation, unless you make your own numbers, subscribe to numbers/data not datamined by many people, or have a lot of imagination. Datamining is beaten to death, in my opinion. Things that are found to work over the past year may not work over the upcoming year. Even if they hit the same percentage, the price often goes down because others find the same thing and bet on it.

[garyoz]

If you're datamining, you probably won't find anything useful that will produce standalone positive expectation, unless you make your own numbers, subscribe to numbers/data not datamined by many people, or have a lot of imagination. Datamining is beaten to death, in my opinion. Things that are found to work over the past year may not work over the upcoming year. Even if they hit the same percentage, the price often goes down because others find the same thing and bet on it.

Agreed, as the sample size increases you will regress toward the mean of the takeout rate. However, some people who post here do seem to do well with their proprietary approaches.

[BIG49010]

You can spend some serious money with bris, the DRF files are a $1.00 and the charts are 50 cents, multiply that by the number of race days for a meeting, then multiply that by the meets you want to play. You will blow past a $1,000 rather quickly if you plan to play to many meetings.


When I built out my first database, I did the previous 3 years, plus the current year and I spent some serious cash for the 4 tracks I play.

[njcurveball]

You can spend some serious money

If I were doing this, 45 days past days for every track for free and the first month for $120 with the HDW products would be a good start to see if this would be something I wanted to do.

Programs like HSH, HTR, and Synergism all have factors built into them that are not available elsewhere.

Since the download is free, I would recommend HTR as the best of the bunch.

[losealot]

Did you mean a year's PP's or a year's charts?
Thanks!

[highnote]

Charts. Sorry if I said PPs.

PPs are derived from charts.

[ZenMasterB]

Hey EagleEye,
I emailed you a couple of days ago.
Check it out.

z

[NoDayJob]

You can get everything you need to do back research, sans Beyer Numbers, at Post Time Daily. Talk to Doug Manley. They'll probably give you a deal if you order enough back races. www.posttimedaily.com

[eagleeye]

First, thanks to everyone that tried to help. It did not go unnoticed. Having been in this business for 33 years now, I thought I had just about seen it all. Yet my wife started collecting data and found some great things in hockey, baseball, basketball and yes, even football. Handicapping things that I'd never thought of. She is really a good find. A good example would be her research in trying to determine what happens when a team that has won 70% or more of its games within the last 30 games is playing a team that has won only 30% of its games over the last 30 games. In other words, this involved a continiously updating program. The results were actually outstanding.

She took hockey and went way into the numbers and came up with some outstanding data as well.

Having said that, she has some good ideas about horse racing. I've been playing the ponies for almost 25 years. I've held my own at the track, but some of her questions have brought up some things that I've never thought of. She is a programming wonder and just wanted some things to play with.

Thanks,

Eagleeye

[highnote]

A good example would be her research in trying to determine what happens when a team that has won 70% or more of its games within the last 30 games is playing a team that has won only 30% of its games over the last 30 games.

So what happens? Inquiring minds want to know. :)

She took hockey and went way into the numbers and came up with some outstanding data as well.

I dabble in hockey. Maybe we can compare notes sometime.

[eagleeye]

I tried to post a reply a day or two ago. Maybe being new has a learning curve. Mine's probably a learning circle.

First ZenMasterB, I have been having trouble with my email account, actually its been with my ISP provider. Please resend your email.

I've been doing this for 33 years and kind of thought I knew it all. Then I got married and my wife started tracking things that I said. I found out a ton of new things. She has looked at things I've never considered. She's looked at things like running a total on baseball and looking at what has happened over the last 5, 10, 20 and 30 games in baseball and if a team has a winning percentage of over 70% and is playing a team that has a losing percentage of 70% or greater what happens?????? She is good. Yes a lot of things have been looked at since the Internet came around (Thanks Al Gore....LOL). She deserves the opportunity to play. Who knows what she'll find........

Thanks,
Eagleeye

[Dave Schwartz]

Mr. Eye,

She's looked at things like running a total on baseball and looking at what has happened over the last 5, 10, 20 and 30 games in baseball and if a team has a winning percentage of over 70% and is playing a team that has a losing percentage of 70% or greater what happens?

That was a great idea! Very "out-of-the-box."


Gets the brain thinking.

Thanks.


Regards,
Dave Schwartz

[Skip]

Point well taken and on target. :)

[formula_2002]

New to the forum and hope I'm putting this into the right place. My wife, a former programmer, would like to do some analysis on the ponies. Is there a place where we can get, or purchase, a copy of a database that would show most of the major fields such as the DRF? Some things that we'd like to see would be track, race number, Beyer ratings, Track speed and variant, calls on the track, etc..

Any help with this would be greatly appreciated.....

Thanks guys,
Eagleeye

Next time marry a scientist who is on good terms with a mathematician!! ;)

Like many here I have thousands of files (I use the Bris data).
I can do some neat programming with the stuff..
End results? Wish I was a scientist... (I know a mathematician)

[prank]

Next time marry a scientist who is on good terms with a mathematician!! ;)

Like many here I have thousands of files (I use the Bris data).
I can do some neat programming with the stuff..
End results? Wish I was a scientist... (I know a mathematician)

It's very funny you mention that. You should visit a statistics department. You don't know how often it is that a clever idea never gets to see the light of real data.

I'm swimming in data, just not horse data. :cool:

[formula_2002]

It's very funny you mention that. You should visit a statistics department. You don't know how often it is that a clever idea never gets to see the light of real data.

I'm swimming in data, just not horse data. :cool:
I'm swimming in Horse data, and have run out of clever ideas (and that is saying a lot).
Post a clever idea and I'll test it.
My money is on the "don't" line ;)

Joe M

[garyoz]

Be careful...there's alot of people who can't handle the truth. IMHO if you think you can download DRF comma delimited files or Bris/TSN files and bounce a bunch of variables together and think you are going to find the road to profitability, I hope that you play the tracks I'm playing. You are much better moving up the curve by using the work already undertaken at HTR or HSH. Database handicapping is a very tough approach.

[formula_2002]

Be careful...there's alot of people who can't handle the truth. IMHO if you think you can download DRF comma delimited files or Bris/TSN files and bounce a bunch of variables together and think you are going to find the road to profitability, I hope that you play the tracks I'm playing. You are much better moving up the curve by using the work already undertaken at HTR or HSH. Database handicapping is a very tough approach.
I gather from what you say, HTR and or HTS might not employ quantitative analysis?
Could that be?
If so, what discipline does it follow?

Thanks
Joe M

[Tom]

HTR, and I would assume HSH, has proprietary ratings that can be used in your research - thinking not available in the raw data offered by DRF or BRIs ( other than Beyers, PP ratings, etc.)

[garyoz]

Tom has it right. Why not stand on the shoulder's of programs that have done yeoman's work in the area of analysis? HTR is well established and has a well known export function and even holds workshops on data analysis. HSH is about to release a new version and, over decades of research, the program has proprietary variables and methods on the cutting edge of handicapping. Maybe Dave Schwartz would be willing to disclose information about the number and scope of new variables in the new version.

This is much different than analyzing DRF or Bris comma delimited files. The easy stuff is already old news.

Formula, they do use a quantitative approach, albeit, in a variety of formats.

[formula_2002]

Tom has it right. Why not stand on the shoulder's of programs that have done yeoman's work in the area of analysis? HTR is well established and has a well known export function and even holds workshops on data analysis. HSH is about to release a new version and, over decades of research, the program has proprietary variables and methods on the cutting edge of handicapping. Maybe Dave Schwartz would be willing to disclose information about the number and scope of new variables in the new version.

This is much different than analyzing DRF or Bris comma delimited files. The easy stuff is already old news.

Formula, they do use a quantitative approach, albeit, in a variety of formats.

I just did a quick read of HTR User guide and downloaded the HTR@ program and flipped through a few screens.

I'm sure you see value here, but upfront, it looks to me not unlike data presentations I have seen from Colts Neck Data Service, All_Ways, Computrack, and few others that I have DEEPLY investigated over the recent years.
I know its a quick judgement, but my gut says pass.
Hows that for quantitative analysis analysis? ;)

[DanG]

I just did a quick read of HTR User guide and downloaded the HTR@ program and flipped through a few screens.

I'm sure you see value here, but upfront, it looks to me not unlike data presentations I have seen from Colts Neck Data Service, All_Ways, Computrack, and few others that I have DEEPLY investigated over the recent years.
I know its a quick judgement, but my gut says pass.
Hows that for quantitative analysis analysis? ;)
Please pass this judgment around…

I would like to keep HTR from the masses. ;)

[cj]

HTR, and I would assume HSH, has proprietary ratings that can be used in your research - thinking not available in the raw data offered by DRF or BRIs ( other than Beyers, PP ratings, etc.)

This is the key. I'm using my own proprietary stuff and there is plenty of profit to be found. If you stick with stuff that every Tom, Dick, and Harry can put in a database and research you are likely going to come up empty.

[formula_2002]

Just because I like to do this kind of stuff,
I exported the 752 horses contained in HTR's data base.

I concentrated on HTR's odds line ("Kline"), which I would think to be a good indiator of the data value from which it was derived.

when the acutal odds >=Kline odds 242 plays, 23 winners returned a total dollar odds of $194, for a .89 roi.

when actual odds >=Kline odds x 1.11, 22 winners returned 1.01 roi
when actual odds >=Kline odds x 1.20, 19 winners returned 1.12 roi.

when actual odds >=kline odds x 1.20, and odds<=10-1, 10 winners returned .83 roi.

-----------------------------------------------------------------

In the entire sample there were 18 winners in 340 horses whose odds were >10-1 .

when actual odds >=kline odds x 1.20, and odds>10-1, there were 9 winners in 103 horses.

HTR pick twice the number of winners in 30% of the horses.
_------------------------------------------------------
Using the same format as the above, anyone willing to post how the last 752 horses in their data base did? ;)

[DanG]

Just because I like to do this kind of stuff,
I exported the 752 horses contained in HTR's data base.

I concentrated on HTR's odds line ("Kline"), which I would think to be a good indiator of the data value from which it was derived.

when the acutal odds >=Kline odds 242 plays, 23 winners returned a total dollar odds of $194, for a .89 roi.

when actual odds >=Kline odds x 1.11, 22 winners returned 1.01 roi
when actual odds >=Kline odds x 1.20, 19 winners returned 1.12 roi.

when actual odds >=kline odds x 1.20, and odds<=10-1, 10 winners returned .83 roi.

-----------------------------------------------------------------

In the entire sample there were 18 winners in 340 horses whose odds were >10-1 .

when actual odds >=kline odds x 1.20, and odds>10-1, there were 9 winners in 103 horses.

HTR pick twice the number of winners in 30% of the horses.
_------------------------------------------------------
Using the same format as the above, anyone willing to post how the last 752 horses in their data base did? ;)
You’re obviously a very educated man and I’m just surprised that you would take such a small slice of an even smaller slice and attempt to draw conclusions.

The K-Line in large samples, (the way I use it) is a deadly predictor of the public. The “K” rating it’s derived from is a probability rating, not an attempt at profitability in a stand alone format. (Although profit exists within filtering of course)

HTR must have 1,000 pieces of information when properly exported and all the permutations of custom work the user wants to apply. In addition; 752 horses is what…1/4 of a busy Saturday?

Three + years into buying software based information and I’m still astounded how few people really understand its role. I’m not saying you’re among them BTW; I have a hunch you have traveled down about every handicapping road several times by now. Between the vendors who have sponsored this site and the path I’ve chosen, there is still a significant edge in 2007 over a good portion of the public...if you’re willing to work at it. :)

[formula_2002]

Dan, I’ve made no conclusions, simply reported what I found.
These results, in my estimation were, on the surface, quite favorable, but the issue is always how well future results will replicate previous results.
That’s why I asked for further results.

I think I have zeroed in on a respectable premise in saying that the accuracy of the probability is a reflection of the data and formulas used to determine the probability.

There can be no wiggle room here. It’s either accurate (to the point of profitability) or its not. And you can not start to break things down by other factors in conjunction with the singular probability without an equal amount of data both upstream and down stream of the new premise.

PS, can you quantify the term "Deadly Predictor"

[DanG]

PS, can you quantify the term "Deadly Predictor"
Formula; Brief example;

In my spot plays that I often pre-bet I use the K-line (in addition to other predictive factors) to anticipate the line from the public. It’s been a very useful “value line” in large samples.

Avg K-Line / Avg off- odds / Avg MLO. The deviation from off-odds and sample size. I stopped at 30-1 to save space and to deal with the majority.

(When compared to the morning line as a whole, the K-line is 655% more accurate in predicting the off-time odds on average.)

KLINE ODDS MLO SAMPLE K_DEV MLO-DEV
0 0.6 1.8 418 0.6 -1.2
1 1.5 2.2 39584 0.5 -0.8
2 2.7 3.1 94618 0.7 -0.4
3 3.7 3.6 88476 0.7 0.0
4 4.7 4.2 86587 0.7 0.5
5 5.7 4.8 82853 0.7 0.9
6 6.9 5.4 74776 0.9 1.5
7 8.1 6.0 59226 1.1 2.1
8 9.3 6.6 47549 1.3 2.6
9 10.3 7.2 40127 1.3 3.1
10 11.5 7.8 37851 1.5 3.7
11 12.6 8.3 37094 1.6 4.3
12 13.8 8.8 37165 1.8 5.1
13 15.1 9.2 32594 2.1 5.8
14 16.4 9.7 27324 2.4 6.6
15 17.5 10.2 26181 2.5 7.3
16 18.6 10.6 19309 2.6 8.0
17 19.6 11.1 16997 2.6 8.5
18 20.4 11.5 12758 2.4 8.9
19 21.1 11.8 12530 2.1 9.3
20 21.9 12.3 12256 1.9 9.5
21 22.9 12.6 10663 1.9 10.2
22 23.5 12.8 11939 1.5 10.6
23 24.3 13.0 9057 1.3 11.3
24 24.8 13.2 10047 0.8 11.6
25 25.2 13.5 7518 0.2 11.7
26 26.1 14.0 7057 0.1 12.1
27 27.2 14.5 6688 0.2 12.7
28 28.2 15.0 4864 0.2 13.2
29 28.6 15.0 6445 -0.4 13.6
30 29.0 15.1 6961 -1.0 14.0
99 58.7 25.8 47792 -40.3 32.9

[formula_2002]

Dan, thanks for the information, but what does the ROI look like when the kline is, say >=off-odds x 1.20.

It certainly would be a good thing to know the final odds range when playing into a pic three ..four..five etc.pool and knowing the probably payoffs before making the bet.
I'd work out the standard deviations and come up with a probability for a set of odds ranges, do the math and seek out the probable overlays.

Do they post the probable payouts before the close of the 1st leg?
Other than that, the odds board is a good indicator of the final odds, especialy after the gate opens. :)

Actual it would also be a good thing for those Pinnacle Match plays.

very impressive, but I would still need to know the standard deivations of the kline and off-odds and then see if there were profits to be made

[DanG]

Dan, thanks for the information, but what does the ROI look like when the kline is, say >=off-odds x 1.20.

It certainly would be a good thing to know the final odds range when playing into a pic three ..four..five etc.pool and knowing the probably payoffs before making the bet.
I'd work out the standard deviations and come up with a probability for a set of odds ranges, do the math and seek out the probable overlays.

Do they post the probable payouts before the close of the 1st leg?
Other than that, the odds board is a good indicator of the final odds, especialy after the gate opens. :)

Actual it would also be a good thing for those Pinnacle Match plays.

very impressive, but I would still need to know the standard deivations of the kline and off-odds and then see if there were profits to be made
Formula;

I don’t apply it as your suggesting in terms of return. I wish I could accommodate your query, but I have to take advantage of this rare down day in NY & Cal to catch up on some long overdue maintenance work. It’s all spinning away in the background, that’s why I can bother you good people with some hot air. :) I happened to have that odds display handy, so I threw it out there.

Best of luck in your quest for the grail! :ThmbUp:

[njcurveball]

I just did a quick read of HTR User guide and downloaded the HTR@ program and flipped through a few screens.




Funny I did the same more than once on my trip through the computer highway and decided HTR just wasn't what I was looking for.

For many, this program will not be what they want. You cannot "fit" your style into a computer program. The program must fit your style. If it doesn't you will never embrace the resulting data enough to make money from it.

In my opinion, the HTR program and User Group fits well with the "do it yourself" type of handicapper. Someone who is more interested in having many tools and deciding when and where to use them.

Funny, I see you fitting that mold Formula, yet you have chosed one tool, a limited job and make conclusions that may or may not go forward.

One thing is certainly true and that is trying to use final odds is a mine field with simulcasting. Many winners in your test may not have been qualifiers with 1 minute to post. OR many losers may not have been. You have no way to be sure and different places update the odds at different times.

When I am at the track, a horse can be 5-2 on the TV there. Then when I look at my laptop, 4NJBETS has not updated and they are 2-1 or 3-1.

Bottom line is that you really need to give any program 2 or 3 months to see if it fits with what you want. I have used many other programs (Synergism and HSH are two good ones), but HTR is the best fit for my game.

AND the last thing we would all want is everyone using the same program, betting the top choices when they are overlays with 1 minute to post and hoping the odds won't drop.

Jim

[shanta]

I'm swimming in Horse data, and have run out of clever ideas (and that is saying a lot).
Post a clever idea and I'll test it.
Joe M

Hi Joe

Can I ask how often is the winner of the race in the top 3 contenders of your mix? Do you track this? If yes do you have a circuit(s)/track(s) with a good % in top 3?

Can you access the pp's for these past races?

Forget database and backfitting stuff for now.

Do you know % of wins top 3 and can you access pp's for races run? Say 200 races?

Richie

[Tom]

That can be done easily.
Hardest parts deciding what factor to use.;)

[shanta]

That can be done easily.
Hardest parts deciding what factor to use.;)

How bout if it is NOT a "factor"? At least a factor that is not part of any software's "base' that can be called up. :)

However something that is EASILY seen when looking at pp's

[TurfRuler]

I'm looking for an edge
I"m looking for an edge
I'm looking here and there
I'm looking everwhere
I'm looking, i'm looking, i'm looking....

But I've found that the past performance that is printed everwhere gives me the same results

Losing by a nose a whisker and even a hair.

[Tom]

How bout if it is NOT a "factor"? At least a factor that is not part of any software's "base' that can be called up. :)

However something that is EASILY seen when looking at pp's

Possibly. Depends on what it is.
And then on my ability to query it:rolleyes:

[shanta]

Possibly. Depends on what it is.
And then on my ability to query it:rolleyes:

check your PM box in a few amigo

[formula_2002]

Hi Joe

Can I ask how often is the winner of the race in the top 3 contenders of your mix? Do you track this? If yes do you have a circuit(s)/track(s) with a good % in top 3?

Can you access the pp's for these past races?

Forget database and backfitting stuff for now.

Do you know % of wins top 3 and can you access pp's for races run? Say 200 races?

Richie

Sure, my top three pick win to their sum of probabilities (1/final odds+1) x (1-track take).
Generally returns an roi of .80 to .84

[shanta]

Sure, my top three pick win to their sum of probabilities (1/final odds+1) x (1-track take).
Generally returns an roi of .80 to .84

I don't know what any of that means Joe. :) I look at horses not "probs".

Check your Pm