Building Databases

[Nosebob]

Can anyone offer advice on how to establish or acquire a database against which various ideas and theories can be tested? I have been involved in handicapping for about 25 years, and understand the basics, but on a "low tech" level.

My past research has always been done the "old fashioned way", using old copies of the DRF which I have saved over the years. The problem of course is that I cannot process enough samples working with pencil and paper to get meaningful results. (To say nothing of getting a lot of ink on my hands). There must be a better way.

From other postings, I see a lot of references to large databases, but very little discussion of how these databases were established.

I don't approach this with the idea that it will be simple or easy, and know that any worthwhile results will take considerable time and other resources. Any help or references in learning where to start will be appreciated.

Nosebob

[cj]

The starting point is that you need lots of computer friendly (comma delimited) past performances and charts for the same dates.

[Cratos]

Can anyone offer advice on how to establish or acquire a database against which various ideas and theories can be tested? I have been involved in handicapping for about 25 years, and understand the basics, but on a "low tech" level.

My past research has always been done the "old fashioned way", using old copies of the DRF which I have saved over the years. The problem of course is that I cannot process enough samples working with pencil and paper to get meaningful results. (To say nothing of getting a lot of ink on my hands). There must be a better way.

From other postings, I see a lot of references to large databases, but very little discussion of how these databases were established.

I don't approach this with the idea that it will be simple or easy, and know that any worthwhile results will take considerable time and other resources. Any help or references in learning where to start will be appreciated.

Nosebob

I would recommend that you build a relational database which focuses on the mathematical relations. Another database (which I would not recommend) would be the flat database which has the data stored in a long text file, called the tab delimited file.

[BillW]

As Craig stated, it requires daily collection of comma delimited data files for past performances and results (charts). Typically one would use these for day to day handicapping in conjunction with a handicapping program that understands the same files.

In my case I use HDW data files that are part of a subscription to the HTR handicapping program suite. Each day as I download files for all tracks running I import them into my database and sooner or later there is enough to do some research (presently I have all PP's/results for the US/Canada since March '06).

The easiest way is to use a program like HTR, JCapper or HSH that have database research capability built in. I know there are others but these are the major programs that I am aware of. The more creative handicappers use their own database tools such as the Microsoft Access database application (or oddballs like me that use Postgresql :)) to perform their research.

[Milkshaker]

I think Nosebob's question (especially the tone of it) mirrors my feelings toward delving into database research: I am curious to expand my knowledge, but am unsure how to take the next step from being computer-literate to being savvy about acquiring the data and then fine-tuning an existing program to test some of my theories. How fluent in programming do you have to be to work with it in various esoteric ways?

For example, here's the type of query I've been dying for an answer to: I play various small circuits, and my goal is to identify some of the worst favorites a particular track can offer for the purpose of betting against them.

How would I go about finding out something like "Which particular class, sex, and distance at Delta Downs has produced the most beaten favorites in the past three years?" And how narrowly could I then drill down further specifics within the same set of results, like if I wanted to know which jockeys & trainers really stood out in these categories?

Are there built-in features of certain programs that would allow me to formulate such a query, or would I have to have to somehow code a query like this from scratch?

I am respectful enough about database analysis to know that it does not take too big a leap to get in over one's head.

[Donnie]

The easiest way is to use a program like HTR, JCapper or HSH that have database research capability built in. I know there are others but these are the major programs that I am aware of.

Would have to agree 100%. This is a great way to start. Then delve deeper as your needs grow.

[traveler]

Milkshaker - here is a link to an HTR article that relates to lousy favourites,
http://www.homebased2.com/km/pdf/HTRMonthlyReport-NOV2002.pdf. I have never used HTR so I'm not endorsing it but it is very highly thought of by many.
This might give you an idea of what you can get out of some of the high-end programs. I use HSH and you should be able to find what you are looking for with Dave's software. Happy New Year!

[JustRalph]

What BillW and Donnie Said!! :ThmbUp:

[Tom]

I think Equisim has BD export also?
Just start out simple and grow into it - you will be amazed how quick things can fall into place.

The hardest part of the DB is feeding the damn thing.

[Track Collector]

Nosebob,

Just a note to tell you that this database journey will take time and will have a significant learning curve involved. One of the biggest challenges to a database user (and handicapper) is when can they "trust" their database to yield profitable results. Many angles can appear to be profitable in the short run, but end up negative in the long run. Once you identify an angle you want to play, start out with small wagers and slowly increase them over time if the angle continues to be profitable. (If a given angle turns out to be unprofitable, and I think all database handicappers unfortunately experience this more often than we care to admit, you will be glad that you had lesser money on them rather than more money. :) ) Do be advised too that even angles that show long-term profitability are going to have significant short-term losing streaks.

Using databases to handicap can be very rewarding. Best wishes with your new efforts!

[Cratos]

In reading through the responses I see more “how to use a database” as oppose to “how to build a data base.” Admittedly, the poster did also ask how to acquire a database.

But from my point of view I would start with a set of variables that I believe to be salient to the outcome of a race and build a set of formally described tables from which the data can be accessed or reassembled in many different ways without having to reorganize the database tables.

In 1984 I started my database construction with five variables (pace, weight, class, distance, and style) to understand their interrelationships mathematically on the outcome of a race. Over the years I have added more variables and changed some algorithms as I learned more about the desired outcome.

However a DBMS should be built or acquired based of the user’s intent.

[eqitec]

One of my databases is from Tampa Bay Downs results and includes about 51,000 starts and 3,013 winners from the the beginning of the '06-'07 meeting up till yesterday. 1,986 of the 3,013 winners were not favorites.

I found your question intriguing so I designed and ran a report from that database to show Summaries of Classes of Races Not Won By The Favorite.

The attached report required about 15 minutes of work, more to pretty up the report than for the actual processing of the data. It would take a bit more work to include surface and distance criteria; however, this should give you an idea of the value of using databases for handicapping research.

I am available to teach you or others on-line at mutual convenience how to build such databases using Filemaker Pro to meet your own handicapping research needs. Contact me privately if interested.

[Nosebob]

Thanks to all who responded with well thought out answers, based on an obvious abundance of knowledge. I think the best path to what I am looking for will be through one of the services mentioned.

I have a good project for the coming year(s).