133 things that I know about horse racing (fishing for new DB ideas)

[LottaKash]

I've got to hand it to the Computer-Handicapping guys....They have all that neat information at their finger tips to aid them in their journey to "steady profitability"....I get jealous of that sometimes...

Me, I am now, and will probably always be, a diehard "pen and paper" handicapper....And, that is both a plus and a disadvantage at times...

At my age, the learning curve is just too steep at this juncture of my game to try anything more tricky than some good old fashioned simple Relational Databasing, for record keeping you know....So now that I have this base line "matrix" to work with, I am quite ready to go on to recording this information in various ways....I want to do this in order to reinforce what I know and believe to be true, and yet discover some things that will correct for any misgivings or untruths about what I belive to be true....

Below, I have posted a matrix of all the pertinent data that I truly believe could have any possible significance or bearing as to the winning of a horse race by a qualified candidate.....Each race is a unique situation, and with each of them, will come a unique way of using the matrix in that particular situation....

================================================== ====

http://i405.photobucket.com/albums/pp137/lottakash/loopholes-1-1.jpg
================================================== ====

This is the "current state of my "Handicapping-Reality"...."LOOPHOLES", as I have dubbed them...It represents everything that I know...It comes with a high price tho...Lots of blood, sweat and tears....Even so, I am still passionate about all of this tho, even after the 50-yrs of doing this....I made my first wager at 17yrs old...I have been hooked ever since....

This has all to do with Harness Racing tho, still, I have been involved with both sides of the different breeds long enough to know how similar the handicapping needs and requirements are, regardless of the breed....So I would ask that you don't let that throw you, this all about finding new ways to Database....The only glaring and important difference between the two breeds, imo, are the strategies that a driver will use to get a horse home first....I mean you just can't open the holes up in a harness race as easy as a T-bred can....Logistics....That is why the Driver vs. the Jockey has so much more an effect on a harness-horse than a Jock does, imo...It is more akin to driving a car vs. a horse carrying a man on his back.....Still pace is pace, and speed is speed, and ABility, Form, and Angles are analgous to both breeds...

A little over 3-yrs ago, I revamped my particular way of going, and adopted some new "Pace Numbers" and began incorporating them into the way I had been doing things for so long.....A risky venture, to say the least, as I had been doing just fine at that time....But, I knew there was more, and I wanted it....So the journey began, and as time went on, I began to understand the true significance of the numbers that I had employed and how potentially powerful they were when used with certain "Labels" in certain situations....Along the way, a tidal wave of "Labels" were born and began to flow across my desk daily as I began to identify and quantify all the things I had, all these years, believed to be predictive to winning a harness race....As my "label-list" began to swell, I soon came up with a language all my own....The marriage of the "Labels" to "Pace Patterns" had begun....How sweet and rewarding this is, to any Pen and Paper Handicapper....I can now speak "fluent" LOOPHOLES....And, it is teachable now....Meaning, if I taught anyone my language and the pace patterns, most times, we would be on the same page in any given handicapping situation...It is that repeatable and reliable...Not perfect mind you, but I am still amazed at how much easier it is to tweak any idea these days....The Pace Patterns, were the missing ingredient all these years....I now have a way to measure the ongoing (race to race) form cycles of any horse....The Pace Patterns are my baseline now, and they keep me in check and in good stead.....And, my labels are magic when they are there....And when they are not there, it is pass, pass, pass, simple...

I have become a "Guerilla Handicapper"since...Ever waiting and watching for Loopholes to appear....I know I will never get rich at this game, I have tried that...But, I am not shooting for that any longer.....Just a way to supplement my "fixed income" retirement....(and it's working)

================================================== =====

Here is where I need some new insights and knowledge....I have never been a serious databaser, and never took the time to learn all about them....I know I should have done more but I just never did...I could've been really good at it if I did, but I just never took or had the time with my busy work and family life...

So, keeping in mind that I am a die hard P&P player, I am looking for a Relational Database that HAS, an EZ Front End to it.....Meaning, I had downloaded a database called "Database Oasis", but the trial wore off before I could give it a proper ride.....I had all the front end setup and I was doing ok with it, until it died of expiration date....

I was fascinated with the way the front end was setup....It was a P&P cappers dream....Once you had the front end part setup by using drag & drop boxes, it was all pushbuttons from thereon....This kept it P&P for me....

Now I am not so sure what to use, or even if the $119 price Oasis was worth it for "only" the ease of use, and still not being sure if it is relational rather than just a flat file db....

I can think of a dozen new DB's that I would like to create & implement, but only if the learning curve is not so steep or tedious....I just want to push the buttons, is all....

Is there anything like that out there.....?

[plainolebill]

LottaKash, DatabaseOasis sounds like an interesting product, according to the website the pro version is a relational db.

What kind of files did you use?

Thanks

[DeltaLover]

- For a relational db you can check:

http://www.mysql.com/

Very simple to use and very powerful

- You should at least consider a non relational db like:

http://www.mongodb.org/

Besides that though I am not sure that a db will cover your needs as at least you will need a statistical package like:

http://www.r-project.org/

Also
http://www.gnu.org/software/octave/
might be helpful



(DatabaseOasis and other similar proprietary software (like liberty basic, clarion etc) should be avoided as they are much inferior, cost money, do not adhere to standards etc etc...)

[Red Knave]

according to the website the pro version is a relational db.

Also, they say they'll give you another 15 days if you email and ask them.

[BIG49010]

I've used an old relational database Paradox for this kind of data for 20 years.

You can pick up used copies on the web for cheap, and the PAL language is very easy. I still run my race cards this way everyday, and the program performs decent on Window 7.

[plainolebill]

'Easy' is a relative term - to me wiring a house, building cabinets, overhauling a motorcycle engine is easy, but at age 70, any kind of code writing is not 'easy'.

Like LK, I don't want a steep learning curve.

[BIG49010]

Query

SAR12STS | FINISH TODAYS DATE 1 OR 2 | TRAINER | ODDS |
| Check 1 | Check ~xxx| CALC AVERAGE |

SAR12STS | DAYS BETWEEN RACES |
| >89 |

Endquery
DO_IT!

You kind of spell everything out, so it is somewhat simple, but I guess it depends on the person. I don't consider myself a programer at all, and I have hundreds of queries that look at every angle you can think of.

[BIG49010]

This kind of database handicapping took me a lot of years to program, but it has been worth every moment of it. I am glad there is little interest in this type of handicapping, my ROI will continue to be good for years.

[traynor]

@ LottaKash

Are you sure you want a database? Creating the intitial database is a snap, but writing the queries needed to extract information from that database (for something as complex as handicapping horse races) is definitely not. As Big49010 poihted out, it could take years to program it and fiddle with it to get it to do exactly what you want it to do. So, the question is, what do you want it to do?

Unless you are already familiar with writing stored queries for databases, you may be better off with something else. "What else" depends on what you need to do after all the stuff goes in. That is the easy part. If you want to locate specific patterns in the past performance records, for example, you might be better off with an application than with a database. It is a lot easier and a lot faster to code functions that extract information from race lines than it is to create all the queries necessary to extract the same information from a database.

[bob60566]

@ LottaKash

Are you sure you want a database? Creating the intitial database is a snap, but writing the queries needed to extract information from that database (for something as complex as handicapping horse races) is definitely not. As Big49010 poihted out, it could take years to program it and fiddle with it to get it to do exactly what you want it to do. So, the question is, what do you want it to do?
Unless you are already familiar with writing stored queries for databases, you may be better off with something else. "What else" depends on what you need to do after all the stuff goes in. That is the easy part. If you want to locate specific patterns in the past performance records, for example, you might be better off with an application than with a database. It is a lot easier and a lot faster to code functions that extract information from race lines than it is to create all the queries necessary to extract the same information from a database.
What type of application I have SQL or is it certain program.

Mac :)

[traynor]

What type of application I have SQL or is it certain program.

Mac :)

SQL uses queries, like:
Select (whatever) From (whatever) Where (conditions are)

In most programming languages, especially some of the user-friendly types, it is easier and faster to use conditionals, like:
If (condition1) AndAlso (condition2) AndAlso (condition3) Then (do stuff. Put it in a file, write it to the screen, or whatever)

It may sound intimidating if you have never done it before, but (at least in my opinion, based on having done a whole lot of research on race results) I have never found SQL to be anywhere near as fast and easy as using conditionals.

You can get downloads of Visual Studio 2010 free from Microsoft, and build your own application fairly easily. They have lots of "getting started" video tutorials, and so on. Pick the stuff you need to know to do what you need to do, and put it together.

http://www.microsoft.com/visualstudio/en-us/products/2010-editions/express

If I were starting from scratch--knowing nothing about programming, databases, or SQL--I would go straight for learning basic programming, rather than even considering SQL. Learning to use Visual Basic to build your own application seems a lot easier than trying to do the same thing with SQL and a bunch of data dumped into a database.

You don't need to learn to be a programmer. Visual Basic is a tool, and all you need to learn is enough to do what you want to do. You can always learn more, but the amount of programming skill you need to get started is minimal.

[traynor]

For example, I use something similar to what you developed. I search past performance data for "attributes"--things that winners have more of than losers. To extract those attributes, I load a past performance line (essentially the race line).

Jul232012,Mea,R11,Pace,5280,Fast,6900,6,KallaAttac k,27.59,57.79,85.38,114.99,2Z,1,1,2,2,1,0,0,1.5,0. 05,114.99,29.60,2.30,RandyTharps,RandyTharps,16.40 ,50.40,1.0000

That data is split into "elements" where the commas are. Each element has a number, starting with 0. Element(1) is the second element in the line "Mea" so all I need to do extract the equivalent of a SQL query to locate data in a table with a Select statement is:

If Element(1) = "Mea" AndAlso Element(3) ="Pace" and so on. That extracts only pace races at The Meadows. Refine your search by tacking on additional conditions, like: AndAlso Element(17) = 1 AndAlso Element(25) < 3. That extracts only winners of pace races at The Meadows that had post time odds less than 3-1.

The advantage is in the ease of creating complex searches that look for stuff that is not quite as simplistic as most databases are designed to handle. SQL seems easy initially, but bogs down badly for anything beyond basics. It can handle it, but it takes a lot more time, programming skill, and knowledge to create a relatively sophisticated (or complex) query in SQL than it does to create the same thing using conditionals in Visual Basic.

[bob60566]

For example, I use something similar to what you developed. I search past performance data for "attributes"--things that winners have more of than losers. To extract those attributes, I load a past performance line (essentially the race line).

Jul232012,Mea,R11,Pace,5280,Fast,6900,6,KallaAttac k,27.59,57.79,85.38,114.99,2Z,1,1,2,2,1,0,0,1.5,0. 05,114.99,29.60,2.30,RandyTharps,RandyTharps,16.40 ,50.40,1.0000

That data is split into "elements" where the commas are. Each element has a number, starting with 0. Element(1) is the second element in the line "Mea" so all I need to do extract the equivalent of a SQL query to locate data in a table with a Select statement is:

If Element(1) = "Mea" AndAlso Element(3) ="Pace" and so on. That extracts only pace races at The Meadows. Refine your search by tacking on additional conditions, like: AndAlso Element(17) = 1 AndAlso Element(25) < 3. That extracts only winners of pace races at The Meadows that had post time odds less than 3-1.

The advantage is in the ease of creating complex searches that look for stuff that is not quite as simplistic as most databases are designed to handle. SQL seems easy initially, but bogs down badly for anything beyond basics. It can handle it, but it takes a lot more time, programming skill, and knowledge to create a relatively sophisticated (or complex) query in SQL than it does to create the same thing using conditionals in Visual Basic.
Thanks
I will print this off and have sent email to my sql guy on this subject seems like winter project.

Mac :)

[bob60566]

For example in race lines can i get it show me horse that was 5th at the stretch and finished 5th and last three running lines within 45 days to 60 days.
Have now downloaded Visual and will figure it out.

Mac :)

[traynor]

Thanks
I will print this off and have sent email to my sql guy on this subject seems like winter project.

Mac :)

If someone else is doing your SQL programming, that is something entirely different. I thought you were going to do it yourself--in which case it would probably be easier and quicker to learn basic programming in a language like Python (very easy) or Visual Basic (pretty easy).

If you are hiring out the programming, the best advice I can give you is to develop a very explicit set of instructions defining exactly what goes in, what needs to be done to it, and what you want to come out. About 98% of the "problems" people have in contracting programming work is caused by their failure to clearly define what they want to accomplish with that work.

[traynor]

For example in race lines can i get it show me horse that was 5th at the stretch and finished 5th and last three running lines within 45 days to 60 days.
Have now downloaded Visual and will figure it out.

Mac :)

Easily. Each of those figures is an "element." One of those elements is the stretch call position (16) and beaten lengths (21). The first element(0) is the date of the race--that format is the result of an internal calculation the computer uses and is simple to extract as "within 45 days to 60 days" or any other range of values.

I use the date format shown because I want to be able to compare dates at a glance when the race lines show up on my monitor and that format is my own personal preference. Dates can be displayed in a number of other formats that may work better for you.

Understand that "basic programming" is something you can learn in a matter of hours, and usefully apply. There are a lot of bells and whistles that may take longer to learn, but the basics will do all, or almost all, what you need to do. The advantage of learning to do it yourself is that when you get another idea, or want to look at something else, or to analyze some other factor, you realize that it only takes a couple of lines of code to do some pretty amazing things.

For example, the app I use myself loads the information for today's races, and generates a list of the bets I want to make according to criteria I have defined. I don't need to look at individual horses, or at individual races, unless I want to for some reason. The upside is that the processes I use are systematically and consistently applied--whether I am rushed for time, feel good, feel bad, or whatever else that causes people to make incredibly foolish wagers and lose money.

If you decide to learn the basics of Visual Basic, I have a lot of "boiler plate" code that I can send you to save you a LOT of time and effort. It can be cut and pasted into your own VB application, or at least will give you an idea of how a particular thing is done. For example, converting "program time" (like 2:01.4--expressed in minutes, seconds, and fifths) to a format the computer can use (like 121.8--expressed in seconds and tenths).

[bob60566]

Thanks excellent information and as usual the learning curve and is something that will suit my needs.
Does it take Bris single files.

Mac :ThmbUp:

[DJofSD]

I've used an old relational database Paradox for this kind of data for 20 years.

You can pick up used copies on the web for cheap, and the PAL language is very easy. I still run my race cards this way everyday, and the program performs decent on Window 7.
Funny you should mention Paradox. I am just about to chuck all of my old Paradox materials into the trash and recycling bin. Really liked the QBE.

[DJofSD]

If someone else is doing your SQL programming, that is something entirely different. I thought you were going to do it yourself--in which case it would probably be easier and quicker to learn basic programming in a language like Python (very easy) or Visual Basic (pretty easy).

If you are hiring out the programming, the best advice I can give you is to develop a very explicit set of instructions defining exactly what goes in, what needs to be done to it, and what you want to come out. About 98% of the "problems" people have in contracting programming work is caused by their failure to clearly define what they want to accomplish with that work.
Hiring out: I would also add they need to understand relational data base design. Designing applications and writing code is one thing, architecting and implementing a relational data base is something else. Heaven help you if the design is poor or just out and out wrong.

If Jeff sees this thread, he might add some helpful statements about the importance of sound design.

[BIG49010]

Funny you should mention Paradox. I am just about to chuck all of my old Paradox materials into the trash and recycling bin. Really liked the QBE.

You might want to google Paradox for Sale, there are people looking for copies of it. I've been amazed that it continues to work on every new computer I get, and every version of Windows too.

[traynor]

Thanks excellent information and as usual the learning curve and is something that will suit my needs.
Does it take Bris single files.

Mac :ThmbUp:

You said you were doing this for harness races. Does BRIS have harness data? Usually each data source (like BRIS) will have a listing that shows what sequence (and format) their data is in. The downloaded data can then be "parsed" (extracted and put into a form that can be read by the application.

Are you doing this for harness races, thoroughbred races, or both?

[DJofSD]

You might want to google Paradox for Sale, there are people looking for copies of it. I've been amazed that it continues to work on every new computer I get, and every version of Windows too.
Gee whiz, you're right. Never thought about that.

Not great prices but better than tossing it and getting nuttin'.

Thx.

[traynor]

Hiring out: I would also add they need to understand relational data base design. Designing applications and writing code is one thing, architecting and implementing a relational data base is something else. Heaven help you if the design is poor or just out and out wrong.

If Jeff sees this thread, he might add some helpful statements about the importance of sound design.

That is a given. It is also why I suggested the first step is a crystal clear knowledge of what is to go in, what is to be done with it, and what is expected to come out. There is no more complexity in designing a RDBMS app than in designing a desktop app. (Don't tell the IT staff ... it will give them performance anxiety.)

[traynor]

Hiring out: I would also add they need to understand relational data base design. Designing applications and writing code is one thing, architecting and implementing a relational data base is something else. Heaven help you if the design is poor or just out and out wrong.

If Jeff sees this thread, he might add some helpful statements about the importance of sound design.

I had to study the results of every race run in Australia for five years, that was in a database that was supposed to be "well architected." It may have been well designed for dumping stuff in, but it certainly was not for getting anything out beyond the simplest relationships, and it totally botched most of those. Specifically, the basic database design was good, but had no facility for recognizing such minor details as the entry of parsed datalines being bumped one column to the left or right on occasion. Among many other things.

Whose fault? Not the database designer, who followed explicitly a set of instructions of what the client wanted. It is the nature of the beast in horse racing (especially in Australia) that the format of data can (and often does) change just enough to thoroughly gum up the works.

[DJofSD]

That is a given. It is also why I suggested the first step is a crystal clear knowledge of what is to go in, what is to be done with it, and what is expected to come out. There is no more complexity in designing a RDBMS app than in designing a desktop app. (Don't tell the IT staff ... it will give them performance anxiety.)
I believe we are in complete agreement.

One thing I learned a long time ago, things change, especially when it comes to data bases. Having the experience and understanding of the fundamentals of a good design will not insulate you from changes but it will make things a lot easier to handle when they occur.

IMO, starting with a good design of the data base will make the design of the app almost as easy as pie -- not that making a pie is all that easy.

[DJofSD]

I had to study the results of every race run in Australia for five years, that was in a database that was supposed to be "well architected." It may have been well designed for dumping stuff in, but it certainly was not for getting anything out beyond the simplest relationships, and it totally botched most of those. Specifically, the basic database design was good, but had no facility for recognizing such minor details as the entry of parsed datalines being bumped one column to the left or right on occasion. Among many other things.

Whose fault? Not the database designer, who followed explicitly a set of instructions of what the client wanted. It is the nature of the beast in horse racing (especially in Australia) that the format of data can (and often does) change just enough to thoroughly gum up the works.
I put those kind of problems into that bucket of things that can and will change. If I understand correctly, the so called front end needed to be tweaked in order to handle how the data was being presented.

And, yes, getting things out is important. What the hell good is having your data in a data base if you can't query it or extract it for analysis? Reminds me of the roach motel -- data goes in but doesn't come out. (Or was that the black hole analogy?)

[traynor]

I put those kind of problems into that bucket of things that can and will change. If I understand correctly, the so called front end needed to be tweaked in order to handle how the data was being presented.

And, yes, getting things out is important. What the hell good is having your data in a data base if you can't query it or extract it for analysis? Reminds me of the roach motel -- data goes in but doesn't come out. (Or was that the black hole analogy?)

That was the whole point of asking if what he really wanted (needed) was a conventional database. I have used for quite awhile (and continue to use) plain text files, because they are the fastest and easiest to parse. I am (almost) converting to XML, in an attempt to overcome the basic problem of mixed data sources and mixed data formats. The outside view is that it is simple to "create a database" and that is essentially true. Overlooked are the issues of "what happens when the data format changes" and that whole "getting stuff out" stage.

Front-end design is only a part. That can be handled fairly easily. The problems arise when the design has been centered on one particular data format, (almost) fully automated, filled with spiffy validators and exception handlers, and "similar appearing" (to the app) data is entered. Not the gross stuff like a string where a double is expected, but a value like 29.4 (odds to one) where the app expects a last quarter time (also 29.4). Extracting data becomes a bit frustrating when virtually every element of data has to be secondarily validated as being within an "acceptable" range, and even then can be seriously tweaky. Especially with way over a million past performance lines, each with more than a hundred elements.

When it is my money being wagered, I like to be able to depend on the numbers I use.