Some Software development surprises

[Capper Al]

I'm a one man development shop. Over the last few years I spent over 2,500 hours developing my software. The program is working fine. I figure for version I, it is about 90% finished. Here are some of the amazing things that I have found about developing your own app.


It takes longer than one thinks it will.
We always need to spend more time testing than we do.
The product is never finished. I had to qualify my 90% complete with version I. I know there will be a version II.
The biggest surprise was that after developing the specifications and writing the code, I still had to learn how to use it to gamble. I wasn't expecting this.
It never ends. There's always one more idea to add in. One just has to draw the line and call it version I.
I'm not selling it. Maybe tip sheets but not the code or the app. I know this is a commercial website. Other developers feel free to add in your experiences.

[bigmack]

How much better can software get than the offerings by Massa, Schwartz & Platt? I don't know why people bother when products exist that are VERY affordable.

[guckers]

Thanks for the insights Capper Al,

I envy that you are this far along, even though you may think that you have much more to do.

I've been gathering 10 years of data and modeling the database over the past three months. I only work on it about 20 hours a week, so it feels like a slow pace but I should hopefully have my db complete and populated with the horses/races that I am interested within another three months. After that, I am just going to analyze races.

Can you tell me what your software does? Also, what language did you develop in?

[guckers]

How much better can software get than the offerings by Massa, Schwartz & Platt? I don't know why people bother when products exist that are VERY affordable.

I wouldn't say that they are very affordable, but its all relative and I would also say that software can always be better. I reviewed what was out there before choosing to develop my own software and still decided to develop because....

1. I'm a developer
2. I need to know intimately how the app functions
3. I will have control over the look/feel of the app

[Dave Schwartz]

G.,
Those who can roll their own usually are driven to. Although my largest single group of clients is professional software developers who don't want to reinvent the wheel.


Dave Schwartz

[Capper Al]

I wouldn't say that they are very affordable, but its all relative and I would also say that software can always be better. I reviewed what was out there before choosing to develop my own software and still decided to develop because....

1. I'm a developer
2. I need to know intimately how the app functions
3. I will have control over the look/feel of the app

You hit the nail on the head.

[Capper Al]

G.,
Those who can roll their own usually are driven to. Although my largest single group of clients is professional software developers who don't want to reinvent the wheel.


Dave Schwartz


Always a good point.

[Capper Al]

Thanks for the insights Capper Al,

I envy that you are this far along, even though you may think that you have much more to do.

I've been gathering 10 years of data and modeling the database over the past three months. I only work on it about 20 hours a week, so it feels like a slow pace but I should hopefully have my db complete and populated with the horses/races that I am interested within another three months. After that, I am just going to analyze races.

Can you tell me what your software does? Also, what language did you develop in?

I develop in C++ and use Excel for presentation. My system is derived from my old paper and pencil methods. I'm about 3 months from starting my database. Not enough time.

[Capper Al]

How much better can software get than the offerings by Massa, Schwartz & Platt? I don't know why people bother when products exist that are VERY affordable.

Got to code and handicap.

[sjk]

My experience is that you will reach a point where the risk of changes turning out to be counterproductive outweighs the potential benefit.

I have not used the commercially available products so it is hard to know what is better, but there is a value in being different from what others are using even if no better.

[Capper Al]

My experience is that you will reach a point where the risk of changes turning out to be counterproductive outweighs the potential benefit.

I have not used the commercially available products so it is hard to know what is better, but there is a value in being different from what others are using even if no better.

I'm pretty much there already. I just need to study the heck out of anything new before I make changes. Need to get that database going.

[ranchwest]

I develop in C++ and use Excel for presentation. My system is derived from my old paper and pencil methods. I'm about 3 months from starting my database. Not enough time.

IMHO, the database design is the most critical part of software development. If you mess it up, it is not nearly as easy as it might seem to correct the issue.

[Capper Al]

IMHO, the database design is the most critical part of software development. If you mess it up, it is not nearly as easy as it might seem to correct the issue.

Agree. I have developed large databases and data warehouses for Fortune 500 companies and the Federal government. Some take what I call the statistical approach toward building their system. They build the database first, query it, and build their applications from their query findings. A logical approach used in the commercial and academic world. This method the stats determine the game.

I have taken what I call a creative or handicapper's approach. My app is developed through hunches and paper and pencil methods from trial and error. This method the game determines the data. Both roads cross in the end. The statistical method ends up querying on hunches, and the handicapper's approach queries for statistical validity of their original hunches.

[cj]

IMHO, the database design is the most critical part of software development. If you mess it up, it is not nearly as easy as it might seem to correct the issue.

I always say, with today's storage capacities, include EVERYTHING.

[Fingal]

G.,
Those who can roll their own usually are driven to. Although my largest single group of clients is professional software developers who don't want to reinvent the wheel.


Dave Schwartz

True. The idea isn't to reinvent the wheel, only to expound on it.

[ranchwest]

Apparently I wasn't very clear about the perils of database design.

I suggest studying database normalization. Reading the works of CJ Date might help.

Improper database design can lead to issues such as difficulties in structuring and/or executing queries.

I wasn't referring to which data is being stored.

Also, take into consideration the eventual size of the database and use a database product that will accommodate a database of that size.

[Red Knave]

I suggest studying database normalization. Reading the works of CJ Date might help.
These rules are older and, I think, more accessible.

Rules here (http://en.wikipedia.org/wiki/Codd's_12_rules)

If you break a rule, make sure you know why.

[ranchwest]

These rules are older and, I think, more accessible.

Rules here (http://en.wikipedia.org/wiki/Codd's_12_rules)

If you break a rule, make sure you know why.
Excellent suggestion.

[Capper Al]

Apparently I wasn't very clear about the perils of database design.

I suggest studying database normalization. Reading the works of CJ Date might help.

Improper database design can lead to issues such as difficulties in structuring and/or executing queries.

I wasn't referring to which data is being stored.

Also, take into consideration the eventual size of the database and use a database product that will accommodate a database of that size.

OK, the technical part about building the database is important. What's more important is the data inside. More than not, if there is something wrong in the structure it can be corrected. But if the data is wrong, it is of no use.

[Dave Schwartz]

Knave,

I read that document last night. Wow, those are very difficult standards to live up to!


Dave

[2low]

How much better can software get than the offerings by Massa, Schwartz & Platt? I don't know why people bother when products exist that are VERY affordable.

The fun of doing it yourself from scratch. It was (is) a learning opportunity for me.

[ranchwest]

OK, the technical part about building the database is important. What's more important is the data inside. More than not, if there is something wrong in the structure it can be corrected. But if the data is wrong, it is of no use.

Yes, the data has to be useful to be useful. :)

However, if you've done a lot of program coding around a structure and then find the structure is not as useful as it should be, it can be about like painting a floor and backing yourself into a corner -- you may work your way out, but it may take a while.

[ranchwest]

Knave,

I read that document last night. Wow, those are very difficult standards to live up to!


Dave

That's why designing a database is more challenging than most people realize. There are no absolute rules, but there are some significant suggestions available.

[Dave Schwartz]

Ranch,

LOL - I have been designing and implementing database professionally for 30 years and I can tell you I have never seen a single database in all of that time that would meet all the points.


Dave

[ranchwest]

Ranch,

LOL - I have been designing and implementing database professionally for 30 years and I can tell you I have never seen a single database in all of that time that would meet all the points.


Dave

That's why I said there are no absolute rules.

[guckers]

Ranch,

LOL - I have been designing and implementing database professionally for 30 years and I can tell you I have never seen a single database in all of that time that would meet all the points.


Dave


The rules are too frigid, but the guidelines provide a basic direction the dba should take. The database needs to be built based on the needs of the system and there is often some ambiguity with that.

One thing is for certain, make sure you have your system well documented or conceived of in your head. DB structure changes and bugs in your code are a lot harder to fix further down the development cycle.

[Capper Al]

Ranch,

LOL - I have been designing and implementing database professionally for 30 years and I can tell you I have never seen a single database in all of that time that would meet all the points.


Dave

Agree. I too have been designing databases for over 30 years. No problem with the structure guidelines. It's still about the data first. What are we going to track in this perfectly designed database is the most important question.

[Capper Al]

My experience is that you will reach a point where the risk of changes turning out to be counterproductive outweighs the potential benefit.

I have not used the commercially available products so it is hard to know what is better, but there is a value in being different from what others are using even if no better.

What I like about the Internet is meeting up other people who share similar experiences. There probably isn't another person within a 150 miles of me that could have known these perils about programming handicapping software. The Internet brings people together.

[Dave Schwartz]

What are we going to track in this perfectly designed database is the most important question.

I am not sure that a "perfectly designed database" will fit the criteria laid out in the article.

In a truly "normalized" database, no piece of information is ever stored twice. That means (to a horse racing programmer) that there is a file for tracks, races, horses, jockeys, trainers, speed ratings, class ratings, etc.

Why speed/class ratings? Depending upon the source of the rating, it might change from the original rating on the day after the race based updates. I found this was one of the most difficult aspects.

The most difficult aspect is names of the participants. Jockey, trainer and even horse names change from time to time. At one point (when I maintained a normalized database) I had a jockey-trainer alias file that had over 50,000 aliases in it!


I have decided that my next database design will be like a spreadsheet. There will be a database for each day of racing (as our software has now). These days are then accumulated into a database (again, as our software does now).

The difference will be the ease with which I can rebuild that database with a finite number of computational factors. Maybe I should emphasize a finite but large number of factors.

Thus, if I want to rebuild my entire database, adding a dozen new columns, I could rebuild it from scratch in just a couple of hours.

[sjk]

What I like about the Internet is meeting up other people who share similar experiences. There probably isn't another person within a 150 miles of me that could have known these perils about programming handicapping software. The Internet brings people together.

I enjoy reading of your work as well. There don't seem to be many of us doing databases and I echo your thought that it is great to have a place to share experiences.

[raybo]

Yeah, I have tried several times to create a racing database and after reading several books on relational database design, SQL, etc., etc., I found that the "rules" for relational database design must be applied sporadically, because, for one thing, duplications are almost guaranteed.

The last one I tried, I created 5 tables, and that was as far as I got, the joining and all that stuff just completely overwhelmed me.

I like Dave's idea for a new approach using spreadsheets and dumping all the outputted data into a real database. Seems like that would certainly improve the updating of the database.

[Capper Al]

I'm glad I didn't rush into DB design. Fields evolve as we develop and test. Hopefully, after I completed testing most of these fields will be calculated the same ever after.

[Dave Schwartz]

I like Dave's idea for a new approach using spreadsheets and dumping all the outputted data into a real database.

Yes, but then you have to write that one too.

:)

[Handiman]

Dave quit responding to these guys and get on that book. I don't want to have to have my cousin in Reno whose a hooker come kick your butt! :lol:

She's only 4' 7" but she says she's pretty sure she can take you.

I want my Christmas present. :ThmbUp::ThmbUp:

Handi:)

[raybo]

Yes, but then you have to write that one too.

:)

I realize that. :mad:

[Dave Schwartz]

Book is finished, H.