I not only agree but suspect the team leads are well aware of this.
I think they are able to work around this by employing multiple observers who collectively make a lot of observations over the course of a meet.
If I were modeling horse physicality myself
I would create three columns in a database table:
phys01 value=ObserverId (a number unique for each observer.)
phys02 value=0 for horses where no physicality observation was made. Value=1 for horses where a physicality observation was made.
phys03 value=0 for horses where no physicality observation was made. Value=Some_Number (a physicality score created by the observer making the observation.)
From there, once you've gathered observations from multiple observers for a meet or two, it shouldn't be difficult to perform significance testing by running the observations data through a stat package.
If the observations of an individual observer aren't statistically significant after a couple of meets, or have far less significance than those of the other observers:
You'll know. At which point you can decide to cut that observer loose.
But if the observations of an individual observer are statistically significant or consistently better than those of the other observers:
You'll know that too. At which point you can implement that observer's work in your model. And possibly make that observer an offer he or she isn't likely to refuse.
Agree here as well.
But if you do find an observer who provides statistically significant insight:
Now you have information not readily available to others.
From there, you should be able give your model a leg up over the competition.
-jp
.
