How to tell if impact values are significant - answered

[podonne]

Hello,

I asked this question before, how to tell if an impact value was "significant", and it looked like the responces were based on personal experience, not statistics, so I thought I would do my own experiment and post the results.

I have a program that makes up impact factors and see's if they work, so I just recorded about 45,000 IVs and plotted them. I had to seperate the samples to >1 and <1 since the skews were so very different, but based on this I got the following:

If the IV is < 1, you can say with 95% confidence that an impact factor is "real" and not the result of random variations, if the IV is < 0.54.

If the IV is > 1, you can say with 95% confidence that an impact factor is "real" and not the result of random variations, if the IV is > 1.74.

Enjoy,
Phil

[Tom Barrister]

The skewing is different, partly because the reciprocals are not linear, compared to the number 1. In other words, the reciprocal of 2 is 0.5; the former is linearly twice as far from 1 as the latter. Also, skewing will generally occur on a greater scale on the plus side (>1) due to various factors.

While my statistics are a bit rusty, 0.54 and 1.74 seem to be a bit high for significance in a sample size of 45,000. I would eyeball the "correct" 95% significance level IV's to be about 0.85 and 1.20. Anybody else?

[podonne]

Thanks for your comment.

I got a standard deviation of 0.228087 on the downside and 0.371433
on the top side of 1.0

It sounds like your numbers are based on somewhere around one standard deviation, which would be like a 68% confidence level, if I remember my statistics correctly. (which is doubtful) :)