Below Analyst Expectations?
When quarterly results for public corporations are announced, the impact of the reported earnings per share, revenue growth and other such key financial summaries on movement in share prices depend more on how these compare to analyst expectations rather than absolute results. One often sees results which appear to be quite Impressive and yet there is a drop in stock prices when results are released. The market had moved to a larger price level based on some kind of a consensus analyst valuation, leading to a correction, as the reported results, though impressive, were below these expectations. A similar expectation can be derived for planned studies.
Planned studies are sized based on limits to the false positive rate, typically a two-sided 5%, and the power or the ability to detect aggregate effect, which usually varies from 80 to 90%. Power is the chance of seeing a reported p-value less than 0.05, indicative of a statistically significant effect. So with 50% power one would tend to hit 0.05 and be lower or higher 50% of the time – an expected p-value of 0.05. With 80% or higher power we would expect to hit a much lower value as we now could have p-values greater than 0.05 only about 20% of the time with smaller values 80% of the time – an expected p-value much lower than 0.05. This expectation at the planning of a study can be used to roughly gauge a study’s eventual aggregate estimated effect relative to the anticipated effect. Additional assessments of aggregate and individual effect can be made for comparative trials using the calculator at this page. The expectation computed is valid for group sequential trials with interim analyses with futility assessments and no stopping for efficacy, and can be used to gauge expected effect in trials allowing for early termination for effect if the trial does not terminate early. For reported trails which were terminated earlier one can instead gauge aggregate and individual effect at the same page noted above.
Expected P-Value Calculator
A brief note on the computation is in this document. The calculator below has, as a default, the expected p-value for a study planned to provide 85% power to detect aggregate effect using a two sided test at a 5% significance level. The expected p-value of 0.00273 is less than a tenth of what is adequate to claim a statistically significant aggregate signal.
Edit the blue cells in the spreadsheet and enter your data and the calculations in the bottom box of the spreadsheet will refresh.