Question
Do probabilities recorded before readout predict whether trials meet their primary endpoint better than the CE-1 base rate and an uninformative forecast of 50 percent?
Clinical Evidence Research · CE-2 · Protocol in preparation
Prospective Readout Forecasts
Can probabilities recorded before readout predict trial outcomes better than the base rate?
Studies of expert forecasts for clinical trials have found that averaged expert judgment often does not beat simple benchmarks. CE-2 measures whether a structured method does, with every estimate recorded before the outcome is known
Sample
Phase 2 and Phase 3 trials with a registered primary endpoint and an expected primary completion within twelve months of the protocol date. Inclusion rules are fixed before the first estimate.
Method
For each trial, two researchers record independent probabilities, followed by a combined estimate. Each carries a timestamp and is stored before the result is public. Outcomes are resolved by the CE-1 rules.
Scoring
Brier score per trial and in aggregate. Calibration and discrimination are reported separately. Comparison against the CE-1 base rate and against 50 percent.
Use
Resolved estimates enter Equity Research Scorecards for healthcare companies and Quantitative Research studies of readouts as market events.
Status
Protocol in preparation. Starts after CE-1 has published its reference classes.
References
Benjamin, D. M. et al. (2022). Principal investigators over-optimistically forecast scientific and operational outcomes for clinical trials. PLOS ONE 17(2), e0262862.
Atanasov, P. et al. (2020). Wisdom of the expert crowd prediction of response for 3 neurology randomized trials. Neurology 95(5), e488–e498.