Read on Twitter
Interesting Q during discussion of @nilanjan10c's #HarvardPQG20 keynote this morning: how do we characterize uncertainty in individual disease risk estimates? /thread @bpasaniuc @amitvkhera
I think there are two separate kinds of uncertainty here: aleatory and epistemic uncertainty.
Aleatory uncertainty refers to inherent randomness: I can say a fair die will come up "one" 16.7% of the time, but I can't say for sure that the next roll will or will not come up "one."
Individual risk estimates themselves contain information about aleatory uncertainty. So do population summaries like AUCs.
Communicating and understanding risk--exactly what a 12% lifetime risk of breast cancer means--can be challenging. But do-able via graphics and analogy.
(These challenges are not restricted to medical decision making. Did I look up a risk model and do formal decision analysis when deciding whether to buy home title insurance? No, no I did not.)
Epistemic uncertainty refers to model uncertainty. Based on my training data, I may decide this die is a fair six-sided die. But if I had different training data, or modeled fair/not fair differently, I might have a different prediction of the probability of rolling a "one."
Communicating this kind of uncertainty is arguably harder and not typically done. It's one thing to take standard errors from a regression of disease incidence on polygenic risk to construct confidence intervals around individual risk estimates...
But how do you account for the fact that you could have (or did) constructed your PRS using 50 different methods, or regressed incidence on a cubic function of PRS instead of linear, etc?
There was a nice discussion of these issues in the context of election prediction models recently. (Too soon, I know) https://twitter.com/JessicaHullman/status/1303384797243486208?s=20
This is a long winded way of saying: I dunno how best to characterize uncertainty in individual disease risk estimates. Thoughts? /end
