Federated Conformal Prediction: Marginal and Training-Conditional Validity

In this talk, I present the objective of conformal prediction and overview some properties of the well-studied split estimator. Then, I introduce a quantile-of-quantiles estimator that allows constructing prediction sets in a one-shot federated learning setting. We investigate the properties of this estimator and how it can be used to build confidence sets with probabilistic coverage being marginally or training-conditionally valid. Over a set of experiments, we empirically verify the quality of our results and show that it is possible to output prediction sets with desired coverage, in only one round of communication, while recovering performances close to the centralized split method.