Predictive clinical decision support (CDS) systems are increasingly used to guide care, yet patients underrepresented in training data face representational harms when these tools generate overconfident outputs despite insufficient evidence. Current mitigation efforts rely on pre-deployment validation metrics such as discrimination and calibration. While necessary, these static measures are limited by their inability to remediate persisting representational debt post-deployment. Individual clinical encounters present a critical mechanism to do so in real time – a missed opportunity among current approaches to improve data equity and enhance patient trust in the process.

I propose a governance framework that shifts from static bias mitigation toward dynamic, patient-centered intervention during clinical encounters. The framework integrates uncertainty quantification methods, including conformal prediction, to identify patients for whom model predictions are unreliable due to data sparsity. Rather than producing a potentially harmful overconfident output, uncertainty serves as a signal: the CDS flags the encounter, prompting the provider to initiate an informed consent protocol. This protocol communicates the limitation to the patient and empowers them to voluntarily contribute their data.

The framework addresses three interconnected challenges: detecting representational uncertainty in real time, establishing an ethical mechanism for data contribution that centers patient autonomy, and creating a self-improving system that progressively reduces bias through participatory engagement. By transforming algorithmic uncertainty into opportunities for transparency and shared decision-making, this approach reframes the relationship between patients, providers, and predictive systems – building trust in clinical AI not through opaque technical fixes, but through governance that makes patients active partners in improving the systems that serve them.