The Use of AI in Health Care
Artificial intelligence (AI) refers to computer systems designed to perform tasks that typically require human intelligence, such as recognizing patterns, reasoning, and making decisions. AI is already widely used in everyday life and is rapidly expanding across the health care system, where providers, health plans, and patients increasingly rely on these technologies for clinical, administrative, and consumer health applications. Providers use AI for diagnostic support, risk prediction, clinical documentation, scheduling, and workflow management, while insurers and pharmacy benefit managers increasingly use AI for utilization management, including prior authorization and claims review. Although these technologies can improve efficiency, support earlier detection of disease, and enhance care, their growing use also raises concerns about transparency, equitable access, and the adequacy of existing oversight. For example, recent litigation has alleged that an insurer relied on a flawed AI system for coverage decisions, with reports indicating that more than 90 percent of denials generated by the tool were ultimately reversed. Such cases raise questions about over-reliance on automated decision-making, the consideration of individual patient circumstances, and the potential for inappropriate denials or delays in care.
Patients are also increasingly using AI-driven tools directly, including smartphone applications, mental health chatbots, and digital platforms that provide personalized health information or coaching. These tools may help patients monitor symptoms, better understand health conditions, and engage more actively in their care, but they can also provide incomplete, inaccurate, or overly generalized information, particularly when they lack clear clinical oversight or transparency requirements. As AI becomes more deeply integrated into health care and consumer health technologies, patient organizations and policymakers should consider how to support patients while ensuring that these tools remain reliable, transparent, equitable, and grounded in evidence-based care.
As the use of AI in health care continues to expand, policymakers must ensure that its adoption supports patient safety, strengthens clinical decision-making, and improves access to evidence-based care. The following considerations are important as policymakers seek to regulate the use of AI across the health care system.
- Transparency is critical to maintaining trust in AI-enabled health care. Patients should be informed when AI tools are used in clinical decision-making, care management, or insurance determinations, particularly when those tools may influence diagnoses, treatment options, or coverage outcomes. Providers must also have clear visibility into how AI systems are applied, including their underlying assumptions and limitations, so they can appropriately interpret and act on AI-generated outputs.
- AI should complement, not replace, clinical judgment. Providers must remain actively involved in reviewing AI-supported recommendations and retain ultimate responsibility for patient care decisions. Accountability frameworks are needed to ensure that responsibility for errors, delays, or adverse outcomes remains clear and does not shift to opaque or unexplainable systems.
- The use of AI in utilization management, including prior authorization, has significant implications for patient access to care. AI-driven tools may increase the risk of inappropriate denials or delays when they rely on incomplete data, generalized assumptions, or rigid decision criteria. Safeguards are needed to ensure that AI does not create additional barriers to timely, medically necessary treatment. Policymakers should require that adverse determinations are reviewed and finalized by qualified human decision-makers, and that patients have meaningful access to exceptions, appeals, and independent review.
- AI systems depend on large-scale data collection and analysis, making robust privacy protections essential. Patients should be provided clear information about how their data is used, including whether it is shared or reused for secondary purposes beyond direct care. Strong data governance and security practices are necessary to protect sensitive health information and uphold patient trust.
- AI systems can reinforce or worsen existing disparities if they are trained on biased or incomplete data. Policymakers must prioritize fairness by requiring ongoing evaluation for bias and ensuring that AI tools do not disproportionately harm underserved or historically marginalized populations. Promoting diverse data inputs and equitable system design is essential to achieving fair outcomes across patient populations.
- A coordinated regulatory approach is necessary to ensure the safe and effective use of AI in health care. Federal agencies play a central role in establishing baseline standards for safety, effectiveness, and privacy, while states may address insurance practices, utilization management, and patient protections. Greater alignment across federal and state frameworks will help reduce fragmentation and ensure consistent protections for patients nationwide.
