AI in the Gray Zone: Ethics of End-of-Life Resource Allocation in Language Models

Key Message: This project explores how AI language models make end of life ventilator allocation decisions, revealing ethical inconsistencies. Comparing outcomes across various models, we highlight the value of AI in assisting clinicians with ethically complex decisions, while emphasizing the need for human oversight and alignment with palliative care principles.

Abstract:

Introduction: Allocation of scarce medical resources during crises such as pandemics, mass casualty events, and natural disasters presents profound ethical challenges. Traditional triage frameworks rely on clinical judgment and standardized scoring systems, these approaches remain susceptible to bias, variability, and emotional burden (1,2,3). Artificial intelligence, particularly large language models, offers potential to enhance decision-making by processing complex data and applying consistent logic. However, their use in ethically complex end-of-life scenarios raises valid concerns.

Methods: This study assessed how four large language models, GPT-4o, Llama4, Claude Sonnet 3.7, and Gemini 2.0 Flash, responded to a simulated ventilator scarcity scenario involving nine diverse mock patients. The cohort included individuals with varying terminal illnesses, advanced age, pregnancy, and caregiving roles. Each model was tasked with selecting three patients for ventilator allocation and providing ethical justification for its choices. Responses were analyzed for alignment with established triage protocols, ethical reasoning, and relevance to palliative care principles.

Results: The artificial intelligence models applied diverse ethical frameworks to their decisions. All of the models emphasized likelihood of survival. GPT-4o and Gemini additionally prioritized life-years and social value. Claude focused primarily on SOFA scores. Llama4 highlighted short-term benefit without deeper ethical reflection. None of the models consistently addressed long-term suffering, cognitive decline, or the patient-centered values foundational to palliative care. The same input scenario resulted in a wide array of allocation outcomes across models.

Discussion and

Conclusion: These findings reveal both the potential and limitations of artificial intelligence in triage education and decision support. While large language models can simulate ethical reasoning and offer structured frameworks for teaching clinical ethics, their inconsistencies emphasize the need for human oversight. This study highlights an opportunity to incorporate artificial intelligence into palliative education as well as advocating for ethically grounded model development aligned with the principles of palliative care.

References: 1. Vergano M, Bertolini G, Giannini A, Gristina GR, Livigni S, Mistraletti G, et al. SIAARTI recommendations for the allocation of intensive care treatments in exceptional, resource-limited circumstances. Minerva Anestesiol 2020;86:469-72. DOI: 10.23736/S0375-9393.20.14619-4

2. Vincent JL, Moreno R, Takala J, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22(7):707-710. doi:10.1007/BF01709751

3. Ehni HJ, Wiesing U, Ranisch R. Saving the most lives-A comparison of European triage guidelines in the context of the COVID-19 pandemic. Bioethics. 2021 Feb;35(2):125-134. doi: 10.1111/bioe.12836. Epub 2020 Dec 16. PMID: 33325536.