Artificial intelligence is reshaping how people live, work, and make decisions. It offers genuine opportunities to improve productivity, expand access to services, and support scientific and social progress. But those benefits are not automatic. The central question is whether AI will be developed and deployed primarily to maximize commercial returns, or whether it will be governed to serve public welfare under meaningful human oversight.
The recent Utah case illustrates the tension. In January 2026, the Utah Office of Artificial Intelligence Policy launched a pilot programme with the health-tech company Doctronic to automate renewals of refill or repeat medications for 190 chronic conditions, based on previous prescriptions by licensed physicians. The programme was designed to reduce clinicians’ workload and improve patients’ access to care, with Doctronic reportedly operating “within strict parameters and with physician oversight.” In April 2026, however, the Utah Medical Licensing Board called for its suspension, arguing that the pilot “compromises patient safety.” The board warned that AI or financial motivations should not override the obligation to protect patients.
This case does not show that AI has no place in healthcare. It shows that oversight must be substantive rather than nominal. Efficiency and access are important goals, but in high-stakes settings such as medicine, they must be balanced against safety, professional accountability, and patient trust. The Utah episode is a useful reminder that pilot programmes should include clear evaluation criteria, independent review, and mechanisms for rapid correction when risks emerge.
Industry has strong incentives to expand AI into healthcare and other sectors. At the same time, medical professionals, clinical practitioners, and researchers have called for a “human loop” across the AI life cycle—from problem formulation, data collection, storage, and management to model development, validation, deployment, and final decision-making. This is a reasonable safeguard, and it applies beyond healthcare. In the use of AI in the name of productivity and profit, people’s welfare should not be treated as secondary.
AI systems can also reproduce existing biases. Research has documented that language models and algorithms may reflect demographic, racial, gender, sexual, social, cultural, and class biases in data collection, classification, model development, and validation. When such biases are not addressed, AI-led processes can produce inaccurate diagnoses, flawed recommendations, and discriminatory outcomes. These risks can undermine public trust and weaken AI’s potential as a skill and as a knowledge tradition.
Automation can be valuable. It can reduce boring, tedious, and repetitive clerical work, freeing people for more meaningful tasks. But successful adoption depends on human oversight, developer accountability, and transparent governance. The goal should be democratic control: people and public institutions should set the rules, rather than simply adapting to technologies introduced in the name of efficiency.
A more balanced approach would prioritize transparent and accountable development, broad and equitable access, independent evaluation, and human review in high-stakes decisions. Commercial innovation can play a role, but profit should not be the primary criterion in domains such as health, law, education, and public administration. AI should serve people—not only the balance sheets of AI companies.
---
*Academic based in UK
Comments