Utah is set to become the first US state to allow AI systems to examine patients and prescribe medication without direct human oversight. The move marks a significant regulatory shift in how autonomous AI can operate in clinical care, raising questions about safety, liability, and the role of human clinicians.
SynthePulse Insight · AI deep reading
Version 1 · 1 source
Utah has launched a one-year pilot allowing the AI of healthcare startup Nolla Health to examine and prescribe for patients with mild to moderate acne without direct human supervision, becoming the first state to replace doctors with AI. But the project's scope, drug types, and doctor review ratios are all strictly limited, and long-term safety and efficacy data are still lacking.
Utah will become the first state to allow AI to examine patients and prescribe medications without direct human supervision. This pilot is launched by healthcare startup Nolla Health, and patients must subscribe to its app at $4.99 per month. The project is currently only one year long and is limited to patients with mild to moderate acne.
In terms of the process, after users scan their face with the app and fill in their medical history and personal information, the AI analyzes the skin, identifies acne problems, assesses the degree of skin oiliness, and generates a numerical severity score ranging from clear skin to severe acne. The AI then combines the score with the medical information provided to recommend drugs from eight approved treatment options.
It is worth noting that although the project will eventually be fully automated, prescribing authority is being rolled out in phases: the first 100 prescriptions must still be reviewed by a real doctor before being submitted to a pharmacy; in the second phase, prescriptions for the next 400 patients will be submitted directly to pharmacies by the AI, with doctors conducting weekly retrospective reviews; in the final phase, doctors will review at least 10% of the total prescriptions issued each month.
Limiting the scope to mild to moderate acne, the duration to one year, and the drugs to topical rather than oral medications—these constraints together form the safety boundary of the pilot. Nolla Health CEO Luis Wenus says the adverse effects of these eight approved drugs are limited to local skin irritation or dryness; the project's prescriptions are also limited to topical medications rather than oral pills.
It should be noted that the above two points are company claims, not independently verified conclusions. The report itself does not explain the specific technical details or accuracy of the AI system, nor does it provide data on the long-term safety and efficacy of AI prescribing.
The design of the doctor review ratio is also noteworthy: from reviewing every prescription, to weekly retrospective reviews, to reviewing at least 10% each month, the intensity of review decreases with each phase. Whether retrospective reviews can promptly catch AI prescribing errors is a question the report does not answer.
While introducing the pilot, the report also mentions previous risk cases involving AI health advice: last year, a 60-year-old man followed ChatGPT's suggestion to replace salt in his diet with sodium bromide, leading to bromism, a psychiatric disorder that has been rare since the 19th century.
Another case is Google quietly removing its AI search feature What People Suggest, which provided health advice for patients with similar conditions, in March of this year; previously, the company had also removed some content because AI Overviews displayed inaccurate information that could endanger health.
These cases do not directly prove that the Nolla Health pilot has the same problems, but they explain why a very small-scale acne pilot is being treated as a sensitive event: it touches on the existing controversy over the reliability of AI in medical advice.
Judging from the project's design of expanding AI prescribing authority in phases and gradually reducing the doctor review ratio, this may be the first step toward AI replacing doctors, or at least being used to treat mild cases. This judgment is an inference, not a fact confirmed by the report.
What supports this inference is the direction of the authority design: the AI moves from being checked prescription by prescription by doctors to submitting prescriptions directly to pharmacies, with the doctor's role reduced to sample-based review. But the evidence against overinterpretation is equally clear: the pilot is only one year, limited to mild to moderate acne, limited to topical medications, and the final phase still retains at least 10% doctor review each month.
The report itself also describes the project as "small in scale" and says it may become the first step toward AI replacing doctors or at least being used to treat milder conditions—this too is a speculative statement.
Whether the pilot can truly achieve full automation remains unclear. There is a lack of data on the long-term safety and efficacy of AI prescribing, and whether the adverse effects of the eight approved drugs are limited to local irritation or dryness is only the CEO's claim.
Whether the project will expand to other conditions or other states is unknown; the report does not explain the specific technical details or accuracy of the AI system; and whether retrospective doctor reviews can promptly catch AI prescribing errors is also uncertain.
Until these questions are answered, the more prudent reading is this: it is a clearly bounded regulatory experiment, not proof that AI can already practice medicine independently.
This article is based on a TechSpot report (included via the Hacker News AI filter feed). The report is secondhand, and its descriptions of the Nolla Health pilot process, phased prescribing authority, subscription price, applicable conditions, and number of drugs are marked as confirmed facts in the evidence pack; while the two points that "the adverse effects of the eight approved drugs are limited to local skin irritation or dryness" and "prescriptions are limited to topical medications" come from statements by Nolla Health CEO Luis Wenus and are company claims, not independently verified. The ChatGPT bromism case and Google's removal of the What People Suggest feature mentioned in the report are confirmed narratives in the report, but a causal relationship with the Nolla Health pilot has not been proven. The judgment about "the first step toward AI replacing doctors" is an inference and should not be regarded as an established fact.
The real significance of Utah's pilot is not that AI can already practice medicine independently, but that regulators have for the first time allowed AI to prescribe directly within clearly defined boundaries: a one-year term, mild to moderate acne, topical medications, and a doctor review ratio gradually reduced from 100% to 10% per month. It is a controlled experiment, and long-term safety and scalability remain unresolved questions.
Primary report
Primary source