New report finds AI scribe industry operating with no meaningful oversight. Patients locked out of care for refusing AI scribes.
A Melbourne psychiatrist has been refusing new patients who decline allowing an AI scribe tool to record and transcribe their sessions, a practice a new report from Digital Rights Watch finds is symptomatic of an industry operating in a regulatory vacuum.
Digital Rights Watch’s new report, “OFF THE RECORD: AI scribes in Australian healthcare”, finds this is not an isolated case. According to the Royal Australian College of General Practitioners, nearly half of all GPs now use AI scribes, with uptake doubling in the last year. Yet the laws and regulatory frameworks that should govern the use of these tools are not keeping pace with the uptake, leaving patients suffering privacy, accuracy, and consent problems.
We read clinics’ privacy policies. Of the 60 policies we reviewed, only 4 make any mention of AI scribes. Of the 7 AI scribe companies contacted by Digital Rights Watch, only two responded to our questions about their data practices, and none were willing to disclose the underlying model their product uses.
Our report raises serious concerns about accuracy and bias. AI scribes have demonstrated lower transcription accuracy for patients with non-English accents, speech differences, or from minority backgrounds. AI hallucinations can result in incorrect diagnoses, wrong medication decisions, or harmful content in medical records.
Digital Rights Watch is calling for urgent action across multiple fronts: the TGA should register and regulate AI scribes in healthcare; the OAIC should issue and enforce binding guidance on consent-gathering, and the Attorney-General should prioritise Privacy Act reforms, including mandatory privacy impact assessments before AI tools are introduced in clinical settings, as well as a patients’ right to refuse AI scribes with no impact on their care.
Quotes attributable to Tom Sulston, Head of Policy for Digital Rights Watch:
“We all need healthcare sometimes. We shouldn’t be forced to provide AI corporations with our most private information in order to get it. Everyone in Australia should be able to refuse the use of an AI scribe when they access healthcare.“
“AI corporations operate in their own interests: not clinics’, not doctors’, and certainly not ours. Regulators need to work together to make AI scribes work for regular people. That means treating them as medical devices, increasing transparency and reporting requirements, and demonstrating that they work for all Australians equally.“
Quote attributable to Ed Santow, Director of the Human Technology Institute and former Human Rights Commissioner
“This report shines a light on the use of AI scribes by doctors and other clinicians. Digital Rights Watch makes clear that any benefits that clinicians experience from the use of AI must not come at the expense of the standard of care, or right to privacy, of patients.
In recent years, the growth of AI scribes in health care has increased almost exponentially - with very limited regulatory oversight to protect patients and their personal information. There is a clear need for stronger oversight of this practice, and clearer rules to ensure that AI scribes may be used only with the free and informed consent of patients, and only in ways that uphold the community’s most basic health care and other rights.”
Quote from a patient who was not given the chance to refuse consent for an AI scribe:
“When I shared my story with Digital Rights Watch I honestly didn’t think it was that big a deal, just yet another frustrating thing I came across while trying to access healthcare. As it turns out this has generated a huge amount of discussion and it’s made me realise how serious it really was. I see it as part of a bigger picture - ‘forced inevitability.’ The plan is clearly to slowly normalise this technology across society until it is impossible to opt out of, and it’s up to us to stop this happening.”
Quote attributable to Sophie Dutertre, Coordinator, Projects & Training at The Centre for Culture, Equity & Health
“We need to ensure that the use of AI scribes in health settings does not further disadvantage patients of culturally and linguistically diverse backgrounds, who speak with an accent or who use an interpreter. It is an equity issue."