I know a bot posted this but can we keep Lemmy clickbait free please? Just downvote all the low effort clickbait titles?

Edit: to save you a click… The AI transcriber hallucinated that she was micro-dosing magic mushrooms and the doctor obviously didn’t bother to read the transcript. Post kidney surgery bleeding was blamed on her ‘drug use’

As far as I'm concerned, using AI of any sort inside of a doctor's office is a grave violation of HIPAA

TL;DR: AI transcript said I’d been a smoker since the age of 11, never smoked a day in my life.

My doctor asked me if she could use AI transcription software because she had hand surgery and couldn’t type. I said yes and reviewed the statements and while it was fairly accurate, it did have one egregious error on it.

When asked if I had ever smoked, in person, I replied with a simple “no,” and did not proceed to have any follow up conversation until her next question. The AI transcript for mysterious reasons decided my answer was “yes, since the age of 11.” 💀💀💀

Had to message them to fix it and while they did fix it asap, my first instinct wasn’t to check the transcript and I only did so out of curiosity. I can’t imagine what other people’s experiences may be.

Article:

Error by AI scribe during medical appointment leaves patient devastated

By the Specialist Reporting Team's Paige Cockburn

• Topic:AI

Posted 13 Aug 2026 at 2:55pm13 August 2026 at 2:55pm, updated 13 Aug 2026 at 4:04pm13 August 2026 at 4:04pm

An error in Rebecca Green's medical records falsely claimed she had taken psychedelic mushrooms. (ABC News: Madeline Grace)

When Rebecca Green arrived for her first appointment with a urologist, she was asked whether she consented to having the appointment transcribed by artificial intelligence (AI).

She said yes without hesitation.

"I don't want to be in her bad books or be that annoying patient who takes up more of her time," she said.

But that decision would cause her a lot of stress, with the AI scribe making up the serious but false claim that Ms Green was taking illegal drugs.

"I was gobsmacked and in tears … it made up that I take [psychedelic] mushrooms. I have never done mushrooms in my life,"

she said.

More GPs and hospitals are using AI scribes. What are the risks?

AI tools are becoming more prevalent in healthcare, helping overworked doctors to carry the workload. But it has come with concerns.

New research by a privacy watchdog has raised serious concerns about AI being used in medical settings due to its propensity to "hallucinate" false symptoms or conditions, and warned patients were at risk.

Ms Green only picked up the mistake after her kidney stone surgery in March, when she read the post-operative letter her specialist sent to her GP.

It stated Ms Green micro-dosed mushrooms and that could be the reason for prior bleeding around the kidneys.

"For this letter to say something that wasn't based in any evidence but could potentially have huge consequences for me … it was quite scary," she said.

"It felt very impersonal to me. I wasn't getting the level of care and familiarity from the doctor that I would be hoping for in such a vulnerable situation."

Rebecca Green discovered the error on her medical record after reading a post-operative letter from her specialist.  (ABC News: Madeline Grace)

Ms Green was particularly stressed as she was receiving workers' compensation and felt any mention of illegal drugs on her medical record could impact her case.

Australia's peak regulator of medical practitioners, AHPRA, said clinicians must always check all output from an AI scribe for accuracy to ensure they meet their professional obligations. Ms Green said it was clear her doctor did not do this.

"If you had checked that letter you would have been able to correct that before it was sent out … why aren't you in tune with each of your patients and what's going on?"

What to know before saying yes to your doctor using AI

Health practitioners are using generative AI to help them record and summarise patient information, but some patients are saying no.

After Ms Green made a complaint, the urologist sent a letter apologising.

"Please be assured that we take the accuracy of our documentation very seriously, and we are truly sorry that this occurred," the letter stated.

The doctor said she had not been able to determine how the claim about mushrooms came to be included, but it appeared to be from an error during the "dictation or transcription process".

The urologist corrected the correspondence and in her letter said she would review how AI was used in her practice.

The doctor did not provide any responses to the ABC's questions about what changes, if any, have been made since.

AI confuses which breast has cancer

Ms Green feels lucky she spotted the error, but is worried some people might not be aware of mistakes on their medical record, which could have repercussions for their health, their career or their insurance.

This week the ABC revealed instances where private health insurers were asking providers for their patients' full clinical history as part of "aggressive" auditing practices.

AI scribes have become prevalent within the healthcare sector. The Royal Australian College of GPs (RACGP) last year estimated that 40 per cent of GPs were regularly using them, but that number was considered conservative.

The peak body for GPs in Australia has estimated that 40 per cent are regularly using AI scribes at work. (ABC News: Tim Swanston)

Research shows having AI transcribe an appointment can reduce the administrative burden for GPs and allow them to focus more on their patient, rather than the computer.

But for every benefit, there are risks.

A recent report by advocacy group Digital Rights Watch (DRW) found Australians were being exposed to unregulated AI medical scribes, sometimes without their consent, and errors had the potential to change clinical decision-making.

Do you have a story to share about AI in health care?

Email Paige Cockburn at cockburn.paige@abc.net.au

Examples of mistakes found in the DRW report included a case where a scribe recorded the wrong breast in a diagnosis of breast cancer and said someone had epilepsy, when they did not.

Chief executive of the Consumers Health Forum of Australia Elizabeth Deveny said she often heard complaints of AI mixing up words and patients only finding out by chance.

"People report both frustration in finding the original source of the error and then getting the record changed," she said.

Elizabeth Deveny says it is frustrating for patients who have to get their records fixed when errors are made. (ABC News: Darryl Torpy)

Dr Deveny, who is currently involved in a major research project on AI medical scribes, said doctors may be too trusting of AI.

"There's evidence that for the first few weeks of using AI, [people] check things pretty carefully and then at a point they stop checking because they assume it will be right."

Perth GP Sean Stevens, who has been using a scribe for three years, said it was incumbent on the doctor to check every transcript, but believed the technology was becoming more accurate.

"Three years ago, I would have to correct multiple things on each transcription. These days it's like one correction every third or fourth transcription."

Sean Stevens says AI transcriptions are getting more accurate, but doctors still need to check them. (Supplied)

Dr Stevens is a big advocate of tech; he is the RACGP's Digital Health and Innovation chair as well as an advisor for one of the biggest AI scribes, but he admits AI has its weaknesses and can produce incorrect results that sound plausible.

"The classic is, it will get the side of the body incorrect and say right when you've said left … so you need to check things pretty closely, especially things like drug doses,"

he said.

He said using AI did not mean he could squeeze more patients in, but it allowed him to run on time more often and produce more thorough referrals.

Consenting to AI's involvement

The DRW found huge discrepancies in how consent is being obtained, with some doctors only putting a sign in the waiting room, which does not meet privacy law requirements.

Other times consent must be given when the appointment is booked online, and alarmingly some patients are even being refused appointments if they do not consent.

Dr Deveny said consent must be enthusiastic and fully informed, and patients should feel comfortable asking questions, even if it feels intimidating.

What the Consumer Health Forum recommends asking at your next medical appointment:

• Are you using AI today and if so, what are you using it for?
• Will the doctor review everything before it goes into my record?
• Will I be able to get a copy of the notes if I want them?
• What's the process if I think something's wrong?

The DRW report also identified that people might not be forthcoming about sensitive issues, like family violence, when they know AI is listening.

Dr Stevens said that was why he asked for consent in every appointment, even with regular patients.

"It's very important patients disclose any and all things that are relevant to their health … and that they know they can say no [to AI] … as a doctor you have to respect that," he said.

TGA flags breaches will be dealt with

There are around a dozen AI medical scribes available in Australia, but none are approved by the regulator — the Therapeutic Goods Administration (TGA) — because if the product claims to be solely for transcription and summarising, it is exempt from oversight.

The TGA only considers tools that do things like suggesting a diagnosis or treatment plan as a "medical device".

AI changing the way GPs tend to their patients

Some GPs are now relying on artificial intelligence to take notes during patient consultations. So, how safe is our sensitive information?

Currently, no AI scribes have gone through the regulation process, despite some claiming to offer clinical suggestions.

The TGA has acknowledged this means some scribes were potentially in breach of the law.

The TGA has spent the last year reviewing AI scribes and says its findings will be released in the coming weeks, and any breaches will be acted upon.

Chief executive of Patients Australia Lisa Robins believed all scribes should be regulated, as summarising a patient still involved decision-making by AI.

Dr Stevens had concerns that extending regulation would only stifle future innovation.

The DRW is calling on the newly established AI Safety Institute to test AI scribes and publish a list of approved tools.

This would be similar to New Zealand's approach, where the national AI advisory group reviewed tools and last year gave approval for clinicians to use two AI tools: Heidi Health and iMed.

Ms Deveny said ensuring doctors were using AI safely could also be enforced in other ways, such as compulsory checks and balances built into the software, or during practice accreditation, which happens every three years.

The error on her medical record has left Rebecca Green reconsidering if she would give consent to AI transcription again.  (ABC News: Madeline Grace)

Ms Green said her trust in AI has been splintered by what happened and now would only give consent for AI if she had an established relationship with the doctor.

"I love AI … but the essential ingredient is the human. At the end of the day what's happened here is more about the lack of human involvement,"

she said.

Australia's peak regulator of medical practitioners, AHPRA, said clinicians must always check all output from an AI scribe for accuracy to ensure they meet their professional obligations.

Show me one single AI user that checks the output of their AI, and I will show you my unicorn

The patient was devastated? You should have seen how many times the AI apologized.

Wow. They are streamlining US "healthcare" by putting the AI claim-denier 5000 INSIDE the exam rooms.

Just say no.

We use a software to help us take notes is that okay? FUCK NO.

Please just bring back traditional voice to text, with its dumb but decipherable errors instead of these plausible left field LLM "hallucinations".

I'd rather the computer write that I'm coming in for a "score throat and fever" than the sort of random ass shit LLMs can throw in there.

I read "medieval" appointment and was thoroughly confused, and intrigued.

I've used AI for medical transcription before and it is absolutely incredible. It's not just a word for word text, but a detailed clinical breakdown of everything discussed.

All you have to do is simply read the damn thing thoroughly and correct whatever minor errors pop up.

Takes 95% less time.

Luckily, I know something that would probably help her get over the trauma...

I'm looking at her and telling you she's done shrooms in her college days.

"I was gobsmacked and in tears … it made up that I take [psychedelic] mushrooms. I have never done mushrooms in my life,"

bruh

It stated Ms Green micro-dosed mushrooms and that could be the reason for prior bleeding around the kidneys.

I want to know if the AI also hallucinated the connection between psilocybin mushrooms and kidney bleeding (!), or if this was hallucinated by the human doctor.

I wonder how many will be killed by AI Slop... and if we'll ever be able to quantify it.

On some sidenote news, oracle pushing AI into the health sector.

"Please be assured that we take the accuracy of our documentation very seriously, and we are truly sorry that this occurred," the letter stated.

Yeah. That's why you do AI: because you take your patients' documentation really seriously...

Dumbest part about this is that auditory dictation software has existed for DECADES and WORKS. It's even based on machiblne learning models, just ones that have been trained for that specific purpose, not this LLM human-impersonating trash.

I love how the article just glosses over the fact that this woman reported doctors were treating her poorly because her notes mentioned she'd consumed psychoactive substances. Like apparently that's just an acceptable and a valid way to treat patients.

I know healthcare workers get compassion fatigue but it's just fucking depressing how normalised shitty behaviour is from them towards the vulnerable people rekying on them.

This type of hallucination is a problem, but is not my biggest concern.

I just want to converse with a human who is listening to me. It's a bonus if that person has some familiarity with me but it's not essential.

I certainly do not want an LLM inferring what might be wrong with me.

I feel like this shouldn't be too much to ask, but it's becoming that way.

Clinicians should be well educated on how this technology works but they aren’t. They’re given basic legal liability nagging (which, tbf, absolutely says to review the output). For my own doctors almost all of them use this and I refuse consent every time they can’t answer very basic questions: are recordings encrypted end to end, or at all? Are recordings destroyed after generation, or at all? Are the recordings processed by the emr vendor (typically epic or oracle) or a third party? Who is the third party? Are generated notes used for reinforcement learning and/or model refinement? Etc. so then it’s quite easy to say no; I will not consent to a technology that you don’t understand the basic security of because you don’t “do computers”

The penalties for violating this should be severe and supervision is even more necessary. I know for a fact when you use these tools you must attest that you reviewed the output for accuracy with every single note submission, because the people selling the models don’t want to touch that liability at all. What does this matter though if enforcement never occurs? The government is severely lacking at managing this kind of behavior. The only hope is insurers going at it, not out of altruism but because it gives them a reason to revoke payment for services

If only someone could've seen this coming!!! /$

I've seen enough errors made by humans that I always check my record after an appointment.

She wonders why she isn't getting the familiarity she wants from her doctor, and feels that the dr didn't review the notes well enough. Well that's probably because the doctor sees dozens of patients a week and doesn't have time / isn't paid to do notes. Medical errors aren't new, they're very common. Picking a few cases of AI errors is meaningless - stats on error levels with and without AI would be more useful.