859
Who's going to tell me how to eat though? (sh.itjust.works)
submitted 1 year ago by to c/whitepeopletwitter@sh.itjust.works
87 comments fedilink hide all child comments
 
you are viewing a single comment's thread
view the rest of the comments
[–] 10 points 1 year ago (12 children)

It's not just far. LLMs inherently make stuff up (aka hallucinate). There is no cure for that.

There are some (non llm, but neural network) tools that can be somewhat useful, but a real doctor needs to do the job anyway because all of them have various chances to be wrong.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 12 child comments
    •
  • [–] -2 points 1 year ago (11 children)

    Not only there's a cure, it's already available: most models right now provide sources for their claims. Of course this requires the user the gargantuan effort of clicking on a link, so most don't and complain instead.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 11 child comments
    •
  • [–] 5 points 1 year ago (10 children)

    This is stupid. Fully reading and analyzing the source for accuracy and relevancy can be extremely time consuming. That's why physicians have databases like UpToDate and Dynamed that have expert (ie physician and PhD) analyses and summaries of the studies in the relevant articles.

    I'm a 4th year medical student and I have literally never used an LLM. If I don't know something, I look it up in a reliable resource and a huge part of my education is knowing what I need to look up. An LLM can't do that for me.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 10 child comments
    •
  • [–] -3 points 1 year ago (9 children)

    And why are you assuming that a model that is designed to be used by physicians would not include the very same analysis from experts that goes into UpToDate or Dynamed? This is something that is absolutely trivial to do, the only thing stopping it is copyright.

    AI can not only lookup reliable sources, it will probably be much better and faster than you or I or anybody.

    I'm a 4th year medical student and I have literally never used an LLM

    It was clear enough from your post, but thanks for confirming. Perhaps you should give it a try so you can understand it's limitations and strengths first-hand, no? Grab one the several generic LLMs available and ask something like:

    Can you provide me with a small summary of the most up to date guidelines for the management of fibrodysplasia ossificans progressiva? Please be sure to include references, and only consider sources that are credible, reputable and peer reviewed whenever possible.

    Let me know how it did. And note that it probably is a general purpose model and trained on very generic data, and not at all optimized for this usage, but it's impossible to dismiss the capabilities here...

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 9 child comments
    •
  • [–] 4 points 1 year ago (5 children)

    It's called RAG, and it's the only "right" way to get any accurate information out of an LLM. And even it is not perfect. Not by far.

    You can use it without an LLM. It's basically keyword search. You still have to know what you are asking, so you have to study. Study without an imprecise LLM that can feed you false information that sounds plausible.

    There are other problems with current LLMs that make them problematic. Sure you will catch onto those problems if you use them, and you still have to know more about the topic then them.

    They are a fun toy and ok for low-stakes knowledge (ex cooking recipies). But as a tool in serious work they are a rubber ducky at best.

    PS What the guy couple comments above said about sources, that's probably about web search. Even when an LLM reads the sources it can missinterpet them easily. Like how apple removed their summaries because they were often just wrong.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] -3 points 1 year ago (4 children)

    Let's not move the goal post. OP post is about med students using GPT to pass their exam in a successful manner. As another comment put it, it's not about Karen using GPT to diagnose pops, it's about trained professionals using an AI tool to assist them.

    And yet, all we get is a bunch of people spewing vague FUD and spitballing opinions as if they're proven facts, or as if AI has stopped evolving and the current limitations are never going to be surpassed.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 4 points 1 year ago (1 child)

    The current limitations of LLMs are built in how they fundementaly work. We would need something completely new. That is a fact.

    Honestly the thought of med students using them to pass exams scares me.

    Sure, use them to replace CEOs of some unimportant companies like facebook. But they are not for jobs where other peoples lives are at stake. They inherently halucinate (like many CEOs). It is built in in how they work.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 0 points 1 year ago

    I don't think the bar will be where you're setting it.

    Suppose a new cancer drug or something comes out that significantly improves the life expectancy and quality of patients. In rare cases however, it can cause serious liver complications that may be fatal. Should this drug be used, or not?

    It's not trivial, but there's a chance that it would in fact be used.

    My point with AI hallucinations is that they're the same. If at some point it's proven that it leads to better patient outcomes, but can have side effects, should it be outright discarded?

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 1 year ago (2 children)

    Some of my classmates used chatGPT to summarize reading assignments and it garbled the information so badly that they got things wrong on in-class assessments. Aside from the hallucinations and jumbled garbage output, I refuse to use AI unless there is absolutely no alternative on an ethical basis due to the environmental and societal impacts.

    As far as I'm concerned, the only role for LLMs in medicine is to function as a scribe to reduce the burden of documentation and that's only because everything the idiot machines vomit up has to be checked before being committed to the medical record anyways. Generative AI is a scourge on society and an absolute menace in medicine.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 1 year ago (1 child)

    Except that your opinion, and mine, are irrelevant.

    New AI models, and new studies that evaluate them, will continue to be produced. If and when those studies show that using AI leads to better outcomes, then that will become the gold standard. That's just how evidence based medicine works.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •