133
AI Images Can Obscure the Reality of Factory Farms | Prompt revision steered AI-generated images towards idyllic scenes even when told to be realistic (sentientmedia.org)
submitted 3 weeks ago by to c/technology@lemmy.world
16 comments fedilink hide all child comments
 

An estimated 99% of farm animals in the United States are raised in concentrated animal feeding operations, or CAFOs. Often called factory farms, these facilities raise livestock in densely packed, enclosed spaces, with some housing tens of thousands of animals at one time.

[...]

In a new Technology in Society paper, Sheng and co-authors analyzed 4,800 independently generated images and found that ChatGPT defaulted to pastoral imagery when prompted to depict dairy and pig farms, even when they requested more realistic images. However, when researchers disabled the underlying prompt revision, the generated images depicted more intensive livestock farming.

all 16 comments

sorted by: hot top controversial new old
[–] 26 points 3 weeks ago* (last edited 3 weeks ago)

I don't know how someone can think "AI CAN obscure reality ..."

Obviously the fundamental truth is "AI does obscure reality ..." (Often this will be inconsequential, but that doesn't make it less true).

(Edit: especially an academic, FFS)

  • permalink
  • fedilink
  • source
    •
    [–] 24 points 3 weeks ago (4 children)

    well, to be fair, we trained most models away from non-sfw images, so ugly images get cleaned up to look nice. That's a consequence of making image models that don't do things you don't want your kids to see

  • permalink
  • fedilink
  • source
  • hideshow 4 child comments
    •
  • [–] [S] 4 points 3 weeks ago*

    It's a little bit more specific than that, prompt rewriting is a step they added intentionally to remap prompts. So not exactly trained to make them less about reality, but actually changes your prompt to the image generator to be less about reality

    Sheng found an explanation deep within the technical documentation for DALL-E 3, OpenAI’s text-to-image AI model for ChatGPT: An internal language model rewrites or expands user prompts to add more details before images are generated.

  • permalink
  • fedilink
  • source
  • parent
    • [–] 8 points 3 weeks ago (4 children)

    “The moment that I found out it is actually a prompt revision that is erasing all the reality, I was absolutely shocked,” says Sheng. “That’s insane.”

    If you're going to insist on using an image generation program instead of actual photos, at least put in the effort to use it correctly. Share some photos, a URL to search results, etc. and instruct it to use them as inspiration.

  • permalink
  • fedilink
  • source
  • hideshow 4 child comments
    •
  • [–] [S] 6 points 3 weeks ago (1 child)

    Image search results are getting flooded with AI generated images. Even using filters to block domains with known AI-generated images still has a lot leak into it. There's a good chance you'll just be asking the models to use AI-gen images as a reference by doing that

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
    • [–] 6 points 3 weeks ago (4 children)

    Why are they using AI photos for this at all? In what context could that possibly be acceptable?

  • permalink
  • fedilink
  • source
  • hideshow 4 child comments
    •
  • [–] 7 points 3 weeks ago (3 children)

    You need to read the article.

    Factory Farms aren't making the AI images, researchers are. The complaint, apparently, is that AI image generators aren't trained on enough animal cruelty to realistically depict a modern slaughterhouse.

    This is a problem because the author of this article thinks that in the future, people will rely on AI image generators to understand the world around them, and honestly, if that's already the assumption, then we are cooked.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 7 points 3 weeks ago (2 children)

    No... Images of realistic farms can be generated by these models, but they are set up to intentionally block that realistic depiction.

    Look at the examples in the article. The service is injecting extra instructions into all these requests invisibly.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] [S] 1 point 3 weeks ago (1 child)

    But to be clear this means realistic as in depictions of factory farming at all, not necessarily still perfect representations of it either

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 3 weeks ago

    I'm searching around but I can't find it, there was some discussion of students doing research and how depicting fantasy farms undermines it's actual use as a research tool.

    The output of these models is highly editorialized and just like the recent court case the publishers should be liable for misinformation and slander generated by these things.

  • permalink
  • fedilink
  • source
  • parent