That’s not how multimodal models work. The image is transformed into a vector embedding, just like the text is.

https://www.ionio.ai/blog/a-comprehensive-guide-to-multimodal-llms-and-how-they-work

They “see” it just like they “understand” the text. Both Grok and Gemini are cited in the article and both are multimodal in this way. They get it wrong because AIs are prone to hallucinate when they’re at the limits of their knowledge.

While growing up I used tech to make myself more intelligent, I wanted to understand how it worked and functioned. Least on a software level I'm a programmer not hardware (though may do some small projects to see at some point).

I always wonder why we fell so far, like from growing up I needed to figure out computers and ensure I could use them to their fullest. These days appears most people of a similar age don't care cause it works.

While I see that point. it hasn't worked that long and at this point we seem to have people not knowing where to save a word document. That's from 3rd party stories but seems believable. So we went from some of the most curious people understanding their tools to people going 'meh it works' without understanding a damn thing.

It only worked decently cause the operators understood it and we put in our time. Shit did I just get a reason for Matrix operators name? Like a dual meaning besides the phone one. Huh never thought of that but back in the day operators had some answer usually....

I mean yeah thats the entire point of the article. All the slop machines were wrong and actual manual research was required to find the truth.

The cemetery image, it turns out, is authentic. Researchers have cross referenced the photo of the site with satellite images that confirm its location, and it can be cross-referenced again with dozens more images taken of the same site from slightly different angles, and again with video footage – none of which experts say show signs of tampering or digital manipulation. The “factchecks” by Gemini and Grok are just one example of a tidal wave of AI-generated slop – hallucinated facts, nonsense analysis and faked images – that are engulfing coverage of the Iran war. Experts say it is wasting investigative time and risks atrocities being denied – as well as heralding alarming weaknesses as people increasingly rely on AI summaries for news and information.

Yes, that flaw is exactly what they're pointing out. They're saying, "we asked AI a question we knew the answer to and it was wrong."

Leaving out the context is bad.

the news story is about how LLMs are resulting in people being fed false "fact-checks", being told the image is fake when it's not

it's not strictly about whether the image is real or not, which the facts are clear about

It absolutely matters if AI images of war are being passed off as real.

Misinformation and propaganda were already a huge issue and AI just ramped it up.

You can give Gemini the exact same prompt and context 100 different times and you might get 95 very similar responses and 5 wildly different responses.

I don't understand why people think a random text generator can ever be relied on for truth. It has no concept of truth. It is a random text generator. A pretty consistent one, but still fucking random. It has no intelligence. It is not intelligent. Stop acting like it is. Its conclusions are meaningless. They do not contain actual meaning. They are random.

oof, might need to work on your tech and media literacy there 🫠