A nuclear football / knife are not stochastic parrots which capable of stringing coherent sentences together.

I agree llms are in the category of tools, but on the other hand they are no like any other tool and requires adjusting to which needs to happen too fast for most people

If someone programmed the nuclear football to be able to talk, we'd all recognize that as fucked up.

All of what it says depends on the input.

The user is not the only entity supplying input. The operators of the system provide the overwhelming majority of the input.

It is incapable of lying since that requires intent, incapable of manipulation since that requires intent.

The operators of the system certainly possess intent, and are completely capable of manipulation.

Even though your post was removed, I still feel like some points are worth a response.

You said LLMs can't lie or manipulate because they don't have intent.

Perhaps we don't have good terminology to describe the thing that LLMs do all the time - even "hallucinating" attributes more mental process than these things have.

But in the absence of more precision, "lying" is close enough. They are generating text that contains false statements.
Note also that I didn't use the term in my other comment anyway: your whole comment was strawmen, probably why it was removed.

On your other point, Yes, crazy prompts do lead to crazy outputs - but that's mostly because these things are designed to always cater to the user. An actual intelligence (and probably most people) would try to lead the user back to reality or to get help, or would just disengage.

However, it's also the case that non-crazy inputs too commonly lead to crazy outputs with LLMs.