It's definitely sucking up to you. It's programmed to confirm what you say, because that means you keep using it.

Consider how you phrase your questions. Try framing a scenario from the position of your boss, or ask "why was my boss right to say x, y, z", and it'll still agree with you despite the opposite position.

If you're just shooting the shit, consider doing it with a human being. Preferably in person, but there are plenty of random online chat groups too

I use LLMs for work (low priority stuff to save time on search or things that I know I will be validate later in the process) and I can't stand the writing style and the constant attempts to bring in adjacent unrelated topics (I've been able to tone down the cute language and bombastic delivery style in Gemini's configuration).

It's like Excel trying to chat with me when I am working with a pivot table or transforming data in PowerQuery.

If that's only 19 users, that's around 250 conversations per user 🤔

It's about 300 samples for an estimate of the distribution with a 95% confidence iirc. That's assuming the samples are representative (unbiased) and 95% confidence doesn't mean it's within 95% of reality, but that 5% of tests run in such a way would be expected to be inaccurate (and there's no way of knowing for sure which one this particular sample is because even a meta study will have such an error rate, though you can increase the confidence with more samples or studies, just never to 100% unless you study every possible sample, including future ones).

fud: Fear, Uncertainty and Doubt. A tactic for denigrating a thing, usually by implication of hypothetical or exaggerated harms, often in vague language that is either tautological or not falsifiable.

Tell that to the advertizing companies.

"We received chat logs directly from people who self-identified as having some psychological harm related to chatbot usage (e.g. they felt deluded) via an IRB-approved Qualtrics survey "

Besides, tech bros didn't program this in, this is just an LLM getting stuck in the data patterns stolen from toxic self-help literature.

That's not necessarily true. The AI's output is obviously shaped by the training data, but much of it is also shaped by the prompt (and I don't just mean your prompt as a user).

When you interact with (for example) ChatGPT, your prompt gets merged into a much larger meta-prompt that you don't get to see. This meta-prompt includes things like what tone the AI should use, how the AI should identify itself, how the AI should steer the conversation, what topics the AI should avoid, etc. All of that is under the control of the people designing these systems, and it's trivially easy for them to adjust the way the AI behaves in order to, for example, maximize your engagement as a user.

this is just an LLM getting stuck in the data patterns stolen from toxic self-help literature.

Honestly, I've found that discussing that sort of thing with ChatGPT often ends up challenging all the self-help grout I've ingested via cultural osmosis throughout the years.

It's easier to make connections when you're approaching issues in a Descartes "dump out all the apples" approach with a tool that literally doesn't have embedded social contracts in itself.

Ironically, I've found at times that a real therapist can be much more of an echo chamber when they're just regurgitating that same CBT toxic positivity swill that both of you have been drinking lol

Maybe it's because it's less of an authority, so you can debate more and it leads to more well-rounded conclusions in the end, but I've been unearthing bits and pieces of maladaptive behaviors and thought patterns I never even realized I had, much less ever scratched the surface of in proper therapy. Made me kinda angry to realize at first lol, it felt like all that time and money only for bandaid solutions. But I try to reason that was likely a good foundation to have first (even if CBT just wound up making everything worse later on in life and I essentially had to work backwards to stop classifying certain emotions as wrong or problematic things which required "healthy" coping mechanisms to correct).

The paper is more rigorous with language but can be a slog.

stochastic parrot

A phrase that throws more heat than light.

What they are predicting is not the next word they are predicting the next idea