When I've seen my parents use it, it brings up things it could not possibly know from the current conversation, so its definitely inserting previous context at the very least.

I have literally no idea what you're so confidently talking about by "pure LLM". It's hooked into other stuff too that lets it e.g. properly do math (I'd assume it can by this point?), generate/analyze images, etc. But that has fuck-all to do with whether it can bring previous context into a conversation. Go read the 2017 paper "Attention is All You Need"; literally the whole point of a transformer model is that it can assess the surrounding context of input tokens out to a basically arbitrary depth thanks to its heavy parallelization. It's obviously imperfect; go watch the most recent Kitboga video (clickbait, sorry) where he creates Gertrude the Pig by leading a scambot away from its training data.

In fact, older LLMs (similarly "pure", again whatever that means) had far worse problems with being cursed by memory. That's why the LSTM was invented from the RNN: because the model needed to have a way to forget accrued past context over time, or the results would quickly corrupt on longer inputs. (Keep in mind that with LSTMs, I'm mainly talking about performing more basic things like sentiment analysis.) That didn't make it "impure"; it just made it not as good at its job as a GPT.

Anyway, unless previously prompted to change to this tone: no, in 2026, it's not just going to start saying "I'd call the police 😂" in response to high component prices.