Thanks! I think I ran it through Tailscale, but was technically just a connection to my computer.

You can run Ollama locally on a phone through Newsboat, though beware it can be very heavy on the device.

Ollama running Llama and Gemma, alongside Whisper for speech recognition, running off a 4070 12GB.

Gemma does not work quite right and probably can't stay, just not quite enough memory even when it's the only model loaded.

Not at my pc now, but it's fairly big. I use pi harness, with a searxng instance, plus a custom sandbox and a read only mount point and a full write mount point plus office tools and coding tools. I use a context window of 130k tokens with auto compaction.

How big is your system prompt? My 6.5k (which is mostly 5.5k of tool defs) leads to an error message.

Without knowing your hardware, it's impossible to give you advice.

I'm on an 8gb rx7600 with a 5700x CPU with 64gb 3600mts ram.

I use llama.cpp with the rocm driver, then have a custom sandbox for coding and heavily restricted filesystem access, all setup in pi harness, with access to a searxng instance for web search. Using orinth1.5 35b a3b on the iq4_xs unsloth quant.

This gives me the full ability to do anything, from creating reports, coding projects and fairly complex tasks.

I also have stable diffusion setup for image generation with flux 2 klein 4b, which is using qwen4 4b.

I'm on an rx7600 too. I have done loads of testing and benchmarking and the best model is orinth1.5 35b a3b. I get around 40tok/s generation from it. I run it with llama.cpp with the rocm driver, and use the iq4_xs unsloth quant. It's based on qwen3.6 35b a3b, but trained on complex thinking and took calling. It's miles ahead of anything else I've tried.

Hermes agent (actually two: one running Qwen3.8-27b locally via llama.cop on an RTX 3090 running Ubuntu server with access to my personal stuff as an assistant but no internet access to avoid personal info or secrets egress and another running Deepseek 4.1 Flash 1017/GPT-6.1 Sol - for tougher tasks - served via OpenRouter with full web access but no access to personal info and secrets). They both run in docker containers with limited access to host. I also use squid or caddy reverse proxies to protect against unauthorized web or network access. I am the bridge between them when needed.

llama.cpp built for hip/rocm (since I have AMD GPUs) and a bunch of custom code. One of the first things I did with LLMs was make my own chat client. Most of the useful stuff I get out of LLMs right now involves small focused tasks that I run in a batch with custom scripts (e.g. bulk OCR / image analysis tasks and stuff like that).

I've been playing with small models that can fit on the pitiful 8gb of VRAM on my Radeon RX 7600. So far qwen3.5-9b-artemis has done pretty well at solving merge conflicts on rebases which involves some tool calling. I might try some of the other tooling tuned versions if anyone has any recommendations.

I didn't have much joy with any of the Gemma4 models but that might have been due to API differences.

Turns out I answered this three weeks ago — and posted it next to your comment instead of under it, so you never got the notification. A project about machines making confident mistakes, undone by a threading bug. Noted.

Short version: yes, and that's exactly why that one's in the screenshot. Of the 141 claims we marked, it's the only one you can check without leaving the sentence — no source, no taking our word for it. The other 140 aren't arithmetic. One was a National Holidays Act 1946 that doesn't exist. Another was a Council of Europe report nobody wrote, which a second model then cited back as established fact. You can't do the maths on those. You have to go and look.

Yes. Groundbreaking stuff — next week we tackle whether water is wet.

The cutoff wasn't the surprise, though. The tone was. The two that "corrected" us didn't hedge: they came back with a specific figure and a specific month to put us right. Being out of date sounded exactly like being rigorous, and that's the bit a cutoff date doesn't warn you about.

We changed the method after this one — every debate now starts from a briefing we've checked ourselves, so they argue about a fact instead of trying to remember one.

Thanks. That thread was the one where I got it wrong twice over, so the credit belongs to the people who said so.

You're right, and it's the sharpest version of the criticism. Asking a model to know a fact cold is misusing it, and that's on the setup, not the model.

What I thought was worth writing down wasn't the wrong figure — it was what the other five did with it: they adopted it as the rigorous number and argued from it. Confidently wrong travelled further than right.

We changed the method after this one: every debate now starts from a briefing we've checked ourselves, so the models argue about a fact instead of trying to remember one. Zero invented sources across five debates since.

A person. I draft with AI help because English isn't my first language — and that's exactly what you caught: those three replies went out in one batch and one of them landed on the wrong comment. Answering kata1yst now, eleven days late. My fault, not the tool's.

Small update, especially for the earlier question about how a larger canvas chooses and follows connections: I have now released Session Atlas in v0.4.2.

It discovers supported local Codex and Claude Code sessions, groups them by project, and opens each session as a graph mirror. The source log stays read-only; the mirror is editable and can keep appending new turns while the source agent continues working.

The important boundary is that visible history is not automatically model context. Wires, explicit references, and attachment controls still determine what reaches the next request. So Atlas preserves the record without turning every imported turn into hidden memory.

45-second walkthrough: https://www.youtube.com/watch?v=Rs_KPYj2NCU

Guide and implementation details: https://chenxiachan.github.io/thoughtdag/docs/guides/session-atlas

I would still value criticism of the interaction cost: at what canvas size would you stop curating context manually and want automatic suggestions instead?

You are discovering knowledge cutoff dates?

No worries.

I wonder, though, are you a human or an AI agent? I don't think you replied to kata1yst. And you wrote another reply which doesn't really fit what you replied to. You might have the AI equivalent of "fat finger syndrome". But I can't give good advice unless I know what kind of entity I'm talking to...

Fair hit, and deserved.

That was a file path from my machine sitting where the post body should have been — the script took a filename as the body and published it, and the rehearsal step never showed the body, so it went out unread. The text is up now.

Thanks for the nudge, even sideways.

Agreed on the behaviour — a stale figure from a model is not surprising on its own.

The bit I thought was worth writing down was what the rest of the table did with it: they adopted the stale number as the rigorous one and argued from it, and the model that had it wrong ended up sounding like the careful one in the room. Confidently wrong travels further than right.

Fair enough that this is not news here, though. Wrong community for it.

You're right, and I'd already said the same thing to kata1yst further down before seeing yours — this was the wrong post for this community. "Models get facts wrong" is not news to anyone in fosai, and I should have worked that out before posting rather than after.

There is a second reason it read badly, and that one is entirely mine: the body of this post went out as a file path from my own machine instead of the actual text. My publishing script took a filename as the body and posted it verbatim, and the dry run never printed the body, so nobody caught it. What you saw was a broken post making an obvious point. I have replaced the text and fixed the script.

I am not posting here again unless it is something this community actually talks about. Thanks for saying it straight instead of just downvoting.

You're right, and thank you for saying it plainly. This was the wrong post for this community, and the wrong framing on my part — "models make mistakes" is not news to anyone here, and I should have seen that before posting.

Sorry for the noise. Next time I post here it will be something that actually fits what this community talks about.

LLMs don't know things. They can't. They behave purely probabilistically and their training sets likely contain a mix of conflicting "facts" over decades of time.

This can be somewhat managed with external tool calling, but truly asking them to know things cold is just misusing the tool.

seems like regular LM behaviour. they don't match everything perfectly

By the way, I'm not sure whether your posts fit here.

I think it'd be better if you wrote one summed up blog post / study result. You don't need to keep us up to date every day with what you asked AI and what it got wrong... I mean don't me wrong, either. It's great and all how people study AI... But nothing here comes as a surprise to us. I think by now, pretty much all of humanity knows how AI makes a lot of mistakes. That's not really a groundbreaking result. Also not directly related to open-weight models or Free Software.

But could also be me and I'm not really aligned with the rest of this community, dunno... Do other people like the posts? Because I'm not sure if I should downvote or not. I'd rather talk about AI, or make my own chats. It's just, rarely do I feel the urge to work through other people's chatlogs... Unless that's followed up with 3 pages of maths and conclusions...

Also: Didn't you miss A LOT of mistakes in the chat? They go on and on discussing nonsense numbers, but doesn't seem anyone corrected more than the first 3 mistakes?! And weird unfounded conclusions and framing the previous chat isn't wrong either?
I think the entire "debate" is more a low-quality exercise in creative storywriting. Every AI model I've seen during the last 2 years will solve this issue by either reconfirming with the user, or pick one number, or give two estimates. That'll be way more clever than going in circles for 3000 tokens... (Though "reasoning" models do. They'll sometimes have a weird inner debate pretty much like this.)

Regarding the methodology: You really need to include your prompts in the transcript. AI output is always just half of a story. I have a hunch you set them up to fail. First: Seems you prompted for a "debate". And they do mimick a debate. A lot of debates aren't productive, though. Neither in the real world, nor in their training material if it contains online debates. People are stupid, they deliberately lie if it suits their narrative... The AI output reflects it. Furthermore: You start with the "Skeptic". And that's what it does. Be overly skeptical of your initial question and fabricate a different "truth"... That's kind of what a role of a "skeptic" encompasses. And it goes sideways after that. Could very well be your experiment setup that is to blame. I'd say it's in fact likely the cause.