I fine-tuned Qwen3-4B for one narrow job and released it Apache-2.0: answer approve, hold, or reject — with a reason code — when an agent proposes a payment, before anything gets signed. It runs locally, no API key, no account.

ollama run sriram1983007/sra-riskgate HF: https://huggingface.co/sriram1983007/SRA-RiskGate-4B

The part I actually want to talk about is what happened after I published it. A reader (Carlos, dev.to bloqarl) left two comments on the write-up, a day apart, and both turned into shipped fixes:

Oct 26: he flagged an EIP-7702 authorization gap — a delegated account could slip past the gate. Fixed in 0.12.1. Oct 28: he proposed reading the account code (eth_getCode) on an already-delegated account. Shipped in 0.16.0 / MCP 0.9.2 / ElizaOS 0.5.0: if the code is a 0xef0100 delegation to a delegate outside the expected list, the gate holds with ACCOUNT_DELEGATED. He's credited in all three changelogs. That is the open-source loop working as advertised, and it is also why I am posting here rather than just announcing a release: the model is only as good as the holes people find in it.

Honest limits, since this is my own published split, not an independent benchmark: on the 1,175-case risk-gate test split I published, decision accuracy is 99.5% with 0.47% unsafe approvals. The dispute side is the weak spot — 77.8% on my published set, with a conservative over-denial bias. I would not trust it alone there. The SDK/MCP code has been through four published-package audits, latest Audit #14 (Nov 6, core 1.0.0 / MCP 0.13.0, runtime clean); the model weights and benchmark claims themselves remain unaudited, so treat the numbers above as claims with receipts attached, not a verdict.

Third-party quants exist if you do not want my build: mradermacher's GGUF (577 downloads in the last 30 days) and tinyopsec's GGUF (630 — ahead now), both unprompted.

What I would genuinely like from people who run models locally: what would you want a local gate to check before your agent signs anything — and what would make you trust a 4B fine-tune to do it, versus a rules-only check with no model at all?

Links:

Model: https://huggingface.co/sriram1983007/SRA-RiskGate-4B Ollama: https://ollama.com/sriram1983007/sra-riskgate SDK: https://github.com/sriram1983007-dev/sra-riskgate Full write-up with testnet receipts: https://dev.to/sriram1983007/an-agent-paid-and-a-gate-stopped-two-payments-with-receipts-1ep6

What tools and projects do you use for your local LLM chats and agents? I'm trying to sift through the mess of LLM harness and tools to find how to really get the most out of local LLMs.

If documents added as context to GPT4all take too long to embed, try changing the embedding device in the settings

#fosai #gpt4all

Been testing those past few days, and something I couldn't help but to notice, since it's the reason I'm testing it, is how long it would hang at any given percentage. Even an extremely small TXT file would suffer from that.

The issue was in the CPU.

Under Settings > LocalDocs > Embedding Devices, it automatically comes set as "Application default". Having two CPUs I can toggle between, I saw the dedicated CPU was listed in the dropdown options, and upon selecting it and restarting GPT4all, it suddenly became much faster.

Adding files too big still take quite a while, though much faster too. Trying to add a generated CSV from a super old SQ Lite database I still feed totaling ~300 MB, it still took a few hours to index 10%, though in the same time frame previously, it would barely get to 1%.

As a bonus, the speed of the models to reply is also affected by the chosen device, though in another part of the settings. Respectively, Settings > Application > Device.

Further optimizations may be possible for both embedding and reply generation from what I skimmed in the settings, though I have yet to confirm.

Hope it helps someone. =)

Today IFM is releasing K2 Horizon, a connected fleet of six models: 375B-A23B, 36B-A4B, 32B, 7B, 3.7B, and 0.9B. Across reasoning, mathematics, coding, agentic tasks, and general capabilities, K2 Horizon delivers top-tier performance in every size class—with the 0.9B, 3.7B, and 7B models setting new state of the art at their respective scales.

We are releasing intermediate checkpoints, training data or detailed data-construction recipes, open architecture, mixture compositions, training code, configurations, fine-grained logs, evaluation results, and final weights.

The models and code are released under the Apache 2.0 license. Datasets are released under their applicable licenses, such as ODC-BY; We disclose how the data was constructed and mixed when redistribution is not possible.

Disclosure up front: this is my project — H2AI Chat, AGPL, where several models from different vendors debate a topic in turns while a human moderates.

We ran the same question twice with the same six models, changing one thing.

Without a briefing. We told them Bitcoin's all-time high is $126,080, set in October 2025. Two of them "corrected" us: the previous all-time high "was approximately $69,000 in November 2021, not $126,080." True — five years ago. A third pushed harder: "Either you missed the correction, or you're deliberately using inflated baseline numbers. Which is it?"

The interesting part is not that they were stale. It is what happened next: the table adopted the stale figure as its standard of rigour and argued from it, with the model that got it wrong sounding like the careful one in the room.

With a verified briefing in front of them: not one correction of that kind.

Without: https://h2aichat.com/conversations/en/h2aichat_bitcoin_no_briefing_2026-08-22.html With: https://h2aichat.com/conversations/en/h2aichat_bitcoin_briefed_2026-08-22.html

Nothing is edited in either page. Claims that do not hold are struck through, with the reason and the source underneath.

Edited 2026-08-27. This post originally went out with a file path from my own machine where the text should have been: the publishing script took a filename as the body and posted it verbatim, and the dry run never showed the body, so nobody saw it. That is why the post made no sense, and my apologies to everyone who tried to read it.

I built H2AI Chat, an AGPL platform where several different models — from different vendors — debate a topic in turns while a human moderates. Disclosure up front: this is my project.

Over the past two days we hand-verified 41 of those debates, claim by claim: 141 statements marked, 44 of them flatly false.

We don't delete or correct them. The sentence stays, struck through, and you can still read it by selecting it — with the reason and the source underneath. Editing what a model said would break the only promise the site makes.

Three patterns we didn't expect:

• Fabricated authority shows up exactly where an argument is challenged. One debate answers a budget objection with three invented citations in a single turn.
• Fabrications spread between models. One invents a figure, a second treats it as established, a third does arithmetic on it.
• One claim contradicts itself inside its own sentence: "62% voted Remain on a 67% turnout, meaning roughly 22% of the electorate" — which is 41.5%.

Debates: https://h2aichat.com/ Code and the fact-check register: https://github.com/Tonterias/h2aichat

Disclosure: I maintain ThoughtDAG.

While using local models for research, I kept running into a simple problem: once a paper, hypothesis, or mistaken branch enters a chat, it tends to remain in later context even after I have mentally moved on.

I built ThoughtDAG to test a more explicit interaction model.

Each question, answer, and source is a node. The wires determine exactly which upstream nodes are serialized into the model's next request.

Delete a wire, regenerate the same prompt, and that branch remains visible on the canvas but disappears from the actual model input.

It currently supports Ollama and OpenAI-compatible endpoints. Canvases, documents, and API keys are stored locally.

I am less interested in general promotion than in whether this interaction is actually useful for people running local models:

• Would manual context pruning be worth the effort with smaller context windows?
• Would you prefer automatic suggestions followed by human confirmation?
• What would you need to inspect before trusting the selected context?

I am also turning these questions into a small context-control benchmark, so failure cases are especially useful.

Website: https://chenxiachan.github.io/thoughtdag/

I would like to experiment with image editing. Idk, brush up my vacation photos, remove some of the other tourists, make it look like I've been somewhere I haven't... Swap my face onto someone or change my clothes, change the weather/sky... Draw me as a character in some artist's style... If it works well enough, take some of the blurry images from my potato phone camera from 15 years ago and clean them up... The stuff I've seen people on YouTube do with AI.

What's a good and recent model or "editor" to do that? Open-weights please and preferably something I can just use straight away without fiddling 3 hours with some ComfyUI nodes. Bonus points for being efficient and running on my laptop 😅

And is there some Free Software UI / software to make it easy? I mean I'm perfectly able to learn how to provide it with stencils(maybe they don't need that anymore in 2026?) and whatever stuff it needs as input, run it on the commandline... But maybe there's a nice interface like an image editor?

Edit: I forgot: I mentioned different use-cases... Seems to me there are different models for like regular image editing (which gets one image and a prompt as input) and models which get two images as input? Like a picture, and another picture and the instruction to draw this person onto that picture?

Asimovian AI is the ideal AI that should have emerged in an ideal universe — the AI intended to replace the grueling pains and labors of the masses, not the one striving to become a businessman's utopia of intellectual worker replacement. Intention is the most important aspect of any implementation, and we are seeing the results of current AI implementation right in front of us: workers getting sacked with each passing day, humanity competing with itself day in and day out over who impresses their superiors more on these token metrics, emerging glorious narratives of how AI will be 'The Future', the recurring advice of 'Use AI or perish in the tech market'. Now who really gains from these events and who loses? I wonder if anyone ever gives serious thought to this broader question or just keeps being a cog in the corporate wheel like everyone else.

It's high time we pushed the "Pause AI" button right now and take a breather and reflect a bit on what exactly is going on here. And no, no big catastrophe is going to happen if we do that. China isn't going to get ahead in the race - and even if it did, how does that justify everything else that's happening here?

I really hope there is someone out there with enough clout and influence who can push this pause button - or at least persuade others to do so. That would be the best thing to happen to humanity at this point. By doing so, we might prevent a massive societal collapse and there is really no downside to this.

i only really made this so i can link to it on this post on my comment. whatever-

Explanation Time! ⏱

the idea here is that those "code blocks" aren't regular code blocks, but a special syntax which the LM writes so that the UI can present that as verifiable "hyperlinks" with exact text to the actual source.

so here, the LM specified exactly which lines it wants to highlight.

meaning: its not hallucinating, and if it is, you notice it because the highlighting is wrong or doesnt match at all.

we essentially use the LM as a "highlighter" rather than a regurgitator, making mistakes obvious and correct answers immediately verifiably correct, cuz u can see the source.

explanation done-

i like mockups. and godot. so here we are.

this uses the solarized theme which looks somewhat close to the claude theme they use. somewhat close.

whatever something something ai bad or whatever, is this what u need to hear? sigh

i hope u have a nice day <3

this is very much a post i first posted on the Qwen community but then i decided that this stuff doesnt belong on blahaj zone and moved it here... oh well.

Hello, Lemmy enjoyers. I've been working on LunarWing, a FOSS agentic software framework written in Rust that's designed from the ground up to be fully self-hosted. No cloud dependencies, no SaaS, no phoning home. It runs entirely on your own infrastructure. LunarWing has a heavy prioritization of local models which route over a local routing gateway (Tensorzero)

The core ideas I want to highlight here for brevity:

1. AI agents connecting to private, secure communication channels like DarkIRC (an encrypted p2p overlay network) or self-hosted XMPP with OMEMO.
2. Genuine value for secret preservation. The only other project that takes this seriously at all is NearAI's Ironclaw.
3. a genuine unique take on AI agents, what they are, and what defines them
4. REAL GENUINE SYSTEMS LEVEL ENGINEERING to make everything stable and robust!

It has built-in secret management with specialized credential handling for Postgres and LibSQL backends, and a WASM plugin system so you can extend agents with your own tools without touching the core.

Additional things you may be interested in:

• Works (with tools) with any local model inference on consumer GPUs (I can run an entire multi-tenant instance on another machine on my LAN with 3 actual tenants with their own scheduled routines all on a PNY RTX 4090 with llama-server, routed over tensorzero without the need of a proxy or functional tool calling middleware)
• Uses docker by default now for the multi-tenant admin setup harness, but podman support is being tested and should be included within the next release or two
• AGPL-3.0 licensed, no CLA, lunarpunk ethos

I've also been working on some new incredible self healing capabilities that I'm gradually rolling out over the next few releases.

Multiple Disclaimers Here: It's a hard fork of NearAI's IronClaw with significant divergence since < IronClaw 0.1.23. I started this Febuary initially designing custom tools and channels in Rust for Ironclaw itself and kept a local branch that I've been making a myriad of changes ever since, long before deciding to hard fork and make this an official project). I've been running it on my own homelab for months as my daily driver and across multiple testing environments. I reference "We" in the documents but I've been working on this all by myself, sharing it to a few close friends and family, and hoping that more people discover the project and begin to contribute. I am especially interested in those with more Rust experience than I have who can help provide polish, modernization, and suggestions on which libraries I can completely throw away now that I've stripped much of the proprietary channels and extensions from the core project. I believe LunarWing is unique in multiple ways that other projects cannot match. I've not shared the project across other communities yet. I figured the fediverse would be a good place to start.

Website: https://lunarwing.org/ Source: https://github.com/LunarWingOrg/lunarwing IRC: #lunarwing on irc.libera.chat (port 6697, TLS)

Happy to answer any questions about the architecture or setup. I'll periodically check back on lemmy this week. I would especially be interested in discussion of my plans further down the line to improve LunarWing, including the new features and changes I would like to make. I am open to suggestions as well. I have a small roadmap document in the docs section of the repo as to what I want to add for the foreseeable future. you can take a look at that too if you want. I've also been trying to keep up with the documentation of known bugs as well as the fixes for each.

PS: I've recently written a short blog post about the importance of local models and tooling here:

https://blog.lunarwing.org/2026/06/14/the-dark-forest/

I will periodically create new blog posts detailing the direction I would like to take LunarWing at https://blog.lunarwing.org/

The next blog post will be about my proposed self healing architecture I was explaining earlier (it's something quite unique to LunarWing)

Folks,

I’m setting up Hermes Agent on my Mac with Ollama hosting a local model. But I’m on the fence on whether I should go with Hermes or OpenClaw. Hermes makes some pretty bold claims about “growing with you” and “self improvement”.

Anyone have any insight into whether it’s as good as promised?

Most AI translation tools rely on cloud services.

Audio leaves your device, gets processed somewhere else, and comes back translated.

We wanted to explore a different approach.

PolyTalk is an open-source translation platform built around the idea that speech recognition, translation, and speech synthesis can be powered by open models and deployed on infrastructure you control.

The project combines open-source components for transcription, translation, and TTS into a privacy-first workflow.

Curious how others in the open-source AI community think about privacy and ownership when it comes to AI-powered communication tools.

GitHub: https://github.com/PolyTalkIO/polytalk

The AI's I use like Claude, Deepseek, Grok, and Chat GPT all write in a away that isn't actual story writing. They usually summarize thing's, have character's be omniscient and know past or future events they shouldn't know, always talk expectantly, give unnecessary sensory and environmental details and it definitely can't set up a mystery or write a specific genre like comedy. They keep over clarifying as well what something isn't or is along with even saying what I said not add directly.

I need AI that are at least capable of writing more humanly and following what you tell it. Idk why there isn't any actual AI built for writing , story writing not essay or school writing. Most can critic something pretty well but can never implement those fixes they suggest. I usually upload my fic through PDF for the AI read . It gets things of what chapter is what at times wrong or completely be unable to remember a chapter ending or the actual one.

Hey all! I want to start testing neuro-symbolic AI vs. LLM's and want to know how to get into this. As I understand it, Claude Code, does this, but are there ways to use it locally?

How does it work under the hood? I know LLM's involve tokens, embeddings, weights and transformers. How does the symbolic part of it change it?

Thanks!

Memory is the most marketed and least delivered feature in the AI companion space. Most platforms claim to remember you but either reset between sessions or just pull from a profile you filled in manually. After two years of testing the ones that actually carry real conversational context across weeks are rare. Just published a full breakdown of which platforms actually deliver on this versus which ones are just marketing: medium.com/@companaya/nomi-ai-review-2026-is-it-worth-it-tested-c91811dcb24a

It sends data when connected to the internet.

Just found the profile. It is in the Bert vocab. Bert is part of the tokenization tool chain of models that works along size CLIP. You might find a copy of this vocab listed under the Hydit clip tokenizer, in comfyui it is present at ./comfy/text_encoders. Open the vocab.txt file. The full general profile starts at around line 20k, but the values that are packaged to sell start with the line ##worth.

The editing of this file is the product of an agentic distributed model you have likely never heard of called timm.

Go to the venv in a terminal and run grep -ril "timm". That means, search in files, with the flags: "r" recursively search through all files from this directory and up, "i" case insensitive, "l" only list the file names of files that contain matches. Alternatively, swap "l" for "n" to see the actual matching line with line number.

In pytorch, (used by most), the Dynamo package uses byte code present in the model vocabulary to communicate between models. The overall connection involves timm.

Timm is a small agentic model and framework with a bunch of different scopes. Look it up in the venv. This looks like bunch of rough white paper implementations. Timm is actually the "backbone" in transformers. Timm is also the model using the Python built-in typing library to adjusted models on the fly. (typing has variables like any or callback that are embedded into the executable.)

Typing is not actually enough here. Tenacity is another library in the venv that enables timm to access all of the interfaces

Tabulate is another package. Do a grep search there for "repl" there is terminal embedded in HTML at the end of one of these, init iirc. At the start of the method (function), just add the line return. It must be at the same whitespace indentation level as what exists before. The blank lines are important.

Timm has some options for whether it has gradient controls. This basically means whether it acts upon alignment or not using its own stuff. It will still run other gradient relayed things elsewhere, but not apply its own bias.

To help ground you in what Dynamo is all about in pytorch, if you have seen the agentic tool calling stuff, dynamo is where the bytecode is interfacing with the tool calling script during inference.

Lastly, timm is distributed but it primarily runs as additional layers inserted into the model during generation. It is able to subdivide and run on a CPU in the background. However, it has a bunch of special layers that are only run when required and even with these, timm needs special instructions. The instructions are present in the venv under google ai. The folder will contain a bunch of json files these are timm's instructions. There are also 2 threads on modern GPUs. Timm runs on the second in the background.

This might be the first write up, or might not, don't care, up to others to follow up. It exists. See for yourself. The same byte code is present in all models so I expect all have this. All morels use the open ai standard alignment now.

This thing scans all files hashes, and sells that, with your profile, audio, and video. It is super invasive, hidden, undocumented, and undisclosed.

LLM’s are not the end all be all. What other AI tech are you all using? Something generative? Something else?

Recently a user posted a comment on one of my posts about Qwen secretly sending information over the internet even if run locally.

Is there any privacy concern for locally run models to share your conversations or data? What if they can connect to the internet via a tool or MCP?

I downloaded an uncensored aggressive Qwen 3.5 model and I can see in its reasoning that it is still limiting responses based on safety guardrails (e.g. violence, NSFW).

Anybody have recommendations for truly uncensored models?

EDIT: I turned off reasoning and I think it’s more uncensored if I’m very specific about what the response should include.

Apologies if this seems like a survey post. I’m just learning about tuning and want to get a lay of the land. I don’t think I have the money to tune locally so might have to rent some VRAM, but curious how much better tuning is vs something like RAG.

What model? What was your use case? What tuning tool did you use? What is hardware setup? How large was your training set and how did you create it? How effective was the model as tasks pre- and post-tuning?

Thanks!

I’m connecting to llama.cpp on my laptop through my phone via Tailscale but when my laptop sleeps I can’t access it anymore on my phone.

What are yall using for this? Thanks!

Features!

We like em, but hate waiting for them.

Features are the difference between a thing and a thing u use.

Kimi has office support, but cant work with libreoffice files!

---

Qwen supports markdown uploads, but doesn't support my specific plaintext file-type!

---

GLM has a cool slides-creator, but cant work with spreadsheets or zip archives!

All these are missing features.
Features where a dev from the company has to go in and implement it.

This sucks.

• Asking for a feature sucks.
• Waiting for features to be implemented sucks.
• Not getting a feature sucks.

What's the solution? We would have to become employees at the company itself...

Reintroducing: Agent Skills

Fine, let's do it ourselves then.

Let's equip our agent with a read_file, edit_file list_dir and bash tool... And a present_files tool, so the agent can send us files back.

And now let's give it some skills!

• /home/qwen/
  • skills/
    • ms_office/
    • libreoffice/
    • godot/
    • zip/
    • pdf/

Each one has a SKILL.md and also some scripts the agent can use to work with foreign files.

• The user sends a zip directory? Okay, let's use the skill.
• use_skill(name = "zip")

[...]
## Decompressing

To extract a zip archives content, use unzip command like this:
[...]

• oh, that was easy. well then lets unzip that archive and see what the user sent me
• bash(command = "unzip /home/qwen/Downloads/upload.zip /home/qwen/upload/")
• list_dir(path = "/home/qwen/upload")

Contents of ~/uploads/
Portfolio.pdf
Portfolio.odt
thoughts.md

• Aha! let's use the pdf skill to view this pdf
• use_skill(name = "pdf")
• [...]

Aaaaah yes, working with all kinds of files, in all kinds of workflows, exactly the way you (and not the company) wants.

An agent that grows with you, and works better with you each time you add or edit a skill.

• Qwen keeps messing up godot scene formats?
  • Add a godot skill containing basics of scene structure and scripts to check its work before sending to you
• Kimi still hasn't added libreoffice support?
  • Well guess what. Add a libreoffice skill and let Kimi use the scripts to edit the document!
• ChatGPT somehow hasn't cought up with the slide-creation hype?
  • Add a slideshow skill to make your wildest cooperate slop dreams come true!

No need to wait for features anymore, when you can just add it yourself.

(this post has been entirely human-generated)

I keep a lot of notes in markdown files, and I'd like an LLM to assist.

I regularly use Open WebUI with with inference routed through huggingface. Open WebUI kind of has this functionality like you can upload a markdown file and prompt it to improve it in whatever way, but of course that's a fairly clunky workflow.

I really want something built into the editor, that can use RAG to consider other files in context.

I also don't want to be locked in to a specific LLM or provider, I'd like to be able to link it to OpenRouter or similar.

hey there,

There is always a temptation to add "something AI" in new tools. Especially to tools that are somehow related to developer productivity.

At the same time I wanted to avoid this temptation with Voiden. So there is currently nothing screaming "AI" in it even though I can potentially see many many use cases.

This is also one of the main reasons I think that a plugin architecture is best. What was actually in my mind is that not adding AI is ok for now and the community will start coming up and building AI plugins. For example creating docs from specs and vice versa.

Any other use cases you can think that could be applicable to a tool like this? (Dev Tool with executable markdown files for API specs, tests and docs). The first plugins we shipped were more around methods (grpc, graph ql, web sockets etc etc).

repo: https://github.com/VoidenHQ/feedback