1
 
 

"Wowie look at my tool call format": this sucks and is getting me nowhere.

always on HF (huggingface), always pulling the latest llama-cpp to test, quantize and run the latest smol text predictors and image segmenters.

my brain is mush, my mood on the subject is low and my interest in scaffolding and harnessing is decreasing at a rapid pace.

real agents need real models: large ones. deepseek V4 flash, minimax M3, Kimi.

i dont want to pay for those, that defeats the "fun playing" part of it and makes me think of money (yes, I know its cheap, I'd be paying like 2€ a month, but still!)

if i make a godot agent, nobody cares about it, its "efficiency", how its animated, how easy and fun it is to set up and how fluid the text parsing and tool call previews are. there wont be a youtuber person saying "ooh thats smooth.... how are other agents so much worse?" about it.

only vibe-coding claude-suckling losers are gonna use it, complain about no free tier and leave.

the only one who cares is me, and i should spend that effort elsewhere and stop caring about this stupid hypie bs.

large-scale stuff is fun, whatever, but its out of my interest pool, so i dont care.

smol local models suck, which is where i live, so:
my experience sucks as much as the local models do.


why is it so difficult to distance myself from this stuff?
is it

  • fear?
  • actual interest?
  • weird investment thoughts?
  • the idea of making something good with it?
  • the wish of local LMs to become viable? (yes it is)

its not like im profiting in any way, nobody cares about what i do and i also stop caring.

but what do i follow instead?

... watch.... gameplay videos? naw

... work on Bloom. yea fair. whatever I do, I should always be working on Bloom instead. (its a game about some butterfly character exploring a 3D mountain in the Austrian alps and-- collecting nectar from flowers and such. I still only have concept art.)

im sitting alone on the balcony, while peeps are celebrating in the hall and I dont want to celebrate with them.

they are partially celebrating our replacement.

this is the softest "overthrow the world" scenario ive heard of so far.


its not all overblown. its not all bs. its not all cash-grabby CEO sloppery. its not all useless chatbots in places you dont want them. its not all "optimize the human away" agendas. its not all just there to be there.

but oh boy 👦 it sure seems so from the outside.

i believe that language models in particular have actual good uses.
they can simplify computer navigation, help or come up with scientific / mathmatical breakthroughs and they can help with understanding subjects on a deeper level, without having to pay a personal tutor or have smart friends (which would obviously be preferred).
is that worth training on the entire internet? im not sure.
if chatgpt ++ pro max and claude opus mythos plus times 10 xhigh 3x speed were free to use and run locally, maybe we (or I at least) would think differently about this. because.... let's be real: ai is not "eating all of our water". most data centers use basic fans or closed-loop water cooling. just like any water-cooled PC at home does and the main environmental problems stem from the power consumption, which is an energy-source problem and less of an LM-specific problem.

if THIS was whats possible with local language models: that would be great!
no more man-in-the-middle company, no bad feels about continuous compute waste, no more feeling of being sold back all of digital humanities labour with no compensation.

but thats not where we are. these capabilities are behind 200€/month paywalls, proprietary models and... awfully excited CEOs selling their text predictors as the second coming of jesus Christ (our all lord and saviour), god (whos less interesting) and mosis (I think he can split oceans. pretty cool).


i need a permanent vacation from local language models.

if anyone reads this, give us your thoughts! or give them to me at least, if no one else is around, I will read them.

this was human-written in case anyone thought otherwise.

2
 
 

yes, thats right! i know that all yall ai-haters can't wait for me to stop posting about this absolutely overblown subject of tool-call parsers and


I ALSO CANT WAIT.

its not even that interesting.

ANYWAY!

u know what's sucks for LMs to generate?...

tool calls. they hate it.

But! u know what doest feel like a tool call but actually is one?

think about it.

its something u have likely already used plenty times yourself.

its! - - -

drumroll

The Slash Command!

wowie zowie how have i missed this one.

now oooobviously basic slash commands would only suffice for very basic calls like

/read path="file.txt" from=0 to=100

or

/use_skill name="scenes-and-nodes"

(skills rule)

but like - pffff this gets exhausting quick when writing larger stuff, like files!

/write path="file.txt" content="oh god\nit begins.\n\nthe never-ending c-escapes"

yea this sucks. No human would want to write this to do stuff.

lets spread out those args, and pull the idea from my last post (about multiline-strings as codeblocks)

/write
path="file.txt"
content=

now this! ~
feels writable.

I feel free in this code block.
and im sure qwen does too.

now looks at this! ~ 🥁 ~ ~ absolute bliss! ~ ~ ~oh what a shock!! :o

watch where u walk! >o<

see here for context

what an awesome song. anyway-


and best of all:

we are saving TWO TOLENS (!!!) compared to my last format per tool call:

[use_skill  
name="theme-creator"  
]

compared to:

/use_skill  
name="theme-creator"

do u see it? thats TWO (2!!!!) TOKENS freeing up infron of our eyes:

  1. \n
  2. ]

yes, we are saving two tokens per tool call.

looking at YOU, opencode, which puts 16k tokens JUST for the system prompt! >:(

why dies this work? wellll-

  • slash commands are well-known, unlikey previous format
  • multiline args appear to be a fitting addition (essentially replacing the with \n as a separator)
  • code-block multiline-strings had to be included cuz- yea, those LMs loooove generating em and hate c-escapes. keep messing it up, even big LMs.

absolute insanity. here a sys prompt to make an LM work with the format

You are agent with tools
read(path:str,from:int,to:int)->file content between from-to lines
write(path:str, content:str)->confirmation
search(query:str, is_regex:bool=false, type:Array[str]=["file_content","file_path"])->results (type can include file_content,file_path,dir_path
...

To call, write slash command at end of msg:

/write
path="/home/maria/file.txt"
content=
` ` `
content
here
` ` `

values are json-style, multiline-strings always in code blocks.
can call multiple in one msg

thats enough for basic tool usage with todays models. yes, even with 4B ones.

487 characters long, tokenizes to 118 tokens (when converting the triple ` strings to actual code blocks).

it works! makes the LM generate the calls correctly, even with banter as a prefix.... to keep itself on-course.

horray we solved climate change /s

oh well, time to actually implement this stuff tho....

its not difficult, but when ya want stable partial-tool-call-parsing, its a bit annoying.

UPDATE: wowie writing string parsers is difficult.

3
 
 

im so sleepy.

agents driven by language models (LMs) call functions to do stuff. Functions like these:

  • read_file(path, from, to)
  • write(path, content)
  • list_fir(path, show_hidden = false)
  • edit_file(path, old_string, new_string)

this is not a simplification btw.

so far, LMs were told to generate any of these formats to call a function:

  • json, which sucks cuz of c-escape
  • xml which sucks cuz of closing tags
  • yaml which sucks cuz of multilinear strings being indented (sucks for LMs)
  • just bash which sucks cuz of security

wow these all have problems with something, hm?

worst of all: we wanna save tokens wherever possible. so if an LM has to generate a full </parameter> for each argument in a function, that adds up quick

Introducing: my new format

wowie let's have a look at this format!

[write path="file.txt" content=

content of file here
horray newlines
no c escape! cool, i can regex all I want [\s\S]*

]

now isnt that simple?

  • no string escape problems
  • no xml-closing tags
  • no json-brace-foolery
  • no... | symbols for multilinear strings

now, of course, this is a new format. so language models suck at generating it, right?

WRONG

even a local 2-bit quant of qwen 3.6 35B-A3B aligned to it super easily.

and! even a dense Qwen3 4B model at Q4 quant worked with it flawlessly. I'm tired and need to sleep.

now congratulate me! say "horray wow ur such a genius ohmygod we are gonna save so many tokens and thus möney".

go, go head. im not gonna ask an LM to do it, that much is clear.

or, even better: tell me what SUCKS about this, im always open for critical feedback.

id rather be wrong than believe im right all the time.

4
5
 
 

wow! agents sure can do stuff.

Agent Basics

so like - claude code is getting a lot of attention lately. its some coding agent thing using their models.

lemmy just define what an agent is real quick:

agents dont just yap but they actually cook, as in: they can do stuff.
this works by having a secret back and fourth with the computer, its essentially doing function calls, but we call them "tool calls" because... why not.

Here an example of what this might look like(every "quote" is the "user" role and every bullet point is the assistant response)


(here are some hidden instructions on how to make tool calls) hey, plz put the stock prices of fairphone over last 2 years in some txt plzplz <3 also gimme a kiss when ur done >////<

  • Sure! First off all all, imma look up the stock prices.
    get_stock_prices(company="fairphone", start="01-01-2024", end="01-01-2026", increment="month")

<tool_response>
blablabla some stock prices here...
</tool_response>

  • Ah, okay, I now have the stock prices, not imma make that txt
    create_file(path="/fairphone_stock_prices_2025.txt", content="the actual file content here")

<tool_response>file /fairphone_stock_prices_2025.txt has been created sucessfully. You can send the file to the user using the send_file(path: str, subtitle: str) command</tool_response>

  • Now I will send you the file
    send_file(path="/fairphone_stock_prices_2025.txt", subtitle="heres ur file")

<tool_response>"/fairphone_stock_prices_2025.txt" has been prepared to send and it will be attached to the last message you send</tool_response>

  • Hey user, i made that txt u asked for! The file is attached below for download. Oh yea, also kisses u <3

Agents scaling Problem

this agentic loop works pretty well. but it doesn't scale well to hundreds - or even thousands of tools.

agents must be fed with tool-definitions, which have:

  • name
  • description
  • arguments (usually with descriptions and types)
  • example tool uses (optional)

for each individual tool.

now, all this stuff stuffs the LMs context, resulting in

  • higher costs (because of more input tokens)
  • it just confuses the model. could you remember hundreds of functions all at once?... i couldn't.

aaaaa How do we solve this?

introducing!!!!!!!!!!!!

docs!!! but for language models.

yes! that's what anthropic is trying to propose with SKILL.md. here's what that looks like:

  • u just give the agent a filesystem with some docs and some useful files.
  • a "skill" is just a directory with a SKILL.md file in it. that file must include a yaml header with name and description
  • every single SKILL.md file header in the filesystem is automatically loaded into the agents context along with the files location and an instruction "Read any skill file if it appears useful" or something similar

your filesystem may look like this

  • /
    • skills/
      • stock_prices/
        • SKILL.md
        • stock_stools.py
        • report_format.md
      • stack_overflow/
        • SKILL.md
        • stack_overflow_tools.py
      • creative_writing/
        • SKILL.md
        • themes.md
        • preferred_format.md

the /skills/stock_prices/SKILL.md may look like this:

***
name: stock_prices
description: Useful for detailed stock analysis
***
## Getting stock Prices
Use ./stock_prices_tools.py to run code which gets you the stock prices of registered companies:

` ` `python
import "/skills/stock_prices/stock_tools.py" as stonks
stocks_for_a_year = stonks.get_prices(company="nvidia", from="01-01-2001", to="01-01-2020", increment="1 month")

stocks_today = stonks.get_price(company="nvidia")

print("Stock prices from 2001 til 2020:\n" + str(stocks_for_a_year) + "\n\ntodays price: " + str(stocks_today))
` ` `
no need to read the file itself.

## Creating reports
Read ./report_format.md for the users preferred structure of a stock prices report

[...]

so... we simply decentralize the knowledge and the tools, by just putting them into files, so that things can be read once they become useful.

but Isn't this super unsafe?

this approach assumes that the given agent can write and run python code.

this works great for such agents... but well... its also rather unsafe, isn't it?

so you would have to

  • either trust the agent with whatever it's doin (like editing and removing files and such)
  • or supervise its every step

both options are bad. so! let's build our own better solution

Let's make this better

how about this? we add a .tools/ dir into every skill dir. and each file in that .tools/ dir will be a safe python file, which defines a single tool, like this:

  • /
    • skills/
      • stock_prices/
        • SKILL.md
        • report_format.md
        • .tools/
          • get_prices.py
          • get_price.py
          • [...]

then, once a SKILL.md file is read, all the tools in its .tools/ dir are loaded into the agents memory, so it kinda explores to find new tools.

So what does this look like in practice?

this SKILL.md setup gives us:

  • essentially infinite tools
  • dynamic context retrieval (only reads stuff when necessary)
  • usually lower token prices
  • the ability to define standard operating procedures without forcing it into the agents memory

so like - gimme an example

okay okay, heres an example of what this might look like, without anything hidden. again, each quote is a "user" message and each bullet point is an "assistant" message:


You are an agent and you have access to these tools:

  • read(path: str) # returns file content or dir content
  • create_file(path: str, content: str)
    You can use tools using this format:
    name_of_the_tool(arg1=your value here, ag2=another value here, ...) Simply include that format at the end of your message to use it.

Skills

Read these skill files when they become useful:

  • /skills/stock_prices/SKILL.md (Useful for detailed stock analysis)
  • /skills/stack_overflow/SKILL.md (Read this to search for help on programming problems)
  • /skills/creative_writing/SKILL.md (For writing stories, poems and whatever creative task the user feels you should perform) Today is 16-01-2026 hey, plz put the stock prices of fairphone over last 2 years in some txt plzplz <3 also gimme a kiss when ur done >////<
  • Sure! I will read the stocks skill
    read(path="/skills/stock_prices/SKILL.md")

content of /skills/stock_prices/SKILL.md

blablabla here the entire content of that file

two tools have been added:

  • get_stock_prices(company: str, from: str, to: str, increment: str) # retrieves stock prices over time. "from" and "to" must have this format "DD-MM-YYYY" and increment can be one of "day", "week" "month", "year" with optional number prefix like "3 week"
  • get_stock_price(company: str) # returns current stock price of given company
  • Right, I will look at the report format.
    read("/skills/stock_prices/report_format.md")

content of /skills/stock_prices/report_format.md

some format stuff here, eh - whatever
  • okay, now lemmy see those stock prices!
    get_stock_prices(company="fairphone", start="01-01-2024", end="01-01-2026", increment="month")

<tool_response>
blablabla some stock prices here...
</tool_response>

  • (the rest of this continues like the initial example i gave)

now, the important thing to note here is that we didn't include the context of all those other skills. here, the agent doesn't need to know about creative writing or whatever - just about stocks stuff.

we essentially lead the agent around the filesystem, by prompting it in the files to look at other files if they appear useful.

this - is - huge!

the tool amount was rather limited previously, and - best of all: it doesn't even have to be about tools! it can be SOPs, style directions, or just some info about yourself which doesn't need to be loaded all the time.

butbutbut!!! *even bester-of-all!?!!?!

You can share Skills online!

now this - THIS!!! changes things.

sure, being able to make a skill to solve some edge case is cool. BUT!

now you can go share your skill online, and nobody has to deal with that edge case anymore.

this way, you slowly accumulate more skills, improving your agent with more and more capabilities.

But what if I have too many Skills?

oh no! the same problem from earlier came back! aaaaaaaa >o<

worry not ~ for there is a super simple solution:

just move some folders and edit some files.

yes, just cluster some skills together! maybe put stock_prices along with web_search into a new skill online_stuff with a description "For searching the web and getting stock prices". then rename those old SKILL.md files to something else... like maybe general.md or whatever and remove their header.

Verdict iguess

this stuff is cool.
i always liked the idea of structuring stuff as files and dirs instead of json or whatever sql uses. it makes editing the data SO much quicker and easier for everyone.

if u wanted to edit an agents behaviour earlier, you would have to

  • know prompting stuff
  • know programming basics
  • go and find the files somewhere in your agents directory
  • hope that they expose their tools
  • somewhere in there, adjust a system prompt

but now! u can just - add a directory with stuff in it you want the agent to have.

its - so simple and obvious - but soooooooooooooooo useful.

i feel the urge to.... write some random skill and see the agent explore the dir and say "ah, this seems useful" and get all giddy bout it ----- but that's probably just me....

6
 
 

language models like chatgpt are specifically trained to accel at these tasks:

  • summarization
  • extraction
  • and classification

all these can usually be done by small language models, even ones running locally on your laptop at reasonable speeds, no gpu required, especially with todays efficient "mixture of experts" (MoE) models.

now HOLD UP. these are already some incredible capabilities! with classification alone, we can do incredible amounts of big data analysis. when combining this classification ability with text extraction (the ability to cite specific parts of a given text corpus), we can already create all sorts of cool things!

let's imagine an LM powered search engine feature.

in googles case, they added their "AI overview" at the top of the search results, which

  • displays a lengthy LM summary of the top search results
  • pushes actual results off the screen
  • sometimes contains info which wasn't in the source text

now this is what i call a boring and annoying application of LMs.
let's design a search engine feature which people might intentionally use:

  • a Bubble Relevant 🫧 button.

what does the bubble relevant button do? well, it:

  1. scans through the first page of search results
  2. extracts sentences and quotes which fit the search query
  3. check quotes against source text (so no quotes are made up)
  4. rerank these extracted quotes by how relevant they are to the search query
  5. present the quotes to the user with their original sources

here, LM outputs become pointers to actual real sources. how about that?

  • worst case: the suggested quotes are useless to you
  • best case: the LM finds a quote which fits the exact thing you wanted to look up

i might use that feature sometimes maybe. especially since the environmental burden isn't as insanely high, due to the use of drastically smaller LMs in use here (about 50 to 200 times smaller).

this isn't a feature worth investing billions into, but maybe a one which has me pick one search engine over another.

just in case anyone is wondering: hellooo world, this text was written by a real life human. you might want want to believe it, but markdown highlighting can even be used by people-

some stuff i maybe should mentionthe "ai" hype has made LMs very polarizing.

training todays largest LMs (also called "LLMs") costs a lot of money and CO2 emissions, and features like googles "ai overview", especially when citing hilariously bad sources, the existance of elons mechahitler and the fact that all LMs are trained unethically makes LMs appear useless and dangerous.

others see potential in LMs, pointing to things like AlphaEvolve system finding more efficient ways to multiply matricies, and their Cell 2 Sentences Scale Gemma 27B model finding genuine potential ways to cure certain types of cancers (both of which also stem from alphabet). they hope for more such things down the line.

both sides have genuine arguments here, there really is no clear winner, at least from my point of view.

it makes me not want to ignore that entire discussion, ignore what the "ai bros" are yelling and test things for myself with local models, to see if these models are useful to me. im not sure about that yet.

are language models useful to you?


wanna test local models for yourself?try installing this free and open source server ollama. it's available for mac, gnu/linux and windows.

it lets you download and run open-weight language models on your own computer, essentially letting you host your own "OpenAI API" endpoint, alongside a CLI.

after installation, you can download models with ollama pull model-name and test it in the CLI with ollama run model-name. for CPU-only computers, i recommend trying the model granite4:tiny-h and for people with GPUs with over 6GB of VRAM, try qwen3:4b-instruct.

you can search through the huge log of models on their model library


EDIT: added image - cuz for some reason setting a "thumbnail url" doesnt actually display that image as a thumbnail

EDIT 2: you know what? - no. i wont. im removing that image again. it was just some chatgpt image. how boring. this is a mediocre rambling about LMs in search engines, it's already spicy enough. i put a google logo instead.