Why does everything need to be done with an LLM?
post
Is naming things hard? Sometimes absolutely! Should you off load that cognitive load to AI? Only if you want to get worse at writing and maintaining code.
Naturally, the LLM can generate interpretations and probabilities which are not correct. Yet, we believe that an LLM is a good-enough measure of semantic ambiguity for our needs.
You’re asking an LLM to invent a probability distribution over meanings. Embedding models are explicitly trained to represent semantic similarity geometrically. Why not use the right tool for the job over "slop"?
- Download you an embedding model — you can run it on garbage, or pay for an API if you want.
- Embed the identifier
- Embed the symbol’s implementation summary (or docstring)
- cosine(name_embedding, description_embedding)
- Have LLM suggest alternatives and repeat
- Rank names by semantic similarity.
ETA:
I present: https://gitlab.com/MagickShel/namer
It doesn't generate the list of names to test, though. I leave that to the imagination of the user.
ETAA:
There is a class of naming it doesn't work great on. I probably overstepped and was a bit of an ass. This just gave me the inspiration to do basically the thing you did here, the way I have told myself for months it should be done.
Also, there is a class of naming - jargon and acronyms - where your approach works better. I still prefer something more deterministic, but credit where credit is due.
Good lord, let's waste more energy and destroy more communities because now naming things is too hard? I hope this is a joke.
On some days I wish back the ASM times. 3 chars that's enough for a variable name. All those modern 4 char names ... Pah .... /S
Yet, we believe that an LLM is a good-enough measure of semantic ambiguity for our needs.
AI and believers, name a more iconic duo
Cool idea but Im pretty sure you can extremely optimize the ambiguity calculation by just vectorizing the terms and looking them up in a vector database.
all 6 comments