40
Anyone messed with local cli LLMs? (sh.itjust.works)
submitted 1 year ago by to c/linux@lemmy.world
22 comments fedilink hide all child comments
 

Let me preface by saying I despise corpo llm use and slop creation. I hate it.

However, it does seem like it could be an interesting helpful tool if ran locally in the cli. I've seen quite a few people doing this. Again, it personally makes me feel like a lazy asshole when I use it, but its not much different from web searching commands every minute (other than that the data used in training it is obtained by pure theft).

Have any of you tried this out?

you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 1 year ago* (2 children)

If by "CLI"", you just mean "terminal", I've used ellama in emacs as a frontend to ollama and llama.cpp. Emacs, can run on a terminal, and that's how I use it.

If you specifically want "CLI", I'm sure that there are CLI clients out there. Be almost zero functionality, though.

Usually a local LLM server, what does the actual computation, is a faceless daemon, has clients talk to it over HTTP.

EDIT: llama-cli can run on the commandline for a single command and does the computation itself. It'll probably have a lot of overhead, though, if you're running a bunch of queries in a row

the time to load a model is significant.

  • permalink
  • fedilink
  • source
  • hideshow 2 child comments
    •
  • [–] 2 points 1 year ago (1 child)

    What's the difference in a command line interface and a terminal?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 1 year ago

    If you're being rigorous, a "CLI" app is a program that one interacts with entirely from a shell command line. One types the command and any options in (normally) a single line in bash or similar. One hits enter, the program runs, and then terminates.

    On a Linux system, a common example would be ls.

    Some terminal programs, often those that use the curses/ncurses library, are run, but then one can also interact with them in other ways. This broader class of programs is often called something like "terminal-based" "console-based", or "text-based`, and called "TUI" programs. One might press keys to interact with them while they run, but it wouldn't necessarily be at a command line. They might have menu-based interfaces, or use various other interfaces.

    On a Linux system, some common examples might be nano, mc, nmtui or top.

    nmtui and nmcli are actually a good example of the split. nmcli is a client for Network Manager that takes some parameters, runs, prints some output, and terminates. nmtui runs in a terminal as well, but one uses it theough a series of menus.

  • permalink
  • fedilink
  • source
  • parent