-9
Best resources to learn GenAI workflows for the job hunt? (lemmy.zip)
submitted 2 months ago by to c/ask_experienced_devs@programming.dev
16 comments fedilink hide all child comments
 

I'm an experienced SSE and I'm about to hunt for a new job for the first time in a few years. I've so far been able to avoid most AI use, with the exception of testing out some code completion with copilot a couple years ago.

Basically every job listing I'm seeing expects some level of AI adoption ("we'll set you up with either Claude or Copilot, so much choice!") so I figured it's time.

What's the best way to get an intro to these tools? I'm curious how people are actually using them in their workflows successfully. I want to give it a fair shake and form my first-hand opinion on these things as tools.

EDIT: I'd also like to hear from those downvoting this post.

you are viewing a single comment's thread
view the rest of the comments
[–] [S] 3 points 2 months ago (5 children)

Thanks for the response.

llama-server (which is one of the tools that comes with llama.cpp) includes a web UI. You can also use llama-cli on the command-line (but that gets old pretty quick).

Will check it out. Also came across opencode which is like a TUI for dozens of models, which seems interesting.

debugging assistance

Are you just using it like a chatbot in this situation? Like as a rubber ducky like you mentioned? And just paste code into the prompt?

here’s a screenshot of how it looks now

How are you giving it things like binary files? Does it actually understand them? Is that a feature of only certain models?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 4 points 2 months ago (4 children)

    Are you just using it like a chatbot in this situation?

    For short snippets, I usually just paste the code in (e.g. using triple backticks to indicate a codeblock with Markdown if I want distinguish it clearly from other text in my prompt). It's also possible to drag-and-drop files into the web UI that ships with llama-server, and I also have some of my own custom tooling.

    How are you giving it things like binary files? Does it actually understand them? Is that a feature of only certain models?

    Vision Language Models can work with images. Both Qwen and Gemma can do it. In the web UI, you can just drag and drop images in and they'll be added as attachments that get processed along with your prompt text. There's limits to what they can understand -- I recommend experimenting with them to get a sense of it -- but for things like OCR, they just do it...

    I've also implemented handling for passing images into models via code if you're curious about that as well -- basically you just add the file as base64 encoded data in the JSON structure that you POST to the server.

    llama-server's web UI can process PDFs too, but I'm not sure on what it does for that exactly under the hood.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] [S] 2 points 2 months ago (3 children)

    Neat that markdown helps.

    I also have some of my own custom tooling.

    Not asking you to share code or anything, but would you mind explaining some thing(s) you've created to help you?

    Thanks again for the responses. They've been super helpful.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 3 points 2 months ago (2 children)

    OK, so there's a few things I've built. First, I want to clarify that there's the concept of a "tool call" in LLMs. i.e. the LLM generates a specific sort of response which is intended to be interpreted as a request to run a function (either in llama-server [if you enable that] or in your client). The LLM needs to be provided with a list of the tools that are supported (including the name of the tool, the description, and arguments) for it to know that it can call them.

    Basically, you send a prompt, the LLM "thinks" for a bit, it spits out a tool call as the response, the client code sees the tool call, runs the appropriate function, then submits the result back as another prompt to the LLM and it continues on generating a response.

    I've built custom tools (in my own custom client) for accessing limited parts of my file system in a particular read only fashion. (I do not trust these things to write to my disk autonomously...)

    For example, I have a find_files tool that LLMs can call with a project name (which I need to provide -- restricted to a few specific locations based on a config file) and which basically runs a listdir and returns the results as JSON to the LLM. I also have a read_source_code tool that allows specifying a project and a file and it will return the text of that file.

    This is technically redundant with features that are available out-of-the-box in llama-server but I'd rather run llama-server locked down in a container and provide it access with no more than I want it to have from code I control.

    The custom client I wrote includes my own web UI (which is simpler than the one that comes out of the box with llama-server but is tailored to my specific tastes), and the code to run those custom tool calls. The first pass was also my first serious experiment with vibe coding; I wrote a very simple skeleton by hand and then iterated on it allowing the LLM to write most of the code so I could see what that process was like and how it would go wrong. As you might expect, that turned into a complete mess and I scrapped and rewrote pretty much the whole whole thing after a few weeks, but it was quite educational!

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •