7
What's the next gen AI tech you're using or experimenting with? (lemmy.world)
submitted 6 months ago by to c/fosai@lemmy.world
24 comments fedilink hide all child comments
 

LLM’s are not the end all be all. What other AI tech are you all using? Something generative? Something else?

you are viewing a single comment's thread
view the rest of the comments
[–] [S] 2 points 6 months ago (7 children)

I want to but I don’t think I have the hardware to support it. Need at least a decent GPU, right?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 7 child comments
    •
  • [–] 2 points 6 months ago (5 children)

    For playing around, no. My GPU is 9 years old and has 8GB VRAM. I generating my 384x512 profile picture with KoboldCPP (Vulkan) and Z-Image-Turbo model locally.

    I just regenerated my profile picture with KoboldCPP a bit differently. I used some clearer LoRA configurations than I did initially. For Z-Image-Turbo it’s CFG 4 and for Flux2 Klein 4b CFG 1, both used Euler. For Anime V1 LoRA I started with anime illustrious, and for Koni Animestyle LoRA with anime_style, . :

    384×512 & Z-Image-Turbo & Anime V1 & 8 steps (2min 42s):

    384×512 & Z-Image-Turbo × Anime V1 & 25 steps (8min 16s):

    384x512 × Flux2 Klein 4b × Koni Animestyle × 4 steps (22s):

    384x512 × Flux2 Klein 4b × Koni Animestyle × 8 steps (35s):

    384x512 × Flux2 Klein 4b × Koni Animestyle × 25 steps (1min 28s):

    Upscaling it with RealESRGAN x4plus anime 6B upscaler only takes 10 seconds longer with this dimension.

    KoboldCPP outputs how long generation takes like:

    [11:09:15] Generating Image (8 steps)
  |==================================================| 480/480 - 0.00MB/s
  |==================================================| 480/480 - 0.00MB/s
  |==================================================| 480/480 - 810.94MB/s
  |==================================================| 480/480 - 0.00MB/s
  |==================================================| 8/8 - 19.64s/it
[11:11:57] Generating Media Complete
  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] [S] 2 points 6 months ago (4 children)

    Those are pretty good results! Unfortunately, I don’t have a GPU at all yet. Any recommendations on something capable yet affordable? What would you buy next?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 3 points 6 months ago* (last edited 6 months ago) (3 children)

    I stick to what I have as long as it continues to work for me, therefore I haven’t looked into GPUs the last years. New Radeon RX 580 is still at 200€, I paid 300€ 8 years ago. Looks like RX 580 2048SP 8GB is actually one of the cheapest one can get currently, but not entirely sure. (My GPU has 2304 shading units.) It’s so old that ROCm support was dropped and you really need software that can do AI via Vulkan

    edit: looks like Intel Arc A750 is 5 years younger, has better performance and more memory bandwidth

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] [S] 1 point 6 months ago (2 children)

    Thanks! I’ll make sure to include Vulcan in criteria for a new machine and I’ll look at intel options. Only been looking at NVIDIA. Even 4GB VRAM will be a huge upgrade for me.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 6 months ago (1 child)

    All modern GPU drivers support Vulkan, at least on Linux. I don’t know how bad Vulkan does in comparison to CUDA, since I’ve never used that stuff. The bigger issue is software-side support. Ollama, llama-cpp and KoboldCPP all support Vulkan by now. ComfyUI doesn’t seem to support it.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •