The web-ui is the thing you type on :) You host it yourself. llama.cpp is the back end runner…it just so happens that it now has a in-built front end too. You can see it below
(Most things run llama.cpp underneath btw and then slap something else on top)
Probably you’re going be better served with Jan.ai until your up on your feet; it’s a little friendlier / less cryptic when starting out. Jan has both llama.cpp AND a different web-ui and stuff on top. All of it always on your machine.
The web-ui is the thing you type on :) You host it yourself. llama.cpp is the back end runner…it just so happens that it now has a in-built front end too. You can see it below
https://github.com/ggml-org/llama.cpp/discussions/16938
(Most things run llama.cpp underneath btw and then slap something else on top)
Probably you’re going be better served with Jan.ai until your up on your feet; it’s a little friendlier / less cryptic when starting out. Jan has both llama.cpp AND a different web-ui and stuff on top. All of it always on your machine.
https://www.jan.ai/docs/desktop/quickstart
As I recall, Jan has has a few one-touch install models (older but pretty decent ones; worth trying when just starting out)
Are you running llama.cpp via Docker or you compiled on your machine?
I run it bare metal. No docker.
cool, i’m doing the same. i thought you had to run it via Docker, but I’m running via powershell and the web UI.
Got it. I’ve built simple webpages and opened those files in my browser to preview them, but didn’t know those would connect to anything.
I’ll check out Jan and Llama and see which works for me.
Thanks!!