Running on your hardware is always going to be less efficient than running in a data center. You can run small models on green data centers - best of both worlds (unless you need privacy then run local).

Ever since I switched to Cortecs I'm spending like 20 cents / month for my LLM usage cuz I mostly use small models and try using them responsibly.