[–] 1 point 1 day ago

That is factually incorrect.

Help page from Kagi (I'd understand if you don't trust it), fit for non-techie consumption. https://help.kagi.com/kagi/privacy/how-does-privacy-pass-work.html

So here's the RFC, which defined the protocol in the first place (Kagi didn't invent it, just implements it) https://www.rfc-editor.org/info/rfc9576/

As such they have absolutely no way of 'deciding' to look up a searchers account based on the token used.

That ofcourse does not reduce other ways anyone can track you like fingerprinting.

  • source
  • parent
  • context
  • [–] 2 points 1 week ago* (1 child)

    To expand on this, using tokens as a unit of billing is quite reasonable as it is also the unit of (inference) cost for LLMs.

    The inference is essentially nothing but a loop of "Hey if the first X tokens are this, what would token X+1 most likely be?" until the LLM says its done. Every single loop is a ton of math which, on ridiculoudly expensive hardware, uses electricity and produces heat. So its a combination of those costs, and then we haven't even taken into account the cost to train the model.

  • source
  • parent
  • context
  • [–] 1 point 2 weeks ago

    We have a few dgx sparks in the office that are gathering dust right now because nobody wants to bother with them (and because IT doesn't allow them on the network because Linux is scary so they're a pain to get set up).

    Matching the level of productivity of frontier models on the local models is possible to a limited degree (depending on the exact task), but takes getting used to. For the local models you have to lean into the fact that tokens are 'free' so its fine to burn a lot of them. (Whereas with frontier models the recommendation is to take a higher model for complex tasks because a lower would take more tries to get it right)

  • source
  • parent
  • context
  • view more: next ›