no comments (yet)

sorted by: hot top controversial new old
there doesn't seem to be anything here
15 comments in c/technology@lemmy.world view that post
[–] [S] 73 points 11 hours ago

Two million bait secrets went into the AI supply chain, and some came back as attempted logins. Apparently »zero data retention« sometimes means »we retain zero responsibility.« If your confidential prompt gets passed through five middlemen chasing the cheapest GPU, privacy isn’t a setting. It’s a game of telephone with burglars.

  • source
  • [–] 33 points 9 hours ago

    Surprise surprise

    Never feed any important data to the machine

  • source
  • [–] 27 points 8 hours ago (2 children)

    Those are some huge canaries

  • source
  • hideshow 2 child comments
  • [–] 16 points 9 hours ago (1 child)

    Once again, money trumps all other considerations. Privacy, security, it doesn't matter. I wonder what will be next.

  • source
  • hideshow 1 child comment
  • [–] 10 points 8 hours ago

    What will the oligarchs take after privacy and security? I can't predict the order, but some of the highlights: International Criminal Court, International Court of Justice, Geneva Convention, life, liberty, pursuit of happiness, égalité, fraternité, sovereignty, dignity, the social contract, overtime pay, and unions.

  • source
  • parent
  • [–] 16 points 8 hours ago
    [–] 7 points 2 hours ago (4 children)

    Moore said that the most effective means for preventing leakage is to host models using on-premises servers. That option is expensive, though. For those who can’t afford a self-hosted system, the next best option, Moore said, is to receive service directly from a first-tier provider that provides “one contract and a privacy policy to read.” Moore said that even then, there’s no guarantee no data will leak, but that the measure minimizes the chances. AI users should also avoid accepting default provider or reseller pools and instead pick upstream providers themselves.

    I'm kind of confused about what the researchers were using? They weren't going to chatgpt.com and seeing the info stolen, they were doing it a more convoluted way?

  • source
  • hideshow 4 child comments
  • [–] 7 points 2 hours ago (3 children)

    A routing service is in the middle between you and the AI. You send your prompt to the router. It relays that to ChatGPT, Gemini, Grok, Qwen, etc.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 1 point 1 hour ago* (2 children)

    But when would somebody be using that? What is the context, as someone who has never heard of this.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 1 hour ago*

    There are several public open weights models that anyone can host. There are a large number of companies offering e.g. DeepSeek or GLM. A router can automatically pick the cheapest offer and detects and works around outages of individual companies.

  • source
  • parent
  • [–] 1 point 1 hour ago

    Opencode/zen go is the best monthly subscription plan. Also has decent free tier. Openrouter is similar. Its 10-30+ different model providers under one api including some closed US ones. It's about juggling usage limits and routing simple requests to simpler models to preserve better model usage limits.

  • source
  • parent
  • [–] 1 point 1 hour ago

    How else can anyone make money from AI, though? :P

    Very big deal that privacy policies mean nothing, though.

  • source
  • [–] 1 point 2 hours ago
    12 comments in c/technology@lemmy.zip view that post
    [–] 31 points 15 hours ago (1 child)

    The clearest example involves a Chinese AI lab. This lab’s systems fetched the secret URLs from more than half of the prompts that we sent to its models between July and October, including roughly two-thirds of the requests marked as “no data collection” or Zero Data Retention. The scraper typically showed up two or three days after the request.

    This wasn’t limited to requests sent to the lab directly. Several U.S. hosting providers resell this lab’s models, and prompts sent to those providers triggered the exact same scraper – the prompts were flowing back to the lab. A customer who picked a U.S. provider specifically to keep their data in the U.S. didn’t get what they paid for. Later on, a git client on a different cloud started trying to clone the private repositories referenced in our prompts, but only for prompts that used this lab’s models.

    The large U.S. providers have their own issues. One continued to crawl a secret URL more than two months after we used it in a single inference call to their search endpoint. Another runs a scraper that comes back for our URLs days after each request, regardless of which router we used to reach it.

    The strangest incident involved a company chat. Twenty-four days after we sent a prompt to one Chinese provider, the secret link from that prompt was posted into a Feishu chat channel (Feishu is known as Lark outside of China). Within two minutes, the link was opened from three desktops and three phones in Beijing and Shanghai. Our disclosure to this provider only included the first eight characters of that link, so whoever shared it was reading from a stored copy of the full prompt, more than three weeks after the request was sent to a U.S.-based provider with the ZDR flag.

    I wish they had named these providers. I know they aren't supposed to, but still.

  • source
  • hideshow 1 child comment
  • [–] 27 points 17 hours ago (1 child)

    I feel like if you ever put any credentials or PII in any prompt that isn’t staying local, you should just immediately consider it compromised.

  • source
  • hideshow 1 child comment
  • [–] 18 points 14 hours ago

    Most people just have no idea what they are doing. It's incredibly common for workers to put sensitive private information into prompts or upload documents, without even thinking about where they might end up and what's happening with the data. You can not be certain that e.g. a government employee or someone working in healthcare, at an insurance company, etc. isn't forwarding your private data to some AI service they don't understand.

  • source
  • parent
  • [–] 25 points 20 hours ago

    Shocking but not surprising.

  • source
  • [–] 6 points 16 hours ago

    The worst part is that some of the patterns match what you'd expect an AI breakout to be.

    The attacker is using the AWS keys to spin up more AI hosting, in theory to spread further

  • source
  • [–] 10 points 15 hours ago

    Anything going off to someone elses server is not in your control any more.

  • source
  • [–] 10 points 16 hours ago (3 children)

    Lol I run the ai horde which is just Foss and crowdsourced and we're upfront we cannot guarantee privacy of prompts, but we're still better than this shite.

  • source
  • hideshow 3 child comments
  • [–] 8 points 15 hours ago* (2 children)

    Thing about hosting AI Horde is you don't know what unspeakable prompt your machine will process. Or from where. While I don't mind this personally (go to town with depraved prompts, it's not my business), I'm somewhat scared of legal repercussions, getting mixed up in a honeypot, or something like that.

    That, and I can't afford to run my GPU like that during the RAMpocalypse :(

    I do agree that it's better though.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 6 points 7 hours ago* (1 child)

    The AI Horde does allow for running in non-NSFW mode, and the NSFW mode itself uses both a blocklist of common keywords and a image recognition based approach to prevent generation of CSAM. IIRC, it is also the basis for blocking out CSAM in lemmy too - there was a short and very ugly phase where federation of illegal stuff made admin lives not fun. I am a daily user of lemmy since the Reddit API exodus, and haven't yet stumbled across a single instance of CSAM while using accounts on 3 different instances, so it seems to work pretty fine.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 3 points 5 hours ago*

    I was thinking of the text generation part of the Horde specifically, which doesn’t have such a toggle/filter AFAIK and tends to get used for NSFW in general.

    Again, not that I personally mind others’ wild fanfiction and RP.

  • source
  • parent
  • [–] 1 point 26 minutes ago

    Have you read about EvilTokens?

    Sekoia TDR uncovers EvilTokens, a turnkey Microsoft device code phishing kit sold as Phishing-as-a-Service.

    Device code phishing tricks users into entering a code on the legitimate Microsoft login page, granting the attacker's device access.

    Harvested access and refresh tokens can be exchanged for a Primary Refresh Token, enabling 90-day silent access bypassing MFA.

    The backend exposes REST endpoints for token weaponisation, cookie generation, Graph reconnaissance and Azure enumeration.

    Campaigns deliver phishing pages through PDF, HTML, XLSX, SVG and DOCX lures, hitting organisations across every continent.

    Distinctive URL patterns, Cloudflare Workers domains and the X-Antibot-Token header give defenders reliable tracking heuristics.

    New widespread EvilTokens kit: device code phishing as-a-service

  • source
  • 4 comments in c/fuck_ai@lemmy.world view that post
    [–] 10 points 5 hours ago
    [–] 6 points 4 hours ago (2 children)

    Ars is trustworthy, but where is the original research paper/source link? Did I miss it in the article?

  • source
  • hideshow 2 child comments
  • [–] 14 points 4 hours ago* (1 child)

    “We sent nearly two million AI prompts containing unique, trackable secrets to AI providers, then watched where those secrets surfaced,” Moore, the chief research officer of security firm Dragos, wrote Thursday. “What we found revealed security breaches, prompt theft, and widespread failures to protect customer data.”

    https://www.dragos.com/blog/ai-supply-chain-security-canary-tokens

  • source
  • parent
  • hideshow 1 child comment
  • 2 comments in c/pulse_of_truth@infosec.pub view that post
    [–] 13 points 19 hours ago

    I wish I could claim to be surprised, but I'm really, really not.

  • source
  • [–] 8 points 17 hours ago

    Respect my privacy. Make no mistakes!

  • source