31
How Claude marks AI-generated content: (1) watermarks embedded in text, and (2) signed provenance metadata attached to files. (support.claude.com)
submitted 3 months ago by to c/technology@lemmy.world
10 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 8 points 3 months ago (4 children)

The text version must necessarily constrain the output, which will impact readability and semantic meaning, whatever they say.

  • permalink
  • fedilink
  • source
  • hideshow 4 child comments
    •
  • [–] 13 points 3 months ago* (2 children)

    Not necessarily, there are valid "characters" that are not rendered in most text parsers, or are rendered as whitespace. A great example of this is the byte-order-mark (BOM). You can embed a BOM code in the whitespaces of a text string and it looks exactly the same to a human as one without.

    I assume that's what they're talking about here

    Here's an example of a steganography encoding technique using this method. https://330k.github.io/misc_tools/unicode_steganography.html

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 3 points 3 months ago* (1 child)

    Maybe, if it's created after the text is generated. Keep in mind there really is no white space to an LLM. There's a lot of holes here. If it isn't embedded within the text itself, it is trivially easy to strip such markings. If it is, it at least requires a human to paraphrase.

    On the other hand, if that's all it is, it would be easily detectable to disallow AI from editing human-written code while allowing it to revise its own output.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 3 points 3 months ago*

    A model could hypothetically be trained to insert zero-width characters (I doubt any have though). But any other layer could also very trivially insert these codes. The inference engine could be designed to delay output streaming by however many tokens is required to embed their coding and ninja-insert them during the decode stream. A proxy between the inference engine could insert them. A harness could insert them. Hell, even the rendering javascript frontend in your browser could insert them.

    Either the inference engine or proxy would be the prime target if they want to enable this on api responses as well as copy/paste from a chat interface. They could also do a combination of the above depending on final output mode.

    You are definitely correct though that it'd be trivial to detect and strip by someone aware of it.

  • permalink
  • fedilink
  • source
  • parent