all 28 comments

sorted by: hot top controversial new old
[–] 68 points 7 months ago (2 children)

So the innovation in Claude was to write 95% of the prompt for the user and make you use like 10k tokens

  • source
  • hideshow 2 child comments
  • [–] 11 points 7 months ago (1 child)

    The problem is that words don't have meaning in the genAI field. Everything is an agent now. So it's difficult and confusing to compare strategies and performance.

    Claude Code is a pretty solid harness. And a harness is indeed just prompts and tools.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 149 points 7 months ago (5 children)
  • [–] 85 points 7 months ago (3 children)

    I’ve literally seen someone include “Don’t hallucinate” in an agent’s instructions.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 39 points 7 months ago (2 children)

    Asking Claude to not hallucinate is like telling a person to not breathe. it's gonna happen, and happen conistently.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 49 points 7 months ago
    [–] 58 points 7 months ago (2 children)

    Just write good code. It’s as simple as that, right?

  • source
  • hideshow 2 child comments
  • [–] 72 points 7 months ago*
    [–] 20 points 7 months ago (1 child)

    "Don't put in any of the Top 10 vulnerabilities. But if you put any from the 11th place and down, that's okay, I don't even know what those are."

    (Also, getting flashbacks from Shadiversity plugging "ugly art" and "bad anatomy" in the negative prompt as he was no doubt silently wondering why it didn't work)

  • source
  • hideshow 1 child comment
  • [–] 36 points 7 months ago

    They are spending thousands of dollars in tokens and write the most complicated prompts in order to avoid writing good specifications.

  • source
  • [–] 13 points 7 months ago

    Oh boy, if there's an OWASP top 11th vulnerability, we're cooked /j

  • source
  • [–] 11 points 7 months ago

    That's a what if, just in case it gains sentience. Gotta make sure we get good code even as it enslaves or extinguishes us.

  • source
  • [–] 11 points 7 months ago

    "Claude, add to this prompt all the instructions necessary to stop you from making mistakes or writing insecure code"

  • source
  • [–] 9 points 7 months ago

    I sort of get the need to do this, but it's so silly to be. Reminds me of how giving Stable Diffusion negative prompts for "bad" and "low quality" would give you better results.

  • source
  • [–] 9 points 7 months ago

    Programming is the use of logic and reasoning. There will always be a use for that. Even without tech.

  • source
  • [–] 13 points 7 months ago (1 child)

    That may actually work a little?

    I mean, it scraped the entirety of StackOverflow. If someone answered with insecure code, it's statistically likely people mentioned it in the replies meaning the token "This is insecure" (or similar) should be close to (known!!) insecure code.

  • source
  • hideshow 1 child comment
  • [–] 17 points 7 months ago

    I was part of that OWASP Application Security Verification Standards compliance at my work. At a high level, you choose a compliance level that suitable for the environment you expect your app to be deployed in, and then there's a hundred pages of 'boxes to tick'. (Download here.)

    Some of them are literal 'boxes to tick' - do you do logging in the proscribed way? - but a lot of it is:

    • do you follow the standard industry protocols for doing this thing?
    • can you prove that you do so, and have protocols in place to keep it that way?

    Not many of them are difficult, but there's a lot of them. I'd say that's typical of security hardening; the difficulty is in the number of things to keep track of, not really any individual thing.

    As regards the 'have you used this thing in the correct, secure way?', I'd point my finger at something like Bouncy Castle as a troublemaker, although it's far from alone. It's the Java standard crypto library, so you think there would be a lot of examples showing the correct way to use it, and make sure that you're aware of any gotchas? Hah hah fat chance. Stack Overflow has a lot of examples, a lot of them are bad, and a lot of them might have been okay once but are very outdated. I would prefer one absolutely correct example than a hundred examples have argued over, especially people that don't necessarily know any better. And it's easy to be 'convincing but wrong', and LLMs are really bad in that case. So 'ticking the box' to say that you're using it correctly is extremely difficult.

    I see the Claude prompt is 'OWASP top 10', not 'the full OWASP compliance doc', which would probably set all your tokens on fire. But it's what's needed - the most slender crack in security can be enough to render everything useless.

  • source
  • parent
  • [–] 4 points 7 months ago (3 children)

    So I don't know if all the other replies are pretending to be stupid, but the shown prompt is not stupid.

    If you include stuff like that section in your prompt, then it has been shown that the AI will be more likely to output secure code. Hence of course the section should be included in the prompt.

    If it looks stupid but it works, then it is not stupid.

  • source
  • hideshow 3 child comments
  • [–] 17 points 7 months ago (2 children)

    Firstly, it can work and still be stupid.
    Secondly, since the chat bot is more likely but not certain to write secure, bug-free code, it does not in fact work and is therefore, by your own reasoning, stupid.
    But so is asking a chat bot for code to begin with, so there wasn't ever really a way around that.

  • source
  • parent
  • hideshow 2 child comments
  • [–] -4 points 7 months ago (1 child)

    since the chat bot is more likely but not certain to write secure, bug-free code, it does not in fact work

    Humans are not certain to write secure, bug-free code. So human code is useless, by the very same metric?

    What kind of "logic" is that?

  • source
  • parent
  • hideshow 1 child comment