[–] 2 points 3 months ago

It should theoretically be impossible with the way you set it up.

You explicitly, opt in, expose which parameters are available to the LLM, it cant invoke any params beyond that.

And you can set params to hard values that arent exposed.

Finally, you can use the enum config to further specify a whitelist of valid values for a param, anything else gets rejected.

This pretty much locks most tools down tight.

Though, wouldn't hurt to add integration tests covering scenarios like this explicitly.

  • source
  • parent
  • context
  • [–] 3 points 3 months ago (1 child)

    That's a good idea. Be careful of the more advanced models finding ways to abuse/circumvent it though, even with only a limited set of parameters. Eg. A lot of git commands can be tricked into dropping into less which can drop into a full shell.

    They know all the gtfobins.org tricks.

  • source
  • parent
  • context
  • [–] 9 points 3 months ago (1 child)

    This is why all of the claims that "Claude broke containment" are pure hype. They know how to truly design a secure box. They just don't want to and put out headlines to boast its hacking skills.

    You want it secure for experimental model security testing?

    • No VMs
    • Physical hardware
    • Physically disconnected from the internet

    For the rest of us who ain't doing that shit, a properly segregated dev network with firewalled access will suffice.

  • source
  • [–] 5 points 3 months ago* (last edited 3 months ago)

    sure. it can be safe on the host machine too, if you remove its ability to manipulate files outside of those you want it to. the reason it can run bash is because most people don't bother to change the very permissive defaults their harness comes with.

    personally, I use Hermes, so the default is it asks permission for every single command it runs. if I didn't want that, I have a couple ways to to prevent the agent from removing files like this. I can do it with Python like so:

    if "rm -rf" in command.lower():  
        return DENY  
    

    or via regex like so:

    \b(rm)\s+-rf\b, or \b(rm)\s+(-[A-Za-z]*r[A-Za-z]*f|-rf|-fr)\b to catch common variants. or you can just use a shell command filter to block all shell commands. you can also make a whitelist and add the commands that you want the agent to be able to run without oversight to that.

  • source
  • parent
  • context
  • [–] 14 points 3 months ago (2 children)

    This is pretty much entirely why I built https://github.com/SteffenBlake/cli-mcp-mapper

    I map bespoke CLI commands, with parameters, to MCP tools, and then 100% just turn off the ability to invoke the CLI itself entirely.

    That way it can only explicitly invoke the whitelisted commands I have granted it access to run, with only the parameters I have allowed it to.

    So for example I can explicitly expose git commit, status, stage, unstage, but everything else git related is not available, not even git push or pull.

    This substantially seems to help prevent the LLM from going off the rails on stupid CLI nonsense, like when it starts cating files instead of just fucking reading them.

  • source
  • [–] [S] 2 points 3 months ago

    Yeah, I'm not arguing against efficiency on the software side. The way I read the article isn't really that it advocates for brute force, but rather that we should be focusing on general solutions because they scale. The work on efficiency should focus on how to make search and statistical inference more efficient. Incidentally, we see a lot of that work happening in China right now precisely because they don't have as much compute to work with.

  • source
  • parent
  • context
  • [–] 2 points 3 months ago (1 child)

    No, I agree, there is huge potential to get back on the curve or even blow right through it, but currently and over roughly the last decade, moores law has been dead. At some point in future compute will grow massively. Humans have only been around for 10,000 years, and silicon tech isn't even a blim in that history. Give it another 10, 100, 1000 years, we will be able to brute force anything we currently think is out of scope. But in the short term, the here and now, it's worth exploring more elegant solutions that don't require compute beyond our current availability

  • source
  • parent
  • context
  • [–] [S] 3 points 3 months ago (2 children)

    On the other hand, Moore's law only applies to silicon. We already know a number of other viable substrates, and some are already being put into production as we speak. A substrate with different physical properties has the potential of making silicon look like vacuum tubes going forward. On top of that, architecture is evolving as well. Huawei's tau law being a great example. Turns out the scales we're dealing with are so small that the time it takes a signal to travel from one end of the chip to another becomes significant. So, if you fold the chip you can dramatically improve its speed by connecting the opposite ends together. And these things are obviously mutually reinforcing. So, there's little reason to think that the current plateau we're hitting is permanent.

  • source
  • parent
  • context
  • [–] 1 point 3 months ago (3 children)

    One of the reasons people keep making the same "mistake" is because moores law isn't actually a law, and just an observation that was true from the 1960s to 2010s, so rightly assumed moores law wouldn't go on forever so relying on ever increasing availability of compute wasn't the best strategy.

    Moores law has been dead for years now and the growth in compute is gradually slowing further. Yes, compute will continue to increase, but the pace is slowing and there is potential for it to stagnate.

  • source
  • [–] 2 points 3 months ago

    Don't over-focus on chess. The Lesson is that if you take two identical machine-learning labs which currently have a working model today, and ask them to come back and show you how much they improved in a year or so, the lab which forward-ports the model to newer hardware will tend to outperform the lab which tries to tweak the model to fit the problem domain more precisely. Sheer amounts of compute dominate any sort of domain-specific (here, chess-specific) knowledge. Further, there's a suggested mechanism, Moore's Law, which is tapering off now but used to be exponential; how could a polynomial amount of effort ever compete with exponentially-better new hardware?

  • source
  • parent
  • context
  • [–] 4 points 3 months ago (2 children)

    When a simpler, search-based approach with special hardware and software proved vastly more effective, these human-knowledge-based chess researchers were not good losers. They said that ``brute force" search may have won this time, but it was not a general strategy, and anyway it was not how people played chess. These researchers wanted methods based on human input to win and were disappointed when they did not.

    careful; lemmings HATE ai and and pro-ai stance is going to get you blacklisted. lol

  • source
  • [–] 2 points 3 months ago

    ever since getting my own diagnosis, it's been fascinating to me how many core concepts that either define or decry neurodivergency are aligned with socialist movements.

    these ones stuck out to me as i was reading your response:

    and a lot of opportunists intentionally add noise and misinformation/divisive rhetoric because informed group coordination scares them for some reason.

    coordination through practiced collectivism is at the heart of socialism and marxists/socialist/humanist thinkers fo the past have spent a significant amount of effort in explaining how/why opportunists behave in this manner.

    i’d say escaping information bubbles is the same kind of experience of finding something you didn’t know was there, because our brains make sense of the world regardless of the things we don’t know we aren’t seeing.

    the growing acceptance of the palestinian genocides proved how big information bubbles in the west can grow and how easily they can be popped .... and restored

    easy to get silo’d into some local-eco-niche or another, and apes going to do ape brain fighting cause we refuse to recognize how bad our ape brains generally are in how we define and grow into ourselves along with those around us

    witnessing the people in the country next door (mexico) do honest-to-god protests that ended up giving them a socialist leaning government that has been actively and quantitatively improving thier lives is a mindfuck when you consider american ape brains fighting to defend ineffective no-kings "pep rallies" and reinforcing a government that's actively genociding multiple others around the world.

    i feel like people need a new ‘basic education’ around systems and framing specifically, just because most of our experience is defined by these complex dynamics and interactions. although i think a lot of this unsettles some people because we’re very hackable as a species.

    our education system is captured by the hands of people who understand how hackable we are and you can see how unsettling it is to them in unhinged rants like marco rubio's recent manifesto about "communists" in our shores.

  • source
  • parent
  • context
  • [–] 3 points 4 months ago (1 child)

    been a thing since early internet forums. anywhere groups can gather and do the in-group out-group thing, you always have to be aware of this. will also note it's also the inverted experience of 'the double empathy problem' where the autistic lens can get socially shut out due to lack of social affirmation in the face of non-autistic interpretations being socialized.

    here is a good breakdown by someone else

    although to expand around it,

    as the 5'th species of ape, we really like to do in-group out-group competition, even when you're just socially confabulating your own excuses to look down on others and dismiss their experience to boost inner group cohesion. the best perspectives are able to make sense of the experiences in groups without completely ignoring the experiences of other groups.

    similarly why progressive groups can't successfully scale their efforts. local group tribalism tends to be toxic as hell, and a lot of money being put into divide and conquer that makes it really hard for progressives to work together.

    there's also a whole essay to be had around what intelligence is, which is why the group confabulations can often result in main-character syndrome that you see in the 'star children.' similarly someone less robust in their thinking can be affirmed into some confabulation or psychosis through an LLM. we are highly confabulatory, and highly social.

    sooo we're very socially vulnerable creatures. even the autistic people when they are allowed to collect in groups larger than they usually see, (yay internet!) they can find affirmation that people who do not comprehend their experience could never affirm. like in collective affirmation being able to get past the double empathy barrier. could also spiral into pathological supremacist behaviour cause monkey brain. this interaction also gives us tools like the double empathy problem, which can be applied as a function to much more broad systems interactions than just the neuro-developmental attempt at communication struggling against an overly rigid neurotypical social bubble affirming itself to save energy. in simple terms, inertia of group beliefs can accidentally overwrite important signals from smaller groups/individuals.

    there's also a good framing for rigidity(through inertia) vs plasticity and elasticity in predictive processing/active inference in how systems are shaped.

    which i believe will also be very valuable in framing and shaping pathology on the 'individual' level as well. quotations because an individual is still a lot of systems interacting.

    that one's definitely a bit technical, but i recommend any of her work generally in communicating some of the complexity within social systems.

    if you've read all of this you are appreciated. world is complicated, and holding all of the tools to not be whisked into some simplification or another is very energy expensive, and we are blind to our blindspots. and a lot of opportunists intentionally add noise and misinformation/divisive rhetoric because informed group coordination scares them for some reason.

    usually frame most of the sense-making here as clearing up blindspots you didn't know you had.

    if you've never tested your literal blindspots, look up the test, you just need two horizontal dots on a paper and to move it until the one not-centered vanishes. i'd say escaping information bubbles is the same kind of experience of finding something you didn't know was there, because our brains make sense of the world regardless of the things we don't know we aren't seeing.

    easy to get silo'd into some local-eco-niche or another, and apes going to do ape brain fighting cause we refuse to recognize how bad our ape brains generally are in how we define and grow into ourselves along with those around us. the actual complexity i'm skipping over feels like an ocean, but i feel like people need a new 'basic education' around systems and framing specifically, just because most of our experience is defined by these complex dynamics and interactions. although i think a lot of this unsettles some people because we're very hackable as a species.

    when life grows into a shape that defines other life as something other than itself, often something to be exploited. either you get parasites that lay eggs in living creatures as part of a stable system, or cancer that refuses to recognize that the environment it is exploiting is necessary for the continued survival of the cells that became it. there's a reason that communicating past scales and group identities is actually very important as a big large ecosystem full of different varied local ecosystems, even if it's an online chat group.

    hope this all makes sense!

  • source
  • parent
  • context
  • view more: ‹ prev next ›