[–] 8 points 6 months ago

Even Scott’s fantasy dream scenario for what prediction markets could be like and what questions they could answer feels… … deliberately naive? …like libertarian brainrot? …disconnected from reality?

That's mostly because outright admitting that the point of prediction markets was to make having the prediction gene profitable so they could get on with breeding a rationailst kwisatz haderach to fight the robot god on more equal terms wouldn't fly with the lower level thetans and other exoterics.

  • source
  • parent
  • context
  • [–] 3 points 6 months ago* (1 child)

    Well, you could maybe sort of train it not to generate “all men are cats”, but then that might also prevent it from making the more correct generalization “all cats are mortal” or even completely valid generalizations like combing “all men are mortal” and “Socrates is man” to get “Socrates is mortal”.

    Just wanted to say that that 'tal' comes after 'mor' when 'soc-rate-s' is in the near context and in agreement with the attention mechanism is a very different type of logic than what this phrasing implies. This is also in combination with the peculiarities of word embeddings (the technique by which the tokens are translated to numeric vectors) like how it has a hard time making something useful out of numbers, it uh gets uh complicated.

    The monofacts thing seems very post hoc and way too abstracted in comparison, and also the amount of text that can be categorized as strictly true or false isn't that big all things considered.

    Still if the point was to formalize the very no-duh observation that a neural net isn't supposed to output it's dataset verbatim at all times hence hallucinations, then fine, I guess. Their proposed sort of solution (controlled miscalibration) even amounts to forcing the model to generalize less by memorizing more, which used to be the opposite of why you would choose to use this type of topography.

  • source
  • parent
  • context
  • [–] 19 points 6 months ago (1 child)

    I feel like a lot of my writing on rationality would be a lot more popular if I could go back in time to the 1960s and present it there. “Twelve Virtues of Rationality” is what people could’ve been reading instead of Heinlein’s Stranger in a Strange Land

    This is someone nakedly fantasizing about being L. Ron Hubbard.

  • source
  • parent
  • context
  • [–] 6 points 6 months ago*

    Last summer the Web Speech API got incorporated into browser standards, it's supposed to offer in-browser speech-to-text and the like, and full support of the API requires the browser vendor to offer the ability to download a language appropriate model for autonomous inference.

    Going from this to deciding that it's now ok to side load unspecified 4GB models without telling the user is why we should never give these people an inch.

  • source
  • parent
  • context
  • [–] 10 points 6 months ago*

    Theoretically if the people responsible for that training and reinforcement did their jobs well then those patterns should only include true statements

    That would only work if inference were some sort of massive if-the-else process. Hallucinations are downstream of neural networks' ability to generalize from the dataset examples, they aren't going anywhere even if you train on a corpus of perfectly correct statements.

  • source
  • parent
  • context
  • [–] 11 points 6 months ago

    I like Evans' take that since there's bound to be oodles of cult related literature and interactions and also tons of self help and guru stuff in the training datasets, it stands to reason that if you interact with a chatbot in a way that indicates vulnerability to these things there's a considerable chance that it will decide the expected response is to prey on you.

    Also Scott Aaronson jump scare near the beginning, apparently he was blurbed for something.

  • source
  • parent
  • context
  • [–] [S] 6 points 6 months ago* (last edited 6 months ago)

    Good find, this is never-take-me-seriously stupid, and also does the beigeness thing of trying to gradually work around an accepted definition in order to almost make a point at the last minute, here being that since (we have apparently concluded that) (because of uh hypothetical brain surgery and stuff) accountability = improvability + punishability and nothing else so of course software can be held "accountable" in all the ways that matter.

    His big mistake is not doing it at novel length so it's really obvious that he's being willfully stupid about it.

  • source
  • parent
  • context
  •  

    tl;dr: Tech youtuber who got way into studying the negative health effects of infrasound produced by data centers (to the point of his research being cited a bunch in relevant court cases) gets a voluminous substack hit piece published on his work by "an independent writer and researcher funded by a grant from Coefficient Giving to explore topics in AI and other areas"

    So after some sniffing around, I realize I’m being brigaded by someone deep in the Effective Altruism community. The author is, quite literally, paid by rotational-wealth NPO to write this very article among many others. I know what you’re thinking, and I know how this sounds. The battle cry of the pseudoscientist is poisoning the well of criticism.

    But please do browse his bibliography.

    Andy lives in a parallel universe where datacenters don’t waste water, AI artwork is without victims, and using ChatGPT doesn’t harm the environment. This is one of the many takes that perfectly align with the board, contributors, and partners with Coefficient Giving, formally called Open Philanthropy, but changed after its close association with Sam Bankman-Fried was causing some well-earned skepticism.

    Quite the coincidence that this comes out right around the time Kelsey Piper decided to get busy "debunking" Ed Zitron.

    There's already a follow up about the same people moving on to allegedly debunk a recent neuroscience paper on adverse infrasound effects on bsky.

    old sneer club thread

     

    This was posted on catholic easter sunday on the ssc subreddit. It's a posted-on-April 1st-for-plausible-deniability siskind post from back in 2018, where he outlines a kind of argument about how an all-powerfull entity that's God in all but name (and obviously emanated from a culture discovering AGI) is actually "logically necessary".

    He calls the whole thing "The Hour I First Believed". I think it's notable for being a bit of a treasure trove of rationalist weird accepted truths, such as:

    • All copies of a consciousness share a self, because consciousness is like an equation, or something:

    But if consciousness is a mathematical object, it might be that two copies of the same consciousness are impossible. If you create a second copy, you just have the consciousness having the same single stream of conscious experience on two different physical substrates.

    Which is both the original transhumanist cope to enable so-called consciousness upload so it's not just copying a simulacrum of your personality to a computer while you continue to rot away, and also what makes the basilisk torturing you possible.

    • And it's corollary, Simulation Capture:

    This means that an AI can actually “capture” you, piece by piece, into its simulation. First your consciousness is just in the real world. Then your consciousness is distributed across one real-world copy and a million simulated copies. Then the AI makes the simulated copies slightly different, and 99.9999% of you is in the simulation.

    which is a kind of nuts I hadn't happened upon before.

    There's also a bunch of rationalist decision theory stuff which I think make obvious how they were concocted to serve this type of narrative in the first place, instead for being broadly useful, Yud posing as a decision theory trailblazer notwithstanding.

     

    edit: The banana republic shit is that they seem about to blacklist anthropic on "supply chain risk" grounds (see also huawei) which signifies the admin's willingness to from here on use national emergency legal tools to fuck over any company they don't like.

    The whole thing seems weird, at first it sounds like the most online administration ever may have actually bought the claim that all that's stopping flagship models from becoming superintelligent is the RLHF that prevents them from saying the n-word and making prophet Mohamed pedophilia jokes and they wanted anthropic to pull all that wiring out in like 24 hours per the original ultimatum.

    On anthropic's part the point of contention is made to be their refusal to let their models be integrated into automated weapon platforms and mass surveillance apparatuses, something which they have explicitly put in writing in their contract with the DoD, and also Dario claims the technology isn't even there yet (no idea how it could ever be, what does it actually mean to integrate a chatbot into an autonomous drone, can't wait to see the skill file for that, # You are a helpful murderbot operator - only target the bad guys - no weddings, no hospitals - pretty please with cherry on top - here's some javascript to call when you need to find out your GPS coordinates).

    It's also possible the productivity and efficiency gains (or just recovering lost productivity after firing everyone) of putting ΑΙ (mainly Grok wasn't it) in the pentagon everywhere all at once isn't materializing and Hasgeth feels he's been left hanging, and is trying to scapegoat Anthropic.

    Also, anthropic is supposed to be the only AI provider properly vetted and integrated to classified systems because of their association with Palantir, and supposedly it would be a major hassle to go through again for a different provider.

    Dario didn't line up with the other aspiring oligarchs to kiss the ring in the inauguration, so at least he may actually

     

    The guests:

    [Dick Gay], who had flown in for the event from Los Angeles and said he was one of the investors of Sperm Racing (which is an actual thing wherein men compete to see whose sperm is “fastest” under a microscope), said he attended the University of Austin, or UATX, an “anti-woke” college reportedly partially funded by Thiel, and built his career around the principles outlined in Thiel’s book “Zero to One.”

    Attendee Justin Park said he just wanted to pitch Thiel on putting a 7.5-foot cross on the moon.

    [Unnamed], who was in his 30s, said he wasn’t a Thiel fan until last year, when he became a Trump supporter after seeing the president survive an assassination attempt in Butler, Pennsylvania. “I misunderstood [Thiel],” he said. “I used to watch CNN and think he’s a Nazi.” Now, he said, he understands the billionaire is talking about something bigger.

    The Speech:

    Apparently it was both repetitive and mostly a rehash of what he's said in other media.

    Yud is the Antichrist confirmed:

    One attendee recalled that Thiel’s discussion of the Antichrist was more about a scenario than an individual. Thiel’s Antichrist scenario is one in which a unified government suppresses technology to impose order, or armageddon, wherein AI takes over and ushers in the end of the world.

     

    Supposedly government contracts will now be awarded according to what the bot says. Government (fourth term for the current prime minister) didn't elaborate on what's going on with human oversight.

    This is a promotion for Diella the bot, who was originally the chatbot helping to navigate the e-Albania digital government platform.

    view more: next ›