The research community is in uproar after OpenAI released a trove of more than 700 mathematical preprints entirely generated by AI on 6 October. The San Francisco, California-based maker of ChatGPT posted the preprints on the software repository Github.

Although some mathematicians celebrated the solution of longstanding problems, others took to social media to complain about being scooped. Some were incensed at what one physicist called a ‘slopocalypse’, even if the mathematical content could end up being formally correct.

all 48 comments

sorted by: hot top controversial new old
[–] 140 points 1 month ago* (9 children)

I saw the Wired article yesterday, it's worse than it sounds.

For at least one of the problems an OpneAI researcher initiated a partnership with a mathametician, loaded all of his work into a chatbot without his knowledge, and then published the result as 100% from the chatbot in this "trove".

When the mathematician confronted the OpenAI employee, he was told sharing credit was "too complicated" so they wouldn't acknowledge he had already mostly had it done. When he tried to push back, OpenAI then threatened his career if he tried to tell people what had happened.

It's is very very unlikely that was a single case. If that happened once, they definitely at least tried it more than once.

Even if it actually solved anything, taking in years of human work and then spitting out one last step is nowhere near what they're portraying this as. They're intentionally making it sound like they just gave the chat or the problem and it solved what a human couldn't.

All it did was steal human work and rush publication

  • source
  • hideshow 9 child comments
  • [–] 12 points 1 month ago

    it might not even be the last step in a way. A person may have it solved but worries about publishing until they check and double check and run it down a few times to be sure and then of course there is the writing. So really it might just be acting as a technical writer for the researcher at best.

  • source
  • parent
  • [–] 6 points 1 month ago (1 child)

    All it did was steal human work and rush publication

    Ok, but literally all it does is steal human work and provide an output.

    I wonder how the mathematician expected it to go. Was he expecting to be directly provided the AI output so he could publish it? Was he going to care if the AI used a novel approach he hadn't employed and was unable to identify where it got it from? We have seen since the beginning that AI has an attribution problem, so it seems like his actual concern was that HE wasn't credited.

    I'm not an AI supporter, but this whole "it did just a small part" complaint is weird to me. I believe I read that most were solved in 3 hours of compute. I'm sure the mathematician wasn't just 3 hours from solving the problem. It did it faster than humans did. Isn't that the point? Does it matter how little the step was if it hadn't been taken yet? It could've taken mathematics years to make that last step. Shouldn't we be excited to see this development? If we're going to have AI, isn't the point that it accelerates problem solving. If AI sped up life saving medical research by years or even just months by doing one last step, would people be having the same reaction?

    I've seen the same people complaining about this regularly talk about using AI for coding. Why is it ok that so many companies have put programmers in this position, but all the sudden it's bad for mathematicians?

  • source
  • parent
  • hideshow 1 child comment
  • [–] 17 points 1 month ago

    Why is it ok that so many companies have put programmers in this position, but all the sudden it’s bad for mathematicians?

    the former has just been normalized and tech bros are hard at work trying to normalize the latter, but it's not ok in either case

  • source
  • parent
  • [+] 2 points 1 month ago* (last edited 1 month ago) (2 children)
  • [–] -1 points 1 month ago (1 child)

    I don't know what you're trying to say, and I feel like you don't either.

    This means people will stop sharing work and insights with each other, which has never before been the case.

    That part was clearly, but obviously false on multiple levels

  • source
  • parent
  • hideshow 1 child comment
  • [+] 4 points 1 month ago* (last edited 1 month ago) (1 child)
  • [+] -8 points 1 month ago

    I’m actually paraphrasing Terence Tao

    That doesn't even mean you're saying the same thing bud...

    Non-mathematicians don’t realize this, but proofs aren’t intrinsically valuable outside of what they mean about the community capable of creating them.

    Do you think "non-mathematicians" think solving a math problem magically gets someone a check? Like, if anythi g you have it backwards

    You're way off on everything though.

    You should paraphrase less, wait till after you understand what you're reading.

  • source
  • parent
  • load more comments (-1 replies)
  • load more comments (-1 replies)
    [–] 52 points 1 month ago (9 children)

    “Why is OpenAI trying to solve hundreds of math problems to be released all at once?”, he added.

    even if the mathematical content could end up being formally correct.

    It should be noted that none of what they released has been verified. Throwing 700 papers out there is intentional to generate buzz, “hey look how great we are”, and to slow attempts at proving or disproving their results. That way even if a few get disproven early they can still say “well that’s just a couple out of hundreds” because it’s gonna take forever to work through the slop dump. It could be a long time before we get a percentage of accuracy and by then most people will have forgotten or lost interest since the headline will be stale by then.

    some mathematicians who might find their research projects pre-empted by the OpenAI release. “We should put that aside for today though. … he wrote, because it shows how quickly the technology has become useful to mathematicians.”

    Yeah this fucks the researchers. Funding could get pulled from related projects leaving their work in hiatus until someone can check the slop for accuracy. Till then someone can just point to this and say “oh it’s already been solved” like it’s a done deal already.

    “Look how useful we are!!!” Turns in unverified crap

  • source
  • hideshow 9 child comments
  • [–] 31 points 1 month ago* (4 children)

    Throwing 700 papers out there is intentional to generate buzz, “hey look how great we are”, and to slow attempts at proving or disproving their results.

    The Gish gallop is a rhetorical technique in which a person in a debate attempts to overwhelm an opponent by presenting an excessive number of arguments, without regard for their accuracy or strength, with a rapidity that makes it impossible for the opponent to address them in the time available. Gish galloping prioritizes the quantity of the galloper's arguments at the expense of their quality.

    The term "Gish gallop" was coined in 1994 by the anthropologist Eugenie Scott, who named it after the creationist Duane Gish, described by Scott as the technique's "most avid practitioner".[1][2]

  • source
  • parent
  • hideshow 4 child comments
  • [–] 8 points 1 month ago (3 children)

    thanks because you know I have seen this with trump and the right in the us but did not think of it in terms of something recognized.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 4 points 1 month ago* (2 children)

    You're more patient than I am; I tune that nonsense right out.

    But I bet if you isolated RW talking points, you'd find they're not even proper arguments so much as emotional triggers. Like a common argument is "That's socialism!" A good ghish gallop's arguments at least have internal validity.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 3 points 1 month ago (1 child)

    well what I mean is they will throw out things and when challenged or fact checked then go to a new thing. sometimes not new but proved false long enough ago that people forgot. but yeah a lot is poorly done to begin with but its using large amounts and going to a new one if its disproved and actually ignoring that they ever used it. Oh and then attacking fact checking places as being political for pointing out facts or yelling at news entities for doing it.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 7 points 1 month ago*

    mathematicians are up in arms

    ... [quotes a physicist]... [paywall]...

  • source
  • [–] 11 points 1 month ago* (20 children)

    Can someone please explain to a non-mathematician what the issue is specifically?

    We all want the problems solved, right? Is there a wrong way to solve them? Or a process that people are supposed to follow not to “scoop” others? A honor system of sorts?

    If a human dropped the same 700 solutions online, would it still be bad?

  • source
  • hideshow 20 child comments
  • [–] 39 points 1 month ago (10 children)

    There are two issues really - one is a perception that they're not vetting any of this output, just dumping it out on the internet and expecting real mathematicians to actually check it all, so yes if a human was doing the same thing it'd probably still be suspect.

    The other is the widely held suspicion that much of the "breakthrough" mathematics here has been stolen from real researchers who've been using chatgpt to develop their theories by talking to it

  • source
  • parent
  • hideshow 10 child comments
  • [–] 5 points 1 month ago (3 children)

    I think a mathematician with no skin in the game should just make a big post saying "All 700 are wrong, no I am not going to show my work, if you disagree prove which one you think is right" put the ball back in OpenAI's court.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 2 points 1 month ago

    Maybe instead they should just generate slop counterproofs for every single one. Then everyone defending OpenAI's dump will have to either read through all the shit or acknowledge that it's not worth the effort to verify slop.

  • source
  • parent
  • [–] [S] 2 points 1 month ago (4 children)
  • [–] 3 points 1 month ago (3 children)

    I need an account to access that.

  • source
  • parent
  • hideshow 3 child comments
  • [+] -6 points 1 month ago

    The other is the widely held suspicion that much of the “breakthrough” mathematics here has been stolen from real researchers who’ve been using chatgpt to develop their theories by talking to it

    I think that was just some rumor. In the end the accusation was that they basically brute-forced a solution using methods developed by someone else and claimed they solved it but those methods were public, not stolen because they used ChatGPT.

  • source
  • parent
  • [–] 18 points 1 month ago (5 children)

    It is a lot of stuff from an unreliable source that basically vomits out stuff that looks correct but is often not and that is a ton of human work to verify that humans were already in the process of doing.

    It is most likely just the stuff humans were close to finishing being scooped up and spit out and not reliably correct.

  • source
  • parent
  • hideshow 5 child comments
  • [+] -4 points 1 month ago (3 children)
  • [–] 7 points 1 month ago

    There's an onus now on those that care about math to check the proofs. If errors are found in one their might be similar errors in another. A serial release would have been cleaner and allowed commentary that might help improve the accuracy or perhaps even the way the model conveyed the data to make it more understandable / digestable.

    Openai dug the problems up from somewhere. They could have collaborated with some of the people that defined the problems in the first place. This is just perpetuation of their culture of theft.

  • source
  • parent
  • [–] 5 points 1 month ago

    We demand rigidly defined areas of doubt and uncertainty!

  • source
  • [–] 4 points 1 month ago

    I worry about what they're not publishing. Cryptography is under-represented in this set.

  • source
  • [–] 3 points 1 month ago

    As an avid dis-liker of AI, this is an interesting development.

  • source
  • [–] 0 points 1 month ago* (last edited 1 month ago) (2 children)

    Here's what the field should do: ignore those paper and never publish them. Then have people verify the math and publish the proofs as their own (whoever can do it first gets the credit).

  • source
  • hideshow 2 child comments