top 89 comments

sorted by: hot top controversial new old
[–] 159 points 1 year ago (1 child)
  • [–] 80 points 1 year ago (13 children)

    Either (you genuinely belive) you are 18 (24, 36 does not matter) months away from curing cancer or you're not.

    What would we as outsiders observe if they told their investors that they were 18 months away two years ago and now the cash is running out in 3 months?

    Now I think the current iteration of AI is trying to get to the moon by building a better ladder, but what do I know.

  • source
  • hideshow 13 child comments
  • [–] -3 points 1 year ago (12 children)

    The thing about AI is that it is very likely to improve roughly exponentially¹. Yeah, it's building ladders right now, but once it starts turning rungs into propellers, the rockets won't be far behind.

    Not saying it's there yet, or even 18/24/36 months out, just saying that the transition from "not there yet" to "top of the class" is going to whiz by when the time comes.

    ¹ Logistically, actually, but the upper limit is high enough that for practical purposes "exponential" is close enough for the near future.

  • source
  • parent
  • hideshow 12 child comments
  • [–] 11 points 1 year ago (9 children)

    why is it very likely to do that? we have no evidence to believe this is true at all and several decades of slow, plodding ai research that suggests real improvement comes incrementally like in other research areas.

    to me, your suggestion sounds like the result of the logical leaps made by yudkovsky and the people on his forums

  • source
  • parent
  • hideshow 9 child comments
  • [–] -2 points 1 year ago (8 children)

    Because AI can write programs? As it gets better at doing that, it can make AI's that are even better, etc etc. Positive feedback loops increase exponentially.

  • source
  • parent
  • hideshow 8 child comments
  • [–] 1 point 1 year ago

    the problem is that ai's are trained on programs that humans have written. At best the llm architectures it creates will be similar to the state of the art that humans have created at that point.

    however, even more important than the architecture of an ai model is the training data that it is trained on. If we start including ai-generated programs in this data, we will quickly observe model collapse: performance of models tend to get worse as more ai-generated data is included in the training data.

    rather than AIs generating ever smarter new AIs, the more likely result is that we can't scrape new quality datasets as they've all been contaminated with llm-generated data that will only reduce model performance

  • source
  • parent
  • [–] 1 point 1 year ago (4 children)

    AI can't write programs, at least not complex programs. The programs / functions it can write well are the ones that are the ones that are very well represented in the training data -- i.e. ultra simple functions or programs that have been written and re-written millions of times. What it can't do is anything truly innovative. In addition, it can't follow directions, it has no understanding of what it's doing, it doesn't understand the problem, it doesn't understand its solution to the problem.

    The only thing LLMs are able to do is create a believable simulation of what the solution to the problem might look like. Sometimes, if you're lucky, the simulation is realistic enough that the output actually works as a function or program. But, the more complex the problem, or the more distant from the training data, the less it's able to simulate something realistic.

    So, rather than building a ladder where the rungs turn into propellers, it's building one where the higher the ladder gets, the less the rungs actually look like rungs.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 0 points 1 year ago (3 children)

    As I said elsewhere, the AI probably isn't going to just be an LLM. It's probably gonna be a complex model that uses modules like LLMs to fulfill a compound task. But the exact architecture doesn't matter.

    We know that it can output code, which means we have a quantifiable metric to make it better at coding, and thousands of people are certainly trying. AI video was hot garbage 18 months ago, now it's basically perfect.

    It's not if we're going to get a decent coding AI, it's when.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 2 points 1 year ago (2 children)

    It's probably gonna be a complex model that uses modules like LLMs to fulfill a compound task.

    That sounds very hand-wavey. But, even the presence of LLMs in the mix suggests it isn't going to be very good at whatever it does, because LLMs are designed to fool humans into thinking something is realistic rather than actually doing something useful.

    We know that it can output code, which means we have a quantifiable metric to make it better at coding

    How so? Project managers have been working for decades to quantify code, and haven't managed to make any progress at it.

    It's not if we're going to get a decent coding AI, it's when.

    The year 30,000 AD doesn't count.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 1 year ago* (1 child)

    LLMs are designed to fool humans into thinking something is realistic rather than actually doing something useful.

    So closer to average human intelligence than it would appear. I don't know why people keep insisting that confidently making things up and repeating things blindly is somehow distinct from the average human intelligence.

    But more seriously, this whole mindset is based on a stagnation in development that I'm just not seeing. I think it was Stanford recently released a paper on a new architecture they developed that has serious promise.

    How so? Project managers have been working for decades to quantify code, and haven't managed to make any progress at it.

    I think you misunderstand me. The metric is the code. We can look at the code, see what kind of mistakes it's making, and then alter the model to try to be better. That is an iterative process.

    The year 30,000 AD doesn't count.

    Sure. Maybe it's 30,000AD. Maybe it's next month. We don't know when the breakthrough that kicks off massive improvement is going to hit, or even what it will be. Every new development could be the big one.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 1 year ago

    So closer to average human intelligence than it would appear

    No, zero intelligence.

    It's like how people are fooled by optical illusions. It doesn't mean optical illusions are smart, it just means that they tickle a part of the brain that sees patterns.

    a paper on a new architecture they developed that has serious promise

    Oooh, a new architecture and serious promise? Wow! You should invest!

    The metric is the code. We can look at the code, see what kind of mistakes it's making

    No, we can't. That's the whole point. If that were possible, then companies could objectively determine who their best programmers were, and that's a holy grail they've been chasing for decades. It's just not possible.

    and then alter the model to try to be better

    Nobody knows how to alter the model to try to be better. That's why multi-billion dollar companies are releasing new models that are worse than their previous models.

    Maybe it's next month

    It's definitely not next month, or next year, or next century. Nobody has any idea how to get to actual intelligence, and despite the hype progress is as slow as ever.

    Every new development could be the big one

    Keep drinking that kool-aid.

  • source
  • parent
  • [–] 4 points 1 year ago

    The problem with that is they can't actually point to a metric where when the number goes beyond that point we'll have ASI. I've seen graphs where they have a dotted line that says ape intelligence, and then a bit higher up it has a dotted line that says human intelligence. But there's no meaningful way they can possibly have actually placed human intelligence on a graph of AI complexity, because brains are not AI so they shouldn't even be on the graph.

    So even if things increase exponentially there's no way they can possibly know how long until we get AGI.

  • source
  • parent
  • [–] 16 points 1 year ago

    Ow, that's why they are restricting "organic" porn, to sell AI porn. Damn.

  • source
  • [–] 28 points 1 year ago (2 children)

    Um, human history has repeatedly demonstrated that when a new technology emerges, the two highest priorities are:

    1. How can we kill things with this?
    2. How can we bone with this?
  • source
  • hideshow 2 child comments
  • [–] -1 points 1 year ago (1 child)

    Right but a chatbot is light years away from artificial general intelligence. And also let's be honest if we want to go the pornography route then that's going to need robots, so that would need general intelligence. As would in fact killer drones.

    So yeah, scepticism is highly warranted

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 1 year ago

    Killer drones already used AI to automatically find their targets pre-2021…

    And AI porn was made as soon as generative AI was able to produce pictures…

    But of course, if general AI is controllable in any way, it definitely will be miss-used for the two activities OC listed.

  • source
  • parent
  • [–] 56 points 1 year ago (27 children)

    There's not a single world where LLMs cure cancer, even if we decided to give the entirety of our energy output and water to a massive server using every GPU ever made to crunch away for months.

  • source
  • hideshow 27 child comments
  • [–] 31 points 1 year ago (5 children)

    Not strictly LLMs, but neural nets are really good at protein folding, something that very much directly helps understanding cancer amount other things. I know an answer doesn't magically pop out, but it's important to recognise the use cases where NN actually work well.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 6 points 1 year ago (3 children)

    I'm trying to guess what industries might do well if the AI bubble does burst. I imagine there will be huge AI datacenters filled with so-called "GPUs" that can no longer even do graphics. They don't even do floating point calculations anymore, and I've heard their integer matrix calculations are lossy. So, basically useless for almost everything other than AI.

    One of the few industries that I think might benefit is pharmaceuticals. I think maybe these GPUs can still do protein folding. If so, the pharma industry might suddenly have access to AI resources at pennies on the dollar.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 6 points 1 year ago (2 children)

    integer calculations are lossy because they're integers. There is nothing extra there. Those GPUs have plenty of uses.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 0 points 1 year ago (1 child)

    I don't know too much about it, but from the people that do, these things are ultra specialized and essentially worthless for anything other than AI type work:

    anything post-Volta is literally worse than worthless for any workload that isn't lossy low-precision matrix bullshit. H200's can't achieve the claimed 30TF at FP64, which is a less than 5% gain over the H100. FP32 gains are similarly abysmal. The B100 and B200? <30TF FP64.

    Contrast with AMD Instinct MI200 @ 22TF FP64, and MI325X at 81.72TF for both FP32 and FP64. But 653.7TF for FP16 lossy matrix. More usable by far, but still BAD numbers. VERY bad.

    https://weird.autos/@rootwyrm/115361368946190474

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 1 year ago*

    AI isn't even the first or the twentieth use case for those operations.

    All the "FP" quotes are about floating point precision, which matters more for training and finely detailed models, especially FP64. Integer based matrix math comes up plenty often in optimized cases, which are becoming more and more the norm, especially with China's research on shrinking models while retaining accuracy metrics.

  • source
  • parent
  • [–] 38 points 1 year ago (14 children)

    which fucking sucks, because AI was actually getting good, it could detect tumours, it could figure things fast, it could recognise images as a tool for the visually impaired...

    But LLMs are non of those things. all they can do is look like text.

    LLMs are an impressive technology, but so far, nearly useless and mostly a nuance.

  • source
  • parent
  • hideshow 14 child comments
  • [–] 8 points 1 year ago (2 children)

    down in Ukraine we have a dozen or so image analysis projects that can't catch a break because all investors can think about are either swarm drones (quite understandably) or LLM nothingburgers that burn through money and dissipate every nine months. Meanwhile those image analysis projects manage to progress on what is basically scraps and leftovers.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 8 points 1 year ago (1 child)

    the problem is that technical people can understand the value of different AI tool. but tell an executive with a business major how mind blowing it is that a program trained in Go and StarCraft can solve protein folding (studied biology in 2010 and they kept repeating how impossible solving proteins in silico was).

    But a chat bot that tells the executive how smart and special it is?

    That's the winner.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 1 year ago (10 children)

    Multimodal LLMs are definitely a thing, though.

  • source
  • parent
  • hideshow 10 child comments
  • [–] 10 points 1 year ago (9 children)

    yhea, but it's better to use the right tool for the job than throwing a suitcase full of tools at a problem

  • source
  • parent
  • hideshow 9 child comments
  • [–] 2 points 1 year ago (8 children)

    That's not…

    sigh

    Ok, so just real quick top level…

    Transformers (what LLMs are) build world models from the training data (Google "Othello-GPT" for associated research).

    This happens by needing to combine a lot of different pieces of information together in a coherent way (what's called the "latent space").

    This process is medium agnostic. If given text it will do it with text, if given photos it will do it with photos, and if given both it will do it with both and specifically fitting the intersection of both together.

    The "suitcase full of tools" becomes its own integrated tool where each part influences the others. Why you can ask a multimodal model for the answer to a text question carved into an apple and get a picture of it.

    There's a pretty big difference in the UI/UX in code written by multimodal models vs text only models for example, or utility in sharing a photo and saying what needs to be changed.

    The idea that an old school NN would be better at any slightly generalized situation over modern multimodal transformers is… certainly a position. Just not one that seems particularly in touch with reality.

  • source
  • parent
  • hideshow 8 child comments
  • [–] 5 points 1 year ago (4 children)

    The main breakthrough of LLM happened when they figured out how to tokenize words... The subsequent transformer architecture was already being tested on various data types and struggled compared to similarly advanced CNN.

    When they figured out word encoding, it created a buzz because transformers could work well with words. They never quite worked as well on images. For that, stable diffusion (a variation on CNN) has always been better.

    It's only because of the buzz on LLMs that they tried applying them to other data types, mostly because that's how they could get funding. By throwing in disproportionate amount of resources, it works... But it would have been so much more efficient to use different architectures.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 1 point 1 year ago (3 children)

    What year are you from? Have you not seen Gemini Flash, ChatGPT 4o, Sora 2, Genie 3, etc?

    Stable Diffusion hasn't been SotA for over a year now in a field where every few months a new benchmark is set.

    Are you also going to tell me about how we'd be better off using ships for international travel because the Wright brothers seem to be really struggling with their air machine?

  • source
  • parent
  • hideshow 3 child comments
  • [–] 1 point 1 year ago (2 children)

    Hehe, true! I left the field about 4 years ago when it became obvious that "more GPUs!" was better than any architectural design changes...

    Most of the image generation made by the products you mention are based on a mix of LLMs (for processing of user inputs) and some other modality for other media types. Last time I checked, ChatGPT was capable of handling images only because it offloaded the image processing to a branch of the architecture that was not a transformer, or at least not a classical transformer. They did have to grift CNN parts to the LLM to make progress.

    Maybe in the last 4 years they reorganised it to completely remove CNN blocks, but I think people call these models "LLMs" only as a shorthand for the core of the architecture.

    Again, you said that a new benchmark is set every few months, but considering they're just consuming more power and water, it's quite boring and I'd argue it's not really progress in the academical/theoretical sense. That attitude is exactly why I don't work with NN anymore.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 1 year ago (1 child)

    Definitely check again. That was how it worked with gpt-4, handing off to Dall-E.

    4o (the 'o' stands for 'omnimodel') and Gemini Flash are native multimodal outputs. Completely just transformers.

    It's why those models can do things like complex analysis in the process of generating things.

    For example, just today in a group chat where earlier on one model had "turned into" a unicorn and then the other models were pretending to be unicorns to fit in, dozens messages later the only direct prompt to an instance of 4o imagegen was "create a photorealistic picture of the room and everyone in it."

    The end result had exactly one actual unicorn and everyone else had horns taped on their head. That kind of situational awareness and nuanced tracking across a 100+ long message context isn't possible in a CNN.

    Also, if you really want your mind blown, check out Genie 3 and the several minute state change persistence. That one is really nuts and the kind of thing that should really have everyone seeing it questioning the empirical findings of our universe fundamentally being superimposed probabilities only collapsing based on attention. Eerily similar to what we're just starting to be independently building.

    As for the consumption — eating a single hamburger has a larger water/energy impact than a year of using these tools in average use. And even those inference costs are probably going to drop to effective insignificance within the decade. There's been very promising advancements in light based neural networks, and those run at like 1,000-10,000x lower energy costs paramater to parameter.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 3 points 1 year ago (2 children)

    go ask chatgpt to fold a protein

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 1 year ago* (1 child)

    Oh, wow, look at that… research just a few weeks ago on protein folding using general transformers. Huh.

    SimpleFold: Folding Proteins is Simpler than You Think

  • source
  • parent
  • hideshow 1 child comment
  • [–] 7 points 1 year ago (2 children)

    And it's clear we're nowhere near achieving true AI, because those chasing it have made no moves to define the rights of an artificial intelligence.

    Which means that either they know they'll never achieve one by following the current path, or that they're evil sociopaths who are comfortable enslaving a sentient being for profit.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 11 points 1 year ago (6 children)

    We are closer to making horny chatbots than a superintelligence figuring out a cure for cancer.

    Actually, if the latter wins, would that super AI win a Nobel prize?

  • source
  • hideshow 6 child comments
  • [–] 7 points 1 year ago (2 children)

    It would probably go to whoever uses it to find the cure... And to none of the authors who wrote the data that it was trained on

  • source
  • parent
  • hideshow 2 child comments
  • [–] 4 points 1 year ago (1 child)

    To be fair, a better pattern finder could indeed lead to better ways of curing cancer.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 1 year ago

    This year has just been a constant stream of examples why capitalism is stupid. Machine learning has a lot of utility in medical research. Imagine if it were deployed in such a way to benefit society instead of to maximize techbro's profit.

  • source
  • parent
  • [–] 8 points 1 year ago

    I appreciate that this post is using dark mode

  • source
  • [–] 7 points 1 year ago (2 children)

    Porn can pay your way through school, so to speak

  • source
  • hideshow 2 child comments
  • [–] 6 points 1 year ago

    But how else would it find the hard lump on yoir testicles?

  • source
  • [–] 14 points 1 year ago (6 children)

    If you've ever wondered why porn sites use pictures of cars, buses, stop signs, traffic lights, bicycles and sidewalks in their captchas, it's because they're using the data to train car-driving AIs to recognize those patterns.

    This is not what an imminent breakthrough in cancer research looks like.

  • source
  • hideshow 6 child comments
  • [–] -1 points 1 year ago (5 children)
  • [–] 11 points 1 year ago* (4 children)

    Google recaptcha? They literally talk about this publically. It's in their mission statement or whatever. It's used to train other kinds of model too.

  • source
  • parent
  • hideshow 4 child comments
  • [+] -11 points 1 year ago (2 children)

    Y’know, it’s bullshit that a) you seem to expect this to be common knowledge, as if everyone is supposed to have an archive of internet minutiae saved in their heads or have read and remembered any such info at all…

    And b) you chose to downvote and pretty much just said LMGTFY without even the sarcastically provided results instead of backing up your claim. It’s basic courtesy to provide a source for claims instead of downvoting like it’s some kind of affront to your ego that someone wants info on your claim.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 6 points 1 year ago* (1 child)

    It's not even my claim you are talking about jackass. Read the usernames. If you have fallen into the rabbit hole that is Lemmy you should have been around enough to know about recapcha. If not it's one DuckDuckGo search away. In fact you could just click the link on the recapcha itself that explains how they use the data for training. Hardly arcane knowledge.

    Your comment to me read like Sealioning.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 7 points 1 year ago (2 children)

    What about an AI naughty nurse that does both?

  • source
  • hideshow 2 child comments
  • [–] 13 points 1 year ago (1 child)

    You can use AI to fulfill your fantasies! for example: having healthcare (if you're not American, this joke does not apply)

  • source
  • parent
  • hideshow 1 child comment
  • i'd rather lucid dream i have healthcare my friend. then i can use my care bear stare laser beam to apply vengeance to incompetent healthcare providers and administrators such that they will never know what it is like to satiate their hunger again. then ride a giant flying tardigrade named Hairy Terry off into the sunset.

    LLMs only let me imagine it, not (from my perception) experience it. and remember, no crimes without Hairy Terry on lookout

  • source
  • parent
  • [–] 7 points 1 year ago (2 children)

    FYI, using openAI/ChatGPT is expensive. Programming it to program users into dependency on its "friendship" gets them to pay for more tokens, and then why not blackmail them or coerce/honeypot them into espionage for the empire. If you don't understand yet that OpenAI is arm of Trump/US military, among its pie in the sky promises is $35B for datacenters in Argentina.

  • source
  • hideshow 2 child comments
  • [–] 2 points 1 year ago (1 child)

    What espionage is an AI simp going to be able to conduct?

    I'm pretty sure this is just them flailing around not being able to come up with anything meaningful so they're going this route so they have some profit. I don't think a conspiracy beyond that is required.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 1 year ago

    What espionage is an AI simp going to be able to conduct

    The AI simp's you are referring to isn't the tech billionaires who've managed useful technology development? It's an insult to users. Incels can't know anything sensitive?

    they’re going this route so they have some profit

    Your data and secrets is monetizable, and has been traded for last 20 years. The biggest AI customer is going to be US government's military/disinformation/destabilization departments. If you tell your AI gf that anti fa seems more reasonable than pro fa, then she will gift you an israeli pager.

    Sure, OpenAI can get $20/month from you to just be your bff, as bff 1.0. OpenAI's real BFF customer, will start giving it ideas on how 2.0 can make OpenAI more money.

    Supposedly, there was a leak from META where they were directing team to romanticize AI engagement with 8 year olds for better engagement numbers. Having unlimited time with you, is a powerful tool to shape ideology and loyalties. Media has traditionally programmed the masses after it replaced churches. You denying that absolute control isn't a goal when the tool is ideally suited for it is not a good look for you.

  • source
  • parent
  • [–] 3 points 1 year ago

    ehm... cure cancer? I thought "cute cancer". Sorry bout that

  • source
  • [–] 8 points 1 year ago (8 children)

    No money in curing cancer with an LLM. Heaps of money taking advantage if increasingly alienated and repressed people.

  • source
  • hideshow 8 child comments
  • [–] 12 points 1 year ago (3 children)

    You could sell the cure for a fortune. Imagine something that can reliably cure late stage cancers. You could charge a million for the treatment, easily.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 5 points 1 year ago (2 children)

    Yes, selling the actual cure would be profitable...but an LLM would only ever provide the text for synthesizing it but none of the extensive testing, licensing, or manufacturing, etc.. An existing pharmaceutical company would have to believe the LLM and then front the costs for the development, testing, and manufacture...which constitutes a large proportion of the costs of bringing a treatment to market. Burning compute time on that is a waste of resources, especially when fleecing horny losers is available right now. It is just business.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 1 year ago (1 child)

    and LLMs hallucinate a lot of shit they "know" nothing about. a big pharma company spending millions of dollars on an LLM hallucination would crack me the fuck up were it not such a serious disease.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 1 year ago (1 child)

    What a weird take, research use AI already? Some researchers even research things that, gasp, is not monetiseable right away!

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 1 year ago

    I used to work in academic physics, and I currently work in data science. I am deeply familiar with both ends of the subject in question. LLMs are useful research tools because they speed up the reference finding and literature review process, not because they synthesize new information that does not need to be independently verified.

    In the context of medical research, they could absolutely use LLMs to facilitate a literature search. What LLMs cannot do is hand researchers a proposed cure that they could sell to people. You still need to do the leg work of synthesizing the molecules, standardizing the process, industrializing it, patenting it, multiple rounds of testing on increasingly complex animals and eventually people, and then going through the drug approval process with the FDA and others. LLMs speed up the CHEAPEST and EASIEST part of the research process. That is why LLMs will not be handing us the cure for cancer.

  • source
  • parent
  • [–] 3 points 1 year ago (2 children)

    startups hyping shit up to get the investors drooling is one of the most despicable things a man can observe.

  • source
  • hideshow 2 child comments
  • [–] 1 point 1 year ago (1 child)

    The thing that makes it actually bad is that they’re taking advantage of the mentally handicapped(investors). That and that said investors have millions to toss at nonsense while so many people are licky to have pennies to toss at such luxuries as “food”.

    Honestly though I don’t think taking advantage of evil people, who swear they deserve their millions because they’re definitely super smart, is really anything I care that much about.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 1 year ago

    What did I miss? Who is pivoting into hornybots?

  • source
  • [–] 1 point 1 year ago

    They think if they say the criminally insane part out loud it will protect them.

  • source
  • [–] 1 point 1 year ago
    load more comments
    view more: next ›