"Apertus: a fully open, transparent, multilingual language model

EPFL, ETH Zurich and the Swiss National Supercomputing Centre (CSCS) released Apertus 2 September, Switzerland’s first large-scale, open, multilingual language model — a milestone in generative AI for transparency and diversity.

Researchers from EPFL, ETH Zurich and CSCS have developed the large language model Apertus – it is one of the largest open LLMs and a basic technology on which others can build.

In brief Researchers at EPFL, ETH Zurich and CSCS have developed Apertus, a fully open Large Language Model (LLM) – one of the largest of its kind. As a foundational technology, Apertus enables innovation and strengthens AI expertise across research, society and industry by allowing others to build upon it. Apertus is currently available through strategic partner Swisscom, the AI platform Hugging Face, and the Public AI network. ...

The model is named Apertus – Latin for “open” – highlighting its distinctive feature: the entire development process, including its architecture, model weights, and training data and recipes, is openly accessible and fully documented.

AI researchers, professionals, and experienced enthusiasts can either access the model through the strategic partner Swisscom or download it from Hugging Face – a platform for AI models and applications – and deploy it for their own projects. Apertus is freely available in two sizes – featuring 8 billion and 70 billion parameters, the smaller model being more appropriate for individual usage. Both models are released under a permissive open-source license, allowing use in education and research as well as broad societal and commercial applications. ...

Trained on 15 trillion tokens across more than 1,000 languages – 40% of the data is non-English – Apertus includes many languages that have so far been underrepresented in LLMs, such as Swiss German, Romansh, and many others. ...

Furthermore, for people outside of Switzerland, the external pagePublic AI Inference Utility will make Apertus accessible as part of a global movement for public AI. "Currently, Apertus is the leading public AI model: a model built by public institutions, for the public interest. It is our best proof yet that AI can be a form of public infrastructure like highways, water, or electricity," says Joshua Tan, Lead Maintainer of the Public AI Inference Utility."

all 15 comments

sorted by: hot top controversial new old
[–] 21 points 11 months ago (5 children)

How does it stack up in benchmarks is the question.

  • source
  • hideshow 5 child comments
  • [–] 10 points 11 months ago* (last edited 11 months ago) (4 children)

    Badly. This was released almost 3 months ago and completely failed to make a splash, just like the other European models you have never heard of.

    There's a lot of denial about this, but you just can't make such models competitive in Europe. The copyright industry is too strong. This combines with a general culture of data ownership and control. Case in point, the copyright industry is especially strong in Denmark, and they are also champions of chat control.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 5 points 11 months ago (3 children)

    Copyright is arguably stronger in the USA, but there's a simple solution: just allow these companies to break the law.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 2 points 11 months ago (2 children)

    I don't know any European country that has anything like the copyright clause in the US Constitution, or anything close to Fair Use. I don't see the argument.

    There's a good chance that European AI companies like Mistral are breaking the law. We will have to see how it eventually goes in court. Recently, there was a decision in a Munich court against OpenAI. By that standard, even Apertus might be in trouble. But I doubt that decision will stand.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 11 months ago (1 child)

    Well yes you'd need to go country by country to truly make an analysis. Maybe I'm talking more about enforcement than actual law, but as someone who has lived half his life on each continent, it's much easier to get away with copyright infringement on the personal level in Europe.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 11 months ago

    Maybe I’m talking more about enforcement than actual law,

    Probably. Different countries in Europe have very different traditions there. I think the former socialist countries are still more relaxed. But the EU-line is rather dominated by countries like Germany.

    Come to think of it. Switzerland officially takes a very lenient approach. It's legal to download media files for personal use. But as you can see here, that leaves research and business hanging.

  • source
  • parent
  • [–] 36 points 11 months ago (5 children)

    Multilingual language large language model.

  • source
  • hideshow 5 child comments
  • [–] 20 points 11 months ago (2 children)

    Let's define a language to interact with this multilingual language large language model. I call it: The multilingual language large language model language.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 8 points 11 months ago
    [–] 7 points 11 months ago

    "Apertus"

    ... or Aperture 😁 ? ...

    GLaDOS is depicted in the series as an artificially superintelligent computer system responsible for testing and maintenance in the Aperture Science Computer-Aided Enrichment Center in all titles. https://en.wikipedia.org/wiki/GLaDOS

  • source
  • [–] -3 points 11 months ago

    Another argument in favor of living in a police state of rich blokes doing banking for mafia and dictators, which doesn't need any surveillance because police already sees everything, and what it doesn't notice, it chooses not to, and also it can legally bend you over when required. I mean, OK, nobody's inviting, LOL.

    I mean, seriously I like such news, every polity gets its turn to be a force for good.

    There's even been news of an attempt by a group of Swiss politicians to create a committee on Nagorno-Karabakh refugees' return and future political existence.

    And they are propping up Taler.

    Just - I value this family of technologies for one main trait, it's destructive to superficial authorization. As in - where in the olden days bot campaigns in social media could just work, possibly unnoticed, now you can be certain it's mostly bots unless you can't transparently establish connection to a person, backed by cryptography.

    But OK, a public confirmation bot is good, especially if it talks in your Jura dialect unintelligible not just to me, but to most native German speakers.

  • source
  • 66 comments in c/opensource@lemmy.ml view that post
    [+] 23 points 11 months ago* (last edited 11 months ago) (1 child)
  • [–] 19 points 11 months ago (20 children)

    Is this hosted somewhere? Maybe distributed? I would love a privacy respecting distributed LLM chatbot.

  • source
  • hideshow 20 child comments
  • [–] 19 points 11 months ago* (18 children)

    In case you're not aware, there are a decent number of open weight (and some open source) large language models.

    The Ollama project makes it very approachable to download and use these models.

  • source
  • parent
  • hideshow 18 child comments
  • [–] 10 points 11 months ago* (last edited 11 months ago) (13 children)

    Ollama has taken a bad turn lately (such is the nature of VC backed software). Maybe recommend kobold.cpp jan.ai for LLM noobs instead

  • source
  • parent
  • hideshow 13 child comments
  • [–] 2 points 11 months ago (8 children)

    there is nothing wrong with ollama it runs models fast and easy add a gguf and youre done unless you want to squeeze out extra performance and have time to figure out your exact flags then use llama cpp otherwise ollama just works for 99 percent of people

  • source
  • parent
  • hideshow 8 child comments
  • [–] 1 point 11 months ago (7 children)
  • [–] 2 points 11 months ago (6 children)

    if you send me a video of you completing the bussin level on geometry dash ill send you 10$

  • source
  • parent
  • hideshow 6 child comments
  • [–] 1 point 11 months ago (5 children)

    handcam necessary or just screen recording w clicks

  • source
  • parent
  • hideshow 5 child comments
  • [–] 2 points 11 months ago (4 children)
  • [–] 1 point 11 months ago* (3 children)
  • [–] 3 points 11 months ago

    I'm keeping an eye on Ollama's service offerings - I don't think they're in enshittification territory yet, but I definitely share the concern.

    I still don't believe the other LLM engines out there have reached an equivalent ease of use compared to Ollama, and I still recommend it for now. If nothing else, it can be a stepping stone to other solutions for some.

  • source
  • parent
  • [–] 1 point 11 months ago (2 children)

    Or just llama.cop they finally got an UI added

  • source
  • parent
  • hideshow 2 child comments
  • [–] 3 points 11 months ago (1 child)

    Other than Apertus, are there any truly open source models - mainly what I want to know is models that list their training data publicly to ensure no theft of art and stuff. (i replied to your comment as you seem to know about these models, I have no clue abou this stuff)

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 10 months ago (1 child)

    I tried Deepseek R1 7B on my MacBook M3 Pro but it is shit compared to ChatGPT unfortunately

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 10 months ago* (last edited 10 months ago)

    There are some factors to consider. Some of the Deepseek quants are based on Llama 3, whereas others are based on Qwen Reasoning.

    You're also not going to get the same quality of the full ChatGPT experience comparing a 7B parameter model to a 500B+ model like ChatGPT.

    Regardless, it's difficult to run the actual Deepseek R1 model as there's not a true quantization or distillation of the original model.

    You can also try GPT-OSS if you want an open source model comparable to ChatGPT. Once again, you're going to have to balance the size and precision of the model with your expectations.

  • source
  • parent
  • [–] 67 points 11 months ago (22 children)

    Apertus was developed with due consideration to Swiss data protection laws, Swiss copyright laws, and the transparency obligations under the EU AI Act. Particular attention has been paid to data integrity and ethical standards: the training corpus builds only on data which is publicly available. It is filtered to respect machine-readable opt-out requests from websites, even retroactively, and to remove personal data, and other undesired content before training begins.

    Available doesn't mean licensed for AI training.

  • source
  • hideshow 22 child comments
  • [–] 19 points 11 months ago

    "Your honor, my archive of Linux ISOs were acquired under the pretense that they were 'publicly available' and the copyright holders didn't 'opt-out' using the 'up-for-grabs.txt' standard I invented."

  • source
  • parent
  • [–] 31 points 11 months ago (18 children)

    and yet it is still a legally unsettled question whether LLM training requires a copyright license at all; and it is my opinion that no one should want that to be the case, why would people on the Internet want to argue for an expansion of copyright law?

  • source
  • parent
  • hideshow 18 child comments
  • [–] 37 points 11 months ago

    Saying an expensive product that requires servers to run is the only thing exempt from copyright is just handing a bunch of giant corporations a get out of jail free card.

    Either reform copyright so more things are public domain or require AI companies to pursue licenses to training data.

    Giving an unfair exemption to copyright laws solely to giant tech companies is just another corporate handout.

  • source
  • parent
  • [–] 20 points 11 months ago*

    What I want is consistency, either apply the law equally and fairly or reform the whole system. Nobody, especially not big business, should be getting special carve-outs to be exempt from copyright infringement outside of 'fair use' considerations.

    In my ideal world, IP law would be framed to protect novel ideas just long enough for inventors or creators to capitalize on their ideas and prevent outright 1:1 copying without any sort of innovative or transformational changes. It would also discourage squatting on things like patents- patent squatting and the like should lead to losing rights.

  • source
  • parent
  • [–] 14 points 11 months ago (9 children)

    As with all things, nuance and context is required. I don't think we should be taxing poor people that heavily (if at all), but does that mean I should be against taxing the ultra-wealthy more? Obviously not.

    I support copyright to protect developers and not hinder users, hobbyists, or the average person. I don't support it to only help massive companies who can manipulate the law to protect them from competition, but also not hinder them from stealing from the masses. They can afford to pay. If AI is actually as valuable as they say, the price of paying for the training data is trivial.

    Copyright shouldn't only be helpful to big businesses. It should be most helpful to the average person. We have the opposite here. I support modifying copyright law to bind big businesses and liberate individuals. I don't need to be totally against it like you imply.

  • source
  • parent
  • hideshow 9 child comments
  • [–] 5 points 11 months ago* (2 children)

    But we can't afford to pay. I don't think open models like the one in the OP article would be developed and released for free to the public if there was a complex process of paying billions of dollars to rightsholders in order to do so. That sort of model would favor a monopoly of centralized services run only by the biggest companies.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 0 points 11 months ago* (1 child)

    The model should take into account income. For an open-source model it should be free. It's using public data to produce a public product. For a for-profit model it should be paid. If they're profiting off of public data then they should have to pay for the right to use it.

    We can't afford to make any of this. We don't have the money for the compute required or to pay for the lawyers to make the law work for us. It should benefit the people, so it needs to change. It needs to be "expanded" (I wouldn't call it that, rather "modified" but I'll use your word) in that it currently only protects the wealthy and binds the poor. It should be the opposite.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 11 months ago

    We can’t afford to make any of this. We don’t have the money for the compute required or to pay for the lawyers to make the law work for us

    I don't think this is entirely true; yeah, large foundational models have training costs that are beyond the reach of individuals, but plenty can be done that is not, or can be done by a relatively small organization. I can't find a direct price estimate for Apertus, and it looks like they used their own hardware, but it's mentioned they used ten million gpu hours, and GH200 gpus; I found a source online claiming a rental cost of $1.50 per hour for that hardware, so I think the cost of training this could be loosely estimated to be something around 20 million dollars.

    That is a lot of money if you are one person, but it's an order of magnitude smaller than the settlements of billions of dollars being paid so far by the biggest AI companies for their hasty unauthorized use of copyrighted materials. It's easy to see how copyright and legal costs could potentially be the bottleneck here preventing smaller actors from participating.

    It should benefit the people, so it needs to change. It needs to be “expanded” (I wouldn’t call it that, rather “modified” but I’ll use your word) in that it currently only protects the wealthy and binds the poor. It should be the opposite.

    How would that even work though? Yes, copyright currently favors the wealthy, but that's because the whole concept of applying property rights to ideas inherently favors the wealthy. I can't imagine how it could be the opposite even in theory, but in practice, it seems clear that any legislation codifying limitations on use and compensation for AI training will be drafted by lobbyists of large corporate rightsholders, at the obvious expense of everyone with an interest in free public ownership and use of AI technology.

  • source
  • parent
  • [–] 5 points 11 months ago* (last edited 11 months ago) (5 children)

    Sadly, we’ll most likely see an influx of regulation right when it’s broadly accessible to the general public to run locally.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 3 points 11 months ago (4 children)

    Yeah, most likely, and it'll only bind users and protect the businesses, as always.

    It already is broadly accessible to the general public. They just don't know about it or just accept using one of the cloud versions. It's trivial to get up and running at this point.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 1 point 11 months ago (3 children)

    That’s news to me, unless you’re only referring to the smaller models. Any chance you can run a model that exceeds your ram capacity yet?

  • source
  • parent
  • hideshow 3 child comments
  • [–] 2 points 11 months ago (2 children)

    This is probably the easiest tool I've used to run them: https://lmstudio.ai/

    There's tons of models available here, some of them fairly large: https://huggingface.co/

    No, I'm pretty sure there's no way to run any larger than your RAM/VRAM, at least not automatically. You can use storage as RAM, but that's probably not a good idea. It's orders of magnitude slower. You're better off running a smaller model.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 11 months ago (1 child)

    I’m not knowledgeable in this area, but I wish there was a way to partition the model and stream the partitions over the input, allowing for some kind of serially processing of models that do exceed memory. Like if I could allocate 32gb of ram, and process a 500gb model but at (500/32) a 15x slower rate.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 11 months ago

    The way that could be done would be significantly worse than 15 slower. That's the issue. Even with the fastest storage, moving things between RAM and storage creates massive bottlenecks.

    There are ways to reduce this overhead by intelligently timing moving pieces between storage and RAM, but storage is slow. I don't know how the models work, if it is possible to know what will be needed soon, so you can start moving it into RAM before it's needed. If that can be done then it wouldn't be impossibly bad, but if it can't then we're talking something like 100x slower maybe. Most of these are already pretty slow on consumer hardware, so that'd be effectively unusable. You'd be waiting hours for responses.

  • source
  • parent
  • [–] 8 points 11 months ago (4 children)

    Why would it be an expansion? If you're using someone else's work, why wouldn't you need a license? If I write a book and publish it under CC-BY-NC, should Google be allowed to take my work for their commercial product without compensation or even attribution? Should Microsoft be allowed to create closed-source commercial Copilot off GPL source code?

  • source
  • parent
  • hideshow 4 child comments
  • [–] 3 points 11 months ago*

    BSD license allows for this and still thrives (PS5 OS, Apple iOS and MacOS [Darwin], TrueNAS, OPNsense, and several enterprise-level commercial router operating systems use it and contribute significant code back into BSD project to ensure CVE safety). I'm not agreeing with it, just providing an alternate perspective.

  • source
  • parent
  • [–] 5 points 11 months ago (2 children)

    It's an expansion to say that LLM training constitutes a derivative work. You are of course entitled to your opinion that it should be the case; all I can say to that is that in the 2000s and 2010s nearly everyone on the Internet tended to argue for more limitations, not further expansions, of copyright law, and I wonder what happened to that attitude.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 7 points 11 months ago* (last edited 11 months ago) (1 child)

    Well, this being the open source community, I would expect most people here to be on the side of respecting the rights of content creators. Like I said, if I write some GPL software, I don't think Microsoft should be able to disrespect my license just because they're also disrespecting everyone else's license too through automation at scale.

    Edit: forgot to mention, since their product is wholly dependent on the other works, that's the very definition of a derivative work. While you could argue it's transformative, it certainly fails the other tests for fair use.

  • source
  • parent
  • hideshow 1 child comment
  • load more comments (1 reply)
    [–] 21 points 11 months ago
    [–] 10 points 11 months ago (11 children)

    I can’t find any hardware requirements for this. What will it take to run this smoothly?

  • source
  • hideshow 11 child comments
  • [–] 8 points 11 months ago

    For fastest inference, you want to fit the entire model in VRAM. Plus, you need a few GB extra for context.

    Context means the text (+images, etc) it works on. That's the chat log, in the case of a chatbot, plus any texts you might want summarized/translated/ask questions about.

    Models can be quantized, which is a kind of lossy compression. They get smaller but also dumber. As with JPGs, the quality loss is insignificant at first and absolutely worth it.

    Inference can be split between GPU and CPU, substituting VRAM with normal RAM. Makes it slower, but you'll probably will still feel that it's smooth.

    Basically, it's all trade-offs between quality, context size, and speed.

  • source
  • parent
  • [–] 11 points 11 months ago* (last edited 11 months ago) (9 children)

    8b parameter models are relatively fast on 3rd gen RTX hardware with at least 8gigs of vram, CPU inferencing is slower and requires boatloads of ram but is doable on older hardware. These really aren't designed to run on consumer hardware, but the 8b model should do fine on relatively powerful consumer hardware.

    If you have something that would've been a high end gaming rig 4 years ago, you're good.

    If you wanna be more specific, check huggingface, they have charts. If you're using linux with nvidia hardware you'll be better off doing CPU inferencing.

    Edit: Omg y'all I didn't think I needed to include my sources but this is quite literally a huge issue on nvidia. Nvidia works fine on linux but you're limited to whatever VRAM is on your video card, no RAM sharing. Y'all can disagree all you want but those are the facts. Thays why AMD and CPU inferencing are more reliable, and allow for higher context limits. They are not faster though.

    Sources for nvidia stuff https://github.com/NVIDIA/open-gpu-kernel-modules/discussions/618

    https://forums.developer.nvidia.com/t/shared-vram-on-linux-super-huge-problem/336867/

    https://github.com/NVIDIA/open-gpu-kernel-modules/issues/758

    https://forums.opensuse.org/t/is-anyone-getting-vram-backed-by-system-memory-with-nvidia-drivers/185902

  • source
  • parent
  • hideshow 9 child comments
  • [–] 2 points 11 months ago (7 children)

    Disagree on Linux nvidia support, it works fine

  • source
  • parent
  • hideshow 7 child comments
  • [+] 2 points 11 months ago (7 children)
  • [–] 2 points 11 months ago (6 children)

    So that's only in the case of sharing RAM? Because the vast majority of people use Nvidia and Linux without issue, myself included.

  • source
  • parent
  • hideshow 6 child comments
  • [–] 3 points 11 months ago* (last edited 11 months ago) (5 children)

    Yes, precisely.

    If you're trying to use large models, you need more RAM than consumer grade nvidia products can supply. Without system ram sharing, the models error out and start repeating themselves or just crash and need to be restarted.

    This can be fixed with CPU inferencing but would be much slower.

    An 8b model will run fine on an RTX30 series, a 70b model will absolutely not. BUT you can do cpu inferencing with the 70b model if you don't mind the wait.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 2 points 11 months ago

    To clarify: NVIDIA does allow system RAM paging (Unified Memory), but it’s not stable or recommended for ML inference. AMD ROCm supports real page-faulting + memory oversubscription. AMD can therefore run larger models on limited VRAM. AMD is not necessarily faster — usually slower. CPU inference is never faster than a GPU.

  • source
  • parent
  • [–] 2 points 11 months ago (2 children)

    No shit lmao, are you going to tell me a horse can't pull an 18 wheeler trailer next?

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 11 months ago (1 child)
    1. You don't need to be rude.

    2. My original comment was in reply to someone looking for this type of information, the conversation then continued.

    3. Disengage: I don't want to deal with it today frankly, I don't have time for rude people.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 11 months ago* (last edited 11 months ago)

    Its a leap to say "nvidia AI support on Linux is bad" when you mean a very particular set of circumstances (which don't apply to someone who would just be getting into it as they're using consumer grade hardware) causes issues.

    I have a 3080ti and run 8b - 12b all in VRAM just fine, which is what a majority of people getting into it would be doing as well, again to my point about pulling an 18 wheeler trailer with a horse, you've got worse problems then nvidia on Linux (if you're trying to run a 70b model on consumer hardware).

  • source
  • parent
  • load more comments (-1 replies)
  • [+] -21 points 11 months ago (7 children)

    Obligatory nitpick: open weights ≠ open source. For it to be open source, they need to release the training data as well as all the parameters they used in training it.

  • source
  • hideshow 7 child comments
  • [–] [S] 71 points 11 months ago (5 children)

    Please read the article before commenting.

    "The model is named Apertus – Latin for “open” – highlighting its distinctive feature: the entire development process, including its architecture, model weights, and training data and recipes, is openly accessible and fully documented."

  • source
  • parent
  • hideshow 5 child comments
  • [+] -11 points 11 months ago (1 child)

    Though I didn't see the link to any repos or anything for confirmation.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 24 points 11 months ago

    Scroll down to "Further Information".

    PublicAI link is a public access point to the model, which can be tried directly at chat.publicai.co. This is also open, see Github.com/forpublicai/ (shown in the links I'm mentioning).

    8b and 70b models and instructions can be downloaded from huggingface, also linked under "further information".

    If you go to the instructs, you'll also be linked to github.com/swiss-ai/

    I haven't looked through it all, but everything appears to be there if you follow the links for additional information.

  • source
  • parent