Office space meme:
“If y’all could stop calling an LLM “open source” just because they published the weights… that would be great.”
Yeah, this shit drives me crazy. Putting aside the fact that it all runs off stolen data from regular people who are being exploited, most of this “AI” shit is basically just freeware if anything, it’s about as “open source” as Winamp was back in the day.
I like how when America does it we call it AI, and when China does it it’s just an LLM!
I’m including Facebook’s LLM in my critique. And I dislike the current hype on LLMs, no matter where they’re developed.
And LLMs are not “AI”. I’ve called them “so-called ‘AIs’” waaay before.
Or as a human without all the previous people’s examples we learned from without paying them, aka normal life.
Would you accept a Smalltalk image as Open Source?
Meta’s “open source AI” ad campaign is so frustrating.
Judging by OP’s salt in the comments, I’m guessing they might be an Nvidia investor. My condolences.
Nah, just a 21st century Luddite.
I mean that’s all a model is so… Once again someone who doesn’t understand anything about training or models is posting borderline misinformation about ai.
Shocker
A model is an artifact, not the source. We also don’t call binaries “open-source”, even though they are literally the code that’s executed. Why should these phrases suddenly get turned upside down for AI models?
A model can be represented only by its weights in the same way that a codebase can be represented only by its binary.
Training data is a closer analogue of source code than weights.
Yet another so-called AI evangelist accusing others of not understanding computer science if they don’t want to worship their machine god.
Do you think your comments here are implying an understanding of the tech?
It’s not like you need specific knowledge of Transformer models and whatnot to counterargue LLM bandwagon simps. A basic knowledge of Machine Learning is fine.
And you believe you’re portraying that level of competence in these comments?
I at least do.
I mean if you both think this is overhyped nonsense, then by all means buy some Nvidia stock. If you know something the hedge fund teams don’t, why not sell your insider knowledge and become rich?
Or maybe you guys don’t understand it as well as you think. Could be either, I guess.
Yeah, let’s all base our decisions and definitions on what the stock market dictates. What could possibly go wrong?
/s 🙄
Because over-hyped nonsense is what the stock market craves… That’s how this works. That’s how all of this works.
I didn’t say it is all overhyped nonsense, my only point is that I agree with the opinion stated in the meme, and I don’t think people who disagree really understand AI models or what “open source” means.
Praise the Omnisiah! … I’ll see myself out.
Even worse is calling a proprietary, absolutely closed source, closed data and closed weight company “OpeanAI”
Especially after it was founded as a nonprofit with the mission to push open source AI as far and wide as possible to ensure a multipolar AI ecosystem, in turn ensuring AI keeping other AI in check so that AI would be respectful and prosocial.
Sorry, that was a PR move from the get-go. Sam Altman doesn’t have an altruistic cell in his whole body.
It’s even crazier that Sam Altman and other ML devs said that they reached the peak of what current Machine Learning models were capable of years ago
But that doesn’t mean shit to the marketing departments
“Look at this shiny.”
Investment goes up.
“Same shiny, but look at it and we need to warn you that we’re developing a shinier one that could harm everyone. But think of how shiny.”
Investment goes up.
“Look at this shiny.”
Investment goes up.
“Same shiny, but look at it and we need to warn you that we’re developing a shinier one that could harm everyone. But think of how shiny.”
American magic bean companies like Beanco, The Boston Bean Company, and Nvidia
Lol
The training data would be incredible big. And it would contain copyright protected material (which is completely okay in my opinion, but might invoice criticism). Hell, it might even be illegal to publish the training data with the copyright protected material.
They published the weights AND their training methods which is about as open as it gets.
Hell, for all we know it could be full of classified data. I guess depending on what country you’re in it definitely is full of classified data…
They could disclose how they sourced the training data, what the training data is and how you could source it. Also, did they publish their hyperparameters?
They could jpst not call it Open Source, if you can’t open source it.
For neural nets the method matters more. Data would be useful, but at the amount these things get trained on the specific data matters little.
They can be trained on anything, and a diverse enough data set would end up making it function more or less the same as a different but equally diverse set. Assuming publicly available data is in the set, there would also be overlap.
The training data is also by necessity going to be orders of magnitude larger than the model itself. Sharing becomes impractical at a certain point before you even factor in other issues.
That… Doesn’t align with years of research. Data is king. As someone who specifically studies long tail distributions and few-shot learning (before succumbing to long COVID, sorry if my response is a bit scattered), throwing more data at a problem always improves it more than the method. And the method can be simplified only with more data. Outside of some neat tricks that modern deep learning has decided is hogwash and “classical” at least, but most of those don’t scale enough for what is being looked at.
Also, datasets inherently impose bias upon networks, and it’s easier to create adversarial examples that fool two networks trained on the same data than the same network twice freshly trained on different data.
Sharing metadata and acquisition methods is important and should be the gold standard. Sharing network methods is also important, but that’s kind of the silver standard just because most modern state of the art models differ so minutely from each other in performance nowadays.
Open source as a term should require both. This was the standard in the academic community before tech bros started running their mouths, and should be the standard once they leave us alone.
Open sources will eventually surpass all closed-source softwares in some day, no matter how many billions of dollars are invested in them.
Never have I used open source software that has achieved that, or was even close to achieving it. Usually it is opinionated (you need to do it this way in this exact order, because that’s how we coded it. No, you cannot do the same thing but select from the back), lacks features and breaks. Especially CAD - comparing Solidworks to FreeCAD for instance, where in FreeCAD any change to previous ops just breaks everything. Modelling software too - Blender compared to 3ds Max - can’t do half the things.
- 7-zip
- VLC
- OBS
- Firefox did it only to mostly falter to Chrome but Chrome is largely Chromium which is open source.
- Linux (superseded all the Unix, very severely curtailed Windows Server market)
- Nearly all programming language tools (IDEs, Compilers, Interpreters)
- Essentially all command line ecosystem (obviously on the *nix side, but MS was pretty much compelled to open source Powershell and their new Terminal to try to compete)
In some contexts you aren’t going to have a lively enough community to drive a compelling product even as there’s enough revenue to facilitate a company to make a go of it, but to say ‘no open source software has acheived that’ is a bit much.
While I completely agree with 90% of your comment, that first sentence is gross hyperbole. I have used a number of pieces of open source options that are are clearly better. 7zip is a perfect example. For over a decade it was vastly superior to anything else, open or closed. Even now it may be showing its age a bit, but it is still one of the best options.
But for the rest of your statement, I completely agree. And yes, CAD is a perfect example of the problems faced by open source. I made the mistake of thinking that I should start learning CAD with open source and then I wouldn’t have to worry about getting locked into any of the closed source solutions. But Freecad is such a mess. I admit it has gotten drastically better over the last few years, but it still has serious issues. Don’t get me wrong, I still 100% recommend that people learn it, but I push them towards a number of closed source options to start with. Freecad is for advanced users only.I reckon C++ > Delphi
Just look at blender vs maya for example.
There are lots of problems with the new lingo. We need to come up with new words.
How about “Open Weightings”?
That’s fat shaming
That sounds like a segment on “My 600lb Life”
It’s not just the weights though is it? You can download the training data they used, and run your own instance of the model completely separate from their servers.
You don’t download the training data when running an LLM locally. You are downloading the already baked model.
Did “they” publish the training data? And the hyperparameters?
I mean, I downloaded it from the repo.
You downloaded the weights. That’s something different.
I may misunderstand, but are the weights typically several hundred gigabytes large?
Yes. The training data is probably a few hundred petabytes.
Oh wow that’s fuckin huge
Yeah, some models are trained on pretty much the entire content of the publicly accessible Internet.
Arguably they are a new type of software, which is why the old categories do not align perfectly. Instead of arguing over how to best gatekeep the old name, we need a new classification system.
… Statistical engines are older than personal computers, with the first statistical package developed in 1957. And AI professionals would have called them trained models. The interpreter is code, the weights are not. We have had terms for these things for ages.
Is it even really software, or just a datablob with a dedicated interpreter?
Isn’t all software just data plus algorithms?
Well, yes, but usually it’s the code that’s the main deal, and the part that’s open, and the data is what you do with it. Here, the training weights seem to be “it”, so to speak.
There were e|forts. Facebook didn’t like those. (Since their models wouldn’t be considered open source anymore)
I don’t care what Facebook likes or doesn’t like. The OSS community is us.
what a weird hill to die on
You can do sneaky things with weights that are virtually undetectable.
On the contrary. What they open sourced was just a small part of the project. What they did not open source is what makes the AI tick. Having less than one percent of a project open sourced does not make it an “Open Source” project.
Open weights
Yes please, let’s use this term, and reserve Open Source for it’s existing definition in the academic ML setting of weights, methods, and training data. These models don’t readily fit into existing terminology for structure and logistic reasons, but when someone says “it’s got open weights” I know exactly what set of licenses and implications it may have without further explanation.
