These little buggers are loud, right?
Question about how shared VRAM works
So I need to specify in the BIOS the split, and then it’s dedicated at runtime, or can I allocate VRAM dynamically as needed by workload?
On macos you don’t really have to think about this, so wondering how this compares.
On my 7800, it’s static. The 2GB I allocate is not usable for the CPU, and compute apps don’t like it “overflowing” past that.
This is on Linux, on a desktop, ASRock mobo. YMMV.
It’s typically dynamic
It’s kinda cool but seems a bit expensive at this moment.
For the performance, it’s actually quite reasonable. 4070-like GPU performance, 128gb of memory, and basically the newest Ryzen CPU performance, plus a case, power supply, and fan, will run you about the same price as buying a 4070, case, fan, power supply, and CPU of similar performance. Except you’ll actually get a faster CPU with the Framework one, and you’ll also get more memory that’s accessible by the GPU (up to the full 128gb minus whatever the CPU is currently using)
Calling it a gaming PC feels misleading. It’s definitely geared more towards enterprise/AI workloads. If you want upgradeable just buy a regular framework. This desktop is interesting but niche and doesn’t seem like it’s for gamers.
Soldered on ram and GPU. Strange for Framework.
Ye the soldered ram is for sure making me doubt framework now.
Apparently AMD wasn’t able to make socketed RAM work, timings aren’t viable. So Framework has the choice of doing it this way or not doing it at all.
In that case, not at all is the right choice until AMD can figure out that frankly brain dead easy thing.
Oh yeah I’m sure you could’ve done it no problem
“brain dead easy thing”… All you need is to just manage signal integrity of super fast speed ram to a super hungry state of the art soc that benefits from as fast of memory as it can get. Sounds easy af. /s
They said that it was possible, but they lost over half of the speed doing it, so it was not worth it. It would severely cripple performance of the SOC.
The only real complaint here is calling this a desktop, it’s somewhere in between a NUC and a real desktop. But I guess it technically sits on a desk top, while also being an itx motherboard.
Signal integrity is a real issue with dimm modules. It’s the same reason you don’t see modular VRAM on GPUs. If the ram needs to behave like VRAM, it needs to run at VRAM speeds.
Then don’t make it work like that. Desktop PCs are modular and Framework made a worse product in terms of modularity and repairability, the main sales of Framework. Just, like… wtf. This Framework product is cursed and shouldn’t exist.
There’s little point in framework selling a conventional desktop.
I guess they could have made another laptop size with the the dev time, but… I dunno, this seems like a niche that needs to be filled.
This is where I’m at. The Framework guy was talking about how very few companies are using this AMD deal because the R&D to add it to existing models wasn’t very viable, you really only have the Asus Z13 so I feel like being ahead of the game there will be a benefit in the long run as far as their relationship with AMD. Plus they’re also doing a 12-in laptop now as well, so it’s not like they committed all their resources to this.
Apparently AMD couldn’t make the signal integrity work out with socketed RAM. (source: LTT video with Framework CEO)
IMHO: Up until now, using soldered RAM was lazy and cheap bullshit. But I do think we are at the limit of what’s reasonable to do over socketed RAM. In high performance datacenter applications, socketed RAM is on it’s way out (see: MI300A, Grace-{Hopper,Blackwell},Xeon Max), with onboard memory gaining ground. I think we’ll see the same trend on consumer stuff as well. Requirements on memory bandwidth and latency are going up with recent trends like powerful integrated graphics and AI-slop, and socketed RAM simply won’t work.
It’s sad, but in a few generations I think only the lower end consumer CPUs will be possible to use with socketed RAM. I’m betting the high performance consumer CPUs will require not only soldered, but on-board RAM.
Finally, some Grace Hopper to make everyone happy: https://youtube.com/watch?v=gYqF6-h9Cvg
I definitely wouldn’t mind soldered RAM if there’s still an expansion socket. Solder in at least a reasonable minimum (16G?) and not the cheap stuff but memory that can actually use the signal integrity advantage, I may want more RAM but it’s fine if it’s a bit slower. You can leave out the DIMM slot but then have at least one PCIe x16 expansion slot. A free one, one in addition to the GPU slot. PCIe latency isn’t stellar but on the upside, expansion boards would come with their own memory controllers, and push come to shove you can configure the faster RAM as cache / the expansion RAM as swap.
Heck, throw the memory into the CPU package. It’s not like there’s ever a situation where you don’t need RAM.
All your RAM needs to be the same speed unless you want to open up a rabbit hole. All attempts at that thus far have kinda flopped. You can make very good use of such systems, but I’ve only seen it succeed with software specifically tailored for that use case (say databases or simulations).
The way I see it, RAM in the future will be on package and non-expandable. CXL might get some traction, but naah.
The cache hierarchy has flopped? People aren’t using swap?
NUMA also hasn’t flopped, it’s just that most systems aren’t multi socket, or clusters. Different memory speeds connected to the same CPU is not ideal and you don’t build a system like that but among upgraded systems that’s not rare at all and software-wise worst thing that’ll happen is you get the lower memory speed. Which you’d get anyway if you only had socketed RAM.
Yeah, the cache hierarchy is behaving kinda wonky lately. Many AI workloads (and that’s what’s driving development lately) are constrained by bandwidth, and cache will only help you with a part of that. Cache will help with repeated access, not as much with streaming access to datasets much larger than the cache (i.e. many current AI models).
Intel already tried selling CPUs with both on-package HBM and slotted DDR-RAM. No one wanted it, as the performance gains of the expensive HBM evaporated completely as soon as you touched memory out-of-package. (Assuming workloads bound by memory bandwidth, which currently dominate the compute market)
To get good performance out of that, you may need to explicitly code the memory transfers to enable prefetch (preferably asynchronous) from the slower memory into the faster, á la classic GPU programming. YMMW.
In systems where memory speed are mismatched, the system runs at the slowest module’s speed. So literally making the soldered, faster memory slower. Why even have soldered memory at that point?
I’d assume the soldered memory to have a dedicated memory controller. There’s also no hard requirement that a single controller can’t drive different channels at different speeds. The only hard requirement is that one channel needs to run at one speed.
…and the whole thing becomes completely irrelevant when we’re talking about PCIe expansion cards the memory controller doesn’t care.
Couldn’t you just treat the socketed ram like another layer of memory effectively meaning that L1-3 are on the CPU “L4” would be soldered RAM and then L5 would be extra socketed RAM? Alternatively couldn’t you just treat it like really fast swap?
Could it work?
Yes, but it would require:
- A redesigned memory controller capable of tiering RAM (which would be more complex).
- OS-level support for dynamically assigning memory usage based on speed (Operating systems and applications assume all RAM operates at the same speed).
- Applications/libraries optimized to take advantage of this tiering.
Right now, the easiest solution for fast, high-bandwidth RAM is just to solder all of it.
Wrote a longer reply to someone else, but briefly, yes, you are correct. Kinda.
Caches won’t help with bandwidth-bound compute (read: ”AI”) it the streamed dataset is significantly larger than the cache. A cache will only speed up repeated access to a limited set of data.
Using it as cache would reduce total capacity as cache implies coherence, and treating it as ordinary swap would mean copying to main memory before you access it which is silly when you can access it directly. That is you’d want to write a couple of lines of kernel code to use it effectively but it’s nowhere close to rocket science. Nowhere near as complicated as making proper use of NUMA architectures.
Honestly I upgrade every few years and isually have to purchase a new mobo anyhow. I do think this could lead to less options for mobos though.
I get it but imagine the GPU style markup when all mobos have a set amount of RAM. You’ll have two identical boards except for $30 worth of memory with a price spread of $200+. Not fun.
I don’t think you are wrong, but I don’t think you go far enough. In a few generations, the only option for top performance will be a SoC. You’ll get to pick which SoC you want and what box you want to put it in.
the only option for top performance will be a SoC
System in a Package (SiP) at least. Might not be efficient to etch the logic and that much memory onto the same silicon die, as the latest and greatest TSMC node will likely be much more expensive per square mm than the cutting edge memory production node from Samsung or whatever foundry where the memory is being made.
But with advanced packaging going the way it’s been over the last decade or so, it’s going to be hard to compete with the latency/throughout of an in-package interposer. You can only do so much with the vias/pathways on a printed circuit board.
You are correct, I’m referring to on package. Need more coffee.
Really framework ? Soldered ram ? How dissapointing
The CEO of Framework said that this was because the CPU doesn’t support unsoldered RAM. He added that they asked AMD if there was any way they could help them support removable memory. Supposedly an AMD engineer was tasked with looking into it, but AMD came back and said that it wasn’t possible.
Specifically AMD said that it’s achievable but you’ll be operating at approx 50% of available bandwidth, and that’s with LPCAMM2. SO/DIMMs are right out of the running.
Mostly this is AMDs fault but if you want a GPU with 96-110 GBs of memory you don’t really have a choice.
At first I was skeptical during the announcement and then I saw the amount of ram and the rack. Imho it is not for enduser but for business. In fact we have workloads that would be perfectly fit that computer so why not?
The Framework Desktop is powered by an AMD Ryzen AI Max processor, a Radeon 8060S integrated GPU, and between 32GB and 128GB of soldered-in RAM.
The CPU and GPU are one piece of silicon, and they’re soldered to the motherboard. The RAM is also soldered down and not upgradeable once you’ve bought it, setting it apart from nearly every other board Framework sells.
It’d raise an eyebrow if it was a laptop but it’s a freakin’ desktop. Fuck you framework.
insanely hostile response to something like this. they attempted to have these parts replaceable, AMD physically couldn’t do it. they’ve still made it as repairable as possible, and will without a doubt be more repairable than similar devices using this chipset. fucking relax, being reactionary without being informed is dumb.
I get the frustration with a system being so locked down, but if 32gb is the minimum I don’t really see the problem. This pc will be outdated before you really need to upgrade the ram to play new games.
It’s not just about upgrading. It’s also about being able to repair your computer. RAM likes to go bad and on a normal PC, you can replace it easily. Buy a cheap stick, take out the old RAM, put in the new one and you’ll have a working computer again. Quick & easy and even your grandpa is able to run Memtest and do a quick switch. But if you solder down everything, the whole PC becomes electronic waste as most people won’t be able to solder RAM.
Yeah, it totally fucks repairability. But it sounds like this is not something this company normally does, and not something they could control.
They should at least offer a superior warranty to cover such scenarios.
The hell you mean “not something they could control”? Their whole deal is making upgradeable, repairable devices and ram thats replaceable is no industry secret. Their options should have been make it work or dont make it at all.
If you read into it, this was a limitation on AMD’s part, which they tried to resolve. You don’t have to buy it, and the rest of their lineup should meet your expectations.
You know, they didn’t have to make a product with this specific chip. They did it anyway despite it being inherently incompatible with their former goals. And this is not about us customers, this is about Framework abandoning what they stood for and losing credibility in the process.
Seriously that’s really disappointing. It really seems like investors decides that they needed to “diversify” their offering and they need something with AI now … Framework was on a good path imo but of course a repairable laptop only goes so far since people can repair it and don’t need to replace it every 2 years (or maybe just replace the motherboard) so if you want to grow you need to make more products …
Well tbh they still do have repairable laptops, even new ones and all that, and the “excuse” is that the only way to properly use that specific AMD CPU is with that specific RAM and the non.soldered bus wasn’t enough, but still… i’ll stick to old #ThinkPads, thank you.
I agree. We need less soldered RAM designs. I thought repairability was something they appreciated.
Honestly this is exactly the product I was waiting for minisforum to make. I think this is actually a pretty solid move.
This is not really that interesting and kinda weird given the non-upgradability, but I guess it’s good for AI workloads. It’s just not that unique compared to their laptops.
I’d love a mid-tower case with swappable front panel I/O and modular bays for optical drives; would’ve been the perfect product for Framework to make IMO.
The mini’s are the latest new hotness for desktop computing. I’ve been running a dirt cheap $90US, mini for 2 years now. It fits extremely well on my desk, just tucked in under the monitor leaving plenty of room for all the other tasks I do daily.
Will it play the latest hot new video game? Nope. But it will run OnlyOffice, FreeCAD and FreeDoom just fine.
and more at people who want the smallest, most powerful desktop they can build
Well, there’s this:
Yeah, the screw holes didn’t fit, that’s why. And the cooler didn’t fit the case, obviously. And the original cooler not the CPU’s turbo.
The bowing on that board makes me think it’s not much longer for this world.
What’s crazy is I still can’t make it onto their website without waiting in a 20 minute queue. Stupid.
Are they going to at least make memory modules available for those who want to solder their own?
You can order those directly from chip suppliers (mouser, digikey, arrow, etc.) for a lower cost than you could get them from framework. Also those are going to be very difficult to solder/desolder. You’re going to need a hot air station, and you need to pre-warm the board to manage the heat sink from the ground planes.
This is one stupid product. It really goes against everything the framework brand has identified with.
It’s a straight up gimmick flanderizing the brand identity.
Desktops are already that, though. In order for them to distinguish themselves in the industry, they can’t just offer another modular desktop PC. They can’t offer prebuilts, or gaming towers, or small form factor units, or pre-specced you-build kits. They can’t even offer low-cost micro-desktops. All of those markets are saturated.
But they can offer a cheap Mac Studio alternative. Nobody’s cracked that nut yet. And it remains to be seen if this will be it, but it certainly seems like it’s lined up to.
I’d argue not. It’s as modular/repairable as the platform can be (with them outright stating the problematic soldered RAM), and not exorbitantly priced for what it is.
But what I think is most “Framework” is shooting for a niche big OEMs have completely flubbed or enshittified. There’s a market (like me) that wants precisely this, not like a framework-branded gaming tower or whatever else a desktop would look like.
It’s as modular/repairable as the platform can be
It can’t be. That’s the point.
AMD said no due to the platform and apparently the signal integrity not being up to snuff.
…said no to what?
Modular Ram modules (e.g. dram and I believe lpcamm)
Soldered ram is more efficient because it does not require big connectors and is closer to the CPU and GPU. 3D Vcache Is the ultimate examples or this.
Yes I’m aware. What’s your point?
I guess I’m not sure what you want Framework to due instead. Just not launch this at all? What alternative are you advocating for?
This is a standard a370 mini PC at a high price.
There’s Beelink, Minisforum, Aoostar and many others.
The AI max chips are a completely different platform, more than double the physical silicon size of most minipc chips.
Most miniPC vendors have already announced AI Max products:
https://www.gmktec.com/blog/gmktec-a-global-leader-in-ai-mini-pcs-unveils-the-amd-ryzen-ai-max-395
But Framework released it now.
Nope, they’re just available to pre-order with an estimate of shipping from Q3 this year. They’re not shipping now.
I just checked, the gmktec link above says that their product is supposed to ship in Q1-Q2 this year, earlier than framework’s.The person above you confidently posted that it’s shipping now and got an up vote despite being wrong.
I never understand people who confidently post wrong things that are easily googlable.
Not really sure who this is for. With soldered RAM is less upgradeable than a regular PC.
AI nerds maybe? Sure got a lot of RAM in there potentially attached to a GPU.
But how capable is that really when compared to a 5090 or similar?
Not really sure who this is for.
Second sentence in the linked article.
The 5090 is basically useless for AI dev/testing because it only has 32GB. Mind as well get an array of 3090s.
The AI Max is slower and finicky, but it will run things you’d normally need an A100 the price of a car to run.
But that aside, there are tons of workstations apps gated by nothing but VRAM capacity that this will blow open.
… but only OpenCL workloads, right?
Not exactly. OpenCL as a compute framework is kinda dead.
What types of compute can you run on an AMD GPU today?
No, it runs off integrated graphics, which is a good thing because you can have a large capacity of ram dedicated to GPU loads
Useless is a strong term. I do a fair amount of research on a single 4090. Lots of problems can fit in <32 GB of VRAM. Even my 3060 is good enough to run small scale tests locally.
I’m in CV, and even with enterprise grade hardware, most folks I know are limited to 48GB (A40 and L40S, substantially cheaper and more accessible than A100/H100/H200). My advisor would always say that you should really try to set up a problem where you can iterate in a few days worth of time on a single GPU, and lots of problems are still approachable that way. Of course you’re not going to make the next SOTA VLM on a 5090, but not every problem is that big.
Fair. True.
If your workload/test fits in 24GB, that’s already a “solved” problem. If it fits in 48GB, it’s possibly solved with your institution’s workstation or whatever.
But if it takes 80GB, as many projects seem to require these days since the A100 is such a common baseline, you are likely using very expensive cloud GPU time. I really love the idea of being able to tinker with a “full” 80GB+ workload (even having to deal with ROCM) without having to pay per hour.
Yeah, I agree that it does help for some approaches that do require a lot of VRAM. If you’re not on a tight schedule, this type of thing might be good enough to just get a model running.
I don’t personally do anything that large; even the diffusion methods I’ve developed were able to fit on a 24GB card, but I know with the hype in multimodal stuff, VRAM needs can be pretty high.
I suspect this machine will be popular with hobbyists for running really large open weight LLMs.
This is my use case exactly.
I do a lot of analysis locally, this is more than enough for my experiments and research. 64 to 96gb VRAM is exactly the window I need. There are analyses I’ve had to let run for 2 or 3 days and dealing with that on the cloud is annoying.
Plus this will replace GH Copilot for me. It’ll run voice models. I have diffusion model experiments I plan to run but are totally inaccessible locally to me (not just image models). I’ve got workloads that take 2 or 3 days at 100% CPU/GPU that are annoying to run in the cloud.
This basically frees me from paying for any cloud stuff in my personal life for the foreseeable future. I’m trying to localize as much as I can.
I’ve got tons of ideas I’m free to try out risk free on this machine, and it’s the most affordable “entry level” solution I’ve seen.
And even better, “testing” it. Maybe I’m sloppy, but I have failed runs, errors, hacks, hours of “tinkering,” optimizing, or just trying to get something to launch that feels like an utter waste of an A100 mostly sitting idle… Hence I often don’t do it at all.
One thing you should keep in mind is that the compute power of this thing is not like an A/H100, especially if you get a big slowdown with rocm, so what could take you 2-3 days could take over a week. It’d be nice if framework sold a cheap MI300A, but… shrug.
I don’t mind that it’s slower, I would rather wait than waste time on machines measured in multiple dollars per hour.
I’ve never locked up an A100 that long, I’ve used them for full work days and was glad I wasn’t paying directly.
