Unified Memory Concept but for x86 computers?
I was looking at these ‘AI Cluster’ videos where they are using 4 Mac Pro studios and making these clusters work for AI. They kept on talking about how the ‘Unified memory’ made it possible for these Macs to make it run. By the looks of Nvidia being cheapskates with VRAM on their latest GPUs, do you guys reckon the concept of ‘Unified Memory’ will trickle down to x86 computers eventually to the point that GPUs will no longer have VRAM in them but it’ll be on us if we wish to put in more or less memory in our systems depending on our needs?
Eventual Answer:
It won’t be universal but might be more common (since it already exists).
Strix Halo is the best example of a high-end x86 competitor to the Mac Studio and it highlights both the advantages and disadvantages of that arrangement. You can allocate enormous amounts of memory to the integrated GPU for rendering or AI tasks at a far lower cost than the equivalent amount of VRAM in a discrete GPU. But Strix Halo still gets demolished by a RTX Pro 6000 in GPU-based workloads.
People are playing with clustered Mac Studios because it’s a relatively cheap way to stack tons of fairly fast RAM, which is important since AI workloads are primarily memory bottlenecked. A Mac Studio with an M3 Ultra and 256 GB of RAM runs at about 820 GB/s memory bandwidth and costs less than an RTX Pro 6000 which only provides 96 GB of VRAM. That’s a lot better than Strix Halo which runs at 256 GB/s or standard socketed DDR5 which runs at around 90 GB/s depending on speed.
The RTX Pro 6000 though still crushes all of them at nearly 1,800 GB/s. That’s why actual AI companies are treating cards like that as entry level. Processing speeds are much faster especially since you can stack multiple GPUs using PCIe on a single board with faster interconnect speeds than TB5 on a Mac Studio (and data centers use even faster interconnects than consumer PCIe).
from Reddit