> The only consumer GPU with 32GB of RAM in the current generation is the RTX 5090.
Well not really, there is the AMD Radeon AI PRO 9700, which is a RX 9070 (or possibly the XT, dunno) with 32GB VRAM. It does use slower GDDR6, but it has "held its price better" (for some definition of better) and is still available for like 2.3k€ (this includes taxes). Though I remember when I was thinking of buying one when it was 1.4k not too long ago and didn't :( now I just don't bother looking anymore.
Slower in what specifically? It wouldn't surprise me if the card is up to 3x slower (it has roughly 3x less bandwidth), but do note that we are comparing a 2.3k€ card vs a +9k€ card here (plus on top of that the power usage on the RTX 5090 can be up to 2x under load).
The RX 7900 XTX has slightly higher bandwidth at almost 1kGBps compared to AI PROs almost 650GBps but has only 24GB instead of 32GB, though I can't find the card on sale anywhere rn.
In the end I see it mostly as a "is the massive jump in price worth it, or do you risk it and maybe get multiple AI 9700s instead and deal with ROCm instead". I can't tell how good ROCm has gotten in regards to training and such, but for inference Vulkan on my 9070 XT is pretty solid.
Well not really, there is the AMD Radeon AI PRO 9700, which is a RX 9070 (or possibly the XT, dunno) with 32GB VRAM. It does use slower GDDR6, but it has "held its price better" (for some definition of better) and is still available for like 2.3k€ (this includes taxes). Though I remember when I was thinking of buying one when it was 1.4k not too long ago and didn't :( now I just don't bother looking anymore.
The RX 7900 XTX has slightly higher bandwidth at almost 1kGBps compared to AI PROs almost 650GBps but has only 24GB instead of 32GB, though I can't find the card on sale anywhere rn.
In the end I see it mostly as a "is the massive jump in price worth it, or do you risk it and maybe get multiple AI 9700s instead and deal with ROCm instead". I can't tell how good ROCm has gotten in regards to training and such, but for inference Vulkan on my 9070 XT is pretty solid.