So what is currently the best and easiest way to use an AMD GPU for reference I own a rx6700xt and wanted to run 13B model maybe superhot but I鈥檓 not sure if my vram is enough for that Since now I always sticked with llamacpp since it鈥檚 quiet easy to setup Does anyone have any suggestion?
You must log in or # to comment.
Just pay nvidia their ill-earned ounce of flesh. I say this as a strong AMD advocate.
It鈥檚 clear that AMD isn鈥檛 serious about the AI market. They had years to provide a proper competitor to CUDA or at the very least a 1:1 compatibility layer. Instead of doing either of these things, AMD continued messing with half-assed projects like ROCm and the other one the name of which I don鈥檛 care to look up. AMD has the resources to build a CUDA compatible API in under 6 months but for some reason they don鈥檛. I don鈥檛 know why they don鈥檛, and at this point I don鈥檛 really care.
Buy an AMD GPU for AI at your own risk.