Old mining rig is it still useful?

[Jackal65]

Been looking to maybe buy a second hand or new motherboard that can support up to 19 GPUs. These boards was build for mining so I don't know at what speeds those PCIe ports run at . My goal is build a GPU compute cluster see if I can get AI run a bit faster. I want to run larger models but I can't really think of who to ask or what to ask actually.

The idea is to take 5 30 series cards example 3080 or 3060 "second hand" plug each into PCIe gen 3 and run it on a mining board run Win11 on it as host and run LLM like MusicGen or LLark? I will then save up for 5 more GPUs later on. Or just maybe save up for 50 series card?

 

[wizardofid]

Been looking to maybe buy a second hand or new motherboard that can support up to 19 GPUs. These boards was build for mining so I don't know at what speeds those PCIe ports run at . My goal is build a GPU compute cluster see if I can get AI run a bit faster. I want to run larger models but I can't really think of who to ask or what to ask actually.

The idea is to take 5 30 series cards example 3080 or 3060 "second hand" plug each into PCIe gen 3 and run it on a mining board run Win11 on it as host and run LLM like MusicGen or LLark? I will then save up for 5 more GPUs later on. Or just maybe save up for 50 series card?

You're going to run into bottleneck problems with a mining board. You need at least an intel xeon or epyc, and then a shyte ton of memory. Mining boards are severely limited with regard to memory slots and memory, for example a 5 GPU cluster with 32gig vram per card is 160gig~ total. You need 2 to 4 times in system memory

So no, mining boards is a terrible idea. A compute cluster isn't just about the GPU, it is CPU, memory and then PCIe lanes speed and bandwidth, exactly the things mining boards care the least about.

If you really want to get into not spend over a million

A 100k for a teraflop of AI computing power, this is the way

https://www.wootware.co.za/nvidia-dgx-spark-gb10-grace-blackwell-20-core-arm-cpu-1000-ai-tops-fp4-nvidia-blackwell-gpu-128gb-lpddr5x-unified-memory-4tb-nvme-m-2-ssd-nvidia-dgx-os-ai-workstation-mini-pc.html?gad_source=1&gad_campaignid=22822623294&gclid=Cj0KCQjwjL3HBhCgARIsAPUg7a7M2UGT-32t4uaSozrDZDYPa1IQ6oSr6-PrmK68vaVHPNNmNB7N0k8aAk19EALw_wcB

 

[EmmanuelGoldstein]

Why not buy an old supercomputer and be done with it