Is the Nvidia Tesla V100 still good for AI? - Inspur DGX V100 vs RTX 5090

Duration: 23:29

Thanks to Verda, formerly Datacrunch.io, for sponsoring today’s episode. Check them out at https://verda.com, and use Coupon Code CRAFT-COMPUTING-V100 for $25 off at checkout!

Grab yourself a Pint Glass or Coffee Tumbler at https://craftcomputing.store

Implementing any form of AI workflow into your business is prohibitively expensive. From just the cost of hardware to the power and cooling infrastructure of modern AI servers, it's enough to turn anyone off. But you know me... I love digging up old servers out of eWaste piles and giving them new life. But how well does Nvidia's Tesla V100s stack up to modern cards in AI? Today, we're testing Eight Tesla V100s in 70B and 120B LLMs to see if there's life still in these eight year old GPUs.

But first... What am I drinking???

Deschutes Brewing (Bend, OR) Fresh Squeezed IPA NA (<0.5%)

HUGE THANKS to UnixSurplus for sending over the Inspur DGX V100 system for me to take a look at. Check them out at https://UnixSurplus.com

Or their eBay store: https://ebay.us/6BPOyd

*Links to items below may be affiliate links for which I may be compensated*

Inspur DGX V100 Server from UnixSurplus: https://ebay.us/1U6I0H

Dual Intel Xeon 8260 24-Core / 48-Thread

256GB DDR4-3200 REG-ECC

2x MZ-7LH1T90 1.92TB SSD Drives

8x Nvidia V100 32GB SXM2 GPU

1x 4x 10GB SFP+ Mezzanine Card

OB NIC: 100GB Nvidia CX6 network card