Apple didn't have to go this hard... (RDMA over Thunderbolt)

Duration: 14:24

Apple loaned four Mac Studios to help me test the new RDMA over Thunderbolt support in macOS 26.2.

Is it good for HPC? For Exo and LLMs? Find out in this video.

Resources (some links are affiliate links):

- Mac Studio: https://www.apple.com/mac-studio/

- Exo 1.0: https://exolabs.net

- Apple MLX PR for Thunderbolt RDMA: https://github.com/ml-explore/mlx/pull/2808

- M5 Neural Accelerator speedup: https://machinelearning.apple.com/research/exploring-llms-mlx-m5

- DeskPi Rackmate TL1: https://amzn.to/4p5ek94

- Tupavco mini-rack PDU: https://amzn.to/4j0DpAq

Support me on Patreon: https://www.patreon.com/geerlingguy

Sponsor me on GitHub: https://github.com/sponsors/geerlingguy

Merch: https://www.redshirtjeff.com

2nd Channel: https://www.youtube.com/@GeerlingEngineering

3rd Channel: https://www.youtube.com/@Level2Jeff

Contents:

00:00 - A rack-full of Macs

02:01 - Putting together a cluster

03:16 - The downsides of Thunderbolt

05:09 - One is enough

07:45 - Managing macOS in a cluster (vs Linux)

08:37 - HPL and llama.cpp cluster performance

09:30 - Enabling RDMA

09:51 - Massive LLMs

11:34 - RDMA over TB5 is great... when it works

12:10 - EXO community and OSS

12:51 - Apple could go harder

13:30 - Jack of all trades, not just AI