It looks like AI has followed Crypto chip wise in going CPU > GPU > ASIC
GPUs, while dominant in training large models, are often too power-hungry and costly for efficient inference at scale. This is opening new opportunities for specialized inference hardware, a market where startups like Untether AI were early pioneers.
In April, then-CEO Chris Walker had highlighted rising demand for Untether’s chips as enterprises sought alternatives to high-power GPUs. “There’s a strong appetite for processors that don’t consume as much energy as Nvidia’s energy-hungry GPUs that are pushing racks to 120 kilowatts,” Walker told CRN. Walker left Untether AI in May.
Hopefully the training part of AI goes to ASIC’s to reduce costs and energy use but GPU’s continue to improve inference and increase VRAM sizes to the point that AI requires nothing special to run it locally
Isn’t this the idea behind tinygrad?
https://tinygrad.org/
https://youtu.be/dNrTrx42DGQ
Tinygrad is (so far) software only, ostensibly sort of a lightweight PyTorch replacement.
Tinygrad is (so far) not really used for much, not even research or tinkering.
Between that and the lead dev’s YouTube antics, it kinda seems like hot air to me.
Ready for pickup in 2-4 days. https://tinycorp.myshopify.com/products/tinybox-red
You’re right to be skeptical, but on the other hand, Geohot has done some incredibly impressive things in the past. Like that time he livestreamed buying a new model of Corolla and updating the software on for the comma.ai self-driving hardware to work for it and immediately took it onto the highway.
That’s a premade 8x 7900 XTX PC. All standard and off the shelf.
I dunno anything about Geohot, all I know is people have been telling me how cool Tinygrad is for years with seemingly nothing to show for it other than social media hype, while other, sometimes newer, PyTorch alternatives like TVM, GGML, the MLIR efforts and such are running real workloads.