Tenstorrent debuted this expansion card, called Grayskull, at the Linley Spring Processor Conference in April. It’s the company’s answer to the computer processing units (CPUs) and graphics processing units (GPUs) currently used for AI applications. But GPUs, as the name suggests, were built to rapidly process heavy mathematical computations for rendering graphics in products such as video games. They were not designed to develop AI algorithms.
And yet, the GPU remains the industry standard. These processors are used to train the vast majority of today’s neural networks, even the largest ones like BERT and GPT-1, 2 and 3, which are used for natural language processing.
Computer scientists train these networks by feeding them massive amounts of data to tune their millions — and even billions — of parameters so they can accurately do things like recognize faces and images or generate coherent and convincing text. Some have been trained on all of Wikipedia, or in other cases, by setting up a web crawler and feeding the entire World Wide Web into the training process.
The latest OpenAI model, GPT-3, is capable of amazing feats, such as writing news articles, viral tweets, even poetry. But the model also requires massive amounts of compute. Whereas its predecessor, GPT-2, could run on a few GPUs, training GPT-3 required thousands of GPUs.
And that comes with multiple costs. Financially, only the biggest companies can afford to train networks that immense, let alone pay for the amount of electricity required to run all that equipment.
And that’s the other cost: to the environment. As AI usage grows, its energy consumption and carbon emissions are becoming increasingly clear. According to one study out of the University of Massachusetts, Amherst, the electricity needed to train a transformer — a type of deep-learning model — can cause more than 626,000 pounds of carbon dioxide to be emitted. That’s nearly five times the emissions of an average American car.
A Computerworld article published last year presented the environmental impact in global terms:
“Recent predictions state that the energy consumption of data centres is set to account for 3.2 percent of the total worldwide carbon emissions by 2025 and they could consume no less than a fifth of global electricity. By 2040, storing digital data is set to create 14 percent of the world’s emissions, around the same proportion as the US does today.”
As Tenstorrent Founder and CEO Ljubisa Bajic puts it, getting a neural network to toddler-level intelligence would require a city full of machinery. So, by replacing GPUs with Tenstorrent’s solution, we can meet the growing demand for AI and ML computation, without filling entire municipalities with computers.
Throughout his career, Ljubisa has driven innovation at some of the most dominant computer-chip makers in the world. Before Tenstorrent, he headed power and performance architecture at leading semiconductor company Advanced Micro Devices (AMD). Prior to that, he worked as a senior architect at Nvidia, which has been dubbed “the house that GPUs built.”
Across the company, Tenstorrent’s architects and engineers bring decades of experience developing products at Altera, AMD, ARM, ATI, Bio-Rad, IBM, Intel and Nvidia. They have helped build industry-defining products that are still in use today. So, there’s no better team to tackle the $18.5 billion AI chip market. Gartner expects sales to reach nearly $44 billion in 2024.
At the conference last spring, Tenstorrent presented performance numbers showing how conditional execution enabled Grayskull to perform many times better than today’s leading solution. Results from early pilots with potential customers have also born out Tenstorrent’s promise to spark a paradigm shift in advance compute.
When we first partnered with the company, Ljubisa had two goals: One was to build a solution that outperforms existing products on the market, which they’ve demonstrated. The other, far more ambitious vision was to make neural networks run as efficiently as the human brain — and ultimately, create a more sustainable and achievable future for AI.
Tenstorrent is closer than ever to achieving its loftier goal and now wants to recruit large, influential institutions like top-tier research universities and federal agencies as allies in this mission to course correct AI’s unsustainable trajectory. One widely respected industry watcher who attended the Linley event — Karl Freund, senior analyst at Moor Insights & Strategy — later shared his thoughts about Tenstorrent in Forbes, clearly bolstered the company’s case.
“To my eye, this announcement marks a shift from chips with lots of fast cores and on-die memory & fabric (which describes most of the entrants to date) to a new approach of smart computing where the software, training, and inference chips all coordinate knowledge of the network to reduce the amount of computation. Brute force is great, until something better comes along, and I think it just did.”
