Meta, determined to keep up with its competitors in the generative AI space, is spending billions on its own AI efforts. Part of these billions goes to recruiting AI researchers. But an even larger portion is spent on developing hardware, particularly chips to run and train Meta’s AI models.
Meta today unveiled the latest result of its chip development efforts, conspicuously a day after Intel announced its latest AI accelerator hardware. Dubbed the “next generation” Meta Training and Inference Accelerator (MTIA), the chip is the successor to last year’s MTIA v1 and runs models for, among other things, ranking and recommending display ads based on meta properties (e.g .Facebook).
Compared to MTIA v1, which is based on a 7nm process, the next generation MTIA is 5nm. (In chip manufacturing, “process” refers to the size of the smallest component that can be built on the chip.) The next-generation MTIA is a physically larger design and features more processor cores than its predecessor. And while it uses more power – 90W vs 25W – it also has more internal memory (128MB vs 64MB) and runs at a higher average clock speed (1.35GHz vs 800MHz).
According to Meta, next-generation MTIA is currently deployed in 16 of its data center regions and delivers up to three times better overall performance compared to MTIA v1. If that “3x” claim sounds a bit vague, you’re not wrong – we thought so too. But Meta only said the figure came from testing the performance of “four key models” of both chips.
“Because we control the entire stack, we can achieve greater efficiency compared to commercially available GPUs,” Meta writes in a blog post shared with TechCrunch.
Meta’s hardware presentation – which comes just 24 hours after a press conference about the company’s various ongoing generative AI initiatives – is unusual for several reasons.
First, Meta reveals in the blog post that the next-generation MTIA is not currently being used for generative AI training workloads, although the company says it has “multiple programs underway” exploring this. Second, Meta admits that next-generation MTIA will not replace, but complement GPUs for running or training models.
If you read between the lines, Meta moves slowly – perhaps slower than he would like.
Meta’s AI teams are almost certainly under pressure to cut costs. The company expects to spend an estimated $18 billion on GPUs to train and run generative AI models by the end of 2024, and—with training costs for cutting-edge generative models running into tens of millions of dollars—provides in-house hardware a great challenge and an attractive alternative.
And as Meta’s hardware declines, competitors move on, much to the dismay of Meta’s leadership, I suspect.
Google this week made its fifth-generation custom chip for training AI models, TPU v5p, generally available to Google Cloud customers and introduced its first dedicated chip for running models, Axion. Amazon has several custom AI chip families on the market. And Microsoft jumped into the fray last year with the Azure Maia AI Accelerator and the Azure Cobalt 100 CPU.
In the blog post, Meta says it took less than nine months to “go from first silicon to production models” of the next-generation MTIA, which, to be fair, is shorter than the typical time frame between Google TPUs. But Meta still has a lot of catching up to do if it wants to achieve a certain level of independence from third-party GPUs – and keep up with stiff competition.