The appetite for alternative clouds has never been greater.
Case in point: CoreWeave, the GPU infrastructure provider that started out as a cryptocurrency mining company, raised $1.1 billion in new funding this week from investors including Coatue, Fidelity and Altimeter Capital. The round brings the post-financing valuation to $19 billion and brings the total debt and equity to $5 billion – a remarkable number for a company that is less than ten years old.
It’s not just CoreWeave.
Lambda Labs, which also offers a range of cloud-hosted GPU instances, secured a “special funding vehicle” of up to $500 million in early April, months after closing a $320 Series C round million US dollars. Non-profit organization Voltage Park, backed by crypto billionaire Jed McCaleb, announced last October that it was investing $500 million in GPU-powered data centers. And Together AI, a cloud GPU host that also conducts generative AI research, raised $106 million in a funding round led by Salesforce in March.
So why all the excitement about the alternative cloud space – and the money flowing into it?
The answer, as expected, is generative AI.
As generative AI continues to boom, so does the demand for hardware to run and train generative AI models at scale. Architecturally, GPUs are the logical choice for training, tuning, and running models because they contain thousands of cores that can work in parallel to execute the linear algebra equations that make up generative models.
But installing GPUs is expensive. Therefore, most developers and organizations turn to the cloud instead.
Cloud computing incumbents—Amazon Web Services (AWS), Google Cloud, and Microsoft Azure—offer no shortage of GPU and dedicated hardware instances optimized for generative AI workloads. But for at least some models and projects, alternative clouds can ultimately be cheaper – and offer better availability.
At CoreWeave, renting an Nvidia A100 40GB — a popular choice for model training and inference — costs $2.39 per hour, which equates to $1,200 per month. On Azure, the same GPU costs $3.40 per hour or $2,482 per month; At Google Cloud it’s $3.67 per hour or $2,682 per month.
Since generative AI workloads typically run on GPU clusters, cost deltas grow quickly.
“Companies like CoreWeave participate in a market that we call specialized ‘GPU as a Service’ cloud providers,” Sid Nag, vice president of cloud services and technologies at Gartner, told TechCrunch. “Given the high demand for GPUs, they provide an alternative to the hyperscalers where they have adopted Nvidia GPUs and created a different route to market and access those GPUs.”
Nag points out that even some large tech companies have started relying on alternative cloud providers as they face computing capacity challenges.
Last June, CNBC reported that Microsoft had signed a multi-billion dollar deal with CoreWeave to ensure that OpenAI, the maker of ChatGPT and a close partner of Microsoft, had sufficient computing power to train its generative AI models. Nvidia, the supplier of the majority of CoreWeave chips, sees this as a desirable trend, perhaps for leverage reasons; It is said to have given some alternative cloud providers preferential access to its GPUs.
Lee Sustar, principal analyst at Forrester, sees the success of cloud providers like CoreWeave in part because they don’t have the infrastructure “baggage” that incumbents struggle with.
“Given the hyperscaler dominance of the entire public cloud market, which requires huge investments in infrastructure and a range of services that generate little to no revenue, challengers like CoreWeave have an opportunity with a focus on premium AI Services to succeed without the burden of hypercaler-level investments,” he said.
But is this growth sustainable?
Sustar has his doubts. He believes the expansion of alternative cloud providers will depend on whether they can continue to bring GPUs online in bulk and offer them at competitively low prices.
Price competition could become challenging in the long run as incumbents like Google, Microsoft and AWS increase their investments in custom hardware to run and train models. Google offers its TPUs; Microsoft recently unveiled two custom chips, Azure Maia and Azure Cobalt; and AWS has Trainium, Inferentia and Graviton.
“Hypercalers will leverage their custom chips to reduce their dependencies on Nvidia, while Nvidia will rely on CoreWeave and other GPU-centric AI clouds,” Sustar said.
Additionally, while many generative AI workloads run best on GPUs, not all workloads require them – especially if they are not time-critical. CPUs can perform the necessary calculations, but are typically slower than GPUs and custom hardware.
Even more existential is the danger that the generative AI bubble will burst, which would result in vendors having mountains of GPUs and not nearly enough customers demanding them. In the short term, however, the future looks bright, say Sustar and Nag, both of whom expect a steady stream of rising clouds.
“GPU-focused cloud startups are here to stay [incumbents] There is a lot of competition, particularly among customers who already use multiple clouds and can manage the complexities of management, security, risk and compliance across multiple clouds,” Sustar said. “These types of cloud customers can easily try out a new AI cloud if it has credible leadership, solid financial backing, and zero wait GPUs.”