Tech executives have touted this as the year in which inference—the process by which AI models use computing firepower to respond to queries—will eat up the entire AI industry. That is starting to bear fruit.
Baseten, which specializes in providing software and computing capacity to companies tapping in to lower-cost AI models, is finalizing a $1.5 billion fundraising round. It will be a dual-tiered structure, with some investors putting in money at an $11 billion valuation and others at a $13 billion valuation, according to Baseten.
Much of the focus in the AI industry has been on the biggest labs including Anthropic and OpenAI, which mostly offer closed-source models to customers and users. While undoubtedly more expensive than the freely available open-source models, part of the value proposition of the AI giants is that they provide computing capacity for their users.
“When you go to OpenAI or Anthropic, they obviously have their models, but they kind of give you all their inference infrastructure for free,” said Tuhin Srivastava, co-founder and chief executive of Baseten.
In the past few years, the quality gap between open-source and closed-source models has narrowed. At the same time, big companies and startups are starting to balk at escalating AI costs—leading enterprise customers to look for alternatives, such as open-source models.
Open-source models are created in a way that any users can download and modify them for their own specific needs at no cost.
The problem with open-source models is that they don’t come with the necessary computing to process user queries and carry out tasks. Enter Baseten, which has built an inference software layer on top of the computing capacity it is sourcing from 20 different cloud providers. Its software gives customers the connective tissue to run, optimize and train open-source models with their own data.
Companies focused on inference are becoming breakout ventures. Cerebras, a startup that has designed chips specifically for inference, had a blockbuster IPO in May and currently has a nearly $50 billion market capitalization.
Fireworks AI, another inference startup, raised funds in October at a $4 billion valuation. Factory, a startup founded in 2023 that has created autonomous-coding agents that plug into open-source and closed-source models, reached a valuation of $1.5 billion in April.
Altimeter Capital, Conviction, Spark Capital, Sands Capital and Wellington Management are co-leading the Baseten round. Current customers of Baseten include such companies as Cursor, Mercor and OpenEvidence. The tech-news site The Information earlier reported on Baseten’s fundraising.
The investment from institutional investors such as Wellington Management—which are typically more risk-averse and longer-term holders—is a testament to the staying power of companies like Baseten, investors said. Baseten will be Wellington’s first investment in the AI inference market, said Oz Nur, an investor at the asset manager who is focused on late-stage private companies.
Right now, the most popular open-source models come from China, including DeepSeek and Moonshot AI’s Kimi. U.S. companies are trying to gain ground in this arena, including Nvidia, which recently launched a family of open-source models called Nemotron.
“If you believe open-source models are always a handful of months behind the frontier models, it can serve a lot of use cases to customers while saving some share of their token usage for the frontier models where you need the absolute best,” said Nur.
With Baseten’s AI infrastructure offerings, other companies can tap in to open-source models and create tailored products at a much lower cost.
Most, if not all, of Baseten’s customers are using a mix of open-source and closed-source models for tasks. But tapping in to open-source models even for just a portion of tasks can help with costs.
Srivastava recalled one customer who told him of being able to perform a specific task at 30% of the cost required if a closed-source model had been used.
“At the highest level, what’s happening generally in the market is that the open-source models are getting very, very good,” said Srivastava. “And as open-source gets better, we are growing with it.”