AI will force a transformation of tech infrastructure
Summary
Corporate technology executives need to ensure their cloud and private infrastructure is ready to handle their data needs to come.
AI is about to make the cloud a lot heavier.
Cloud services and private networks for years had to handle relatively limited amounts of data. Now that artificial intelligence and deep learning are driving vast quantities of photos, video, sound and natural language into the mix, however, data that was once counted in gigabytes and terabytes is measured in much larger units of petabytes and exabytes.
Information systems, including the cloud, must expand to store all of that data. Less obvious—and more interesting—is the need to access all of that information at much higher speed and, critically, lower operating cost.
Some companies have already begun trying to develop the next generation of infrastructure. CoreWeave, a cloud-computing provider that gives customers access to advanced AI chips from Nvidia, has focused attention on the emerging market.
CoreWeave in May announced a $1.1 billion equity funding round that valued the seven-year-old startup at $19 billion, as well as $7.5 billion in debt financing from investors including Blackstone, Carlyle Group and BlackRock. Nvidia is also an investor.
CoreWeave in turn is a customer of a startup called VAST Data, which approaches cloud and private-network modernization from the software perspective. VAST has developed what it calls a faster, cheaper and more scalable operating system for all sorts of distributed networks.
“Our vision was to build infrastructure for these new AI workloads," said Chief Executive Renen Hallak, who founded the company in Israel in 2016. In December, VAST said it secured $118 million in a Series E funding round, led by Fidelity Management & Research, that nearly tripled its valuation to $9.1 billion. The company has surpassed $200 million in annual recurring revenue and claims a gross margin of nearly 90%.
Data storage historically has been organized in tiers, in which recent, high-priority data was kept readily accessible and older data was buried further down, according to Hallak. “That’s not the case anymore with these new AI workloads," Hallak told me in an interview at the company’s office in New York.
“Once you have a good AI model you want to infer on all of your history, because that’s how you get value out of it. And then as you get more information you want to retrain and build a better model," Hallak said.
“You read data over and over and over again across many petabytes and in some cases exabytes. And so that’s a very different problem," he added.
Traditional systems also expand by adding nodes that store a slice of the larger data set. The nature of this architecture requires all parts to expend resources communicating with one another and can suffer if a single node develops a problem. As a result, many enterprise systems could only scale to a few dozen nodes, insufficient for AI-driven demands, Hallak said. In the VAST approach, all nodes have access to all of the information at once, improving scalability, speed and resiliency, he said. VAST also unbundles the price of data storage and computing, which it says saves money.
The need for a new tech infrastructure at first summons thoughts of the tech giants, but it will move ever deeper into the economy.
This shift is already under way at some highly data-intensive companies such as Pixar, the Disney movie studio behind this summer’s hit “Inside Out 2." It has been working with VAST since 2018.
Beginning with its 2020 film “Soul," Pixar has employed a technique known as volumetric animation that produces more highly detailed characters and movements. The use of volumetric animation was more extensive in its 2023 release “Elemental," for which Pixar used AI to curate protagonist Ember Lumen’s flames.
“Inside Out 2," which came out in June, had double the data capacity needs of “Soul" and required about 75% more computation.
Pixar’s old system of moving data from high-performance drives to lower-performance drives when not in use didn’t work for rendering volumetric characters, according to Eric Bermender, head of data infrastructure and platforms at Pixar. For AI, Pixar tends to employ on-premises networks as opposed to the cloud.
AI in general doesn’t easily conform to traditional architectures, Bermender said. “These workflows tend to require processing massive amounts of diverse data that is not cacheable, not sequential, and which traditionally would have been stored in lower performance archive tiers," he said.
The upshot for companies is that the adoption of AI must occur in a technology environment that can manage its unprecedented demand for data. It is analogous to an electric car, which requires a total reconsideration of many components in a gasoline-powered car, right down to the tires. To gain traction, AI will need a new set of wheels, too.
Write to Steven Rosenbush at steven.rosenbush@wsj.com
