This audio is automatically generated. Please let us know if you have any feedback.
There’s a race on to build bigger and better cloud data centers, and AWS is revving up the engine of its massive infrastructure.
In the first half of the year, the largest hyperscaler invested more than $50 billion in expanding its pipeline in the US, committing $35 billion to several sites in Virginia, $11 billion to a campus in Indiana and $10 billion to two complexes in Mississippi.
Globally, the company has unveiled plans for new buildings in Taiwan, Japan and Singapore, as well as a €7.8 billion ($8.4 billion) European sovereign cloud region in Germany – all announced since January.
Generative AI workloads are driving the expansion. The company already generates billions in revenue from the technology, said Prasad Kalyanaraman, AWS vice president of infrastructure services, in a blog post on Wednesday. But the company’s infrastructure investments are based on the general appetite of enterprises for cloud services.
“We’re seeing strong demand for non-generative AI and generative AI workloads and that’s driving our capacity needs,” Kevin Miller, AWS VP of Global Data Centers, told CIO Dive. “We’re building data centers based on the growth of the business. Keeping our data centers highly utilized is a fundamental part of our business.”
Cloud usage is steadily increasing as companies use hyperscaler infrastructure to modernize their operations, close on-premises data centers, and move to a service-based IT model. Consumption, in turn, brings revenue. Gartner expects global cloud spending to exceed $675 billion in 2024, up 20% year-over-year.
The three largest hyperscalers have posted revenue gains and all have infrastructure expansions planned. Microsoft, which captured a quarter of global cloud spending in the first three months of 2024, has initiated more than a dozen large data center projects in the past eight months. Google Cloud, the junior player in the big cloud space with 11% market share, recently announced billion-dollar expansions in Virginia, Indiana and Missouri.
Despite the competition, AWS has dominated nearly a third of the growing market for years. According to the company’s first-quarter 2024 earnings report, Amazon’s cloud division generated revenue of $25 billion in the first three months of the year, contributing $9.4 billion to the tech giant’s operating profit.
“The greater the demand for AWS, the more new data centers, power and hardware we need to acquire,” Amazon President and CEO Andy Jassy said during a conference call on quarterly earnings in April.
“We’re doing over $100 billion in annual revenue, but 85% or more of global IT spending is still on-premises,” Jassy added. “And that’s not even counting GenAI, because the majority of that will be built from scratch and in the cloud over the next 10 to 20 years.”
Chips and a dip
AI’s insatiable hunger for computing is fundamentally changing data centers.
Demand for graphics processing units and other high-performance chip technologies slowed data center revenue growth in the first half of the year, according to a report from Dell’Oro Group. The decline was largely due to chip supply bottlenecks triggered by the shift to faster computing power.
It is a temporary slowdown, said Lucas Beran, head of research for data center infrastructure at Dell’Oro.
“The pandemic-related data center buildout is largely complete, but the use of AI has not really started to scale,” Beran said in an email. “Hyperscalers will be the driving force of growth as they have the capital to invest in the CaPex-intensive data center buildout related to AI workloads.”
The trend was reflected in a record quarter for chip sales, the analyst firm noted. Sales of semiconductors and data center components rose 152% year-on-year in the first three months of 2024 as GPU giant Nvidia reported soaring sales.
AWS is no newcomer to the chip business. The company relies on a diversified portfolio of proprietary silicon and Nvidia GPUs to meet the growing demand for AI services.
“There is no compression algorithm for experience,” Kalyanaraman said in his post. “Because we have been building large data centers for more than 15 years and GPU-based servers for more than 12 years, we have a huge existing AI infrastructure.”
The company plans to have its third-generation Trainium 2 chips ready for deployment later this year, Miller said. The company will deploy them in huge clusters to speed up training of the base model.
For more general inference operations, which most analysts believe will account for the majority of enterprise AI usage, AWS has already enabled customized Inferentia chips.
“There have been no dramatic changes in inference,” Miller said. “We have years of experience with this capability and these workloads are similar to other customers.”
Beyond GPUs
The construction boom is an industry-wide phenomenon that consumes land, power and labor. AWS is battling with Microsoft, Google Cloud and numerous smaller competitors for cloud dominance. Almost every provider is investing capital in AI data centers.
“These capital programs are competing for scarce resources on the GPU and chipset side,” Brian Alletto, director of enterprise technology at West Monroe, told CIO Dive. “But I think what we’re seeing in the industry is competition for power systems, cooling systems and expertise to build these things – it’s a busy time for that part of the building industry.”
The major cloud providers benefit from experienced in-house design and engineering teams to lead infrastructure expansion. AWS expects its Mississippi data centers to be ready for capacity expansion as early as next year, Miller said.
“Hyperscalers undoubtedly benefit from economies of scale through lower costs for infrastructure, space and power,” says Beran.
However, in order to convert the funds provided for new construction into operational infrastructure, countless considerations must be made and hurdles overcome.
“We need to look at grid capacity, land parcels, zoning and work with local authorities,” Miller said.
Projects also require labor during and after the construction process.
“Data centers require skilled workers with technical knowledge to operate the facilities and server equipment,” Miller said. “To make all of this possible, it’s all about having the right partnerships with local governments, local community colleges and local energy providers.”
