Nvidia promotes AI factories as 81% of firms underutilize GPUs

Nvidia announced its new “AI factories” platform on Tuesday, promising a turnkey solution for companies to scale deep‑learning workloads, while a VentureBeat Intelligence survey released the same day found that 81 percent of firms running AI in production say their GPUs operate at half capacity or less. The contrast highlights a growing mismatch between the hype of AI hardware and the reality of underused resources.

The survey, conducted in September with more than 500 respondents, reflects a market still grappling with the rapid expansion of AI compute. After the 2022‑2023 boom in generative‑AI models, many enterprises rushed to buy expensive GPUs to keep up with competitors. Yet budget constraints, talent shortages, and the complexity of workload orchestration have left many installations idle. Analysts note that the surge in demand for Nvidia’s H100 chips outpaced the supply chain, prompting firms to over‑provision in anticipation of future needs. The result is a surplus of hardware that sits underutilized, inflating costs without delivering expected returns.

Industry leaders responded to the findings with mixed signals. Nvidia’s CEO, Jensen Huang, said the AI factories will “automate the entire pipeline, from data ingestion to model deployment,” aiming to squeeze more work out of existing chips. Cloud providers such as Microsoft and Google are already offering managed services that bundle GPU access with orchestration tools, hoping to improve utilization rates. Meanwhile, venture‑backed startups are marketing software that can dynamically shift workloads across on‑premise and cloud GPUs, promising better efficiency. Some CFOs, however, remain cautious, noting that without clear ROI, further investment in hardware could exacerbate the under‑use problem.

The next few months will test whether these solutions can close the utilization gap. Nvidia plans to roll out its AI factories to select partners by early 2025, while the VentureBeat survey suggests that more than three‑quarters of respondents still intend to expand their AI infrastructure despite low current usage. If the new platform can deliver measurable gains, it could validate the continued spending spree on GPUs. If not, companies may pause purchases and turn to more flexible, pay‑as‑you‑go cloud models, reshaping the AI hardware market once again.