Elon Musk’s AI start-up xAI plans to expand its Colossus supercomputer tenfold, incorporating over 1 million graphics processing units (GPUs) to outpace rivals such as Google, OpenAI, and Anthropic.
Colossus, built earlier this year in just three months, currently operates a cluster of more than 100,000 interconnected Nvidia GPUs, making it the world’s largest supercomputer. These GPUs are used to train Musk’s chatbot Grok, which trails behind ChatGPT and Google’s Gemini in advancement and user base.
Expansion efforts are underway in Memphis, Tennessee, according to a statement by the Greater Memphis Chamber on Wednesday. Nvidia, Dell, and Supermicro Computer are setting up operations in Memphis to support the growth, while the chamber will create an “xAI special operations team” to provide dedicated services to the company.
The expansion will require significant investment. The latest Nvidia GPUs cost tens of thousands of dollars each, and xAI’s planned growth could demand tens of billions of dollars, factoring in the costs of construction, power, and cooling for the vast servers. Despite the high costs, xAI has raised about $11 billion in capital this year and is valued at $45 billion after recently securing an additional $5 billion in funding.
The demand for GPUs and data center capacity is intense across the AI industry as companies race to build large-scale computing power for their models. OpenAI, the maker of ChatGPT, has a nearly $14 billion partnership with Microsoft, which includes access to computing resources. Anthropic, creator of the Claude chatbot, has received $8 billion in funding from Amazon, which is building a 100,000-GPU cluster for its use.
Unlike its competitors, Musk has opted to independently build his computing infrastructure rather than rely on partnerships. Musk, who founded xAI just over a year ago, faces stiff competition from OpenAI, a company he co-founded in 2015 with Sam Altman before parting ways. Musk is currently suing OpenAI to block its shift from a non-profit to a commercial enterprise.
Jensen Huang, CEO of Nvidia, has praised Colossus as “easily the fastest supercomputer on the planet as one cluster” and noted that a project of its scale would typically take three years to complete. The speed of Colossus’s construction has drawn criticism for potentially bypassing planning permissions and placing heavy demands on the regional power grid.
At an event in Memphis, Brent Mayo, xAI’s senior manager for site builds and infrastructure, defended the project, saying, “We’re not just leading from the front; we’re accelerating progress at an unprecedented pace while ensuring the stability of the grid utilizing megapack technology.”
