OpenAI's Sam Altman has a $7 Trillion Ambition
Who Is Sam Altman?
Sam Altman is the founder and CEO of OpenAI, the company behind ChatGPT, an AI chatbot that produces human-like responses to queries. He founded the company in 2015 with Elon Musk and others to develop a large language model (LLM) that could evolve into AGI (Artificial General intelligence), a generalized intelligence that is aimed to be task-agnostic. Before the founding of OpenAI, he was the president of Y Combinator, a prestigious accelerator known for early investments in Airbnb, Twitch, Stripe, and other notable startups. He dropped out from Stanford University after two years of studying computer science explaining that he had learned more from playing poker with classmates than from attending lectures, as poker taught him to make decisions with imperfect information. Sam Altman’s intense nature that resembles Elon Musk is probably what led him to partner with Musk on the founding of OpenAI.
Today, with the advanced GPT-3.5 and GPT-4 models, OpenAI provides the foundational infrastructure for language-based AI tasks through its APIs for B2B customers and client-facing ChatGPT interface for consumers.
LLMs and NVIDIA
Training and using large language models demand significant computing power due to several reasons. LLMs require vast amounts of text data for training, including web text, books, and specialized data from various domains. Their complex architectures, like the Transformer model, contain billions to trillions of parameters, necessitating advanced computational capabilities.
The training process, involving data preprocessing and fine-tuning, is computationally intensive, requiring high-performance hardware such as GPUs, CPUs, large RAM, and storage solutions. Challenges in development and deployment, like data biases and model complexity, also demand additional computational resources. As LLMs evolve, with a push for more parameters and capabilities, the need for computing resources continues to increase, emphasizing the resource-intensive nature of working with them.
Currently, NVIDIA serves as the leading provider of AI chips, touting a market share of 92% in the data center GPU segment. As evidenced by the explosive usage of AI applications including ChatGPT, the demand for NVIDIA GPUs inside AI data centers has skyrocketed as well.
In the third quarter of fiscal year 2024, NVIDIA reported revenues of $18.12 billion, more than tripling from the year-ago period thanks to the explosive demand for data center GPUs propelled by the usage of AI applications. Its gross margin also expanded more than 20 percentage points, implying that deep-pocketed big-tech customers like Microsoft, Amazon, and Meta could not get enough of its GPUs and had to pay extra for the enormous markup.
According to industry experts, NVIDIA’s chips are in such high demand and constrained in supply due to the following reasons.
High Performance and Vibrant Ecosystem
Because the performance of GPUs is highly relevant to the time it takes to train and infer a large language model, researchers and end users alike prefer high-performance GPUs to reduce training time and accelerate inference speeds. NVIDIA’s GPUs, specifically the H100 model designed for data centers, consistently top benchmarks for AI-based tasks (computer vision, natural language processing, and others) using PyTorch or TensorFlow. PyTorch and TensorFlow are machine learning frameworks used to build deep learning (a type of machine learning) models that can translate to features like image recognition and natural language processing which is required for LLMs. The sheer amount of raw processing power that NVIDIA’s data center GPUs provide is second to none, forcing researchers to choose NVIDIA out of necessity despite their eye-watering price tags.
Another main advantage of NVIDIA compared to its competitors like AMD or Intel is the existence of its vibrant software ecosystem that makes GPUs easier to work with for software engineers. While deep learning frameworks such as PyTorch that allow engineers to use the high-level Python language for AI programming have expanded their support for none-NVIDIA GPUs and accelerators, NVIDIA’s parallel computing platform dubbed CUDA (Compute Unified Device Architecture) supports major languages like C, C++, Fortran, Python, and MATLAB, making it the most accessible parallel computing platform available for researchers and engineers that build AI models. In addition, the CUDA Toolkit includes a comprehensive set of libraries, debugging and optimization tools, a compiler, and documentation. These resources support many applications, from deep learning and linear algebra to signal processing and parallel algorithms. The availability of these tools and libraries simplifies the development of GPU-accelerated applications and helps engineers achieve optimal performance on NVIDIA GPUs.
Limitations in Advanced Packaging Capacity
The major reason why NVIDIA is supply-constrained is due to the capacity limitations in its foundry service provider, TSMC. NVIDIA doesn’t actually manufacture their GPUs; they design their chips in coordination with their hardware business partners like TSMC for fabricating integrated circuits and SK Hynix for high-bandwidth memory. For the latest and greatest H100 data center chips, TSMC is the only foundry company equipped with the advanced technology necessary to manufacture them.
The bottleneck is specifically in the CoWoS (Chip-on-Wafer-on-Substrate) packaging capabilities. TSMC has seen a threefold increase in orders for its CoWoS packaging services, which are critical in the production of AI GPUs. The company is working on expanding its facilities to meet this demand, but it is a long-term process, and the shortage is expected to last until at least 2025.
The CoWoS packaging technology is a complex, multi-step process that is essential for assembling high-performance GPUs, and the current capacity is not sufficient to meet the skyrocketing demand, particularly for generative AI applications. This has led to a situation where Nvidia can only fulfill about 80% of customer needs.
The $7 Trillion Ambition
Unsatisfied with the inflated price tag of NVIDIA GPUs and their scarce availability, Sam Altman decided to embark on his next-generation AI chip startup that seeks to control everything from design to semiconductor manufacturing and is reportedly eyeing $7 trillion in funding from international investors. To put this in perspective, the entire budget of the United States government in 2023 was just under $7 trillion, NVIDIA’s market capitalization was $1.8 trillion as of late, and a little more than $1 trillion was estimated to be spent on semiconductor manufacturing equipment since the inception of the industry.
Although it is unclear where exactly Sam Altman is planning to deploy such a large amount of capital, and whether it’s even an obtainable goal considering that over 5,000 public sector retirement systems in the United States had $5.3 trillion in assets combined in the third quarter of 2022, the current demand for AI accelerators seems to be growing at an astronomical rate thereby prompting him to come up with a potential competitor to NVIDIA to fulfill his dream of developing AGI, which will demand even more computing power.
Nonetheless, it would be much easier for him if getting the $7 trillion funding secured and throwing money at the problems solves them.
Expand Semiconductor Production
Altman is talking to TSMC about forming a chip venture together to create “a network of factories” to produce AI chips needed to power OpenAI’s artificial intelligence models. He urges the industry to act promptly to secure ample chip supply by the end of the decade, given the surge in interest in AI applications since the release of ChatGPT over a year ago. The heightened demand for computing power and processors necessary for AI development has led to a shortage, which Altman has emphasized is already impacting his company.
Fix Talent Shortage
To design new AI chips and expand production capabilities, Altman needs to train and acquire skilled workers en masse. TSMC, the global leader in semiconductor foundry services, has reportedly delayed its construction of an advanced fabrication plant in Arizona citing the lack of skilled labor in the industry. Deloitte thinks that one million additional skilled workers are needed by 2030 to fulfill the growing semiconductor market, which is set to achieve $1 trillion in annual revenue by then. For context, there are fewer than 100,000 graduate students enrolled in electrical engineering and computer science in the United States annually. The lack of skilled workers could potentially mean hindered research and development efforts for advanced artificial intelligence, a major setback for Altman.
With $1 trillion, he could contemplate establishing a university focused on AI, semiconductor design and manufacturing, dwarfing Harvard’s $50 billion endowment and theoretically funding over 40 MIT-sized institutions.
Push the Energy Envelop
Another aspect to consider for developing even more advanced artificial intelligence is the energy constraint. Training and inferencing advanced AI models require massive utilization of AI chips (if we can even produce enough of them), which generate lots of heat and consume substantial amounts of energy.
Speaking at a Bloomberg event on the sidelines of the World Economic Forum’s annual conference in Davos, Altman said that AI will consume vastly more energy than expected, communicating the need for breakthroughs in energy production technologies like nuclear fusion and cheaper solar.
In 2021, Altman invested $375 million in Helion Energy, a U.S. nuclear fusion company, which has since secured a deal to supply energy to Microsoft, OpenAI's primary financial supporter, and computing resource provider.
If you enjoyed this post, please consider taking a look around BoxHero, the easiest inventory management software available.
