October 7, 2023 – In a significant development, Microsoft is gearing up to unveil its first-ever AI-focused chip at its annual developer conference scheduled for next month. This initiative marks the culmination of years of effort and could potentially reduce Microsoft’s reliance on NVIDIA’s AI chips, which have been in high demand due to the surging AI requirements.
Microsoft’s chip, codenamed “Athena,” bears similarities to NVIDIA’s Graphics Processing Units (GPUs) and has been tailor-made for data center servers tasked with training and running large language models, such as OpenAI’s ChatGPT. Currently, Microsoft’s data center servers rely on NVIDIA GPUs to support AI functionalities for cloud customers, including OpenAI and Intuit, as well as AI features within Microsoft’s productivity applications.
Athena is expected to compete with NVIDIA’s flagship microprocessor, the H100 GPU, in the realm of AI acceleration for data centers. This custom-designed chip has been undergoing secretive testing by Microsoft and its partner, OpenAI.
Microsoft initiated the development of Athena chips around 2019, aiming to reduce costs and strengthen its position in negotiations with NVIDIA. Azure currently depends on NVIDIA GPUs to power AI functionalities used by Microsoft, OpenAI, and cloud customers. With Athena, Microsoft can follow in the footsteps of competitors like AWS and Google, offering cloud users AI chips developed in-house.
Although specific performance details of Athena remain undisclosed, Microsoft hopes that this chip will match the capabilities of NVIDIA’s H100. While many companies vie to market exceptional hardware with cost-efficiency, NVIDIA GPUs remain the preferred choice for AI developers, thanks to the company’s CUDA platform. Convincing users to adopt new hardware and software will be crucial for Microsoft.
In addition to potentially reducing its dependence on NVIDIA, Microsoft’s in-house AI chip could come in handy amid GPU supply shortages. Reports suggest that, following their close collaboration with OpenAI, Microsoft ordered at least hundreds of thousands of NVIDIA chips to support OpenAI’s products and research needs. Using its own chips could lead to substantial cost savings.
OpenAI might also be contemplating reducing its reliance on both Microsoft and NVIDIA chips. Recent reports indicate that the AI research lab is considering manufacturing its own AI chips. Recent job postings on the OpenAI website also suggest that the company intends to recruit talent to evaluate and co-design AI hardware.
While Microsoft and other cloud service providers may not immediately halt their plans to purchase GPUs from NVIDIA, in the long run, persuading their cloud customers to opt for in-house chips over NVIDIA’s GPU servers could be economically advantageous. Microsoft is also closely collaborating with AMD on the upcoming AI chip, MI300X. With the skyrocketing demand for AI workloads, this diversified approach offers multiple choices. Competitors in cloud computing are also adopting similar strategies to avoid vendor lock-in.
Amazon and Google have strategically integrated their AI chips into their cloud business promotions. Amazon, in its support for OpenAI’s rival Anthropic, is stipulating that Anthropic use Amazon’s AI chips, namely Trainium and interentia. Meanwhile, Google Cloud has announced that customers like AI image developers Midjourney and Character AI are using the company’s Tensor Processing Units.
As AI chips become integral components of data centers, the potential returns in this field are significant. With this development, Microsoft joins the ranks of its competitors in the race for market share in the booming AI chip sector. With Athena, Microsoft can offer more choices to its cloud customers while charting a more independent course in the next-generation AI infrastructure.
