Microsoft Enters Custom Chip Arena with Innovative AI Accelerator and Proprietary CPU
Microsoft is stepping into the realm of custom silicon development with the introduction of two bespoke chips, specifically engineered to enhance the AI capabilities within its data centers.
At Microsoft Ignite, the company showcased its latest foray into silicon innovation with the launch of two tailor-made chips. These chips are crafted to bolster Microsoft's specialized AI tasks and computing operations in its server infrastructures. This reveal marks a significant milestone in Microsoft's journey toward internalizing the design and architecture of its server ecosystems.
Microsoft tests its custom-designed chips in-house, bringing more control over compute performance and efficiency for AI and data center applications.
Many organizations, including Amazon and Google, have turned to custom chip development to bolster their computing performance. Microsoft’s announcement continues this trend, and will eventually offer more data points to evaluate the tradeoffs between custom and commercial silicon.
Introducing the Maia AI Accelerator and Azure Cobalt CPU
Microsoft’s two new pieces of custom silicon are the Maia AI Accelerator and the Azure Cobalt CPU. Each will eventually play a critical role in Microsoft’s custom-built Copilot and Azure OpenAI servers as developers demand more performance for cloud-based computing.
The Maia 100 chip offers higher performance than commercial chips, allowing Microsoft to optimize its own unique workloads and offer a fully custom solution.
Microsoft doesn’t currently have a comprehensive spec sheet for the Maia and Cobalt chips. We do know, however, that each chip is built on a TSMC 5 nm process, with the Maia AI accelerator including over 100 billion transistors and a 4.8 Tbps per accelerator Ethernet-based protocol.
The Cobalt CPU, on the other hand, is an Arm-based 64-bit, 128-core processor built to complement the Maia GPU in parallel processing tasks. The Cobalt chip reportedly offers a 40% performance improvement over current-generation Azure servers, giving designers access to higher compute performance.
Building Bespoke Servers
With Microsoft’s venture into custom silicon, the company joins others, such as Amazon and Google, who have abandoned commercial chips for purpose-built chips to optimize performance. Especially in the case of cloud servers running advanced AI models, the computation and efficiency improvements of custom silicon can greatly impact the overall performance of a data center.
In Microsoft’s case, the jump to custom silicon is the final component needed for fully custom servers. Microsoft has even adopted a unique approach to cooling the Maia chips. Instead of a large liquid chiller, Microsoft employs custom server racks with space for the compute card and a “sidekick” that contains the liquid cooling mechanism.
An example of the Maia server highlights Microsoft’s fully custom approach. From server rack to liquid cooling “sidekick”, the Maia server includes all Microsoft-designed parts.
Throughout the process, Microsoft has worked with OpenAI to provide feedback on the Maia and Cobalt chips’ performance with large language models. Although the Maia chips may provide higher performance, Microsoft will still use commercial silicon to create a customer-focused solution, adding more versatility to its offerings.
The Trend Toward Custom Silicon
As more companies move toward custom silicon in their HPC and data center offerings, AI developers are reaping the benefits of improved computing power. Companies can also customize their solutions to find the best operating point for efficiency, computing power, and versatility in their cloud computing centers.
Despite the fact that Maia and Cobalt were only recently announced,
Microsoft has reported that it is already designing second generations
of the chips, demonstrating how custom silicon allows developers to
precisely control development times without relying on third parties.
