HomeAI News
Microsoft OpenAI joins forces to suppress NVIDIA, the first self-developed AI chip will be released next month!
72

Microsoft OpenAI joins forces to suppress NVIDIA, the first self-developed AI chip will be released next month!

Hayo News
Hayo News
October 9th, 2023
View OriginalTranslated by Google
The dominant Nvidia has forced both Microsoft and OpenAI to build chips themselves. Perhaps the AI ​​chip war has just begun.

Microsoft's self-developed AI chip will be launched in November!

Well-known foreign media The Information exclusively broke the news that Microsoft plans to launch its first artificial intelligence chip at its annual developer conference next month.

At the same time, OpenAI is also recruiting people who can help it evaluate and design AI hardware.

There is a saying in the industry that "it is easier to sell H100 than to sell water to people who are dying of thirst in the desert."

Whether it is to get out of the exhaustion of computing power, develop your own model more efficiently and at low cost, or to get rid of being exploited by H100 with "profit margin as high as 1000%".

Both Microsoft and OpenAI are trying to "harden" and try to get rid of their dependence on Nvidia's GPUs.

However, according to industry insiders, Nvidia will control the supply of GPUs to companies that have launched their own chips, such as Google and Amazon.

Therefore, the path of "self-research of chips" is a choice with high risks and benefits. After all, no one wants to be further "stuck" in GPU supply by Boss Huang in the future.

Microsoft develops its own AI chip to catch up with Google and Amazon

Similar to Nvidia's GPUs, Microsoft's chips are designed for data center servers and can be used to train and run large language models such as ChatGPT.

Currently, whether it is providing advanced LLM support for cloud customers or providing AI functions for its own productivity applications, Microsoft needs to rely on Nvidia's GPUs to provide computing power.

This new chip, which has been developed since 2019, can obviously greatly reduce Microsoft's dependence on Nvidia GPUs.

According to people familiar with the matter, a joint team composed of Microsoft and OpenAI is now testing it.

Compared with competitors who entered the game early, Microsoft did not start the research and development of AI chips until 2019.

Also in the same year, Microsoft announced that it would invest $1 billion in OpenAI and required them to use Microsoft's Azure cloud server.

However, when Microsoft began working more closely with OpenAI, it found that the cost of purchasing GPUs to support the startup, Azure customers, and its own products was too high.

According to people familiar with the matter, during the development of Athena, Microsoft had ordered at least hundreds of thousands of GPUs from Nvidia in order to meet the needs of OpenAI.

As early as April this year, news revealed the existence of this chip, codenamed Athena.

It is said that Microsoft hopes that this chip, codenamed Athena, can compete with Nvidia's H100 GPU, which has been in short supply.

Currently, Amazon and Google have made artificial intelligence chips an important part of their cloud business marketing strategies.

Among them, Amazon’s investment in Anthropic stipulates that the other party needs to use Amazon’s AI chips, namely Trainium and Inferentia. At the same time, Google Cloud also stated that customers such as Midjourney and Character AI use self-developed TPU.

Although Microsoft is still discussing whether to provide its self-developed chips to Azure cloud customers, the chip's debut at the developer conference may indicate that Microsoft is seeking to attract the interest of future cloud customers.

What is certain is that Microsoft will use the launch of Athena to greatly shorten the distance with the other two giants-Google and Amazon have already adopted self-developed chips on a large scale in their own cloud servers.

In addition, in order to get rid of Nvidia's "stuck neck", Microsoft is also working closely with AMD to develop the upcoming artificial intelligence chip MI300X.

However, Microsoft and other cloud service providers generally indicate that they have no plans to stop buying GPUs from Nvidia.

But if they can convince cloud customers to use more of their own chips, it could lead to huge savings in the long run. At the same time, it can also help them gain more leverage in negotiations with Nvidia.

OpenAI: I don’t want either of these two companies

It is obviously best for OpenAI to reduce its dependence on Microsoft and Nvidia chips at the same time.

According to several job postings on the OpenAI website, the company is hiring people who can help it evaluate and co-design AI hardware.

Reuters also reported that OpenAI is planning to produce its own AI chips.

Previously, CEO Sam Altman had made acquiring more AI chips a top priority for the company.

On the one hand, the GPUs needed for OpenAI are in short supply, and on the other hand, the cost of running these hardware is "staggering."

If the cost of computing power remains high, it may not be good news for the entire AI industry in the long run.

After all, if the Nuggets' "shovel" sells for more than the gold itself, will anyone still be a gold digger?

According to Stacy Rasgon's analysis, ChatGPT costs about 4 cents per query. If ChatGPT's query volume grew to one-tenth the size of Google searches, it would require approximately $48.1 billion worth of GPUs and approximately $16 billion worth of chips per year to keep running.

It's unclear whether OpenAI will move forward with plans for custom chips.

According to analysis by industry veterans, this will be a strategic move with huge investment, and the annual cost may be as high as hundreds of millions of dollars. And even if OpenAI devotes resources to the task, success is not guaranteed.

In addition to complete self-research, another option is to acquire a chip company like Amazon acquired Annapurna Labs in 2015.

OpenAI has considered this path and conducted due diligence on potential acquisition targets, according to a person familiar with the matter.

But even if OpenAI moves forward with plans for custom chips, including acquisitions, the work could take years. During this period, OpenAI will still rely on GPU suppliers such as Nvidia and AMD.

Because even if it is as powerful as Apple, it took three years to acquire PA Semi and Intristy in 2007 and launch the first chip A4 in 2010.

And OpenAI itself is still a start-up company, so this process may be even more difficult.

And the most important moat of NVIDIA GPU is its accumulation of software and hardware ecosystem based on CUDA.

OpenAI must not only design hardware that does not lag behind in performance, but also catch up with CUDA in terms of software and hardware collaboration, which is definitely not an easy task.

However, on the other hand, OpenAI also has its own unique advantages in making chips.

The chips OpenAI wants to build do not need to be like the chips launched by other giants to serve the entire AI industry.

He only needs to satisfy his own understanding and needs for model training and design an AI chip customized for himself.

This is very different from Google and Amazon, which put their own AI chips in the cloud for third parties to use, because there is almost no need to consider compatibility issues.

This allows the chip to execute the Transformer model and related software stack more efficiently at the design level.

Moreover, OpenAI’s leading advantages and planning in model training will allow it to truly solve the hardware problems related to model training with its own exclusively designed chips in the future.

Don't worry about the performance of your own chips that "meet your own needs". Compared with industry giants like NVIDIA, you will have a latecomer disadvantage.

It's all a matter of cost

It is so difficult to design your own AI chip and compete directly with NVIDIA. Why do the giants still give up?

The most direct reason is that NVIDIA GPUs are too expensive!

Plus the cloud provider has to make another profit in the middle. In this way, including OpenAI, the cost for enterprises using the basic model of NVIDIA GPU + cloud provider will definitely remain high.

Some foreign media have made this calculation:

Today, the cost of purchasing an artificial intelligence training cluster using NVIDIA H100 GPU is about $1 billion, and its FP16 computing power is about 20 exaflops (not including sparsity support for matrix multiplication). Three years of renting on the cloud will increase the cost by 2.5 times.

These costs include network, compute, and local storage for the cluster nodes but do not include any external high-capacity and high-performance file system storage.

Purchasing a Hopper H100-based eight-GPU node can cost close to $300,000, including the amortized cost of InfiniBand networking (network cards, cables and switches).

The same eight-GPU node costs $2.6 million to rent on-demand on AWS and $1.1 million to reserve for three years. The price may be similar on Microsoft Azure and Google Cloud.

Therefore, if OpenAI can build a system at a unit price of less than $500,000 (including all costs), its cost will be reduced by more than half, while still being able to control its own "computing power freedom."

Cut these costs in half, and OpenAI’s model size would double while investing the same resources; if the cost could be reduced by three-quarters, it would quadruple. This is important in a market where model sizes double every two to three months.

So in the long run, perhaps the most basic problem that any ambitious large AI model company will have to face is how to reduce the cost of computing power as much as possible.

Getting rid of the "golden shovel seller" NVIDIA and using your own GPU is always the most effective method.

Hot discussion among netizens

Some netizens seem to have different opinions on OpenAI and Microsoft's decision to stop manufacturing AI chips, believing that AI chips are a "trap."

One of the biggest reasons why model companies such as OpenAI are forced to build hardware is that other chip companies are completely incompetent, and Nvidia has almost no competition.

If AI chips were a fully competitive market, companies like OpenAI would not end up making AI chips on their own.

Some netizens with more radical ideas believe that large language models will be integrated into chips in the future, and humans can communicate directly with computers using natural language. So designing chips is a natural choice to get to that point.

References:

https://www.theinformation.com/articles/microsoft-to-debut-ai-chip-next-month-that-could-cut-nvidia-gpu-costs?rc=epv9gi

Reprinted from 新智元 好困 润View Original

Comments

no dataCoffee time! Feel free to comment