TrendForce estimates ChatGPT needs over 30,000 Nvidia GPUs

[Jan]

ChatGPT will need over 30,000 Nvidia GPUs to go commercial

OpenAI will need 30,000 top-end Nvidia GPUs to power a commercial version of its ChatGPT language model, according to an analysis by TrendForce.

The market research firm explained that the generative AI that made ChatGPT and other language models useful required a huge amount of data for training.

 

[Grubscrew]

Instead of "mining mathematics" for bitcoin, slackers can now use their pcs for something good.

 

[Vorastra]

ChatGPT will need over 30,000 Nvidia GPUs to go commercial

OpenAI will need 30,000 top-end Nvidia GPUs to power a commercial version of its ChatGPT language model, according to an analysis by TrendForce.

The market research firm explained that the generative AI that made ChatGPT and other language models useful required a huge amount of data for training.

Imagine all those 80GB H100s running.

 

[Sapphiron]

That's actually not that many. There are individual supercomputers with more than that.

Not to mention all the bitcoin mining operations.

 

[Polymathic]

That's all?

 

[IT_Steven]

Ok and?

 

[2026]

That's actually not that many. There are individual supercomputers with more than that.

Not to mention all the bitcoin mining operations.

What would be interesting is how large is their training data.

10. It was trained on a massive corpus of text data, around 570GB of datasets, including web pages, books, and other sources.

Ok, not as large as I thought it would be, but that is a massive amount of text when you think about it.

The GPT-3 AI model reportedly cost OpenAI $12 million for a single training run.

Quite costly to train it though?

 

[Nerhzelok]

What would be interesting is how large is their training data.



Ok, not as large as I thought it would be, but that is a massive amount of text when you think about it.



Quite costly to train it though?

I asked ChatGPT what 1gb of text from books, articles and website looks like:

• Books: Assuming an average book size of 90,000 words, 1GB of text data would be equivalent to approximately 11,000 books (assuming no images or other multimedia content).
• Articles: Assuming an average article length of 1,000 words, 1GB of text data would represent around 1 million articles.
• Webpages: Assuming an average webpage size of 1,000 words, 1GB of text data would be equivalent to around 1 million webpages.

Considering 570gb of datasets: 62.7 million books or 570 million articles or 570 million web pages.

According to a survey conducted by the Pew Research Center, the average American adult read 12 books in 2019. Assuming an average lifespan of 78 years, this would translate to approximately 936 books over a lifetime.


Bliksem.. Those are all really rough estimates and I did cut some of GPT's output for brevity, so not to be taken seriously but just to put things into a bit more perspective, 570gb of training data is a $#@% load of knowledge.

 

[Vorastra]

That's all?

$450 million just for GPUs is small to you?

 

[backstreetboy]

$450 million just for GPUs is small to you?

M$ gave them a few billion $ already.

 

[Vorastra]

M$ gave them a few billion $ already.

Clearly a small operation.

 

[Polymathic]

$450 million just for GPUs is small to you?

Have you seen what ChatGPT can do and the scale?

 

[Vorastra]

Have you seen what ChatGPT can do and the scale?

You tell me, you think half a billion dollars in GPUs is nothing.

 

[Polymathic]

You tell me, you think half a billion dollars in GPUs is nothing.

For paradigm shifting technology ChatGPT is, that's peanuts.