Nvidia and Microsoft have teamed up to create the Megatron-Turing Natural Language Generation model, which the duo claims is the "most powerful monolithic transformer language model trained to date".
"Each model replica spans 280 NVIDIA A100 GPUs, with 8-way tensor-slicing within a node, and 35-way pipeline parallelism across nodes," the pair said in a blog post [microsoft.com].
[...] However, the need to operate with languages and samples from the real world meant an old problem with AI reappeared [zdnet.com]: Bias. "While giant language models are advancing the state of the art on language generation, they also suffer from issues such as bias and toxicity," the duo said.
Related: OpenAI's New Language Generator GPT-3 is Shockingly Good [soylentnews.org]
A College Student Used GPT-3 to Write a Fake Blog Post that Ended Up at the Top of Hacker News [soylentnews.org]
A Robot Wrote This Entire Article. Are You Scared Yet, Human? [soylentnews.org]
OpenAI's Text-Generating System GPT-3 Is Now Spewing Out 4.5 Billion Words a Day [soylentnews.org]