Generative Pre-trained Transformer - 3

Subscribe to my newsletter and never miss my upcoming articles

gpt3.jpg Generative Pre-trained Transformer 3 (GPT-3) was released by OpenAI in July 2020. Since then there has been much hype around it. In this post i will discuss what it is and how does it impact the future of AI especially from developer's prospective.

What is GPT-3?

GPT-3 is a large open source state-of-the-art language model with more than 175 billion parameters which is quite a big number compared to it's predecessor GPT-2 which "only" had 1.5 billion.

A parameter is like a measurement in a neural network that gives some weight to few aspects of data, for providing that aspect larger or smaller importance which will impact the entire measurement of the data.

GPT-3 was first introduced in May 2020 & was available for the beta testing from July. Before this the biggest language model was Microsoft's Turing NLG which was trained on 17 billion parameters.

Due to the data it is trained on, it is able to generate text that is correct both grammatically and contextually.

image.png

What does it offer?

GPT-3 is capable to perform language translations, 3-digit arithmetic, question-answering, generate programming code from description, as well as several other tasks that require reasoning or domain expertise. The OpenAI researchers stated in their paper. “We find that GPT-3 can generate samples of news articles which human evaluators have difficulty distinguishing from articles written by humans.”

Here are some simple example :

  1. A grammatically wrong english sentence was written and GPT-3 corrected it (written in bold) image.png

  2. Translating english language to french image.png

  3. Getting a bash command from textual description image.png

Conclusion

Apart from a huge technical achievement in the field of AI, there is one more interesting fact about it that it's available as an API.

Being able to run a neural network like this is no easy as it needs expensive infrastructure to keep it running, and if you want to use it at scale it’s a different challenge altogether.

And people who are talking about replacing the developers jobs, according to me that is not going to happen anytime soon because after all it is a trained machine & it lacks creativity and problem solving ability.

Whereas software engineering is a vast field which covers an entire process, from building algorithms,writing code and integrating it (software) with hardware and IT infrastructure and so far no machine has been invented to do so with as much accuracy as needed.

Overall, GPT-3 is a huge step in the AI space, specially for natural language generation, and going forward it will help multiple industries. The biggest application in my opinion will be in chatbots, using it's API we can enhance the conversational experience & can take it to whole new levels.

download.png

Comments (5)

Peter Thaleikis's photo

Is there a chance to try it out ? Sounds really interesting, but I haven't found a way to use / try it.

Show +2 replies
Apoorv Tyagi's photo

Peter Thaleikis you can try filling out this for by debuild.co

If you are lucky enough you will get the access. But as Chris said there is a waitlist

Peter Thaleikis's photo

Ah okay, thanks for sharing Apoorv Tyagi