The newest breakthrough technology in the Artificial Intelligence world, GPT-3, has been making waves ever since its debut in 2020. GPT-3 (Generative Pre-trained Transformer-3) has the ability to create any content that has a language-like structure — human or machine. Although this may seem simple at first, this advancement in AI will undoubtedly play an important role in how we interact with technology in the near future.
To give some background, GPT-3 was created by OpenAI, a research lab in San Francisco that was co-founded by Elon Musk. It is the latest addition in a series of language prediction models, which were created with the goal of emulating a human’s natural speech.
OpenAI originally started as a non-profit. However, with the release of GPT-3 it has shifted its business objective around to making money by using this new language model as its main product. Companies can request access to GPT-3 via its waitlist and then build services upon it via licensing agreements.
So how does it work?
GPT-3 is a language prediction model. This means it takes some input phrase from the user and returns what it thinks is the most useful continuation or response to the input. To give an example, you could ask it to continue on a sentence or give you a summary of a paragraph.
The detailed and relevant responses it provides are only possible because of the comprehensive training GPT-3 underwent during its development. Specifically, the training consisted of providing the model incomplete sentences and having it try to predict what words were needed to fill in the blanks. To achieve this, the AI would scour through billions of examples in its training text to determine what word it should use in response to the original phrase. Each word that the AI generates would be compared to the original sentence, which gets assigned a “score” depending on how well it fits into the sentence, based on comparisons to previously scored data. By comparing these “scores,” the model gradually learns what methods are likely to come up with the best words to fill in the blanks. This process of deciding what word to use is the foundation of being able to make complete sentences for an AI.
A possible sentence could be, “The blue house has a blue floor and ____ walls.” We humans can figure out that the word is most likely “blue” by following the pattern in the sentence, but the AI cannot do that on its own. The AI doesn’t have the prior knowledge that a human does, which is what we use to make educated guesses and follow the pattern. Without this knowledge, the AI could be generating colors, adjectives, or whatever options it thinks could fit in that sentence. What makes GPT-3 special is that it can scour through the billions of words in its dataset and compare each word it comes up with to find the best fit. After enough comparisons, it would most likely output “blue”, like a human.
This weighing process is nothing new, in fact it’s been around for many years. What’s special about GPT-3 is the pure amount of weights in its dataset. No other AI model even comes close, with the closest competitor reportedly having around only a tenth of the data GPT-3 has processed.
How is this technology being used today?
GPT-3 is able to create anything that has a language-like structure. This means it can be put to a wide array of tasks, not just returning results in English. These tasks include:
- Writing essays
- Writing working code
- Generating automatic plots
- Generating website layouts from a written description
- Writing creative fiction
These 5 examples are remarkable by themselves, but they’re just scratching the surface when it comes to the potential applications of this new technology. The possibilities are practically limitless.
To provide some statistics, hundreds of applications are now using GPT-3 in their systems. OpenAI reports that GPT-3 generates an average of 4.5 billion words a day and that number is still increasing.
One of these applications is called Viable. Viable uses GPT-3 to analyze customer reviews. Instead of looking through each review manually, Viable looks through all the reviews and provides a simple summary. You can also ask questions about the reviews, and get a satisfactory response. For example, a query of “What’s frustrating customers about the checkout experience?” could get a reply of “It takes too long to load.” Viable uses GPT-3’s ability to identify themes in text to allow it to function. It’s quite an interesting concept, and I recommend checking them out here.
On the flip side, there are certainly still problems with GPT-3 limiting its implementation. The computing power required to run the model is very high, which makes it expensive, hence its privatization. There is also a risk of generating unsafe outputs, which is arguably the largest of its many flaws. OpenAI has already acknowledged these problems in its blog and is working to fix these problems in the near future.
On the topic of unsafe outputs, a medical technology company called Nabla tested GPT-3 for use as a medical chatbot. When a simulated patient asked for mental health advice, they were told to commit suicide. You can imagine how this could be a potential liability for the company if they were dealing with a real patient.
Multiple other examples of unsafe outputs have popped up too, with the AI generating sexist, racist and/or other negative language when it was asked about Jews, black people, and other minorities that face discrimination. Jerome Pesenti (head of Facebook AI Lab) goes into great detail about this in this series of tweets. These examples clearly go to show that GPT-3 still has its “bad side” and that these are important issues to address when fully implementing this new technology.
How can it be applied to the future?
As AI technology develops, we can expect great things from the next versions of GPT-3. Currently, this technology is expensive to use and it can only be accessed if you have permission from OpenAI. In the future, more powerful technology will allow AI’s to be ever more powerful, so that they’re even more versatile than even the cleverest of humans. This will result in a steady increase of the popularity and widespread adoption of AI technology.
GPT-3 is already on a path to automate and revolutionize many industries. Take software engineering for example. With GPT-3 in their arsenal, software developers will only need to tell GPT-3 what they want in a program and review the code after. This feature saves a lot of time for the programmer and makes their life a whole lot easier. According to infoq.com, AI will be advanced enough that there may be no reviewing necessary, which means even people with no knowledge of coding will get to create whatever they want. This is clearly a step in the right direction.
There’s also some negatives that could come with the progression of GPT-3, one example being its effect on social media platforms. To be more specific, bot accounts have been a prevailing issue in social media for a long time. They are accounts that are not run by humans, but message other people and leave impressions on posts just like a human would. They can lure unsuspecting victims into scams and cause them harm. Right now, bots lack the ability to speak coherently and that makes them much easier to detect. But when GPT-3 goes public, there will be no way to distinguish between the bots and humans. People will start to trust bots like they do humans, and that will make the chances of getting scammed much higher. This is a point of concern, and will define the future of how people interact with AI based systems.
Soon, the world will be filled with AI generated chatter. As AI technology develops, we need to be prepared for massive changes in how we interact with technology.