More than a hundred years ago, French mathematician Émile Borel theorized what might happen if a monkey randomly hit the keys on a typewriter. What is the probability that the monkey’s chaotic keystrokes would end up producing a copy of Hamlet?
So far, no monkeys have managed to replicate Shakespeare’s work (Borel believed it would take an unthinkable amount of time). But computer scientists have developed a new tool that CAN generate coherent pieces of text.
In simplest terms, GPT-3 (short for Generative Pretrained Transformer, version 3) is a piece of software capable of producing text that resembles something created by humans. GPT-3 was developed by OpenAI, a San Francisco-based lab that conducts research into artificial intelligence. Since OpenAI’s launch in 2015 (with more than $1 billion in support from Elon Musk, Sam Altman, and others), the company has focused on creating software that analyzes enormous amounts of source text. Previous models demonstrated a fascinating — but fundamentally limited — ability to convert user instructions into comprehensible paragraphs.
Here is OpenAI’s description of GPT-2 (2019)
GPT-2 generates synthetic text samples in response to the model being primed with an arbitrary input. The model is chameleon-like—it adapts to the style and content of the conditioning text. This allows the user to generate realistic and coherent continuations about a topic of their choosing.
GPT-3 represents a massive leap forward. For example, GPT-2’s programming included 1.5 billion parameters, but GPT-3 has 175 billion. In terms of output, OpenAI claims that GPT-3 can generate content in a variety of genres and formats, including not just text but also music tabs and computer code.
Most artificial intelligence systems function within a very narrow context. Consider, for instance, the programs designed to play games like chess, go, or checkers. GPT-3 engages with a much broader set of ideas and situations. A user provides GPT-3 with a text prompt, and the software will attempt to predict logical content and match the literary style.
To really appreciate the potential for GPT-3, take a look at an experiment conducted by Delian Asparouhov, a principal at Founders Fund. Asparouhov composed a few hundred words of text explaining the process for establishing your startup’s board of directors. After he inputted his work into GPT-3, the program generated more than twice as much text, most of which is fluidly written and factually accurate.
In addition to completing a user’s written composition, GPT-3 can even write an entire document, based on prompts like title, topic, and audience. Some people with early access to GPT-3 have noted the software’s ability to generate creative writing in a number of forms, like short story and poetry.
Beyond text in a literary context, users can leverage text for purposes of coding and developing applications. Preliminary experiments include the automatic generation of new search engines, calendar programs, and games.
In the tech world, we’ve encountered a number of “game changers” like bitcoin, blockchain, and quantum mechanics. So far, though, practical applications of those technological innovations have been far more limited than many pundits expected.
Let’s take a two-minute reality check.
Could GPT-3 really change everything in the tech ecosystem? Perhaps. The early tests show astonishing potential for reducing the time and energy required to generate content and develop applications.
And yet, GPT-3 still faces some serious obstacles before becoming a ubiquitous part of your company’s tech stack. The “magic” generation of text requires the establishment of many parameters. OpenAI expects that custom training a GPT-3 model could cost upwards of $5 million.
Even OpenAI co-founder Sam Altman recognizes the current limitations, as he communicated in a tweet:
“The GPT-3 hype is way too much. It’s impressive (thanks for the nice compliments!) but it still has serious weaknesses and sometimes makes very silly mistakes. AI is going to change the world, but GPT-3 is just a very early glimpse. We have a lot to still figure out.”
1. Do you think your company’s product could be made redundant by advancements in artificial intelligence? If so, you might want to reflect on your long-term goals and brainstorm new directions.
2. Do you think your company could benefit from more robust engagement with artificial intelligence? If so, you should keep your pulse on developments with GPT-3.
3. Before jumping into GPT-3 with both feet, take a moment to stop and think about your connection to artificial intelligence. Are you just trying to stay “on trend” or are you genuinely interested in making this the primary focus of your next 10+ years?
TechToronto is the best way to expand your knowledge and network within Toronto's technology community.
