What is Word Prediction in Generative AI?

Imagine you are playing a game of "fill-in-the-blank" with a friend. You give them a sentence with a missing word and ask them to guess what word fits best. For example:

"The sun sets behind the ______."

Your friend would probably guess words like "mountain," "ocean," or "horizon" based on what makes sense in that context.

In a similar way, a large language model (LLM) like ChatGPT works by predicting which word is most likely to come next in a sequence based on the words that have already been provided. However, instead of using intuition or personal experience, the AI model uses probabilities derived from patterns it has learned from vast amounts of text data.

How Does the Model Predict Words?

Let’s break down how the model uses probabilities to predict the next word.

Example: Word Prediction

Let’s use the following example to illustrate how the model predicts words:

"The city was buzzing with excitement as people gathered for the annual ______."

When given this prompt, the LLM doesn’t just randomly pick words. It evaluates the context—specifically the phrase “annual” and the overall feeling of the sentence—to predict what comes next. Here are some of the possible predictions:

• "festival" — this fits the context of a yearly event.

• "parade" — this also fits, as parades are common at such events.

• "concert" — a valid option if the event involves music.

At this point, the model doesn’t just choose a single word. It might consider multiple options and assign probabilities to each one based on its training.

How Probabilities Work in Word Prediction

At its core, an LLM works by estimating the likelihood of each possible word based on the words that came before it. The model uses probabilities to measure which word is most likely to follow the given sequence.

Let’s say the model looks at the words "annual" and "gathered for the" and decides that the most likely continuation is "festival." It assigns a high probability to this word. But, it doesn’t stop there. It continues to predict the next word, refining its guesses step by step.

For example:

• If it picks "festival," the model may then suggest words like "celebration," "parade," or "performance" to follow.

• If it picks "parade," the model might next predict words like "marching," "floats," or "bands."

The Branching Process of Predictions

Think of this process as a branching decision tree. Each choice the model makes (like choosing "festival" over "parade") leads to new branches, which are the next set of word possibilities. As the sentence evolves, the model considers all the context and refines its choices to make sure the sentence flows naturally.

This branching process allows the model to generate long, coherent sequences of words by continually narrowing down the possibilities as it moves through the sentence.

Human vs. AI Prediction

This word prediction process in AI is not that different from how humans predict the next word when they’re speaking or writing. When you say something like, “I’m going to the store to buy some ____,” your brain will instantly think of words that fit the context: “groceries,” “snacks,” or maybe “milk.”

LLMs work in a similar way, except that they do it by calculating the probabilities of various words appearing in sequence, based on their previous training. The more context the model has, the more accurate its predictions become.

Key Takeaways

• LLMs predict words by calculating the probability of each possible word based on the context of the sentence.

• The model refines its predictions step by step, making the sentence more coherent with each new word.

• The process is like a branching tree, where each word choice influences future options.

• The prediction process is similar to how humans generate language based on context and experience.

Activity: Predicting the Next Word

Let’s practice predicting the next word in a sentence! Try your hand at continuing a sentence based on the given context.

Instructions:

1. Consider this sentence:
   "The rain poured down as the streets began to fill with _____."

2. What word do you think would most logically follow? Write it down.

3. Now, imagine the next word the model might predict after your choice. Continue the sentence with that word.