How does a chatbot create replies?

1. Mia types a question, and the chatbot's reply appears one word at a time. It looks as if someone is thinking and typing. So what is really happening inside?
2. Before anyone could chat with it, the program was shown a huge amount of writing. It did not memorize it all. It adjusted itself to capture patterns, such as which words tend to follow which.
3. Your question is chopped into small pieces called tokens. A token is a word or part of a word. Each token becomes a number, because computers work with numbers, not letters.
4. From the pieces so far, the model works out a chance for every word that could come next. After 'Cats like to', the word 'sleep' is likely and 'fly' is not. Then it picks one, usually a likely one.
5. The chosen word is added to the end, and the whole thing goes back in. Guess, pick, add, repeat. That loop is why the reply appears one word at a time.
6. The model picks words that sound right. It does not check whether they are true, so a reply can be smooth, confident and wrong. It does not know or feel anything. For important facts, check a trusted source.

Tap each part to follow your question from the screen to the reply. This is a simplified model.

• Your question. The words you type are called the prompt. Everything the chatbot writes starts from the prompt and the conversation so far.
• Word pieces. The prompt is cut into pieces called tokens: whole words or parts of words. Each one is turned into a number.
• What it read. Long before your chat, the model was trained on a huge amount of writing. That is where its word patterns come from, including any mistakes in that writing.
• Pattern network. A giant web of numbers, called a neural network. The token numbers flow through it, and out come chances for the next word.
• Next-word chances. For each possible next word the model gives a chance. Words that fit well get a big chance. Words that fit badly get a tiny one.
• Picker. The picker chooses one word using the chances, like a spinner with big and small slices. That is why the same question can get different replies.
• Reply so far. The picked word joins the reply, and the reply so far goes back into the machine to guess the next word. Round and round it goes.

Tap the spinner to pick each next word, or tap a word card to choose it yourself. Then ask a different question and compare the replies. This is a tiny, simplified model.

Experiment: If you ask this little model the same question several times, what will its replies be like?

1. Tap the spinner until the reply ends with a full stop. Read it.
2. Tap the cat question again and watch the machine write a new reply. Is it the same?
3. Ask a different question. Look at how the word cards change.
4. Try the moon cat question. Is the reply true?

What we found: The replies are often different, because each word is picked by chance from the likely ones. A different question gives different chances. And the model happily answers the moon cat question, although no cat lives on the moon. It makes sentences that sound right. It does not check that they are true.

Turn on JavaScript to try it yourself.

This machine has read nothing yet. Let it read up to 6 books, then test it. It must say 'Ice is cold' and 'The sun is hot' at least 75 times out of 100.

A chatbot is built on a program called a language model. Before anyone could chat with it, the model was trained: it was shown a huge amount of writing and practiced guessing the next word, billions of times. Every wrong guess changed it a tiny bit. In the end it had captured patterns, such as which words go together and how sentences are built.

When you ask a question, your words are cut into pieces called tokens. The model uses its patterns to give every possible next piece a chance. Then one is picked, usually a likely one. It is added to the reply, and the model guesses again. Guess, pick, add, repeat: that is why the reply appears one word at a time, and why asking twice can give two different replies.

Here is the important part. The model picks words that sound right. It does not look things up, and it does not check whether a sentence is true. So a chatbot can say something false in a smooth, sure voice. It can even invent things, such as a book that was never written. People call this making things up, or hallucinating.

A chatbot does not know things the way you do. It has no feelings, even when its words sound kind or sad, because those words are patterns too. Chatbots can be very useful for ideas, practice and explanations. For anything important, check with a trusted book, a reliable website, a teacher or a grown-up. Our guessing-game comparison is also simplified: a real model looks at all the earlier words at once, not just the last one.

Key words

Language model

A program trained on lots of text to predict which piece of text is likely to come next.

Token

A piece of text the model works with: a word, part of a word or a punctuation mark.

Prompt

The text you give the chatbot. The reply is built starting from it.

Training

Showing a model huge amounts of text and adjusting its numbers so that its next-word guesses improve.

Temperature

A setting that controls how often the model picks less likely words. Low is steady, high is surprising.

Hallucination

When a chatbot writes something that sounds right but is false or made up.

Check your thinking

1. How does a chatbot build a reply?

   Show the answer

   It guesses a likely next word, adds it, and repeats. The reply is built one piece at a time. Each new piece is a guess based on everything written so far.

2. A chatbot's reply sounds very sure of itself. What does that tell you?

   Show the answer

   Only that the words fit together well. It could still be wrong. The model picks words that sound right together. Nothing inside it checks whether the sentence is true.

3. A model was trained mostly on writing that contains a mistake. What is likely to happen?

   Show the answer

   It may repeat the mistake, because it learned that pattern. A model can only repeat patterns it was given. That is what you saw in the challenge: mistaken books teach mistakes.

4. Why can the same question get two different replies?

   Show the answer

   Each word is picked by chance from the likely ones. The picker works like a spinner: likely words win more often, but not every time.

5. A chatbot writes 'I'm so sorry you feel sad.' What is true?

   Show the answer

   It produced kind-sounding words. It does not feel sorry. Kind words are patterns it learned from writing. They can still be helpful, but there is no feeling behind them.