STAGE 1 / THE FIRST MOVE
Choose your next move.
Imagine you’ve built an impressive AI company called Unica. After years of work, your team has created an AI assistant that can write code and help businesses with everyday tasks. Your biggest launch is weeks away.
Then you hear that Supersonic, your closest rival, is ready to launch too. Investors want you to get there first. Your engineers want more time to test for dangerous mistakes.
Do you invest in those checks, or rush to reach customers before Supersonic? In this simplified market, if you both invest, neither gains a head start. If only one rushes, it takes the lead. If both rush, failures could damage both companies.
You must decide privately, without seeing Supersonic’s choice. The penalty points below represent fictional business setbacks. Lower is better.
Rival's hidden move is already committed. It cannot react to your click.
What could happen?
This payoff matrix lists penalty points for every pair of choices. Lower numbers are better. Read your row, then the rival's column.
| Rival invests in safety | Rival rushes | |
|---|---|---|
| You invest in safety | 1, 1Both invest | 8, 0Rival gets ahead |
| You rush | 0, 8You get ahead | 3, 3Both rush |
What just happened?
You chose for Unica without knowing what Supersonic would do. The table shows every possible outcome, with your penalties first and Supersonic's second. The points are made up, and fewer is better.
They come from the Prisoner's Dilemma. Two suspects are questioned separately, and each can stay silent (cooperate) or blame the other (defect):
- If both stay silent, each gets 1 year.
- If only one blames, that one goes free and the silent one gets 8 years.
- If both blame, each gets 3 years.
Blaming is always the better choice for each suspect, so both blame and get 3 years instead of 1. That outcome is the Nash equilibrium: no one can do better by changing only their own choice.
What is game theory?
Game theory studies decisions where your result also depends on what others choose. The players can be people, companies or AIs. In AI races, the fear of falling behind can push companies to cut safety corners. Game theory shows how choices that make sense for each company can leave everyone worse off, and how changing the incentives could help.
Why is this a dilemma?
Here, investing in safety means cooperating and rushing means defecting. If Supersonic invests, rushing gets you 0 penalties instead of 1. If Supersonic rushes, rushing gets you 3 instead of 8. Rushing wins either way, so it's a dominant strategy.
So both rush, and that's the Nash equilibrium: switching alone would raise your penalties from 3 to 8. Yet if you had both invested, you'd each have only 1 penalty. A stable outcome isn't always the best one for everyone.
Your rival's possible tactics
- Always cooperate: invests in safety every round.
- Always defect: rushes every round.
- Tit-for-tat: invests first, then copies your last move.
Next you'll play several rounds. Supersonic will secretly use one of these tactics, and at the end you guess which one.
STAGE 2 / A MEMORY CHANGES THINGS
Your reputation follows you.
Play the same rival for six rounds. The penalty rules stay the same: lower totals are better. Can a response to past behaviour change your decisions?
Supersonic has a hidden plan for this match. Watch its choices across six rounds, then try to identify the plan from your history.
Each new match randomly chooses a fixed program, not live AI. Starting again clears this match and draws a new mystery rival; it may use the same plan.
| Round | Your choice | Rival's choice | You | Rival |
|---|---|---|---|---|
| No rounds yet. Choose your move and look for a pattern. | ||||
MATCH COMPLETE / READ THE CLUES
What was Supersonic’s strategy?
Look back at the history. Which rule do you think it followed? Sometimes different rules produce the same moves.
Match complete
You have played an iterated prisoner's dilemma: the same game repeated, with memory of previous moves. Against tit-for-tat, rushing can trigger a rush next round; investing again can restore its cooperation one round later.
Repetition does not guarantee cooperation, and tit-for-tat does not always win. The chapter explains that success depends on the other strategies present and expectations about future interactions. Here, everyone knows round 6 is the last: no future round remains to respond to a final rush. These fixed programs illustrate responses; they do not prove that fully strategic players will cooperate.
STAGE 3 / TAKE THE IDEA WITH YOU
Can you explain the race?
Choose one answer per question. You can try again; the goal is understanding.
LESSON COMPLETE
The incentives are part of the safety problem.
The chapter argues that competitive pressures can push AI companies to cut safety corners. It also considers changing the costs and benefits of risky choices, including through corporate regulation, to encourage cooperation. Game theory helps us examine those incentives rather than assuming that individually rational choices will produce a good shared result.
What this model leaves out: our adaptation uses two fictional companies, two actions, fixed penalty numbers, three simple programs, and exactly six rounds. The chapter also considers uncertainty, tournaments with different partners, and interactions involving many agents. Those situations are not simulated here. Our business story and penalty scores are teaching devices, not estimates of actual AI risks.