# Human or AI game, with real photos and classic writing Source: https://decka.dev/ideas/human-or-ai-game A human or AI game to play live. Eight rounds of real photos, AI pictures, classic social science and AI paragraphs, and the research on who gets fooled. A human or AI game puts something on the screen and asks the room one thing: whether a person made it or an AI did. Everyone votes on their phone, then you show the answer. Each round runs on a short clock, and the discussion starts from the rounds the room gets wrong. One well-known version was AI21 Labs' Human or Not?, a two-minute chat with either a stranger or a bot. In its first month it was played by [over 1.5 million users](https://arxiv.org/abs/2305.20010), who guessed right in [68% of the games](https://arxiv.org/abs/2305.20010). Against a bot they guessed right in only [60%](https://arxiv.org/abs/2305.20010). It no longer runs at its old address: [humanornot.ai](https://www.humanornot.ai/) now forwards to AI21's home page. This page has a version you run yourself, in a lecture hall or a staff meeting. The deck has eight rounds, four pictures and four short paragraphs. Some are real photographs and passages from classic books, and some were made by AI for this quiz. The reveal after each round says which. ## How to play the human or AI game 1. Put up one picture or one paragraph. 2. Everyone answers Human or AI on their phone before the clock runs out. 3. Show the answer, and how many got it right. 4. Ask someone who got it right, and someone who got it wrong, what gave it away. The fourth step is the lesson. Ask what gave each round away, and listen for the clues people name: the light is too perfect, the hands look odd, the sentence is too smooth. In the studies further down, people sorting real faces from AI ones were about as accurate as a coin toss, so a confident clue is worth testing. Hearing the clues out loud is how a room tests them. Keep the rounds short. This deck gives a picture thirty seconds and a paragraph of fifty or sixty words a minute. In a large room, read each paragraph aloud once. Do not tell the room how many of each there are, so nobody can answer by counting. ## What is in the deck Eight rounds, alternating a picture and a paragraph. The pictures are four ordinary scenes: a roadside fruit stall, a hummingbird on a branch, a rocky cove and a bowl of peppers. Two are photographs their photographers gave to the public domain. Two were made with Google's Gemini image model for this quiz, from prompts that asked for an ordinary photograph with ordinary flaws. All four were cropped to the same shape and size and stripped of their file data, so the file cannot answer the question. There are no faces. The studies cited below used faces, and a made-up face is fair to test. A real face belongs to a real person, though, and a room deciding whether that person exists is a different exercise from this one. The paragraphs share two old subjects: what poverty takes from people, and how feelings show on the body. Two are real, from Jane Addams's [Twenty Years at Hull-House](https://www.gutenberg.org/ebooks/1325) and Charles Darwin's [The Expression of the Emotions in Man and Animals](https://www.gutenberg.org/ebooks/1227). Two were written by an AI model in the same voice, on the same subjects. The deck's last slide credits every photograph and passage. The answer key is in the speaker notes. ## Can people tell AI pictures from real photos With faces, mostly not, and they are surer than they should be. The studies below tested faces made by one AI model, not scenes like the ones in this deck. Sophie Nightingale and Hany Farid showed [315 people](https://pmc.ncbi.nlm.nih.gov/articles/PMC8872790/) a mix of real faces and faces made by an AI model, one at a time. Average accuracy was [48.2%](https://pmc.ncbi.nlm.nih.gov/articles/PMC8872790/), close to the coin toss's [50%](https://pmc.ncbi.nlm.nih.gov/articles/PMC8872790/). A second group of [219 people](https://pmc.ncbi.nlm.nih.gov/articles/PMC8872790/) first went through a short tutorial on the flaws to look for, and got feedback after every face. They reached [59.0%](https://pmc.ncbi.nlm.nih.gov/articles/PMC8872790/), and did not improve as the trials went on. The same paper asked a third group to rate the faces for trustworthiness. The AI faces averaged [4.82 against 4.48](https://pmc.ncbi.nlm.nih.gov/articles/PMC8872790/) for real ones on a seven-point scale. The authors call that [only 7.7% more trustworthy](https://pmc.ncbi.nlm.nih.gov/articles/PMC8872790/), and statistically significant. It is a small gap pointing the wrong way. Elizabeth Miller and colleagues ran a study of [124 adults](https://aura.abdn.ac.uk/items/338d3219-2e43-4f1c-89ff-0fcd17dec90a) on White faces from the same AI model. The AI faces were judged human [65.9% of the time](https://aura.abdn.ac.uk/items/338d3219-2e43-4f1c-89ff-0fcd17dec90a), and photographs of real people only [51.1%](https://aura.abdn.ac.uk/items/338d3219-2e43-4f1c-89ff-0fcd17dec90a). They call it AI hyperrealism. It did not hold for faces of other races, which the authors trace to a model trained on mostly White faces. One more finding is worth stopping on in a classroom. In Miller's study, [the people who made the most errors were the most confident](https://aura.abdn.ac.uk/items/338d3219-2e43-4f1c-89ff-0fcd17dec90a). The game does not ask how sure anyone is, so ask the room yourself: before each reveal, have people raise a hand if they are certain, then compare with the answer. A wider survey points the same way. Joel Frank and colleagues asked [3,002 people](https://arxiv.org/abs/2312.05976) in the United States, Germany and China to judge pictures, text and audio. Accuracy on images was [below 50%](https://arxiv.org/abs/2312.05976), and AI-made media of every kind was more often taken for human work. All three studies tested faces from an older image model, StyleGAN2. The pictures in this deck come from a newer one, and there are no faces in them. ## Can people tell AI writing from human writing Not reliably, and the clues they trust point the wrong way. Maurice Jakesch, Jeffrey Hancock and Mor Naaman ran six experiments with [4,600 people](https://pmc.ncbi.nlm.nih.gov/articles/PMC10089155/), who read short profiles of the kind people write for dating, rentals and freelance work. In the three main experiments, readers named the source correctly [50 to 52%](https://pmc.ncbi.nlm.nih.gov/articles/PMC10089155/) of the time. Paying them for accuracy did not help, and nor did telling them after each answer whether they were right. The researchers then looked at what readers relied on. Profiles with grammatical slips were taken for AI, though they were [15% less likely](https://pmc.ncbi.nlm.nih.gov/articles/PMC10089155/) to be AI. Contractions read as human, and turned up a little more often in the AI text. First-person writing and talk of family felt human and told readers nothing either way. AI text tuned to fit those beliefs was judged human [65.7% of the time](https://pmc.ncbi.nlm.nih.gov/articles/PMC10089155/), against [51.7%](https://pmc.ncbi.nlm.nih.gov/articles/PMC10089155/) for real profiles. Poetry shows the same pattern. Brian Porter and Edouard Machery mixed poems by ten well-known poets, Shakespeare, Dickinson and Plath among them, with AI poems written in each poet's style. Non-expert readers were right [46.6% of the time](https://doi.org/10.1038/s41598-024-76900-1), below chance, and took the AI poems for human work more often than the real ones. Conversation has gone the same way. Cameron Jones and Benjamin Bergen ran a five-minute test in which people chatted with a person and an AI at once, then picked the human. Prompted to play an introverted young person who uses slang, [GPT-4.5 was picked as the human 73% of the time](https://arxiv.org/abs/2503.23674). The figure comes from a preprint. The paragraph rounds use old writing on purpose, to test a clue people reach for: that old-fashioned prose must be human. Two of the four paragraphs here are AI writing in that voice, so the round tests the clue directly. ## What it means for marking essays The study to show colleagues is from the University of Reading. Peter Scarfe and colleagues wrote answers to at-home exams with GPT-4 and slipped them, unedited, into five undergraduate psychology modules without telling the markers. [94% of the AI submissions](https://doi.org/10.1371/journal.pone.0305354) went undetected. On average they scored [half a grade boundary higher](https://doi.org/10.1371/journal.pone.0305354) than real students. Teachers asked outright still miss most of them. Johanna Fleckenstein and colleagues gave short English essays to [89 trainee teachers](https://hilpub.uni-hildesheim.de/bitstreams/3a70a194-ae71-4ccb-ba3c-7442d4968af2/download) and [200 experienced ones](https://hilpub.uni-hildesheim.de/bitstreams/3a70a194-ae71-4ccb-ba3c-7442d4968af2/download), told them some were written by ChatGPT, and asked which. Trainees spotted [45.1% of the AI texts](https://hilpub.uni-hildesheim.de/bitstreams/3a70a194-ae71-4ccb-ba3c-7442d4968af2/download), and experienced teachers [37.8%](https://hilpub.uni-hildesheim.de/bitstreams/3a70a194-ae71-4ccb-ba3c-7442d4968af2/download). Both groups were confident: experienced teachers put their own confidence on the AI texts at [80.6%](https://hilpub.uni-hildesheim.de/bitstreams/3a70a194-ae71-4ccb-ba3c-7442d4968af2/download). Software does not rescue the reader, and it carries a risk of its own. Weixin Liang and colleagues ran seven AI detectors over [91 essays](https://pmc.ncbi.nlm.nih.gov/articles/PMC10382961/) written by non-native speakers for the TOEFL English test. On average the detectors labelled [61.3% of them](https://pmc.ncbi.nlm.nih.gov/articles/PMC10382961/) AI-generated, and at least one detector flagged [97.8%](https://pmc.ncbi.nlm.nih.gov/articles/PMC10382961/). Every one of those essays was written by a person. The authors [strongly caution against](https://pmc.ncbi.nlm.nih.gov/articles/PMC10382961/) using detectors to judge students' work, particularly that of non-native English speakers. So catching AI by reading for it is not something to grade on. Change the task instead. Have students write some of it in the room. Ask them to talk you through a paragraph they handed in. Mark the drafts and the reasoning as well as the final file. None of that needs a detector, and none of it accuses an honest student. The game is a good way to open that conversation in a department. Run it before the discussion, so the room talks about its own answers rather than a study's. ## How to make your own human or AI rounds **Pick ordinary subjects.** Food, birds, streets, weather, a page about families or work, as this deck does. **Use pictures you are allowed to use.** Wikimedia Commons marks every file with its licence. Look for files marked CC0, the licence this deck's two photographs use. Check the licence on each file's own page, not the site it came from, and write down the source as you go, because you will credit it at the end. **Leave out real people's faces.** A stranger did not agree to be a quiz question. **Ask the image model for an ordinary photograph.** Describe a phone camera, uneven light, a little clutter, nothing posed. This deck's prompts asked for an ordinary photograph with ordinary flaws. **Make every file look the same.** Crop them all to one shape and one size, and strip the file data. Then no picture stands out by its shape, and no file names a camera. **Cut real passages at full stops.** Forty to seventy words, whole sentences, nothing skipped in the middle. Leave out lines your room may know by heart, and anything with a giveaway spelling. **Write the AI ones on the same subjects, in the same voice.** Then search the real book for the AI's best phrases, so no AI round borrows a real sentence. **Mix the answers unevenly**, so the answers follow no rhythm the room could learn. **Credit everything at the end.** Name each photographer, collection and book, and say which rounds were made by AI and with what. ## Running the human or AI game live Decka's [live quiz](/activities/live-quiz) runs all eight rounds from one slide. Everyone joins from their own phone with a code, with no app and no account. Each round puts the picture or the paragraph on the big screen and on every phone, with two answers: Human and AI. When the clock runs out, a picture round keeps its picture and stamps the verdict across it: the answer, then the share of the room that "got it right". A paragraph round names the answer and the share that "answered correctly". The leaderboard comes next, and faster correct answers score more. The deck on this page has the eight rounds ready, with a slide on how to play before them and the research and the credits after. The speaker notes hold the answer key, round by round. Decka runs inside [PowerPoint](/powerpoint) as an add-in and inside [Google Slides](/google-slides) through a Chrome extension, so the quiz can sit in a lecture deck you already have. To write your own rounds, press "Make this quiz free" at the end of this page. The quiz it makes opens with the four paragraphs, ready to edit. ## Frequently asked questions ### What is the human or AI game? A guessing game in which the room sees a picture, a passage or a chat and decides whether a person or an AI made it. Each round ends with the answer, so people find out straight away how good their instincts are. It works as a warm-up, a lesson on AI, or a staff meeting about assessment. ### Is Human or Not still online? Not at its old address. AI21 Labs' Human or Not? ran as a two-minute chat with a person or a bot, and its address, [humanornot.ai](https://www.humanornot.ai/), now forwards to AI21's home page. You can run your own version with a live quiz, using pictures and passages you choose. ### Can people tell AI images from real photos? With faces, not reliably. The studies cited here all tested faces. In Nightingale and Farid's study people sorted real and AI faces with [48.2% accuracy](https://pmc.ncbi.nlm.nih.gov/articles/PMC8872790/), close to chance. In Miller and colleagues' study, White AI faces were judged human [65.9% of the time](https://aura.abdn.ac.uk/items/338d3219-2e43-4f1c-89ff-0fcd17dec90a), more often than real ones. ### Can teachers tell AI essays from student essays? Mostly not. In Fleckenstein and colleagues' study, experienced teachers identified [37.8% of the AI-written essays](https://hilpub.uni-hildesheim.de/bitstreams/3a70a194-ae71-4ccb-ba3c-7442d4968af2/download) and were confident while doing it. At the University of Reading, [94% of AI exam answers](https://doi.org/10.1371/journal.pone.0305354) went undetected by markers. ### Do AI detectors work on student essays? Not well enough to grade on. In Liang and colleagues' study, seven detectors labelled [61.3% of essays](https://pmc.ncbi.nlm.nih.gov/articles/PMC10382961/) by non-native English writers as AI-generated, on average, though people wrote all of them. The authors advise against using detectors to judge students' work. ### Where can I find public domain photos for a human or AI quiz? Wikimedia Commons shows the licence on every file. Look for photographs marked CC0, the licence this deck's two photographs use. Check each file's own licence before you use it, and credit the source on your last slide. ### How many rounds should a human or AI game have? This deck uses eight, as an editorial choice: enough rounds for a room to notice a pattern in its own mistakes, and short enough to fit inside a lesson. Mix pictures and passages, and do not make the answers alternate. ## Activities - [Live Quizzes](https://decka.dev/activities/live-quiz.md): Timed questions, live leaderboard [Human or AI? Eight rounds](https://decka.dev/templates/human-or-ai-eight-rounds-9148b98166cd.md)