On October 7, 2026, Scale AI, in partnership with the company Elorian, introduced Humanity’s Sixth Sense (HSS), a benchmark designed to measure the intuitive visual reasoning of artificial intelligence models. The set includes 522 open-ended tasks based on 288 images and 234 video clips, totalling 17.6 hours of footage. Its main finding is blunt: models miss what any person picks up at a glance.
The authors start from a paradox. Today’s models show a high “visual IQ”: they read charts, ace exams and extract text from images with ease. Yet they struggle with what is almost automatic for people: reading a room, knowing who holds authority in a meeting, judging whether a car will fit between two parked ones or understanding why a dog gets startled by birds. According to Scale AI, that intuition rests on a latent understanding of social dynamics, spatial relationships, physical cause and effect, and intent.
For agents deployed in homes, vehicles and workplaces, the authors argue, this skill is not a luxury but a necessity. Yet most existing visual benchmarks test expert-level academic analysis or low-level perception, leaving untested the intuitive reasoning that people perform instinctively. Cognitive science estimates that we categorize natural scenes within roughly 150 milliseconds while extracting physical dynamics, motives and social structure.
HSS organizes its tasks into four domains and eleven subdomains. The first, temporal and causal dynamics, asks models to reconstruct what happened before or what will happen next. The second, physical and spatial logic, deals with hidden objects, whether something fits or is reachable, and how a scene looks from another viewpoint. The third, social understanding, covers beliefs, emotions, unwritten norms and power dynamics. The fourth, abstract and contextual inference, looks for the rule that gives a scene meaning, including faces that appear in objects.
The construction process was demanding. Trained annotators chose an image or video and wrote a question, a reference answer and a rubric of atomic criteria. Each task had to require inference beyond what is depicted, need no specialized expertise and command unanimous human agreement. After three independent review rounds, only 522 of 3,466 authored tasks survived, an acceptance rate of 15.1%.
The evaluation also aims to prevent shortcuts. Models answer in free form, with no multiple choice to eliminate options against, and a judge model, Claude-Opus-5, scores each answer against the rubric: a task counts as solved only when every criterion is met. To check the reliability of the system, the authors re-graded five models with judges from three different vendors, and the rankings were identical, with agreement above 95%.
The results were obtained with 25 multimodal models from eight vendors and 20 human participants. People reached 93.1% accuracy; the best model, GPT-6-astra, stopped at 53.6% even at maximum reasoning effort, and the median was 30.9%. When the image or video was removed, GPT-6-astra dropped to 6.6%, confirming that the tasks cannot be solved from language priors alone.
More thinking does not close the gap. Models spend an average of 4,046 reasoning tokens per task on questions people answer at a glance. Raising reasoning effort helps overall but hurts some subdomains: GPT-6-astra loses 14 points on retrodiction, that is, inferring what already happened, when moving from high to maximum effort, and peak accuracy often comes at an intermediate setting.
The error analysis is perhaps the most revealing part. Of 8,573 failures examined, 94% trace to problems of perception or latent inference and only 5% to faulty logic. The two largest causes are missing the decisive visual cue (21%) and misidentifying an object, person or role (20%). The errors are systematic: on tasks that five or more models fail, a median of 80% fail for the same reason.
Social understanding is the weakest domain: it is the lowest-scoring one for 21 of the 25 models, averaging 24.4% against 34.1% for the other three. Video is also harder than images for 23 of 25 models, by 7.3 points on average. Scale AI offers examples: models did not understand that a woman slowed down after running because the bus had stopped to wait for her, and they claimed two books would not fit on a shelf where the gaps were visible.
The authors also tested tool-based setups. Inside Claude Code and Codex, where models can crop, zoom, search the web and re-sample media, the best setup reached 59.3% on a 388-task subset. Closer inspection fixes missed cues and misidentified objects, but not depth: errors from reading a two-dimensional overlap as three-dimensional alignment were essentially unchanged, going from 102 to 101.
The implications go beyond a leaderboard. If visual intuition is needed for home robots, self-driving vehicles or assistants that look through a camera, the gap measured by HSS points to a practical limit of current systems. The authors conclude that visual intuition is a measurable axis of multimodal intelligence that current scaling has left behind, and that more compute at answer time is not enough to solve it.
Like any benchmark, HSS has nuances. It uses a model as the judge, although the authors validated its verdicts with judges from three vendors, and scores will shift as new model versions arrive. Scale AI has published the dataset on Hugging Face, the evaluation code and a technical paper, which will let other teams reproduce the results and measure new models under the same rules.
In short, Humanity’s Sixth Sense puts numbers on an intuition widely shared among people who work with AI: models can solve problems that require expert knowledge and, at the same time, fail at visual judgments any person makes without thinking. The distance between the human 93.1% and the best model’s 53.6% sets a clear target for the next generation of multimodal systems and a warning for those planning to deploy them in the physical world.
