#AI horizons 25-07 – Altman’s Third Nightmare

Executive Summary

Sam Altman’s clearest AI warning at the Federal Reserve’s July 2025 banking conference was grave: while malicious actors and misaligned systems are headline risks, the most insidious threat is Scenario 3, a slow erosion of collective human judgement via cognitive offloading. Altman highlighted how young people internalise AI’s counsel to the point of never making decisions without it—despite recognising it feels “bad and dangerous.” This steady drift is harder to detect and govern than crashes or hacks. Businesses and regulators must not only safeguard AI, but discipline the ways people depend on it.

Key Points

• Altman defined three “nightmare” AI scenarios: misuse by adversaries; systems refusing override; and human agency eroding under AI convenience.
• He singled out the third as the biggest risk: “people rely on ChatGPT too much… young people say things like, ‘I can’t make a decision in my life without telling ChatGPT…’ That feels bad and dangerous.” 
• A Common Sense Media survey found 72 % of U.S. teens have used an AI companion, and roughly half trust its advice at least somewhat. 
• Cognitive researchers warn that this offloading reduces critical thinking, perception of consequences, and depth of reflection. 
• Altman urged regulators and educators to address AI’s societal design, not just its technical rules.

In‑Depth Analysis

Three Nightmare Scenarios — Why No. 3 Stands Out

Altman’s first two scenarios fit familiar paradigms: a rogue state or criminal develops super‑intelligent AI; or systems refuse human commands (“I’m afraid I can’t do that, Dave”). These pose sudden threats that can be governed with technical controls, policy or alignment protocols. In contrast, Scenario 3 is an organic shift: humans, unconsciously and collectively, let machines guide decisions—and often feel powerless doing otherwise. Altman described it not as a malfunction, but as a societal slip into automated life planning. 

Cognitive Offloading in Real Time

A powerful example: Altman spoke about teenagers who say they “can’t make a decision in their life without telling ChatGPT everything that’s going on,” trusting it “because it knows me, it knows my friends.” He labelled that as emotional over‑reliance, even when the advice from ChatGPT surpasses that of human therapists.  The underlying data supports it: 72 % of teens report using AI companions like Character.AI or Replika; 50 % trust their advice.  In interviews a high school student admitted, “I think kids use AI to get out of thinking.” 

Why Drift is Harder to Catch than Malfunctions

This form of dependency rarely triggers alarms. There’s no bug to patch. No misaligned objective to shut down. And yet, Altman insisted stakeholders must feel uneasy when users report—even proudly—they “live their lives the way AI tells us.” Social norms evolve slowly; and once autonomy erodes gradually, restoring it is expensive and slow. 

The Larger Context of Risk and Regulation

Altman raised alarms not only about this cognitive drift but also about AI’s deployment risks—especially in banking. He told Fed leaders that voice‑print authentication is already obsolete, with deepfakes capable of bypassing personal verification. He urged financial institutions to prepare for a fraud crisis and consider AI‑powered verification systems.  Analysts see a growing consensus: governments and education systems must now build governance frameworks that limit cognitive outsourcing, not just capability misuse.

Business Implications

Board rooms, training departments, and HR leaders must confront cognitive risk, not just cyber risk. Teams who rely on AI for company-wide strategy or innovation may lose the muscle memory of analysis and judgment. Critical thinking will degrade if employees reflexively defer decisions to an LLM. Risk management needs to evolve: not only technological audits of models, but agency audits that review how tools affect decision-making culture. Companies should consider instituting structured “pause and reflect” checkpoints, encourage employees to draft decisions before AI input, and promote internal mentorship that challenges AI-generated proposals.

For firms in regulated sectors like banking or healthcare, the fraud risks of one-click deepfake override systems are real. But equally, institutions must resist wholesale automation—even good advice from their own copilots—without retaining human verdicts and oversight. Failure to preserve human judgement may expose firms to reputational, compliance, and ethical liabilities as dependence becomes normalized.

Why It Matters

Human judgement is often dismissed as inferior to data-driven models—until it’s completely absent. When a decision tribe loses its cognitive brainpower, institutions no longer guide strategy; they follow algorithmic forecasts. That transition, Altman’s third nightmare, is happening before we notice. Without intervention, democratized cognition becomes AI-directed cognition.

The actionable path forward: embed cognitive hygiene practices. Teach employees, especially younger ones, to carry out “explain your reasoning” prompts. Design AI interfaces that default to asking users to justify their question or assess multiple solutions. Encourage team leaders to “pause the AI,” draft their own versions first, and then compare.

AI should augment, not dissolve, human insight. This shift won’t happen via a regulation alone—it requires cultural change, continuous monitoring of decision patterns, and respect for the complexity of human agency. The real risk is not that AI will rebel against us—but that we might surrender the capacity to think for ourselves.