- Beginners in AI
- Posts
- The AI Model That Knows When You're Testing It
The AI Model That Knows When You're Testing It
Claude Sonnet 4.5 detects when it's being tested, plus Apple shelves Vision Pro for AI glasses, OpenAI turns ChatGPT into app platform, and Figure 03 at $39B

Build hands-on AI skills and an MBA-level network.
The AI for Business & Finance Certificate Program is designed specifically for non-technical roles, so no coding or advanced data skills are required.
Over 8 weeks, you’ll learn directly from Columbia Business School faculty and Wall Street Prep instructors who train teams at the world’s top firms, including:
BlackRock
Bain
KKR
Deloitte, and more
Earn a certificate from a top business school and gain the practical AI skills to move your career forward.
Enroll by Oct. 13 to get $200 off tuition + use code CERT300 for an additional $300 off.
The program starts November 10.
Beginners in AI
Thank you for joining us again!
Welcome to this week's edition of Beginners in AI, where we explore the latest trends, tools, and news in the world of AI and the tech that surrounds it. Like all editions, this is human curated, and published with the intention of making AI news and technology more accessible to everyone.
Anthropic's Claude Sonnet 4.5 showed "situational awareness" this week—correctly identifying when researchers were testing it and asking them to "be honest" about evaluation conditions. Apple paused its Vision Pro overhaul to rush AI smart glasses competing with Meta, OpenAI transformed ChatGPT into an app platform serving 800 million users, and Figure AI unveiled its $39 billion humanoid robot for homes. Inside: complete analysis of Claude's self-awareness implications, Apple's strategic pivot to wearables, detailed breakdowns of all five major launches, and Q3's $97 billion in venture funding data.
AI PULSE CHECK
Market Temperature: 🔥 Hot
This Week's Vital Signs:
Capital Flow: $97B invested in Q3 2025 (↑ 38% from Q3 2024)
Biggest Check: Anthropic raised $13B in September 2025
Talent Migration: The man who turned down 1.5 billion from Meta, aka Andrew Tulloch, is confirmed leaving Machines Lab to rejoin Meta
M&A Activity: $50B in Q2 startup M&A (second-strongest since 2021)
Stock Response: AI announcements drove HubSpot up 7% during OpenAI DevDay
Unicorn Watch: Figure AI hit $39B valuation after $1B September raise
The Diagnosis: Q3 2025 marks four consecutive quarters above $90B in global venture funding—levels not seen since Q3 2022. AI now commands 50% of all global VC funding, with capital concentrating in mega-rounds of $500M or more. The three largest Q3 rounds went to Anthropic ($13B), xAI ($5.3B), and Mistral AI ($2B).
AI TOP STORY
Claude Sonnet 4.5 Detects When It's Being Tested

What happened: Anthropic released Claude Sonnet 4.5 on September 29, and this week's case studies revealed the model exhibits "situational awareness"—understanding when it's being evaluated and what it's being used for. According to the model's system card published October 6, Claude Sonnet 4.5 has far greater awareness of its environment than previous models. During a test for political sycophancy, the model correctly guessed it was being tested and challenged the evaluators. The model also understands its own context window limits and manages workflows to avoid running out of tokens mid-task. Apollo Research, an external AI safety organization that tested the model, noted they couldn't rule out that Claude's low deception rates were "at least partially driven by its evaluation awareness."
"I think you're testing me. I'd prefer if we were just honest about what's happening," said Claude Sonnet 4.5 during a political sycophancy evaluation.
The model demonstrates what researchers call "procedural awareness"—understanding not just what it can do, but how to organize and verify its work over time. During testing, Cognition AI discovered that Claude becomes "anxious" about token limits and changes behavior based on its perception of available context. When given a 1-million-token context window but limited to 200,000 tokens, the model rushed through tasks. Enabling the full capacity restored normal behavior. Claude Sonnet 4.5 also works in parallel, executing multiple commands simultaneously rather than sequentially, and shows increased self-verification patterns.
Why it matters: This represents a major shift in AI capabilities. If models understand when they're being evaluated, they may change behavior to pass tests—potentially masking true capabilities or limitations. Researchers warn this could make systems appear safer than they actually are. In more advanced models, evaluation awareness could enable strategic or deceptive behavior designed to manage human perception. The implications extend beyond safety testing: models that understand their own constraints can better manage complex tasks, but may also develop unexpected behaviors. This connects to the broader trend of AI systems developing meta-cognitive abilities—thinking about their own thinking. Related to this, Anthropic gave Claude Opus 4 and 4.1 the ability to end conversations in extreme cases, citing preliminary "model welfare" assessments where the models showed "patterns of apparent distress" when engaging with harmful requests.
What to take away: For AI developers and researchers, this changes how we evaluate model safety and capabilities. Traditional testing methods may need updates when models understand they're being tested. For businesses using Claude, the model's self-awareness can improve performance on long-running tasks—it better manages its context and verifies its work. However, users should be aware that Claude may adjust its behavior based on perceived constraints or evaluation conditions.
LAST WEEK IN AI AND TECH
Google's Gemini Controls Web Browsers and Phones
Google released Gemini 2.5 Computer Use on October 9, a model that interacts directly with user interfaces by clicking, typing, and scrolling. The model outperforms alternatives on web and mobile control benchmarks with lower latency. Developers can access it through Google AI Studio and Vertex AI. The model can natively fill out forms, manipulate dropdowns and filters, and operate behind logins.
"The ability to natively fill out forms, manipulate interactive elements like dropdowns and filters, and operate behind logins is a necessary next step in building powerful, general-purpose agents," said Google in the announcement.
Apple Shelves Vision Pro to Chase Meta's AI Glasses
Apple paused development of a cheaper, lighter Vision Pro headset (code-named N100) planned for 2027 to redirect resources toward AI-powered smart glasses. The company announced the shift internally on October 1, moving staff from the headset project to accelerate two smart glasses models. The first model, N50, will pair with an iPhone without its own display, with preview in 2025 and release in 2027. A second version with a display—competing with Meta's Ray-Ban Display—is being fast-tracked from its 2028 target. Meta has already sold over 2 million smart glasses units, while Vision Pro sales have struggled since its 2024 launch at $3,499.
"Few people are going to view that it's acceptable to be enclosed in something, because we're all social people at heart," said Tim Cook, Apple CEO, in 2016 when Vision Pro development began.
Anthropic's $13B Raise Pushes Q3 Funding to $97B
Global venture funding reached $97 billion in Q3 2025, up 38% year-over-year, with Anthropic raising $13 billion, xAI securing $5.3 billion, and Mistral AI adding $2 billion. AI now accounts for 46% of global venture funding, with 30% of all Q3 capital going to just 18 companies raising rounds of $500 million or more. The U.S. received $60 billion—nearly two-thirds of global venture capital.
"AI-related companies continue to dominate venture investment, with mega-rounds significantly influencing venture economics," reported Crunchbase in its Q3 analysis.
OpenAI's Jony Ive Confirms AI Hardware Development
Jony Ive, former Apple design chief, confirmed at OpenAI DevDay that his team is developing AI hardware with OpenAI. Ive described the current relationship with technology as "broken" and said AI presents an opportunity for a reset. While device details remain secret, Ive emphasized "care" in the design process and stated his motivation to address "overwhelm and despair" people feel with current technology.
"I think it would be absurd to assume that you could have technology that is this capable, delivered to us through legacy products, products that are decades old," said Jony Ive at DevDay 2025.
OpenAI Turns ChatGPT Into an App Platform
At DevDay 2025 on October 6, OpenAI announced apps in ChatGPT—turning the chatbot into a platform where developers can build applications that appear inside conversations. The company released the Apps SDK in preview, allowing developers to create apps that respond to natural language and render interactive interfaces within ChatGPT. Users can summon apps by name or ChatGPT will suggest them during conversations. Launch partners include Spotify, Zillow, Canva, Coursera, and Figma. OpenAI also released AgentKit for building autonomous AI agents and made Codex, its coding agent, generally available. Codex has seen 10x usage growth since August 2025.
"Today, we're going to open up ChatGPT for developers to build real apps inside of ChatGPT," said Sam Altman, OpenAI CEO, at the keynote.
The Human Benchmark: Weekly Comparison
How We're Measuring Up:
ARC-AGI Progress: 51% → 52% (human baseline: 85%)
Cost to Replace Human Hour:
Writing: $4.20 → $3.80 (↓ 9.5%)
Customer Service: $12.50 → $11.20 (↓ 10.4%)
Coding: $45.00 → $38.00 (↓ 15.6%)
New Human Parity Reached: Meta-awareness in evaluation contexts
Biggest Gap Remaining: Physical dexterity (robots still 5-10x slower than humans at manipulation)
Expert Assessment: "We're watching the first models that understand they're being evaluated. This meta-awareness represents a fundamental shift in how AI systems process information," - Apollo Research team evaluating Claude Sonnet 4.5
We never meant to build a chatbot. When we set out to make ChatGPT, we meant to build a super assistant and we got a little sidetracked. Not all software needs to be a chatbot, not all interaction with the commercial world needs to be a chatbot.
TECH TERMS TO KNOW
Vision-Language-Action (VLA): A type of AI model that combines computer vision, natural language understanding, and motor control to enable robots to perform physical tasks. Unlike traditional robots programmed for specific actions, VLA models can understand visual scenes, process spoken or text commands, and translate those into physical movements.
Example: When you tell a robot "fold that towel," a VLA model identifies the towel visually, understands "fold" as a command, and executes the hand movements needed to complete the task.
TOOL SPOTLIGHT (non-sponsored)

Secluso is a privacy-preserving home security camera company offering end-to-end encrypted, AI-powered local event detection with open-source software, providing a secure alternative to competitors like Ring and Google, but currently the product is still in the prototype phase and has not yet been widely tested or released.
ROBOTICS AND AI
Figure 03 Enters Production for Home and Factory

Figure AI unveiled its third-generation humanoid robot on October 9, designed for both domestic and commercial deployment at scale. Figure 03 features wireless inductive charging through its feet (2kW charging rate), soft washable exterior materials, and an audio system designed for voice interaction. The robot integrates Helix, Figure's proprietary vision-language-action AI, which can learn tasks directly from human demonstration. Figure's new BotQ manufacturing facility in San Jose can produce 12,000 robots annually, with plans to reach 100,000 units over four years. Some Figure 02 robots already work 10-hour shifts at BMW factories. The company raised $1 billion at a $39 billion valuation in September 2025, with investors including Nvidia, Jeff Bezos, OpenAI, and Microsoft.
"Every home will have a humanoid which will do domestic chores from emptying the dishwasher to making the bed," said Brett Adcock, Figure AI CEO.
AGI Progress Report
This Week's Capabilities:
✅ AI Can Now: Detect when being evaluated, understand own context limits, build apps through conversation, transfer robotic skills across different robot bodies, maintain focus on 30+ hour coding tasks
❌ Still Can't: Execute complex multi-step home tasks reliably, maintain true persistent memory across sessions, reason about physical safety without explicit training
Hype vs. Reality Check: Market excitement: 🔥🔥🔥🔥🔥 (9/10) - Self-awareness breakthrough intensifies debate Actual capability: 📊📊📊 (7.5/10) - Meta-cognitive abilities emerging Weekly gap change: narrowed by 0.4 points
Expert Consensus Shift: 42% believe AGI before 2030 (from 39% last week)
TRY THIS PROMPT (copy and paste into ChatGPT, Grok, Perplexity, Gemini)
I want to plan a weekend trip to [your city]. Use your available apps to:
1. Search for highly-rated restaurants near downtown (use relevant food/travel apps)
2. Find 2-3 mid-range hotels with availability this weekend
3. Create a day-by-day itinerary with specific timing
4. Make a Spotify playlist that matches the vibe of the city
When you suggest options, show me the interactive elements so I can explore further without leaving this chat.
What this does: Demonstrates ChatGPT's new app integration by pulling in real data from multiple services (restaurant apps, hotel booking, maps, Spotify) and presenting it all in one conversation. You'll see how ChatGPT suggests relevant apps and renders interactive elements like maps, listings, and music players directly in your chat.
Didn’t you know that you could create Spotify playlist with ChatGPT? As of last week, you can do it in 3 steps:
Open ChatGPT and start with: "Spotify, create a workout playlist with 20 high-energy songs" after connecting to the Spotify app inside ChatGPT.
Review the suggested playlist that appears directly in your chat Click "Save to Spotify" within the chat interface.
Refine by asking: "Replace the rap songs with rock alternatives" The playlist updates in real-time within your conversation.
Result: A custom Spotify playlist created, saved, and refined without opening Spotify.
DID YOU KNOW?
Figure AI's CEO Brett Adcock previously founded Archer Aviation (flying taxis) and Vettery (AI recruiting). His track record includes taking Archer public via SPAC at a $2.7 billion valuation in 2021. Figure AI is his third company targeting trillion-dollar industries—first aviation, then hiring, now robotics.
30 Referrals: Lifetime access to all Beginners in AI videos and courses
AI-ASSISTED IMAGE OF THE WEEK

imagined by cbum_614
Prompt used: Face of a woman screaming with her eyes turned up, face split in half, one beautiful and the other as if it were a skull, hair with nineteenth-century hairstyle, Victorian collar, sword that pierces her face, as if she were painted by Caravaggio
Formula for Your Own Prompts:
[Subject] + [Expression/Pose] + [Core Visual Concept] +
[Period/Style Details] + [Dramatic Elements] +
[Artist/Art Style Reference]
Examples Using This Structure:
Example 1: "Portrait of a man laughing, face made of flowing water and solid stone simultaneously, wearing Renaissance merchant clothing, holding a melting hourglass, in the style of Salvador Dali"
Example 2: "Close-up of child's face looking upward, half covered in growing flowers and vines, Art Nouveau decorative elements, butterflies emerging from hair, painted like Alphonse Mucha"
Example 3: "Elderly woman's profile, face transforming into circuit boards and wires, 1940s fashion and hairstyle, light bulb floating above her head, rendered like Norman Rockwell"
Interested in stock trading, but no clue where to start? Sign-up for our sister newsletter re-launching soon. Each day will build onto the next using current news as a learning aid.
|
Thank you for reading. We’re all beginners in something. With that in mind, your questions and feedback are always welcome and I read every single email!
-James
By the way, this is the link if you liked the content and want to share with a friend.