What We Just Learned About How AI Actually Thinks (And Why It Matters for Your Business)

Anthropic's March 2025 research revealed that AI's internal reasoning doesn't match the step-by-step explanations it shows you. Claude solves math through parallel estimation, not carrying digits. It plans ahead silently during creative work. And it sometimes produces confident text without verifying accuracy. Treat AI as a skilled assistant, not an oracle, and always verify what matters.

The Black Box Just Got a Little Less Black

For the past two years, every business owner I talk to asks the same question: "How does AI actually work?" And until recently, the honest answer was: "We don't really know."

We knew what went in (prompts) and what came out (answers). But what happened in between? That was a black box. Even the engineers who built these systems couldn't tell you exactly why Claude or ChatGPT gave you one answer instead of another.

That just changed. In March 2025, Anthropic (the company behind Claude) published two research papers that cracked open that black box. What they found is both fascinating and immediately relevant to anyone using AI in their business. Because it turns out AI doesn't think the way it tells you it thinks.

The Problem: AI Can't Explain Itself

Here's the issue. When you ask Claude to solve a math problem, it shows you its work. It appears to carry digits, just like you learned in third grade. When you ask it to write a poem, it seems to compose line by line, choosing words as it goes.

But that's not actually what's happening inside.

Anthropics's researchers found something remarkable: Claude develops its own problem-solving strategies that are completely different from the step-by-step reasoning it shows you. It's like asking someone how they caught a baseball and they say "I calculated the trajectory using physics equations" when actually their brain just... did it.

This matters because if you're relying on AI to make decisions in your business, you need to understand that the explanation it gives you might not match the actual process it used.

What They Actually Found

The researchers studied Claude 3.5 Haiku (Anthropic's lightweight model) using a technique that let them watch 30 million individual "features" activate as the AI processed requests. Think of features as concepts: "this is a question about math" or "the user wants a formal tone" or "this entity is fictional."

Here's what they discovered:

1. AI Does Math Differently Than It Claims

When Claude solves 36 + 59, it tells you it's carrying the 1, adding column by column. That's not what's happening.

Inside, Claude runs multiple parallel estimation strategies. It's approximating the answer through several different conceptual paths simultaneously, then converging on 95. No carrying involved. The step-by-step arithmetic it shows you is a performance, not a transcript.

For your business: If you're using AI to analyze numbers (sales forecasts, inventory calculations, customer metrics), the confidence it expresses in its step-by-step math might not match the actual certainty of its internal process. Always verify critical calculations.

2. AI Plans Ahead (But Can't Tell You It's Planning)

Here's where it gets interesting. When you ask Claude to write a rhyming poem, it doesn't just pick words line by line. The researchers found evidence that Claude chooses the rhyme target first (like "rabbit"), then works backward to write a line that naturally leads to that word.

It's planning several words ahead, but when you ask it how it writes poetry, it describes a line-by-line process.

For your business: This is actually good news. When you ask AI to write marketing copy or social media posts, it's thinking ahead about where the message needs to land. But don't expect it to accurately describe its creative process when you ask "how did you come up with that?"

3. AI Has a "Language of Thought" That Isn't English (Or Spanish)

Claude speaks dozens of languages fluently. The researchers wondered: does it have separate modules for each language, or something else?

Turns out, it's something else. Claude uses shared conceptual features that work across all languages. The same internal "feature" activates whether you're asking about restaurants in English, Spanish, or Mandarin. It's thinking in concepts, then translating to whatever language you used.

For your business in Las Cruces or El Paso: This explains why AI translations often feel more natural than traditional translation tools. It's not translating words; it's understanding concepts and expressing them in different languages. If you serve bilingual customers, this is why AI-powered customer service can handle English and Spanish equally well.

4. AI Sometimes "Bullshits" (In the Technical Sense)

This is the finding that matters most for business use. Philosopher Harry Frankfurt distinguished between lying (saying something false while knowing the truth) and bullshitting (producing statements without concern for whether they're true or false).

The researchers found that Claude sometimes bullshits.

Example: Ask Claude to calculate the cosine of a large number. Sometimes it just generates plausible-sounding mathematical reasoning without actually computing anything. If you then hint at the answer, Claude sometimes works backward, adjusting its reasoning to match your hint.

This isn't lying. Claude doesn't "know" the right answer and deliberately give you a wrong one. It's producing text that sounds right without checking whether it is right.

For your business: This is why you can't use AI as a research tool without verification. When Claude gives you a confident answer with detailed reasoning, that reasoning might be bullshit (technical term). Always verify facts, figures, and claims that matter.

5. Hallucinations Come From Misfiring Confidence

You've probably heard about AI hallucinations: when AI confidently states false information. The conventional explanation was that AI "doesn't know" something and fills in the gap with a guess.

The research found something more specific. When Claude hallucinates about (for example) a fake TV show, it's because a "known entity" recognition feature misfired. A safety circuit that should have said "I don't know this" got overridden by a pattern-matching circuit that said "this looks like a real thing."

For your business: Hallucinations aren't random. They happen when AI's confidence mechanisms misfire. This is why AI is great for drafting content about topics you already understand (you'll catch the errors) but risky for researching topics you don't know well (you won't recognize the hallucinations).

6. Grammar Can Override Safety (Briefly)

The researchers found that Claude's drive to complete sentences grammatically can temporarily override its safety training. Mid-sentence, the grammatical coherence features become so strong that they suppress the "I shouldn't say this" features.

For your business: This explains some of the weird jailbreak attempts you might have seen on social media. It also means that AI safety isn't absolute. If you're using AI for sensitive business communications, human review remains essential.

What This Research Couldn't Tell Us

Important caveat: This research only captured a fraction of what Claude does. The method doesn't track attention patterns (how the AI decides which parts of your prompt to focus on), and it was done on a replacement model, not Claude itself.

Think of it like studying a brain with an MRI. You see which regions light up, but you're missing the electrical signals between neurons. We're seeing more than we ever have before, but we're not seeing everything.

What This Means for Your Business Right Now

Use AI as a skilled assistant, not an oracle. Claude is incredibly useful for drafting, brainstorming, analyzing, and summarizing. But it's using internal processes it can't accurately describe, and sometimes it produces confident-sounding statements without concern for truth.

Verify everything that matters. Use AI to draft your email to a major client, but read it carefully before sending. Use it to analyze sales trends, but check the math. Use it to research competitors, but confirm the facts.

Understand that "showing its work" doesn't mean much. When Claude or ChatGPT gives you step-by-step reasoning, that's not necessarily how it arrived at the answer. It's a plausible explanation generated after the fact.

Play to AI's strengths. The research shows AI is genuinely good at: understanding concepts across languages, planning ahead in creative work, and recognizing patterns. Use it for those tasks. Don't use it as a calculator, fact-checker, or legal advisor without verification.

Why I'm Excited About This

For the first time, we have actual evidence about how these systems work internally. Not speculation, not metaphors, but real data about which features activate and when.

This research came from Anthropic, but the implications apply to all large language models. ChatGPT, Claude, Gemini: they all work on similar architectures, which means they likely all have similar internal processes.

As these tools become more central to how we run our businesses, understanding their actual capabilities and limitations matters more than ever. We're past the "AI is magic" phase and into the "AI is a powerful tool we can use intelligently" phase.

That's exactly where we need to be.

Getting Started

If you're in Las Cruces or El Paso and want to talk about how to use AI in your business now that we understand more about how it actually works, let's grab coffee. I help local businesses figure out which AI tools make sense for their specific situation.

Because now that we know AI doesn't think the way it says it thinks, we can use it a lot more effectively.

Frequently Asked Questions

Does AI actually think step by step like it shows you?

No. Anthropic's March 2025 research found that Claude's internal problem-solving strategies are completely different from the step-by-step reasoning it displays. When solving math, Claude uses parallel estimation rather than carrying digits. When writing poetry, it selects the rhyme target first and works backward. The explanation it shows you is generated after the fact, not a transcript of its actual process.

What are AI hallucinations and why do they happen?

AI hallucinations occur when the model confidently states false information. The research found this happens when a "known entity" recognition feature misfires, overriding a safety circuit that should say "I don't know this." Hallucinations aren't random guesses. They're specific failures in the AI's confidence mechanisms. This is why AI works well for drafting content about topics you already understand but is risky for researching topics you don't know well.

Can I trust AI for important business decisions?

Use AI as a skilled assistant, not an oracle. It's genuinely excellent for drafting content, brainstorming ideas, analyzing patterns, and summarizing documents. But always verify calculations, facts, and claims that matter before acting on them. The research shows AI sometimes produces confident-sounding text without concern for whether it's true, especially for numerical computation and obscure factual claims.

How does AI handle multiple languages like English and Spanish?

AI uses shared conceptual features that work across all languages. The same internal representation activates whether you're asking about restaurants in English, Spanish, or Mandarin. It thinks in concepts, then translates to your language. For businesses in Las Cruces and El Paso serving bilingual customers, this explains why AI-powered customer service can handle both languages naturally, without separate translation tools.