Decoding Claude

Sumit
Owner
Insha
Editor
Aryan
Editor

Jun 8, 2026

7 views 6 min read

Stress-testing a chatbot with a chatbot

One of the most underrated uses of a small, fast model is pointing it at another bot and letting it act like a real user.

Take this career-counselling assistant a multi-step flow that guides a student through rapport-building, framework selection, career matching, and more. Testing it by hand is slow, inconsistent, and only ever covers the paths you happen to think of. So we hand the job to Claude Haiku 4.5 instead. We give it a persona: Aarav, a panicking third-year engineering student two months from placements; and let it drive the conversation through the browser: typing his worries, reacting in character, pushing back when advice gets generic. The screenshots show Haiku doing exactly this, working through the flow nodes and replying like a real stressed student: "yeah i get that, but like… i don't really know how to rewrite it."

Why Haiku 4.5 fits

The per-turn job isn't hard, it's repetitive: look at the screen, recall who you are, respond in character. That rewards consistency and instruction-following over deep reasoning Haiku's strength. Speed matters because a conversation runs 20-plus turns of screenshot-read-click-type, and a fast model keeps that loop tight. Cost matters because good testing means more runs persona variants, repeat trials, a fresh run after every change and cheap runs are runs you'll actually do. It also handles the browser automation reliably, which is the other half of the job.

One rule: generate with Haiku, grade with a bigger model. Drive the conversation with Haiku; judge whether the bot actually helped by running an evaluation pass over the transcript with a stronger reasoning model.

Where else this works

The pattern, many bounded calls where throughput and cost matter more than raw intelligence, shows up everywhere:

Synthetic users and red-teaming for support, onboarding, or sales bots, at scale.
Conversational regression testing in CI replay user journeys on every deploy.
Browser and UI automation click-through QA, form filling, navigation checks.
High-throughput classification ticket triage, intent detection, moderation.
Structured data extraction pulling fields from documents and emails into JSON.

The throughline: reach for Haiku when the bottleneck is throughput, latency, or cost and save the heavyweight model for the one step that needs real judgment.

This is the prompt I used

SONNET

The Sonnet Sweet Spot: High-Leverage Use Cases

Claude Sonnet occupies the critical intersection of high-tier reasoning and low-latency execution. It is the definitive model for tasks that require institutional-grade logic without the computational overhead of Opus or the rigid simplicity of Haiku.

When deploying Sonnet, these are the highest-impact operational domains:

1. Product Design & UX Architecture

Sonnet excels at translating abstract user needs into structured interface logic. It is highly effective for mapping out complex, multi-step user journeys and anticipating error states before development begins.

Generating contextual UX copy and tooltips for specialized software (such as AI-driven legal assistants or technical platforms).
Drafting comprehensive design system documentation and component specs.
Conducting accessibility audits and providing reasoned design critiques based on established heuristic frameworks.

2. Software Engineering & Data Architecture

Because of its deep contextual memory, Sonnet is a premier asset for navigating and altering existing codebases.

Executing multi-file code refactoring, generating boilerplate, and debugging across diverse environments.
Structuring pipeline logic for predictive data dashboards and MLOps integrations.
Writing robust unit tests, docstrings, and comprehensive technical documentation.

3. Agentic Workflows & Conversational Systems

Sonnet is fast enough to run in a continuous loop but smart enough to handle ambiguity, making it the ideal engine for autonomous agents.

Powering multi-step conversational bots that require deep intent recognition and nuanced, in-character responses.
Executing automated workflows that require real-time tool use, such as API calls, web searches, or database queries.
Functioning as an internal knowledge assistant capable of synthesizing proprietary company data on the fly.

4. Structured Analysis & Content Scaling

For heavy text processing, Sonnet balances rapid summarization with acute attention to detail, preventing the loss of critical nuance.

Extracting strictly formatted, structured data (like JSON or CSV) from unstructured text, contracts, or emails.

Synthesizing research across multiple dense documents to generate clear executive decision frameworks.

Drafting, editing, and localizing long-form technical reports, articles, and proposals at scale.

The Core Deployment Heuristic If a task requires multi-step reasoning, produces an asset destined for production, or operates within an agentic loop, Sonnet is the optimal choice.

Scale up to Opus only when a problem is highly ambiguous, completely open-ended, or requires profound, exploratory judgment.
Scale down to Haiku when the primary bottlenecks are cost, volume, and raw throughput (e.g., UI automation or mass classification) and deep reasoning is not required.

OPUS :

When the wrong answer looks right

One of the most underrated uses of a frontier reasoning model is pointing it at another system not to generate volume, but to catch the small, expensive mistakes that everything else waves through.

Take a primary-grade science tutor, a CBSE/NCF-SE-aligned assistant that walks a young student through a doubt: reading the question, picking an explanation, simplifying it, and checking understanding. The dangerous failures here aren't crashes or dead ends. They're answers that sound perfectly reasonable and are quietly, scientifically wrong or technically correct but pitched three grade levels too high for a seven-year-old. You can't catch those by clicking through happy paths; you need someone who actually knows the science and the child. So we hand the job to Claude Opus 4.6 instead. We give it a persona: Mrs. Iyer, a veteran primary-science teacher reviewing the tutor before it reaches her classroom; and let it drive the conversation through the interface: asking the deceptively simple questions kids actually ask, probing the bot's reasoning, and flagging the moment an explanation drifts from the curriculum. The logs show Opus doing exactly this working through the tutor's flow and reacting like a teacher who has seen every misconception: "It told the child plants 'eat sunlight.' That's a metaphor a Class 2 student will take literally now they think light is food."

Why Opus fits

The per-turn job here isn't volume and it isn't just context it's judgment. Knowing that "the Sun is a star, not a planet" is easy; knowing whether a given explanation is scientifically sound, age-appropriate for a Class 2 reader, and aligned to the NCF learning outcome all at once is the hard part, and it's exactly where a weaker model nods along. That rewards deep domain reasoning and the ability to hold several constraints in mind simultaneously: Opus's sweet spot. It can play an expert convincingly because it reasons like one, which means the questions it asks are the questions that actually expose weak spots. And because the same expertise that drives the conversation can evaluate it, Opus does the one thing the cheaper models can't: tell you whether the bot was actually right.

One rule: don't just drive the conversation and render the verdict. Run Opus as the expert user on Max Effort with thinking on; then have it grade the full transcript against a rubric of scientific accuracy, age-appropriateness, curriculum alignment marking each answer pass or fail with a reason. Generate cheap with Haiku, find the edge cases with Sonnet, and keep Opus for the judgment call that decides whether any of it shipped correctly.

Where else this works

The pattern putting a frontier model where correctness is subtle and the cost of a quiet mistake is high shows up wherever "plausible but wrong" is the real failure mode:

Expert-grade evaluation: grading other models' transcripts for factual and pedagogical correctness, not just tone or format.
Content QA at the source: checking textbook drafts and rewrites for subtle scientific errors, misleading analogies, and grade-level mismatches before they go to print.
High-stakes domain review: medical, legal, or curriculum content where a confident wrong answer is worse than no answer at all.
Adversarial probing by an expert: the questions only someone who knows the field would think to ask the ones that break a system precisely because they look easy.

The throughline: reach for Opus when the bottleneck isn't speed or even context, but judgment when someone has to know whether the answer is actually right, and be right about it.

Decoding Claude

Stress-testing a chatbot with a chatbot

Why Haiku 4.5 fits

Where else this works

1. Product Design & UX Architecture

2. Software Engineering & Data Architecture

3. Agentic Workflows & Conversational Systems

4. Structured Analysis & Content Scaling

Why Opus fits

Where else this works

Receive the next signal