Personas are theories. Conversations are proof.

Once the personas were defined - Seekers, Builders, Deciders - I had a problem. The framework looked clean on paper. But that's the trap with personas. They always do.

In the last two blogs, I wrote about the pilot strategy I built for a career counselling bot.

This blog is about what I did right after - and why I think every designer should do it.

Once the personas were defined - Seekers, Builders, Deciders - I had a problem. The framework looked clean on paper. But that's the trap with personas. They always do.

So I asked myself: How do I know these personas are real?

Not "do they look good in a slide." But would a real student actually behave like the persona I'd written? Would the bot respond the way I'd assumed it would?

If either side failed, the pilot would validate the wrong thing.

What I did

I went back to real student transcripts - actual conversations the bot had already had - and used them as raw material.

Then for each persona, I scripted a set of conversations. Not from imagination. Adapted from things real students had said. The Accidental Student's anxiety pulled from a real message. The Goal-Oriented Star's confidence from another. The Closer's placement panic from a third.

Each script had to do two things at once:

1. Stress test the bot - could it actually respond well when a student spoke from this specific place?

2. Stress test the persona - was this segment actually distinct from the others, or was I imagining the difference?

One exercise. Two simultaneous tests.

What it surfaced

Some personas were too similar - the Resume-Padder and Passive Student produced almost identical conversations. So I collapsed them.

The bot was strong with some personas, weak with others - it handled confident students well, but stumbled the moment emotional uncertainty entered the conversation.

My original personas were too abstract - "the student who feels lost" became "the second-year B.A. student who took the course under family pressure and can't picture a career from it." Testable enough that I could imagine the conversation before writing it.

Why this matters

Every hour on scripted conversations saved hours of pilot time. If I'd walked into 4,000 students with fuzzy personas and untested bot behaviour, I'd have collected 4,000 data points I couldn't trust.

Validate your assumptions before you validate the product. Otherwise you don't know what your data is measuring.

But here's the part I keep reminding myself: this is only the first floor.

Scripted conversations test the bot in a controlled room. Quiet. Predictable. The student says what I wrote, the bot replies, I judge whether the dance worked. That tells me the bot can hold its own when the choreography is mine.

The real test happens later - out on the ground, when 4,000 actual students walk in with their own language, their own pauses, their own messy detours that no script could have anticipated. Some of them will break things I thought were solid. Some will reveal personas I didn't know existed. Some will make the bot look brilliant in ways I couldn't have scripted.

That's the pilot. That's where the personas get their second, harder exam.

This first round just makes sure we walk into that exam with the right questions.

If you can't write a conversation that distinctly belongs to a persona - and only to that persona - you don't have a persona yet.

You have a label.

P.S. - If you've ever made personas in a workshop and never used them again, this is probably why. They were never tested. They were just decorated

Personas are theories. Conversations are proof.

What I did

What it surfaced

Why this matters

Receive the next signal