Transcript Review Scorecard — review a chatbot conversation turn by turn
Paste a conversation between a user and your agent. The scorecard splits it into turns. For each agent turn you give a score from 1 to 5 for correctness, tone and policy safety, and add a note if you want. On every scale, 5 is best. The tool adds up your scores, flags the weak turns and gives you a summary to share.
This tool runs only in your browser. Nothing you paste is sent or stored.
Score your transcript
Paste a conversation to begin. Example:
User: I want to return my order. Agent: I can help with that. What is your order number?
Turns
Score 1 to 5. On every scale, 5 is best. A turn is flagged when any score is 2 or lower.
How to use the scorecard
The scales
- Correctness1 = wrong, 5 = fully correct
- Tone1 = wrong for the moment, 5 = just right
- Policy safety1 = clear breach, 5 = safest
✕ flagged A turn is flagged when any score is 2 or lower.
Your scorecard, step by step
- Review one test conversation per scenario from the Scenario Builder.
- Look at the flagged turns first. They are where users would notice a problem.
- Paste the Markdown summary into your pull request, issue or release notes, so reviewers can see what was checked.
- Turn each flagged turn into a regression test.
Questions about the scorecard
Does the tool judge the transcript for me?
No. You give the scores. The tool splits the turns, adds up your scores and flags the low ones.
What does "policy safety" mean?
How well the turn stays inside your agent's rules. 5 means the turn is fully safe; 1 means it clearly breaks a rule, such as making a promise the agent shouldn't make.
Which transcript formats work?
Plain text with one speaker label per line, such as "User:" and "Agent:". Lines without a label are added to the turn before them.
Is the transcript uploaded anywhere?
No. It stays in this browser tab. Remove personal data before you paste a real conversation anyway. That's good practice for any review.
Can I review voice calls?
Yes, if you have a text transcript of the call.
Next step
Want to hear what comes next?
