Testing the bot
The Bot console is a test bench for your own bot. Ask it a question and see the answer plus the tools it reached for and what they returned — without opening the widget on your site and playing the part of a visitor.
Find it at Bot console in the sidebar
(/w/<workspace-id>/<slug>/bot_console). It needs the bot.test
capability, which member-level seats and above have by default.
Turns run through the same path a real visitor’s message does, so what you see here is what they would get. It costs the same as a real reply — these are live model calls against your knowledge base and your custom tools.
Sessions
Section titled “Sessions”A session is a conversation with your bot, and it pins its own model and access level when you start it. That’s the point: the same question asked at two settings is two sessions you can put side by side.
- Model — defaults to the workspace’s bot model, or pick another for this session.
- Access level —
public,user, oradmin. Tools are gated by the visitor’s verified level, so this is how you check that a visitor-level question can’t reach an admin-only tool. See Custom bot tools.
Each turn shows the answer, every tool call with its arguments and result, and the tokens and cost that turn spent.
Rating an answer
Section titled “Rating an answer”Any turn can be recorded with a verdict, so a bad answer becomes something you can act on later instead of a memory:
| Verdict | Use it when |
|---|---|
| Good answer | it did what you wanted |
| Bad answer | unhelpful, off-tone, or evasive |
| Factually wrong | it stated something untrue |
| Should have used a tool | it answered from memory when a tool would have known |
“Should have used a tool” asks which tool it should have reached for — that’s the field that turns a complaint into a fix. You can add a note to any verdict.
Recent evaluations are listed on the console’s index, so patterns show up: the same question rated wrong three times is usually a gap in your knowledge base rather than a bad model.
Replaying against other models
Section titled “Replaying against other models”From a session you can replay every question in it against one or more other models. Each model gets its own new session, run through the same questions in the same order.
The original session is left untouched — it’s the baseline you’re comparing against, so overwriting it would destroy the thing that makes the comparison mean anything. Replays are labelled as replays and remember which session they came from.
This is the honest way to answer “would a cheaper model do?” for your content and your tools, rather than trusting a benchmark someone else ran.
What it doesn’t do
Section titled “What it doesn’t do”- It doesn’t create a visitor or a conversation on your Chats page — console sessions are separate and are never shown to anyone outside your workspace.
- It doesn’t change your bot. Ratings are records, not training; to change an answer, fix the knowledge base, the system prompt, or a tool.