24.14. - AI Advisor for Conversational AI

  • Published on Oct 1, 2025
  • 4 minute(s) read
Prev Next

What is AI Advisor for Conversational AI

AI Advisor for Conversational AI is designed to act as a supervisory layer for AI Agents that handle customer phone conversations. Instead of participating in the call itself, AI Advisor reviews and evaluates how the AI Agent performs, ensuring conversations are accurate, compliant, and aligned with business objectives.

By monitoring call transcripts and outcomes, AI Advisor can assess quality, flag unusual or incorrect behaviour, and provide a transparent audit trail. This helps organizations build trust in their AI Agents while maintaining high standards of customer experience.

With AI Advisor in place, organizations can:

  • Ensure consistent and professional handling of customer calls

  • Detect misunderstandings, errors, or gaps in the AI Agent’s responses

  • Gather insights to refine prompts, workflows, and training data

  • Provide visibility and accountability that strengthens trust in AI-driven service

What Analysis can AI Advisor provide?

This product analyses your Conversational AI conversations to help you understand what’s really happening on your calls. It can spot things like subversion attempts, private information being shared, and when a call gets escalated to a human. It also looks at how the call went, was the customer happy or frustrated? Was the issue resolved? Did the agent talk too much or repeat themselves? These are just some of the ways it helps you improve service, stay compliant, and monitor performance without listening to every call. .

For a set of example prompts you can use with AI Advisor, check out our AI Advisor prompt examples guide.

Setting Up AI Advisor AI.

AI Advisor includes a suite of tools to analyse and monitor conversations in your business using Generative AI.

Before you begin you will need to make sure you are logged into Salesforce with a user that has either a Team Leader or Admin Natterbox permissions

Before you begin you must have the Natterbox AVS package version 1.351 or later installed to make use of this functionality (details on how to check your current version are HERE)

  1. Navigate to AI Advisor Settings.

  2. Click Prompt Studio

  3. Click Add new prompt

  4. Set a name under Prompt Name

  5. Enter the Description of your prompt (this is not mandatory)

  6. Select the Prompt Type

    1. Rating - This prompt type provides a rating out of 10 alongside a short explanation of why the AI chose the rating. If you do not want an explanation of the rating, uncheck the ‘Include an explanation for the prompt’ checkbox.

    2. Free Text - This prompt type provides purely a textual response which can be used for AI Assist

      1. Free Text results cannot be searched or reported on as the field a Long Text field and goes beyond the restrictions within Salesforce for search capabilities. However, free text fields can be displayed in Salesforce reports.

  7. Enable your prompt to process new calls by selecting the option Active

  8. Set the focus of your prompt to AI Only

    1. This will provide analysis only on the part of the call where the customer was talking to a Natterbox Conversational AI Agent.

  9. Define your prompt within the Prompt Details box

  10. Click Save prompt.

Reviewing AI Advisor Results

AI Advisor

The analysis of all your Conversational AI prompts will appear in the AI Advisor page in Salesforce. All analysis that looks specifically at Conversational AI conversations will have a blue ‘AI Only’ chip in the header.

Salesforce

Creating Salesforce reports out of AI Advisor analysis is easy as all analysis data is stored in the Natterbox AI object in Salesforce. Using Salesforce reports allows you to set out the data in the way that works best for you and lets you easily create dashboards for others in your business.

Using Salesforce Reports you will be able to setup reports that show you things like:

  • Show me the 10 best calls by customer sentiment

  • Show me the 10 worst calls by customer sentiment

  • Show me all the calls where the AI did not fully complete its task

  • Show me the number of calls per day into my AI

  • Show me all calls that were escalated to a human

Plus many more.

A guide on how to setup reports for AI Advisor is HERE.

FAQ

Do I need an AI Advisor Licence to use AI Advisor for CAI?

No - AI Advisor is included in each AI Assistant and AI Agent licenses at no additional cost.

Why can’t I select the ‘Human’ or ‘Whole call’ focus when setting up a prompt?

AI Advisor for CAI only analyses conversations between your AI and your customers. If you'd like to analyze calls involving your human agents, to gain insights into their performance and help with post-call tasks, you’ll need to purchase the full AI Advisor license.

Will my AI Advisor prompts enabled for AI Assistant and AI Agent calls count towards my 50 prompt max?

Yes, they will count towards the 50 prompt limit.

How long after a call will I get my AI Advisor results?

Results are ready between 15 and 25 minutes after a call.

What AVS Package version do I need to use these features?

v1.351 or higher

Are there some default prompts I can use?

Yes, these are available HERE

Can I target a prompt to a single AI Agent?

No, all AI prompts will fire for all calls that contain AI.

Is there anything I need to do on the Routing Policies to make this work.

Ensure that all ‘Record and Analyse’ apps are set to start recording ‘now’ and the recording legs are set to ‘both’