Visualize Performance of Your Bots With The Ibex Dashboard

I recently finished building a bot using the Microsoft Bot Framework for a major client. One of the standard requirement of the clients that develop Bot applications is to have bot-specific monitoring in place. Managing the bot that you have built involves adding monitoring and management capabilities to it. Adding monitoring and management aspects to your bot ensures that your bot implementation will be successful in the long run.

There are three services supported by the Bot Framework to bake monitoring and management capabilities in your bot.

1. ApplicationInsights: Since your bot is ultimately a WebAPI, you can install and enable the AppInsights agent in your application to gather instrumentation data from the application and the host and send it to the AppInsights service hosted on Azure. The AppInsights service provides rich reporting and dashboards for visualizing the telemetry data.
2. Bot Analytics: Using the logs generated by AppInsights, Bot Analytics supplies conversation-level analytics on users, messages and, channel data.
3. Ibex Dashboard: Using custom data logged in AppInsights, Ibex can provide in-depth information about intent usage, channel usage and, the rate of messages to the administrators. We will discuss how to configure Ibex in this article.

Need for Advanced Analytics

In general, after deploying a bot, the stakeholders of the project are interested in knowing the following about the bot:

1. Are the LUIS intents getting adequately utilized?
2. How successfully is the bot causing conversions, such as the sale of a product?
3. What is the user satisfaction score while interacting with the bot?

To answer these crucial questions, Microsoft built an open source tool called Ibex which is available on GitHub at this link. Ibex is built using Node.js and React. It uses custom logs that its helper library pumps into AppInsights and therefore, it has a hard dependency on AppInsights.

Ibex provides relevant information on intent usage, channel usage, activity, the rate of messages and, sentiment analysis.

Configuring The Ibex Dashboard

the following are the two aspects to configuring the dashboard:

1. Configuring the dashboard web application.
2. Adding instrumentation package to your bot application.

Configuring and deploying the dashboard web application is straightforward, and the steps to do so are documented here. Note that you would need to supply an AppInsights read access key to the application so that the dashboard can read data from the AppInsights logs and display it on the dashboard. You can generate a key from the Azure portal by selecting the relevant options from the AppInsights blade.

Configuring Your Bot

You’d need to configure your bot to push telemetry data to AppInsights in a format that can be understood by the Ibex dashboard. To do that follow the following steps.

• Install the BotBuilder.Instrumentation package. You can read more about this package here.
• Add the following keys to your the web.config file of your bot.

    <!-- AppInsights InstrumentationKey-->
    <add key="InstrumentationKey" value="AppInsight Instrumentation Key" />
    <add key="InstrumentationShouldOmitUsernameFromTelemetry" value="False"/>

The above configuration will enable basic logging from your bot and would not perform any sentiment analysis on the text entered by the users. You can also use the following settings in addition to the previous one to perform sentiment analysis on the user input text.

    <!-- LUIS credentials-->
    <add key="LuisModelId" value="LUIS MODEL ID" />
    <add key="LuisSubscriptionKey" value="LUIS SUBSCRIPTION KEY" />

    <!-- Text Analytics data for message sentiment analysis -->
    <add key="TextAnalyticsApiKey" value="TEXT ANALYTICS API KEY" />
    <add key="TextAnalyticsMinLength" value="MIN LENGTH OF TEXT THAT SHOULD BE PROCESSED FOR SENTIMENT ANALYSIS" />
    <add key="CognitiveServiceApiEndpoint" value="https://XXX.api.cognitive.microsoft.com/"/>

Note that you would need to provision a Congnitive Service in Azure to get the keys and enable sentiment analysis on the input text.

Also, take care of the fact that you need to supply AppInsights Instrumentation Key to the application so that it can log data. You can get the Instrumentation Key from the Azure portal as follows.

Next, add one of the following lines of code to start automatic telemetry collection and transfer.

1. If you want to use Cognitive Service and have configured your bot for it.

       public readonly BotFrameworkApplicationInsightsInstrumentation DefaultInstrumentation = DependencyResolver.Current.DefaultInstrumentationWithCognitiveServices;
   

2. If you don’t want to use Cognitive Services with your bot.

    public readonly BotFrameworkApplicationInsightsInstrumentation DefaultInstrumentation = DependencyResolver.Current.DefaultBasicInstrumentation;
   

In my case, I just added the code to Global.asax file at the very start. Being a static member, the member would be initialized at the start of the application.

    public class WebApiApplication : HttpApplication
    {
        public static readonly IBotFrameworkInstrumentation DefaultInstrumentation = DependencyResolver.Current.DefaultInstrumentationWithCognitiveServices;
		...
	}

• Finally, make your LuisDialog class inherit the InstrumentedLuisDialog class and supply the LUIS model id and LUIS subscription key as the constructor arguments.

    public class LuisDialogBase : InstrumentedLuisDialog<object>
    {
        public LuisDialogBase() : base("Model Id", "Subscription Key")
        {
        }
		...
	}

Demo

After configuring your bot, you can test the bot locally or deploy it on Azure and carry out conversations with it. Following is a brief demo of how the dashboard looks like after a short period of activity.