Category Archives: Microsoft Bot Framework

Remotely controlled bots

You know. Because it’s good to have a fail-safe around in case of Skynet.

But to the point: Let’s say you have a great backend heavy application and you want to deliver a bot experience to broaden your user base and to provide a new way of interacting in your app context. Well your new bot can surely be made to access the application data in your backend, but how can you make it work the other way around – say, in case of notifications?

Controlling a bot from backendThat’s the bot on the right, by the way.

Hence the title – by remote control I simply refer to a backend controlling the bot remotely (over interwebs) by messages that can be interpreted as commands to execute an action.

Backchannel

That’s what we call it and apparently it’s just one word. What we mean by the word is a type of message (like the ones users send to talk to a bot and the bot uses to reply back), but we just put the meaningful content in a different place of the message object (Activity in C#). Namely, we put the message the bot should react to somehow in IMessageActivity.ChannelData instead of IMessageActivity.Text. Tadaa! End of article.

No, but it really is that simple! In a nutshell you devise a simple custom protocol that your bot knows, for example, when the IMessageActivity.Text contains “notification”, you look at the channel data content to see who and with what message to notify. Then let your implementation in the bot code to do it’s job. Still don’t believe me? Look, here’s a sample (in C#).

Ok, you got me. What I failed to mention is that you have to have some Microsoft Bot Framework specific code in your backend. Perhaps the easiest way to implement this backchannel messaging pipeline between the backend and the bot is using Direct Line. And the easiest way to use the Direct Line is by utilizing the ready-made client components for Node.js and C#. If your backend is not compatible with Node or C# components, implementing your own Direct Line connection is quite straightforward (the first link about Direct Line describes the protocol). They are, after all, only HTTP calls. My sample comes with a super simple console app sending notification commands to the bot. You should be able to use the code almost as-is, if your backend is built with C#.

What about security? I’m not an expert, but there are three points here I want to make:

  1. The Direct Line pipeline is secured by a secret key and TLS
  2. The user cannot inject content to the channel data (think of SQL injection vulnerability) as long as the channel (e.g. Skype) is secure
  3. You can encrypt the channel data content

Note that some descriptions of backchannel say that you should also change the value of the Type property of your Activity; from being “message” to “event”. This is a matter of taste. The benefit of this is that you can be sure that your backchannel message is not treated as a regular message (because the type is not “message”).

Where to, sir?

Where am I supposed to place this backchannel messaging specific code in my bot project? To me, this introduces some controversy; The bot framework utilizes Autofac, an inversion of control (IoC) container for dealing with dependencies, and I am not a fan. In my opinion wide use of IoC leads to incoherent code and architecture with little benefits to offer. And it can make writing tests (which I don’t do unlike true professionals I guess) a pain! But that’s just me – maybe my brain is not sophisticated enough to understand these kinds of exquicite concepts.

Just to show I can do things I don’t like I integrated the backchannel bot code using Autofac in my sample. Take a look at GlobalMessageHandlerModule.cs and Global.asax.cs. I’ve created classes derived from ScorableBase, which are automatically invoked when (and only when) I forward the received Activity object to my root dialog in MessagesController.cs. Then if a backchannel message is detected, the specific scorable class (NotificationsScorable in my sample) consumes and deals with the Activity and it is never given to my dialog. Special thanks to my brilliant colleague, Lilian Kasem, for coming up with this idea!

Call me old-fashioned, but I still find the code a lot easier to understand if I simply put this logic to my MessagesController class (or equivalent) before passing anything to any dialog. That’s just the way I roll…

if (we got a valid backchannel message)
{
    // Do what needs to be done
}
else
{
    // Looks like a message from a user, let the dialog handle it
    await Conversation.SendAsync(activity, () => new RootDialog());
}
else ...

See?

Related resources

 

Chatbots as middlemen

Chatbots typically serve their customers on 1:1 basis. They are not unlike digital assistants (Cortana, Siri, Alexa etc.) except that a chatbot is usually designed to execute a small number of prefined tasks well and focus on a narrow subject like filling a pizza order for example.

Building chatbots is easy, but making them clever is more difficult. Despite all the analytics on user behavior, it is still impossible to anticipate every user reaction. As the technology, Conversation as a Platform (CaaP), evolves, creating more intelligent bots becomes easier and easier, but until Skynet grows self-aware humans still serve a purpose.

Imagine a customer service chat on a website. A bot can probably handle most of the problems a customer could have. For instance, implementing a simple FAQ bot is trivial using Microsoft’s QnA Maker. Add some additional intellect including natural language understanding service and whatnot and you have an efficient customer service bot in your hands that 9 out of 10 customers are perfectly happy with. But for that one customer, you might want to consider a fallback: Let the human – in this case a customer service agent – take over to ensure customer satisfaction.

As long as you have the human labour, implementing this isn’t rocket science. What you need to do is as follows:

  1. Make sure your bot keeps track of all the individuals the bot sees (but remember privacy policies!)
  2. Make sure your bot also keeps track of itself. This might sound weird at first, but I’ll make the reason apparent soon.
  3. Design the handover scenario. It could be based on sentiment analysis or simply a request of help by the user.
  4. Implement the message relaying logic (don’t worry – there are samples available!)

How and why to keep track of people and bots

By keeping track I mean collecting the contact information of a user (and the bot – I’ll explain later) from the bot’s perspective. You can’t send a post card to a person without knowing his or her address. The same applies to the bot framework: You cannot send a message to a user without knowing the IDs of the user and the conversation. What you’ll need at least are:

The aforementioned details may be enough, but this depends on the channel (Skype, MS Teams, Slack etc.) You might as well store all the details as shown in the following tables.

Table 1. Identities in Skype (all values are of type string).
Me Bot
ServiceUrl https://skype.botframework.com https://skype.botframework.com
ChannelId skype skype
ChannelAccount.Id 29:1byUvXHHhinNxwnPCHh4MPhpfiJUbadX_Y3_sTkBspdiSke8sX_Ps6riTYRVez5jT 28:f99fa2c3-8834-418e-b293-039205238055
ChannelAccount.Name Tomi Paananen Intermediator Bot Sample
ConversationAccount.Id 29:1byUvXHHhinNxwnPCHh4MPhpfiJUbadX_Y3_sTkBspdiSke8sX_Ps6riTYRVez5jT 29:1byUvXHHhinNxwnPCHh4MPhpfiJUbadX_Y3_sTkBspdiSke8sX_Ps6riTYRVez5jT
ConversationAccount.Name (N/A in direct conversation) (N/A in direct conversation)

The values above are from a direct conversation in Skype between my bot and I. As you can see the channel account ID (read: my user ID) and the conversation account ID match, but that isn’t necessarily the case in other channels.

Table 2. Identities in Slack.
Me Bot
ServiceUrl https://slack.botframework.com https://slack.botframework.com
ChannelId slack slack
ChannelAccount.Id U1F3JK9A9:T1F248PJ8 B2NSU1D4Z:T1F248PJ8
ChannelAccount.Name tomi intermediatorbot
ConversationAccount.Id B2NSU1D4Z:T1F248PJ7:C3B1ZK5D0 B2NSU1D4Z:T1F248PJ7:C3B1ZK5D0
ConversationAccount.Name bottest bottest

So why do we need the bot’s identity stored too? As you can see, the same bot has a different identity on different channel and conversation. When we send a message to a user, we need to specify who the message is from, and some channel, for example Slack, doesn’t allow you to send messages from bots that aren’t actually there. So in order to relay a message from a user to another on another channel (e.g. Skype to Slack) we need to know and use the bot’s identity in Slack in the from field.

Briefly about the technical implementation: All the activities flow through the MessagesController class in a bot built with C# and that’s the ideal place to keep track of everything. As for bots, the bot is always the receiving party when it gets a new activity, and that’s how you store the bot identities. See Sending and Receiving Activities for more information.

Finally, store the records of the users and the bot somewhere in web e.g. Azure Table storage service. Note: My sample stores the data locally (in memory), which is never, ever a good idea, because bots are essentially web apps and can have multiple instances!

Comparison to Node.js

The essentials for relaying messages are the same whether you are building your bot using C# or Node.js SDKs. However, there are differences between the SDKs and some things are handled differently.

Table 3. Node.js counterparts for establishing user/bot identity.
C# Node.js Node.js example
Activity.ServiceUrl  IChatConnectorAddress.serviceUrl session.message.address.serviceUrl
Activity.ChannelId IAddress.channelId session.message.address.channelId
ChannelAccount.Id IIdentity.id session.message.address.bot.id
session.message.address.user.id
ChannelAccount.Name IIdentity.name session.message.address.bot.name
session.message.address.user.name
ConversationAccount.Id IIdentity.id session.message.address.conversation.id
ConversationAccount.Name IIdentity.name session.message.address.conversation.name

Samples

How to create dynamic FormFlow

You might’ve guessed it from the title – Yes, this is one of my no-nonsense posts. Strictly business and by business I mean code-talk.

This post is about a building block in Microsoft Bot Framework called FormFlow and namely how to add dynamic behavior to the flow when building bots using C#. If you are not familiar with FormFlow, I suggest your study the basics before reading further. Just the basics though, that’s enough.

Building simple FormFlows is… well, simple! This is a method that creates a basic form:

public static IForm<MyClass> BuildForm()
{
    var builder = new FormBuilder<MyClass>();

    return builder
        .Field(nameof(MyClass.Property1))
        .Field(nameof(MyClass.Property2))
        ...
        .Build();
    }
}

Not difficult at all! And you can influence the behavior using, for instance, property attributes like:

[Serializable]
public class MyClass
{
    [Prompt("What would you like the value of this property to be?")]
    public string Property1 { get; set; }

    ....
}

But what if you want to do some of the following:

  • Skip the questions you already know the answer to based on what the user said earlier?
  • Define the options presented for the user dynamically?
  • Change the way the question and options are presented to the user?
  • Validate the user’s response?
  • Customize the behavior of the form in the fly?

Some of the aforementioned things FormFlow tries to do for you automatically. However, usually to achieve a great experience you have to do a bit more work, and luckily, it is possible. See the resources available under Microsoft.Bot.Builder.FormFlow.Advanced namespace. One very useful class under there is called FieldReflector. Whilst you can still add ActiveDelegate and ValidateAsyncDelegate using the overloaded Field method in FormBuilder, FieldReflector allows you to do more:

.Field(new FieldReflector<MyClass>(nameof(MyClass.PropertyX))
    .SetType(typeof(MyClass.PropertyX))
    .SetActive((state) => SetFieldActive(state, nameof(MyClass.PropertyX)))
    .SetDefine(async (state, field) => await SetOptionsForFieldsAsync(state, nameof(MyClass.PropertyX), field))
    .SetAllowsMultiple(true) // Single selection vs. multi-selection
    .SetPrompt(new PromptAttribute("What type of values should this property have? {||}"))
    .SetValidate(async (state, value) => await ValidateResponseAsync(value, state, nameof(MyClass.PropertyX))))

…where SetFieldActive, SetOptionsForFieldAsync and ValidateResponseAsync are methods defined and implemented by the developer. See this class implementing the building of the form from my Dynamic FormFlow Sample. The snippet provides solution to all the questions presented in the bullet point list above. In case you are curious how this enables customizing the behavior in the fly, notice that you can run any arbitrary code in your response validation method (ValidateResponseAsync in the snippet).

In my Dynamic FormFlow Sample I use FormFlow to narrow down a spaceship selection from a static catalog of spaceships. Therefore, it is important that I don’t bother the user with unnecessary options; the user might’ve already told me that he/she is looking for a small ship and thus I shouldn’t later ask to select from options only available for large ships. That’s simply bad UX! Some queries I can skip in case there is only one option available. In my response validation method I do a search against the catalog using the details I’ve gathered from the user so far and  if the response is valid, I store the search results for later. With this approach I can narrow down the selection with as few questions as possible and without presenting the user with options that make no sense.

The caveats

Playing with FormFlow is not all bed of roses. I had to fight few errors, that for first seemed odd, until I got the hang of it. Here are some of the things to keep in mind:

  • After you build your form, it is built. It now exists. And you cannot really control the instance anymore. Why is this important to understand? Because you cannot say for certain when the methods (delegates) in your form are called. It’s now in the hands of the Bot Framework. So make sure your methods (delegates) work in any situation! For starters, have null checks.
  • Do not use the IList interface as a property type in the class where you are collecting user input (Spaceship.cs in the sample). It won’t work and you’ll get a FormCanceledException with “Cannot create an instance of an interface” message. Use List instead, it works.
  • Realize that you don’t have to do everything in the form; after the form is complete, you can continue with the data in a Dialog and ask further questions etc. That’s what I did in my sample, see SpaceshipSelectionDialog.cs.

And a top tip: If you are new to FormFlow, implement and test one complex field at a time.

So it’s a magic bullet?

No, it is not. FormFlow is a handy building block, but will not solve all problems. Duh.

If you feel like your FormFlow code is turning into horrible, uncontrollable mess and you feel like you need to compromise the UX, stop. Stop using FormFlow. You can do the same using Dialogs too and with really complex scenarios it will be – most likely – much easier too. FormFlow is a solution for fairly straightforward forms – it was never meant to be used with overly complex flows. Or at least I think that’s the case.

I should have a post about managing dialog flows out soon, but in the meanwhile, here’s the big secret: IDialogContext.Wait(<method name>) lets you define the next method that will process the next user response.