Integration testing

Since a Botfuel Dialog chatbot is a web application, you can rely on powerful programming concepts like unit and integration testing to test your bot.

Goal

In this tutorial you’ll learn how to test your chatbot using Jest, a testing library developed by Facebook. However, any other testing library such as Mocha can be used.

The goal of this tutorial is to learn how to automate the tests you would have to do manually to make sure your chatbot works as expected.

For this tutorial, we’ll start from the starter sample bot. Make sure you have it running by following the Getting Started tutorial up until the installation part.

For instance, the starter sample bot has the following behavior:

Testing sample

Let’s write a test that ensures this simple functionality works.

Configuration

First, we need to install Jest as a development dependency:

npm install --dev jest

Once Jest is installed, add the following test script to your package.json file:

"scripts": {
  "start": "botfuel-run",
  "train": "botfuel-train",
  "test": "jest"
},

You can now run npm test to run Jest.

To run the chatbot in test mode, we’ll need a config file that indicates we’ll use the test adapter. Create a test-config.js file at the root level with this content:

module.exports = {
  adapter: 'test',
};

Writing the test

To simulate a conversation between a human and your chatbot, we’ll be using the play method of the Bot class. This method expects an array of user messages that represents all the messages a user would send to the chatbot. For a user, they can be of the following types:

  • PostbackMessage
  • UserImageMessage
  • UserTextMessage

In our case, they are simple UserTextMessages.

For example, to simulate this conversation:

Testing sample

We’ll write:

const bot = new Bot(config);
const userId = bot.adapter.userId;

await bot.play([
  new UserTextMessage('Hello'),
  new UserTextMessage('My name is Bob'),
]);

The captured output of both the user messages and the bot messages is stored into bot.adapter.log, so we can write the following assertion to test that the bot responds as expected:

expect(bot.adapter.log).toEqual(
  [
    new UserTextMessage('Hello'),
    new BotTextMessage('Hello human!'),
    new UserTextMessage('My name is Bob'),
    new BotTextMessage('Nice to meet you Bob!'),
  ].map(msg => msg.toJson(userId))
);

To compare expected output and actual output, we provide a convenient toJson method so messages can be compared as raw JSON. This method needs userId as a parameter because each UserTextMessage and BotTextMessage converted to JSON stores a userId property tracking the author of the message.

Here, you can also use all types of messages possible for both user and bot.

Let’s write the complete test case!

Create a tests directory at the root level of your chatbot and a hello.test.js file inside it with the following content:

const { Bot, BotTextMessage, UserTextMessage } = require('botfuel-dialog');
const config = require('../test-config');

test('answers greeting and name', async () => {
  const bot = new Bot(config);
  const userId = bot.adapter.userId;

  await bot.play([
    new UserTextMessage('Hello'),
    new UserTextMessage('My name is Bob'),
  ]);

  expect(bot.adapter.log).toEqual(
    [
      new UserTextMessage('Hello'),
      new BotTextMessage('Hello human!'),
      new UserTextMessage('My name is Bob'),
      new BotTextMessage('Nice to meet you Bob!'),
    ].map(msg => msg.toJson(userId))
  );
});

To run the tests, execute the following command:

BOTFUEL_APP_TOKEN=<the BOTFUEL_APP_TOKEN> BOTFUEL_APP_ID=<the BOTFUEL_APP_ID> BOTFUEL_APP_KEY=<the BOTFUEL_APP_KEY> npm test

Your app’s credentials are needed because your chatbot calls NLP APIs.

You will see the following results:

Testing output

Congratulations, you have successfully automated a test case of your chatbot!

Note: Making your bots rely on APIs that are outside of your control make them dependent on the status of these APIs (downtime, API changes) and thus can fail even though you did not change anything in your bot. To make them unit tests (independent of anything other than your code, as opposed to integration tests), you will need to mock the called APIs, which is outside the scope of this tutorial.