The second in a 3 part blog series by Melbourne consultant Jeremiah Mannings
Introduction
This is the second part in a series of Watson Conversation API articles that aims to cover the design and development of a finance-based chatbot. To check out the first article here!
In this part of the series, we are going to take an in-depth look at the usage of entities and dialog in the system. Both of these are key factors in how a WCS chatbot differentiates and processes natural language, and provides the user understanding that helps make the system as natural as possible to interact with.
Links
- Watson Conversation WCS Intent Design
- Part 2: Watson Conversation Entity & Dialog Design
- Part 3: Watson Conversation WCS Further Tips & Tricks (coming soon!)
Entity Design
There have been many entities collected in the system entities for WCS, however, they need to be expanded in order to suit our example finance use case in order to keep consistent and in line with the design of the system. The system entities and the added custom entities can both be utilised in the design. Entities will be used as previously shown to encapsulate the nuances of the user’s intent. Matching both an intent and an entity will result in the answer the user will see. Therefore the entities must be grouped logically, for example, similar types of taxes, or loans etc.
When the entities are well grouped it makes the mapping of similar intents easier, as seen in the examples in the Intent Design section of the previous article. Similar entities are mapped into groups, and then titled as appropriate, such as “loan_type” or “tax_type”, the values within the entity may not be directly related in practice, but they are around the same theme, they can be bundled logically for development and then refined for production.
The system doesn’t consider entities any further than a string match generally, which is to say unless the system you are designing has a type of spell check the system will only match entities when that concept or its children is matched. This means the associations and grouping of entities don’t affect the system whatsoever unless they crossover, which is called crosstraining the system is to be avoided when considering entities.
Primary entity -> Children -> Synonyms of Children
In this design, we are referencing purely the children of the entities (which considers the synonyms of that child), therefore the group is not very important, and the main consideration is keeping it logical. For example:
This is an example I created that may not accurately represent what they will look like upon implementation but it does provide a good view of what a properly setup entity would look like. If you have very general questions you can also reference the entire entity in this example “@loan_type” which would reference all children and all the synonyms of the children. Generally, the logical gates use with entities references the child, so you may reference “@loan_type:home” in order to respond to a users query. Using “@loan_type:home” would look for matches on “home” as well as all of the entities listed under the synonyms for that entity child.
Dialog Design
The dialog design for this example implementation is simple but effective in reducing the large overhead of intents that can be present in some systems. This involves setting up the intents and entities as ‘gates’ of sorts. The logic to the system is as follows:
Match incoming user query to an intent -> Match an entity -> Provide answer
This design also enables chit chat to be added into the bot as needed, as confidence thresholds can be set on the intents to only match when the system is reasonably confident in the match. This makes it much easier to filter out valid questions and chit chat, making for a more realistic system when the user interacts.
This is a simplistic but effective way of designing what is called “dialog flows”. Dialog flows are guides the user walks down when interacting with a system, these are designed to follow a logic path, but also able to be broken out of when needed if the users change direction (or flow). This is a careful consideration with dialog design as users can often chop and change their queries and directions as new information is made available to them, this means when designing a user flow a heavy emphasis on flexibility is present.
If a user can only access certain information by engaging in a logical path you have created this can sometimes cause frustration if they return to the system, as you are forcing them down a path to that information. In order to avoid this, a majority of the nodes you create in the system should be accessible from any stage (using whats call jumps, which are logical access points into dialog flows) which can essentially “jump” into any part of a user flow when the user is trying to access the information.
Examples
If you decide to use gates, they can be effectively setup on more important questions instead of every question, as they represent more development work to add en masse. The general code is using two nodes as ‘gates’ that continue the flow of the conversation, these are done by using the code snippet:
intents[0].confidence>0.5
What this does is pick the first intent surfaced (intent 0 as this is 0 based) and assess the confidence parameter, this can be set greater (>) or lesser (<) or even equal (=) to whatever threshold is used in the system. For a large system such as this one a recommended confidence level is around 50% or 0.5.
The screenshot above shows how a confidence gate is generally used in the system, utilising the continue-forms in the system can be a powerful way of determining the best way to answer the query. The aforementioned confidence threshold code is used to filter the intents when the intent matches at a confidence greater than 50% the conversation then moves on to match the entities and answer the question.
The single node way of doing this is far easier as it combines the intent, entity, and confidence match into a single node. Form this combination you can even add things like multiple entity matches using OR gates, which can further expand the ability of the system to understand users. This is done using three condition triggers and using AND statements, it evaluates to:
#wantloan AND @loantypes:boat AND intent[0].confidence<0.5
This can be used to insert single-line gated nodes into the conversation API. This is an easier way of doing smaller implementations, however, the previous method of gating the queries is a far better way of doing more productionized systems. It can also enhance the ability to write users flows in the system.
I would recommend using confidence quite often, the confidence is partly what sets WCS apart from just a text matching system as it is the output of the advanced natural language processing the system is doing. Having a confidence and tweaking the parameters can help you design a system that will reduce false positives.
Package Design Notes
The design ethos in this example is using a single chatbot, however, when you start designing large enterprise level chatbots the rational choice is to split the bot into mutiple domain trained instances. Technically you can set up multiple pipelines and move the user’s request around as needed using a middleware layer, however as this example is all in a single domain it makes more sense to keep the example bot in the single pipeline.
A quick tip is to use node identification keys which can be a great way to bug test and see how the system is tracking when answer questions. It also provides a way to find nodes other than just the name (which can sometimes get confusing). When the .json package is sent back, the name of the node is not included, and therefore changing individual nodes can be difficult.
Adding a node identification key helps get around this, the NODEID parameter is added to the .json package and mapped in a sheet, that way the development team can logically see what nodes they are hitting when testing.
Adding a NODEID
When adding answers to a node, the json structure can also be modified directly using the code editor. The typical setup includes adding the NODEID parameter to the .json file, and naming the node with the ID.
As you can see here the name of the node has been updated to the NODEID “1111” which can be used to track the node.
Below you can see the .json response of the node, which has been modified to have a NODEID parameter in the output section, this will output this node ID every time the node is hit.
This allows the node to be tracked from the front end. It can also help to provide statistics on which nodes are being hit by recording the number of times a node is accessed. This can be a great way to gain insight into which parts of the system need the most attention when developing and assessing user behaviour.
The tracking of nodes is done via a spreadsheet which resembles the following format:
This helps the system be comprehensible when it’s being updated and reviewed and keeps a single source of truth for the system.
Conclusions and next steps
I hope this second part of the WCS series was useful and helped expanded some the concepts around entities and the use of dialog in the system. With WCS the very best thing you can do is use it! In the next article following this I will provide some useful links to GitHub projects to get started and some pages that will help with tips and tricks. This article should give you a good starting understanding of the major concepts in the system and how they work together to take the user on a journey.
Let me know what you thought of the article and be sure to comment if you have any questions or need any clarification!