As the artificial intelligence is becoming more and more important on our daily job, in this tutorial we will provide some simple steps on how to facilitate automated testing based on an AI Chat Bot. What we are trying to achieve is the simulation of a real QA manual tester.
The scenario covered is an interaction between a QA tester which will send out natural language messages to a bot. “He”/”She” will automatically analyze the messages and perform actions on Wikipedia.org using Selenium WebDriver.
For example, see below how to open the browser:
Enter your message to the bot: open firefox browser On my way, wait for it! Enter your message to the bot: hello SAL SAL Enter your message to the bot: open firefox browser On my way, wait for it!
- Python 3.x
- Python packages: aiml, fuzzywuzzy, selenium
In the above picture, you can find the high-level architecture for QA Chat Bot. The idea is pretty simple: the actual user sends a message to the chatbot through the console -> the message is processed by the main.py script (the entry point of the chat-bot -> BotTrainer class is instantiated and two methods could be used (simple_learning () or brain_learning ()) -> The BotTrainer uses the Kernel class which facilitates the communication with the startup.xml -> Based on the startup file, a specific .aiml file is executed. On the other hand, the chat_executor trigger different QA scenarios (Selenium based scripts that interact with the browser) Based on the keyboard input qa_scenarios and .aiml files are synchronized.
Now let’s take it step by step.
Step 1 – std-startup.xml file
This is the root file used for loading different ‘.aiml’ files. It’s actually a simple .xml file where you configure the actual brain of the bot. It has three important tags:
- category – an aiml unit used to select a specific category to be used by the bot. Each category could contain one pattern and many templates
- pattern – it provides the access to different .aiml files
- template – a group of .aiml files to be used in a category
- learn – the path to the .aiml file
Step 2 – AIML files definition
AIML – Artificial Intelligence Modelling Language is an XML designed to create artificial intelligence based applications. Below you can find an example. Same as the startup XML file any aiml file should contain three tags
- category – a group of patterns and their related templates
- pattern – pattern linked to the input sent by a user to the chatbot
- template – the associated template to a pattern
Step 3 – BotTrainer
This class provides the access to the chatbot Kernel. Based on it we have the following methods:
_start_chat– a protected method used to facilitate the interaction between the user and the QA chatbot. It’s also the place where the
chat_executoris integrated with the bot. More details about it will be provided in step 5.
simple_learning– this method takes the startup.xml file as an input and give the proper response based on its content and more important through the pattern parameter, which delegates the pattern to be used to the user.
brain_learning– similar to
simple_learning, the only difference is that based on
bootstrap() function, we have the option to save the content of different .aiml file to a brain and when adding more and more files we don’t have to wait for loading the previous saved ones, so the learning time will be much faster.
class BotTrainer: def __init__(self): self._kernel = aiml.Kernel() def _start_chat(self): """ Start the communication with the bot """ executor = WikiActions() while True: message = input("Enter your message to the bot: ") if message == "quit": exit() elif message == "save": self._kernel.saveBrain("public/bot_brain.brn") else: chat_executor(message, executor, 70) print(self._kernel.respond(message)) def simple_learning(self, pattern): """ Train the bot based on a pre-defined pattern :param pattern: pattern used to train the bot """ self._kernel.learn("public/std-startup.xml") self._kernel.respond(pattern) self._start_chat() def brain_learning(self, pattern): """ Train the bot based on a pre-defined pattern including a storage option into the brain bot of some actions :param pattern: pattern used to train the bot """ if os.path.isfile("bot_brain.brn"): self._kernel.bootstrap(brainFile="public/bot_brain.brn") else: self._kernel.bootstrap(learnFiles="public/std-startup.xml", commands=pattern) self._kernel.saveBrain("public/bot_brain.brn") self._start_chat()
Step 4 – QA Scenarios
Based on page object pattern, this class is mostly a classical page where different interactions with the browser are implemented. We’ve listed some scenarios just for the sake to evidentiate how the chatbot could interact with the Selenium
class WikiActions: def __init__(self): """ Create a Chrome instance """ self._driver = webdriver.Firefox(executable_path=os.getcwd().split("src") + "/public/geckodriver") def open_page(self): """ Open the wiki home page """ return self._driver.get("https://en.wikipedia.org/wiki/Main_Page") def click_create_account(self): """ Click on create account """ return self._driver.find_element_by_id("pt-createaccount").click() def click_login(self): """ Click on login """ return self._driver.find_element_by_id("pt-login").click() def set_query(self, query): """ Set your query """ return self._driver.find_element_by_id("searchInput").send_keys(query) def click_search(self): """ Press search button """ return self._driver.find_element_by_id("searchButton").click() def set_full_screen(self): return self._driver.maximize_window() def minimize_screen(self): return self._driver.minimize_window() def close_browser(self): """ Quit driver """ return self._driver.quit()
Step 5 – Chat executor
This is actually the chatbot core, where the user input is preprocessed and the actual messages are converted into real functionality.
Each input is linked to a message and if the pattern sent by a user matches a group of strings, a specific method is called. We’ve also integrated the fuzzywuzzy python package in order to be more flexible on pattern matching. In this case, if the actual message is “Serch for dsgn patterns” and the content save into the aiml file is “Search for design patterns” the message is still found and the proper method will be called.
def chat_executor(topic, qa_flow, threshold): """ :param threshold: - Levenshtein distance between two strings - Matches the similarity between user input and its corresponding value saved in bot brain - an alternative to Regex (if any(re.findall(r'set|now', "NOW SEARCH FOR DESIGN PATTERNS", re.IGNORECASE))) :param qa_flow: qa flow to be executed while running the bot :param topic: user input to be processed by chat executor :return: the processed topic """ if fuzz.partial_ratio(topic, "open firefox browser") > threshold: return qa_flow.open_page() elif fuzz.partial_ratio(topic, "set full screen") > threshold: return qa_flow.set_full_screen() elif fuzz.partial_ratio(topic, "now search for design patterns") > threshold: return qa_flow.set_query("design patterns") elif fuzz.partial_ratio(topic, "click search button") > threshold: return qa_flow.click_search() elif fuzz.partial_ratio(topic, "close the browser") > threshold: return qa_flow.close_browser() elif fuzz.partial_ratio(topic, "minimize browser") > threshold: return qa_flow.minimize_screen() elif fuzz.partial_ratio(topic, "now close the browser") > threshold: return qa_flow.close_browser() else: return topic
Let’s see how a simple search flow looks like based on the interaction with the QA chatbot:
Enter your message to the bot: hello SAL SAL Enter your message to the bot: run some tests Ok, it seems you're lazy today Enter your message to the bot: open firefox browser On my way, wait for it! Enter your message to the bot: set full screen Got it! Enter your message to the bot: now search for design patterns Ok, let's do this! Enter your message to the bot: click search button You can find here your results! Enter your message to the bot: thanks You're welcomed! Enter your message to the bot: now close the browser OK, Browser closed! Enter your message to the bot: bye bye Bye by mr. QA! Enter your message to the bot: quit
If you want to see the script running, try it by your self. The link to the actual code is listed in conclusions section.
Now we are done with the QA chatbot. Below you can find my feedback on this approach:
- Easy to understand
- Easy to maintain
- It could be the first step on translating your manual test cases into fully automated commands, so as a user you don’t need any technical background for using or even testing an app based.
- It needs the creation of a real database, as the number of .aiml files could become more complex and much hard to maintain and not the best choice in terms of memory usage.
- The integration of some extra logic is pretty restrictive, so any new functionality must be done from scratch
As a conclusion, I think you should give a try especially if it’s your first interaction the implementation of a QA chatbot, and especially for small applications it’s a good approach.
More details about the implementation could be found here