Artificial Intelligence Methods for Conversational Agents in Healthcare
Facilitators:
Will Kearns, Aakash Sur, BHI PhD Students and Trevor Cohen, BHI Faculty
Details:
Tuesdays, Autumn Quarter: 11:30 am-12:20 pm, Health Sciences Building, T478
Course Description:
Through this course, students will be introduced to reinforcement learning methods and how to apply them to train health dialog systems to address specific problems in healthcare. We will cover a range of machine learning methods including tree search, tree pruning, Markov decision processes, and Q-learning. We will explore both classical methods and recent advances in the development of dialog system components including natural language understanding, dialog management, and natural language generation. The course structure will be a mixture of lectures and interactive coding sessions culminating in the deployment of a health dialog system.
We welcome questions during the class as others might share the same questions. If you need individual help, please see one of the instructors after class or send a question to the group on slack.
Course Reading:
Designing Voice User Interfaces: Principles of Conversational Experiences
Week 1 - 10/1/19
Title:
Introduction to Conversational Agents and Reinforcement Learning
Description:
We will introduce conversational agents operating within a natural language environment, an ideal context for employing reinforcement learning (RL). We will survey the methods of RL and its applications of NLP. Finally, we will cover the software requirements for the class, and ensure students can interactively follow along in coding exercises.
Lecture:
Reading:
Coding:
None
Week 2 - 10/8/19
Title:
Natural Language Understanding
Description:
We will explore how we can train agents to understand their conversational environments.
Lecture:
Reading:
ONENET:Joint Domain, Intent, Slot Prediction for Spoken Language Understanding
Dynamic Integration of Background Knowledge in Neural NLU Systems
Coding:
Train NLU model: with a Pipeline in Rasa
Week 3 - 10/15/19
Title:
Tree Search
Description:
We will model decisions as trees and learn to efficiently search them using classic algorithms such as breadth-first search and depth-first search. In addition, we will introduce heuristic based searches, including A* search.
Lecture:
Reading:
Joint A* CCG Parsing and Semantic Role Labelling
Coding:
Week 4 - 10/22/19
Title:
Advanced Tree Searches
Description:
We will cover how to model two player games as trees, and how the optimal strategy can be recovered from these trees. In addition, we will cover how to prune these trees to limit the total search space using alpha-beta pruning, and heuristic pruning.
Lecture:
Reading:
None
Coding:
In Class Exercise with Grundy’s Game of Nim
Week 5 - 10/29/19
Title:
Dialog Management
Description:
Reading:
Health dialog systems for patients and consumers
Coding:
Train Rasa DM with Interactive Learning
Week 6 - 11/5/2019
Title:
Markov Decision Processes
Description:
In this class, we will extend our tree based decision models to graphs with Markov models. We will learn how to calculate the best route through a Markov decision process (MDPs) using the Bellman equations. Finally, we will extend these ideas to conversational agents using partially observable Markov decision processes (POMDPs).
Lecture:
Reading:
POMDP-Based StatisticalSpoken Dialog Systems:A Review
Training a real-world POMDP-based Dialogue System
Coding:
None
Week 7 - 11/12/2019
Title:
Q-Learning
Description:
Here we will introduce one of the key concepts in RL, Q-learning. This approach overcomes the limitations of MDPs and allows us to conduct on-line or off-line learning without complete information.
Lecture:
Reading:
TBD
Coding:
TBD
Week 8 - 11/19/2019
Title:
Deep Q-Networks
Description:
Moving past basic tabular Q-learning, we will cover current approaches which revolve around Deep Q-Networks (DQN). We will cover popular examples of DQNs used to master video games, and conversations. Finally, we will cover how to efficiently train these models using experience replay.
Lecture:
Reading:
Playing Atari with Deep Reinforcement Learning
Agenda-based user simulation for bootstrapping a POMDP dialogue system
A User Simulator for Task-Completion Dialogues
Coding:
Train dialog policy w/ episodic replay
Train a DQN using an agenda based User Simulator
Week 9 - 11/26/2019
Title:
Advanced Neural Methods for Dialog Systems
Description:
We will cover advanced neural architectures for training dialog systems, e.g. A2C and MemNN models.
Reading:
ConveRT: Efficient and Accurate Conversational Representations from Transformers
The Illustrated Transformer (Transformer Tutorial)
Coding:
TBD
Week 10 - 12/3/2019
Title:
Ethics and NLG
Description:
We will finish off the course with a discussion of ethics in the development of dialog systems using two case studies and then focus the discussion on health dialog systems in particular.
Reading:
The Design and Implementation of XiaoIce, an Empathetic Social Chatbot
Twitter taught Microsoft’s AI chatbot to be a racist asshole in less than a day
Coding:
TBD