Conversational agents have a tough job following the zigs and zags of human conversation. They’re getting better at it — thanks to yesterday’s technology.
What’s new: Amazon recently improved the Alexa chatbot’s ability to identify the current topic of conversation. The system keeps its responses relevant by tracking the back and forth between itself and the user.
Key insight: In conversation, the topic can shift fluidly. The meaning of a word that’s ambiguous in a single conversational exchange, such as “it,” is often clear in light of previous conversational turns. Evaluating several exchanges makes it possible to identify the current topic more accurately.
How it works: The system recognizes 12 common topics (like politics, sports, fashion, books, and movies) and 14 intentions (like information request, opinion request, and general chat). The training data came from 100,000 conversations gathered in the 2017 Alexa Prize competition. Human annotators labeled a topic and intention for each statement.
- Each time a user or Alexa speaks, a 2017-vintage architecture known as a conditional adversarial domain network predicts the current dialog action.
- A pre-trained network extracts word vectors and passes them as a sequence to a biLSTM, a small, efficient recurrent layer that debuted in 2015.
- The biLSTM reads through what has already been said, word by word, forward and backward, to extract conversational features.
- Based on the features and dialog action, the biLSTM predicts the current topic.
Results: Amazon evaluated its topic identifier using a test dataset collected alongside the training data. The system exceeded baseline accuracy of 55 percent to achieve 74 percent accuracy when it used context from five conversational exchanges.
Why it matters: There’s plenty of life left in older techniques. Given the right data, algorithms from years ago can still do well on modern tasks.
We’re thinking: Is it too much to ask that deep learning take its place alongside sports and fashion as one of the 12 topics?